Designing, developing, and implementing data service architecture that ingests real time data streams, parses JSONs, and loads them in to multiple data persistence stores to provide real time and offline analytics. Building fact tables to facilitate quicker and easier data access. Building indices at Elastic Search to support real-time dash boards at Kibana, and building predictive models to support AI/ML. Experience using Scala on Spark, especially building ETL and complex query models. Experience on Hadoop platform including Big Data tools. Experience in developing shell scripts and running cron/oozie/spark jobs on Hadoop platform. Experience with Kafka, hBase, and Hive. Experience on ElasticSearch and Kibana. Experience in accessing and modeling NoSQL data models, especially with Cassandra. Experience with APIs, JSON, OLTP and real-time data processing. Experience with Java Script. Working experience on Linux and Cloud platforms. Knowledge of Big Data Tools. Experience in building reports using Tableau. Knowledge of business intelligence and analytics industry and best practices. Experience using data wrangling, data engineering, and feature engineering software. Experience in interpreting data models to build user friendly visualizations/dashboards. Experience in statistical techniques and quantitative methodologies that are used in decision making applications. Ability to work independently, and multi-task under short deadlines, based upon general direction. Effective verbal and written communication skills.
Hands on experience in:
Scala, Spark, Kafka, Realtime Streaming
Hadoop, Hive, NoSQL, SQL, Cassandra,