Data Engineer

Role : Associate Architect - Data

Experience Level : 6.5 to 12 Years

Work location : Mumbai, Bangalore, Trivandrum

Role & Responsibilities :

• 6+ years of experience in building and managing Data Lakes, Data Warehouse, Data Integration, Data Migration and Business Intelligence / Artificial Intelligence solutions on GCP Cloud.
• Ability to understand business requirements, translate them into functional and non-functional areas, and define non-functional boundaries in terms of Availability, Scalability, Performance, Security, Resilience etc.
• Experience in architecting, designing, and implementing end-to-end data pipelines and data integration solutions for varied structured and unstructured data sources and targets.
• Experience of having worked in distributed computing and enterprise environments like Hadoop.
• Well-versed with various Data Integration, and ETL technologies on Cloud like Spark, Pyspark / Scala.
• Dataflow, DataProc, EMR, etc. on various Cloud.
• Experience of having worked with traditional ETL tools likInformatica / DataStage / OWB / Talend, etc.
• Deep knowledge of one or more Cloud and On-Premise Databases like Cloud SQL, Cloud Spanner, Big Table, RDS, Aurora, DynamoDB, Oracle, Teradata, MySQL, DB2, SQL Server, etc.
• Exposure to any of the No-SQL databases like Mongo dB, CouchDB, Cassandra, Graph dB, etc. Experience in architecting and designing scalable data warehouse solutions on cloud on Big Query or Redshift.
• Experience in having worked on one or more data integration, storage, and data pipeline toolsets like S3, Cloud Storage, Athena, Glue, Sqoop, Flume, Hive, Kafka, Pub-Sub, Kinesis, Dataflow, DataProc, Airflow, Composer, Spark SQL, Presto, EMRFS, etc.
• Preferred experience of having worked on Machine Learning Frameworks like TensorFlow, Pytorch, etc.
• Good understanding of Cloud solutions for Iaas, PaaS, SaaS, Containers and Microservices Architecture and Design.
• Ability to compare products and tools across technology stacks on Google, AWS, and Azure Cloud. Good understanding of BI Reposting and Dashboarding and one or more toolsets associated with it like Looker, Tableau, Power BI, SAP BO, Cognos, Superset, etc.
• Understanding of Security features and Policies in one or more Cloud environments like GCP / AWS / Azure.
• Experience working in business transformation projects for the movement of On-Premise data solutions to Clouds like GCP / AWS / Azure.

Role :

Other Skills :