Job description
Position - Data Engineer
Experience Level : Mid-Senior Level
Experience Required : 3+ Years
Overview :
We are seeking a highly skilled and motivated Data Engineer to join our team. In this role, you
will collaborate with cross-functional teams to design, build, and maintain scalable data
platforms and solutions on the AWS Cloud. You will leverage your expertise in data engineering
tools and technologies to deliver next-generation application data platforms and optimize
current implementations. The ideal candidate should have a strong background in Databricks,
Spark, and Big Data ecosystems, along with experience in data warehousing, including
datamarts and data modeling.
Key Responsibilities :
- Develop and maintain scalable and high-performance data pipelines using AWS Glue, EMR,
Databricks, and Spark.
Design and implement robust ETL processes and frameworks to integrate, process, and analyzelarge datasets.
Build and optimize data models for structured and semi-structured data to support reporting,analytics, and machine learning workflows.
Utilize Python, PySpark, and SQL to develop and optimize data transformation logic.Collaborate with stakeholders to understand business requirements and translate them intotechnical solutions.
Implement best practices for data governance, security, and performance optimization on AWSCloud platforms.
Work with Big Data ecosystems, including Hadoop, Hive, Sqoop, and HDFS, to process andmanage large datasets.
Design and develop streaming data solutions using Spark Streaming, Kinesis, and Firehose.Contribute to the architecture and strategy for modernizing and scaling data platforms.Required Skills & Experience :
3-5 years of hands-on experience as a Data Engineer.Proficiency in Python, SQL, and PySpark.Strong knowledge of Big Data ecosystems, including Hadoop, Hive, Sqoop, HDFS, and HBase.Expertise in the Spark ecosystem : Spark Core, Spark Streaming, Spark SQL, and Databricks.Solid experience with AWS cloud services, including EMR, EC2 / EKS, Lambda, Glue, and S3.In-depth understanding of data modeling, data warehousing methodologies, and ETLprocesses.
Familiarity with data governance, quality, and security principles in cloud environments.Excellent problem-solving skills and ability to work independently or collaboratively in a fastpaced environment.Please revert me back with your confirmation mail and updated CV!!
