Job Title : Data Engineer
Location : Hyderabad (Work from Office)
Experience : 5+ Years
Notice Period : Immediate / Last Working Day till 31st August (Serving Notice the Role :
We are looking for an experienced Data Engineer to design, develop, and optimize scalable data pipelines and cloud-based big data solutions, preferably on Microsoft Azure. The ideal candidate will have strong expertise in ETL development, big data platforms, and performance optimization of large-scale datasets. This role demands hands-on technical skills with tools like Informatica, Apache Hive, HDFS, and Azure Data Lakehouse architectures, along with experience in scheduling and orchestration tools.
Key Responsibilities :
- Design, build, and maintain scalable and reliable data pipelines for analytics and business needs.
- Develop and optimize ETL workflows using Informatica.
- Work with big data platforms such as Apache Hive and HDFS for distributed data processing.
- Manage and optimize data storage solutions with Azure Data Lake and Lakehouse architectures.
- Collaborate with cross-functional teams to ensure data quality, governance, and security compliance.
- Implement performance tuning, data partitioning, and query optimization strategies for large datasets.
- Use workflow orchestration and scheduling tools such as Airflow, Control-M, or Azure Data Factory.
- Support DevOps and CI / CD practices for data pipelines and Skills & Experience :
- 5+ years of experience as a Data Engineer with hands-on expertise in :
1. Apache Hive, HDFS, and distributed data processing.
2. ETL development (preferably with Informatica).
3. Relational databases and data modeling for structured and semi-structured datasets.
4. Azure Data Lake and Lakehouse architecture.
Strong knowledge of performance tuning and optimization for large-scale data Skills :Experience with scheduling and orchestration tools (Airflow, Control-M, Azure Data Factory).Familiarity with data quality frameworks, governance practices, and cloud security standards.Exposure to CI / CD pipelines and DevOps practices in data engineering workflows.(ref : hirist.tech)