Data Engineer

Job Responsibilities -

• Architect and implement scalable offline data pipelines for manufacturing systems including AMHS, MES, SCADA, PLCs, vision systems, and sensor data.
• Design and optimize ETL / ELT workflows using Python, Spark, SQL, and orchestration tools (e.g., Airflow) to transform raw data into actionable insights.
• Lead database design and performance tuning across SQL and NoSQL systems, optimizing schema design, queries, and indexing strategies for manufacturing data.
• Enforce robust data governance by implementing data quality checks, lineage tracking, access controls, security measures, and retention policies.
• Optimize storage and processing efficiency through strategic use of formats (Parquet, ORC), compression, partitioning, and indexing for high-performance analytics.
• Implement streaming data solutions (using Kafka / RabbitMQ) to handle real-time data flows and ensure synchronization across control systems.
• Building dashboards using analytics tools like Grafana.
• Good Understanding of Hadoop ecosystem.
• Develop standardized data models and APIs to ensure consistency across manufacturing systems and enable data consumption by downstream applications.
• Collaborate cross-functionally with Platform Engineers, Data Scientists, Automation teams, IT Operations, Manufacturing, and Quality departments.
• Mentor junior engineers while establishing best practices, documentation standards, and fostering a data-driven culture throughout the organization.

Essential Attributes  -

Qualifications  -

Desired Experience Level  -

Skills Required

Python, Pyspark, Apache Spark, Kafka, Rabbitmq, Sql