As a Data Engineer, your primary role is to create and maintain high-performance data pipelines, warehouses, and cloud-based systems. This includes creating efficient batch and streaming pipelines, using Snowflake for scalable data warehousing, and leveraging AWS for cloud development. You will work with an agile team to ensure the reliability and accuracy of data applications through rigorous testing. In addition, you will contribute to documentation and stay current on upcoming technologies in order to promote continuous development.
Role Responsibilities
As a Data Engineer, you will be responsible for a range of duties related to the development and maintenance of data pipelines, data warehouses, and cloud-based systems. Some of the key job duties are :
Building performant batch and streaming data pipelines : You will be responsible for designing and building efficient data pipelines that can handle large volumes of data, both in batch and real-time formats
Data warehouse development, using Snowflake to build scalable and reliable data warehousing solutions
Cloud-based development using Amazon Web Services (AWS) to build scalable and reliable systems
Maintenance of various data applications, pipelines, and databases
You will be part of an agile development team, participating in daily stand-ups and other team meetings to collaborate and coordinate on development efforts
Writing and executing various types of tests to ensure the quality, accuracy, and reliability of the data pipelines and data applications that you develop
Contributing to documentation and best practice guidelines
Staying up-to-date with new technology and techniques
Skills & Requirements
Experience working with Python and SQL
Snowflake, Kubernetes, Amazon Web Services (AWS) and Airflow experience is desirable
Experience in BI tools like ThoughtSpot, Power BI would be beneficial.
Able to write clean, scalable and performant code
Some knowledge of Linux and the command line
Strong written and verbal communication skills including an ability to effectively communicate with both business and technical teams
Basic knowledge of data science topics like machine learning, data mining, statistics, and visualisation
Note
This role follows UK shift timings and is based in Ahmedabad. Candidates from other locations can apply, as remote and hybrid work options are available.
About us
With over 20 years of experience, MSBC Group delivers cutting-edge technology solutions across industries like financial services, manufacturing, logistics, and startups. Our expertise spans AI, custom software, staff augmentation, and managed services. Operating globally, we drive innovation in trading platforms, application migrations, and real-time analytics, making us a trusted AI and automation partner.
Senior Data Engineer • Pune, India