As a Data Engineer, your primary role is to create and maintain high-performance data pipelines, warehouses, and cloud-based systems. This includes creating efficient batch and streaming pipelines, using Snowflake for scalable data warehousing, and leveraging AWS for cloud development. You will work with an agile team to ensure the reliability and accuracy of data applications through rigorous testing. In addition, you will contribute to documentation and stay current on upcoming technologies in order to promote continuous development.
Role Responsibilities
- As a Data Engineer, you will be responsible for a range of duties related to the development and maintenance of data pipelines, data warehouses, and cloud-based systems. Some of the key job duties are :
- Building performant batch and streaming data pipelines : You will be responsible for designing and building efficient data pipelines that can handle large volumes of data, both in batch and real-time formats
- Data warehouse development, using Snowflake to build scalable and reliable data warehousing solutions
- Cloud-based development using Amazon Web Services (AWS) to build scalable and reliable systems
- Maintenance of various data applications, pipelines, and databases
- You will be part of an agile development team, participating in daily stand-ups and other team meetings to collaborate and coordinate on development efforts
- Writing and executing various types of tests to ensure the quality, accuracy, and reliability of the data pipelines and data applications that you develop
- Contributing to documentation and best practice guidelines
- Staying up-to-date with new technology and techniques
Skills & Requirements
Experience working with Python and SQLSnowflake, Kubernetes, Amazon Web Services (AWS) and Airflow experience is desirableExperience in BI tools like ThoughtSpot, Power BI would be beneficial.Able to write clean, scalable and performant codeSome knowledge of Linux and the command lineStrong written and verbal communication skills including an ability to effectively communicate with both business and technical teamsBasic knowledge of data science topics like machine learning, data mining, statistics, and visualisationNote
This role follows UK shift timings and is based in Ahmedabad. Candidates from other locations can apply, as remote and hybrid work options are available.
About us
With over 20 years of experience, MSBC Group delivers cutting-edge technology solutions across industries like financial services, manufacturing, logistics, and startups. Our expertise spans AI, custom software, staff augmentation, and managed services. Operating globally, we drive innovation in trading platforms, application migrations, and real-time analytics, making us a trusted AI and automation partner.