Senior Data Engineer

Data Engineer Role

Location : India Remote

Are you a passionate Data Engineer ready to design and build cutting-edge, scalable, and reliable ML / AI solutions? Join our dynamic cross-functional feature team in India and drive innovation from concept to industrialization. We're looking for an expert to integrate and optimize algorithms across the entire product lifecycle, ensuring robust and performant data pipelines.

What You'll Do :

• Architect and Implement : Collaborate with AI Product Managers and DevOps to co-realize data and tech due diligence, setting up pristine exploratory environments. You'll design and build efficient data pipelines on demand, meticulously measuring cost-effectiveness across transformation scenarios.
• Innovate and Optimize : Embed critical monitoring metrics into products, design future-proof MVP target architectures, and meticulously package and document your code and infrastructure. Your expertise will optimize ML / AI algorithm performance in distributed systems, enhancing model response times and parallelism.
• Drive Future-Forward Initiatives : Stay at the forefront of technology through continuous technological watch, test new innovations in real-world use cases (MVMs & MVPs), and champion market-shifting technologies that add tangible value. You'll even contribute to selecting new service providers.

Your Technical Prowess Will Shine In :

○ Big Data : Spark is a must, coupled with knowledge of message-based systems like Kafka and Stream. Experience with Azure Databricks is highly valued.

○ Containerization : Proficient in Docker and Kubernetes.

○ Architecture & Cloud Patterns : Expertise in Microservices (Azure Functions), SOA architectures, and API design practices (HATEOAS, GraphQL).

○ Databases : Strong with SQL (Azure SQL Server) and NoSQL (ElasticSearch,

CosmosDB, MongoDB). Proven experience with Snowflake is a significant plus.

○ Deep understanding of data structures and big data-oriented file formats (Parquet / Delta, ORC, AVRO).

○ Familiarity with main Machine Learning algorithms (Reinforcement, Supervised, Unsupervised).

○ Hands-on coding experience with analytics and ML frameworks : NumPy, Scikit-learn, Pandas.

○ Experience with experimental environments : Jupyter Notebook, Zeppelin, MLFlow.

○ Proficiency in deep learning frameworks : TensorFlow, Keras, Caffe.

○ Experience with dashboarding and visualization tools (PowerBI in particular) is a plus.

○ Experience with Azure Machine Learning Services is a plus.

○ Architect-level knowledge of Azure Services, with a sharp focus on data services like Azure Data Factory, Data Lake, Databricks.

○ Solid understanding of governance, networking, and security in the cloud (subscriptions, resource groups, SSOs, VPC, NSGs).

○ Knowledge of other cloud offerings (AWS, GCP) is a plus, but Azure Cloud experience is mandatory.

○ Comfortable with Git for Source Code Management.

○ Adept at creating build and release pipelines in CI / CD tools like Azure DevOps (preferred) or Gitlab.

○ Experience with workflows and ETL / ELT processes.

○ Familiarity with ADLS Gen2, Azure Functions, and Logic Apps.

○ Understanding of Medallion architecture (Bronze, Silver, Gold).

○ Experience with Jira for project management.

○ Familiarity with Azure OpenAI and working with APIs / webapps and other relevant Azure services.

Qualifications :

Your Core Competencies :

measurement, DevOps methodology, coding & scripting, and thorough validation & testing.

Personal Traits :

If you're ready to make a significant impact on our AI / ML journey and thrive in a fast-paced, innovative environment, apply now!