We are seeking a visionary Senior Data Engineer to spearhead our Data Engineering team.
This role requires strong leadership and technical expertise in designing and implementing data architectures that meet business needs.
As a Senior Data Engineer, you will be responsible for leading a team of data engineers and collaborating closely with clients and stakeholders to understand their requirements and develop solutions that meet their needs.
You will also provide technical guidance and support to your team, helping them set up the Data Lake on Cloud and implementing standardized Data Models.
Main Responsibilities :
- Design and build pipelines to bring in diverse data from multiple sources within the organization and external social media and public data sources.
- Collaborate with cross-functional teams to source data and make it available for downstream consumption.
- Work with the team to design effective solution frameworks that meet business objectives.
- Ensure regular communication with key stakeholders and escalate any critical issues or dependencies to the Sponsor and / or Head of Data Engineering.
- Manage timelines, milestones, and decision-making processes to ensure value delivery without compromising quality or budget.
- Develop an appropriate communications plan for initiative execution and delivery, both internal and external.
Required Skills and Qualifications :
Strong problem-solving and analytical skills.Excellent understanding of Azure Databricks platform and experience working with at least one Cloud Infra provider (Azure / AWS).Experience in building data pipelines using batch processing with Apache Spark (Spark SQL, Dataframe API) or Hive query language (HQL).Knowledge of NoSQL databases, including Cosmos DB, Restful APIs, and GraphQL.Experience with Big data ETL processing tools, Data modeling, and Data mapping.Basic knowledge of scripting (shell / bash) and CI / CD tools such as Jenkins, JIRA, Bitbucket, Artifactory, Bamboo, and Azure Dev-ops.Ability to debug, fine-tune, and optimize large-scale data processing jobs.Benefits :
This role offers opportunities for career growth and professional development, working with a talented team of data engineers and experts in the field.
Competencies & Personal Traits :
Strong team player with excellent communication and collaboration skills.Ability to adapt to changing requirements and priorities.