Talent.com
This job offer is not available in your country.
AWS Data engineer / Architect

AWS Data engineer / Architect

Tata Consultancy ServicesBengaluru, Karnataka, India
19 days ago
Job description

Job Summary : In this key leadership role, you will lead the development of foundational components for a Lakehouse architecture on AWS and drive the migration of existing data processing workflows to the new Lakehouse solution. You will work across the Data Engineering organisation to design and implement scalable data infrastructure and processes using technologies such as Python, PySpark, EMR Serverless, Iceberg, Glue and Glue Data Catalog. The main goal of this position is to ensure successful migration and establish robust data quality governance across the new platform, enabling reliable and efficient data processing. Success in this role requires deep technical expertise, exceptional problem-solving skills, and the ability to lead and mentor within an agile team.

Must Have Tech Skills :

  • Prior Principal Engineer experience, leading team best practices in design, development, and implementation, mentoring team members, and fostering a culture of continuous learning and innovation
  • Extensive experience in software architecture and solution design, including microservices, distributed systems, and cloud-native architectures.
  • Expert in Python and Spark, with a deep focus on ETL data processing and data engineering practices.
  • Deep technical knowledge of AWS data services and engineering practices, with demonstrable experience of implementing data pipelines using tools like EMR, AWS Glue, AWS Lambda, AWS Step Functions, API Gateway, Athena
  • Experience of delivering Lakehouse solutions / architectures

Nice To Have Tech Skills :

  • Knowledge of additional programming languages and development tools to provide flexibility and adaptability across varied data engineering projects
  • A master’s degree or relevant certifications (e.g., AWS Certified Solutions Architect, Certified Data Analytics) is advantageous
  • Key Accountabilities :

  • Lead complex projects autonomously, fostering an inclusive and open culture within development teams. Mentor team members and lead technical discussions.
  • Provides strategic guidance on best practices in design, development, and implementation. Leads the development of high-quality, efficient code and develops necessary tools and applications to address complex business needs
  • Collaborates closely with architects, Product Owners, and Dev team members to decompose solutions into Epics, leading the design and planning of these components.
  • Drive the migration of existing data processing workflows to a Lakehouse architecture, leveraging Iceberg capabilities.
  • Serves as an internal subject matter expert in software development, advising stakeholders on best practices in design, development, and implementation
  • Key Skills :

  • Deep technical knowledge of data engineering solutions and practices. Expert in AWS services and cloud solutions, particularly as they pertain to data engineering practices
  • Extensive experience in software architecture and solution design
  • Specialized expertise in Python and Spark
  • Ability to provide technical direction, set high standards for code quality and optimize performance in data-intensive environments.
  • Skilled in leveraging automation tools and Continuous Integration / Continuous Deployment (CI / CD) pipelines to streamline development, testing, and deployment.
  • Exceptional communicator who can translate complex technical concepts for diverse stakeholders, including engineers, product managers, and senior executives.
  • Provides thought leadership within the engineering team, setting high standards for quality, efficiency, and collaboration. Experienced in mentoring engineers, guiding them in advanced coding practices, architecture, and strategic problem-solving to enhance team capabilities.
  • Educational Background :

  • Bachelor’s degree in computer science, Software Engineering, or a related field is essential.
  • Bonus Skills :

  • Financial Services expertise preferred, working with Equity and Fixed Income asset classes and a working knowledge of Indices.
  • Create a job alert for this search

    Aws Data • Bengaluru, Karnataka, India

    Related jobs
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    L&T Technology ServicesBengaluru, Karnataka, India
    Graduate with 8-10 years of experience working as a AWS Data Engineer.Excellent knowledge of PySpark, JavaScript, AWS - RDS, and Redshift.Show moreLast updated: 30+ days ago
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Tata Consultancy ServicesBengaluru, Karnataka, India
    Greetings from TATA Consultancy Services.Pls find the Job Description below.Hands-on experience in Pyspark, Glue.Experience on EMR, S3, IAM, Lambda, Cloud formation, Python.AMI Rehydration, Python,...Show moreLast updated: 30+ days ago
    • Promoted
    Data MDM Architect (Azure / AWS)

    Data MDM Architect (Azure / AWS)

    Tata Consultancy Serviceshosur, tamil nadu, in
    Job Title : Azure / AWS Data MDM Architect.Years of Experience - 10-16 years.We are seeking an accomplished Data MDM Architect with deep expertise in designing and implementing cloud-based Master Data...Show moreLast updated: 9 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Manuh Technologieshosur, tamil nadu, in
    Strong proficiency in Python programming (Pandas, NumPy, PySpark, or similar).Hands-on experience with Dask for large-scale distributed data processing. Proven expertise as a Data Modeler (conceptua...Show moreLast updated: 22 days ago
    • Promoted
    Agentic Data Engineer (AWS)

    Agentic Data Engineer (AWS)

    Sustainability Economics.aiBengaluru, Karnataka, India
    AI, enabling profitable energy transitions while powering end-to-end AI infrastructure.By integrating AI-driven cloud solutions with sustainable energy, we create scalable, intelligent ecosystems t...Show moreLast updated: 9 days ago
    • Promoted
    Senior AWS Data Engineer

    Senior AWS Data Engineer

    TEKsystemsBengaluru, Karnataka, India
    Your skills & Your Experience Experience.Analytics and Data Warehousing.Experience in building end-to-end solutions with the Big data platform, Spark or scala programming.Solid experience in ETL pi...Show moreLast updated: 7 days ago
    • Promoted
    GenAI Engineer

    GenAI Engineer

    Xebiahosur, tamil nadu, in
    Any Xebia Location (Hybrid, 3 days office per week).Building and deploying GenAI solutions leveraging.Collaborating with global teams under US overlap hours. AWS, GenAI, Bedrock, AgenticAI.Ability t...Show moreLast updated: 28 days ago
    • Promoted
    Lead Azure Data Engineer

    Lead Azure Data Engineer

    RandomTreeshosur, tamil nadu, in
    We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies. If you're passionate about building r...Show moreLast updated: 27 days ago
    • Promoted
    AWS Data Engineer for Bangalore

    AWS Data Engineer for Bangalore

    DeloitteBengaluru, Karnataka, India
    Looking for Skilled AWS Data Engineer, Interested candidates please find the JD below : .Bachelor’s Degree and 3+ years’ experience in the implementation of modern data ecosystems in AWS / Cloud platfo...Show moreLast updated: 8 days ago
    • Promoted
    Data Architect

    Data Architect

    InfoBeanshosur, tamil nadu, in
    Data Architect – Snowflake & Matillion.Snowflake, Matillion, Data Modeling, SQL, Python / Bash, Cloud Platforms (AWS / Azure). Immediate to 15 days preferred.The ideal candidate is proactive, self-direc...Show moreLast updated: 18 days ago
    • Promoted
    • New!
    AWS DeVops Engineer

    AWS DeVops Engineer

    Tata Consultancy ServicesGreater Bengaluru Area, India
    Greetings from Tata Consulting Services.TCS is Hiring for AWS DevOps Engineer.Location : Bangalore / Chennai / Gurgaon / Pune. Cloud Platform automation through DevOps scripting o (Terraform, Ansible, HE...Show moreLast updated: 1 hour ago
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Sun LifeBengaluru, Karnataka, India
    To do this, we offer a broad range of protection and wealth products and services to individuals, businesses, and institutions, including : . Life, health, wellness, disability, critical illness, stop...Show moreLast updated: 16 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kanerika Inchosur, tamil nadu, in
    Following are high level responsibilities that you will play but not limited to : .Analyze the Data Model and do GAP analysis with Business Requirements and Power BI. Design and Model Power BI schema....Show moreLast updated: 20 days ago
    • Promoted
    Sr. AWS Cloud Engineer

    Sr. AWS Cloud Engineer

    Mastekhosur, tamil nadu, in
    Cloud Engineer Job description : .Have work experience in the following areas : .Experience in designing, building, and maintaining AWS Cloud Infrastructure. Proficient in AWS services including EC2, S3...Show moreLast updated: 26 days ago
    • Promoted
    AWS Data Architect- Leadership

    AWS Data Architect- Leadership

    QuantiphiBengaluru, Karnataka, India
    AWS Data Architect- Leadership.We are seeking a highly skilled and experienced Technical Leader for our AWS Data Engineering practice. The ideal candidate will be responsible for driving Data Engine...Show moreLast updated: 30+ days ago
    • Promoted
    Cloud Engineer Lead (AWS)

    Cloud Engineer Lead (AWS)

    Datapel Systemshosur, tamil nadu, in
    The Senior Cloud Engineer (AWS) will be responsible for developing, maintaining, optimising and supporting the cloud infrastructure that supports Datapel’s Warehouse Management System (WMS) and rel...Show moreLast updated: 19 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Eucloid Data Solutionshosur, tamil nadu, in
    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 9 days ago
    • Promoted
    Data Engineer

    Data Engineer

    ACL Digitalhosur, tamil nadu, in
    Design, develop, and optimize Spark-based data pipelines on Databricks for large-scale data processing.Design, develop, and optimize AWS pipeline as applicable. Implement and manage GitHub asset bun...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    IntraEdgehosur, tamil nadu, in
    Snowflake, AWS (Lambda, Glue), DBT, and SQL.The ideal candidate will be responsible for enabling seamless data integration, transformation, and analytics to support business intelligence and advanc...Show moreLast updated: 28 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kumaran Systemshosur, tamil nadu, in
    Strong hands-on experience in Databricks.Proven expertise in building and managing data ingestion pipelines.Exposure to Databricks data ingestion jobs along with incident management frameworks.Expe...Show moreLast updated: 9 days ago