Talent.com
This job offer is not available in your country.
Data Engineer AWS / Power platform Engineer

Data Engineer AWS / Power platform Engineer

ConfidentialChennai, Bengaluru / Bangalore
30+ days ago
Job description
  • Design and develop data pipelines for Generative AI projects by leveraging a combination of technologies, including Vector DB, Graph DB, Airflow, Spark, PySpark, Python, LangChain, AWS Functions, Redshift, and SSIS. This will involve the logical and efficient integration of these tools to create seamless, high-performance data flows that efficiently support the data requirements of our cutting-edge AI initiatives. Collaborate with data scientists, AI researchers, and other stakeholders to understand data requirements and translate them into effective data engineering solutions.
  • User will be managing movement, organization and quality assessments of large set of data to facilitate the creation of Knowledge base for RAG systems and model training
  • Demonstrate familiarity with data integration services such as AWS Glue and Azure Data Factory, showcasing the ability to effectively utilize these platforms for seamless data ingestion, transformation, and orchestration across various sources and destinations.
  • Possess proficiency in constructing data warehouses and data lakes, demonstrating a strong foundation in organizing and consolidating large volumes of structured and unstructured data for efficient storage, retrieval, and analysis.
  • Optimize and maintain data pipelines to ensure high-performance, reliable, and scalable data processing.
  • Develop and implement data validation and quality assurance procedures to ensure the accuracy and consistency of the data used in Generative AI projects.
  • Monitor and troubleshoot data pipeline performance, identify bottlenecks, and implement improvements as necessary.
  • Stay current with emerging trends and technologies in the fields of data engineering, Generative AI, and related areas to ensure the continued success of our projects.
  • Collaborate with team members on documentation, knowledge sharing, and best practices for data engineering within a Generative AI context.
  • Ensure data privacy and security compliance in accordance with industry standards and regulations.
  • Qualifications we seek in you :

    • Bachelors or Masters degree in Computer Science, Engineering, or a related field.
    • Strong experience with data engineering technologies, including Vector DB, Graph DB, Airflow, Spark, PySpark, Python, langchain, AWS Functions, Redshift, and SSIS.
    • Strong understanding of data warehousing concepts, ETL processes, and data modeling.
    • Strong understanding of S3 and code-based scripting to move large volumes of data across application storage layers
    • Familiarity with Generative AI concepts and technologies, such as GPT-4, Transformers, and other natural language processing techniques.
    • Excellent problem-solving, analytical, and critical thinking skills.
    • Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams.
    • Preferred Qualifications / skills

    • Knowledge of cloud computing platforms, such as AWS, Azure, or Google Cloud Platform, is a plus.
    • Experience with big data technologies, such as Hadoop, Hive, or Presto, is a plus.
    • Familiarity with machine learning frameworks, such as TensorFlow or PyTorch, is a plus.
    • A continuous learning mindset and a passion for staying up-to-date with the latest advancements in data engineering and Generative AI.
    • Skills Required

      Cloud Computing, Artificial Intelligence, Pytorch, Big Data Technologies, Aws, Tensorflow

    Create a job alert for this search

    Aws Data Engineer • Chennai, Bengaluru / Bangalore

    Related jobs
    • Promoted
    Azure Data Engineer

    Azure Data Engineer

    Texplorers Inchosur, tamil nadu, in
    We are seeking a skilled Data Engineer with proficient knowledge in Spark and SQL.The ideal candidate will be responsible for designing,. Data Pipeline Development : Design, develop, and maintain sca...Show moreLast updated: 18 days ago
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    L&T Technology ServicesBengaluru, Karnataka, India
    Graduate with 8-10 years of experience working as a AWS Data Engineer.Excellent knowledge of PySpark, JavaScript, AWS - RDS, and Redshift.Show moreLast updated: 30+ days ago
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Mastech Digitalbangalore, karnataka, in
    Hybrid model weekly 3 days at office.Python, Pyspark, Glue, Redshift, Lambda, DMS, RDS ,Cloud Formation and other AWS serverless. Seeking a developer who has good Experience in Athena, Python code, ...Show moreLast updated: 30+ days ago
    • Promoted
    AWS Data Engineer for one of the leading Big4

    AWS Data Engineer for one of the leading Big4

    Acme ServicesBengaluru, Karnataka, India
    We are seeking a highly skilled.The ideal candidate will have strong expertise in.AWS cloud services, Snowflake, Python, Airflow, Glue, Redshift, and Kafka. Design, develop, and maintain scalable.De...Show moreLast updated: 25 days ago
    • Promoted
    Agentic Data Engineer (AWS)

    Agentic Data Engineer (AWS)

    Sustainability Economics.aiBengaluru, Karnataka, India
    AI, enabling profitable energy transitions while powering end-to-end AI infrastructure.By integrating AI-driven cloud solutions with sustainable energy, we create scalable, intelligent ecosystems t...Show moreLast updated: 6 days ago
    • Promoted
    Data Engineer

    Data Engineer

    R Systemshosur, tamil nadu, in
    Offshore candidates accepted (Singapore Based Company).Please don't apply if less than 4 years exp in Data Engineer).ETL pipelines, real-time streaming, and data transformations.RDDs, transformatio...Show moreLast updated: 25 days ago
    • Promoted
    Lead Azure Data Engineer

    Lead Azure Data Engineer

    RandomTreeshosur, tamil nadu, in
    We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies. If you're passionate about building r...Show moreLast updated: 25 days ago
    • Promoted
    GenAI Engineer

    GenAI Engineer

    Xebiahosur, tamil nadu, in
    Any Xebia Location (Hybrid, 3 days office per week).Building and deploying GenAI solutions leveraging.Collaborating with global teams under US overlap hours. AWS, GenAI, Bedrock, AgenticAI.Ability t...Show moreLast updated: 25 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Otomeyt AIhosur, tamil nadu, in
    We are seeking a highly skilled 5+.The ideal candidate will have strong technical expertise in Azure, Data Engineering tools, and advanced ETL design along with excellent communication and problem-...Show moreLast updated: 30+ days ago
    • Promoted
    Data Platform Engineer

    Data Platform Engineer

    Intuitive.Cloudhosur, tamil nadu, in
    With the reputation of being a.Digital Transformation challenges across following Intuitive Superpowers : .Application & Database Modernization. Platform Engineering (IaC / EaC, DevSecOps & SRE).Cloud N...Show moreLast updated: 4 days ago
    • Promoted
    AWS Data engineer / Architect

    AWS Data engineer / Architect

    Tata Consultancy ServicesBengaluru, Karnataka, India
    In this key leadership role, you will lead the development of foundational components for a Lakehouse architecture on AWS and drive the migration of existing data processing workflows to the new La...Show moreLast updated: 17 days ago
    • Promoted
    AWS Data Engineer for Bangalore

    AWS Data Engineer for Bangalore

    DeloitteBengaluru, Karnataka, India
    Looking for Skilled AWS Data Engineer, Interested candidates please find the JD below : .Bachelor’s Degree and 3+ years’ experience in the implementation of modern data ecosystems in AWS / Cloud platfo...Show moreLast updated: 6 days ago
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Sun LifeBengaluru, Karnataka, India
    To do this, we offer a broad range of protection and wealth products and services to individuals, businesses, and institutions, including : . Life, health, wellness, disability, critical illness, stop...Show moreLast updated: 13 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kanerika Inchosur, tamil nadu, in
    Following are high level responsibilities that you will play but not limited to : .Analyze the Data Model and do GAP analysis with Business Requirements and Power BI. Design and Model Power BI schema....Show moreLast updated: 18 days ago
    • Promoted
    Senior AWS Data Engineer

    Senior AWS Data Engineer

    TEKsystemsBengaluru, Karnataka, India
    Your skills & Your Experience Experience.Analytics and Data Warehousing.Experience in building end-to-end solutions with the Big data platform, Spark or scala programming.Solid experience in ETL pi...Show moreLast updated: 5 days ago
    • Promoted
    Data Engineer

    Data Engineer

    ACL Digitalhosur, tamil nadu, in
    Design, develop, and optimize Spark-based data pipelines on Databricks for large-scale data processing.Design, develop, and optimize AWS pipeline as applicable. Implement and manage GitHub asset bun...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    IntraEdgehosur, tamil nadu, in
    Snowflake, AWS (Lambda, Glue), DBT, and SQL.The ideal candidate will be responsible for enabling seamless data integration, transformation, and analytics to support business intelligence and advanc...Show moreLast updated: 25 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Innodata Inc.hosur, tamil nadu, in
    CI / CD practices, Databricks (Spark), Python, Github and SQL.The ideal candidate should have hands-on expertise in building and automating data pipelines, managing multi-environment deployments, and...Show moreLast updated: 24 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Eucloid Data Solutionshosur, tamil nadu, in
    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 6 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kumaran Systemshosur, tamil nadu, in
    Strong hands-on experience in Databricks.Proven expertise in building and managing data ingestion pipelines.Exposure to Databricks data ingestion jobs along with incident management frameworks.Expe...Show moreLast updated: 6 days ago