Talent.com
This job offer is not available in your country.
Data Engineer AWS / Power platform Engineer

Data Engineer AWS / Power platform Engineer

ConfidentialDelhi, Hyderabad / Secunderabad, Telangana
30+ days ago
Job description
  • Design and develop data pipelines for Generative AI projects by leveraging a combination of technologies, including Vector DB, Graph DB, Airflow, Spark, PySpark, Python, LangChain, AWS Functions, Redshift, and SSIS. This will involve the logical and efficient integration of these tools to create seamless, high-performance data flows that efficiently support the data requirements of our cutting-edge AI initiatives. Collaborate with data scientists, AI researchers, and other stakeholders to understand data requirements and translate them into effective data engineering solutions.
  • User will be managing movement, organization and quality assessments of large set of data to facilitate the creation of Knowledge base for RAG systems and model training
  • Demonstrate familiarity with data integration services such as AWS Glue and Azure Data Factory, showcasing the ability to effectively utilize these platforms for seamless data ingestion, transformation, and orchestration across various sources and destinations.
  • Possess proficiency in constructing data warehouses and data lakes, demonstrating a strong foundation in organizing and consolidating large volumes of structured and unstructured data for efficient storage, retrieval, and analysis.
  • Optimize and maintain data pipelines to ensure high-performance, reliable, and scalable data processing.
  • Develop and implement data validation and quality assurance procedures to ensure the accuracy and consistency of the data used in Generative AI projects.
  • Monitor and troubleshoot data pipeline performance, identify bottlenecks, and implement improvements as necessary.
  • Stay current with emerging trends and technologies in the fields of data engineering, Generative AI, and related areas to ensure the continued success of our projects.
  • Collaborate with team members on documentation, knowledge sharing, and best practices for data engineering within a Generative AI context.
  • Ensure data privacy and security compliance in accordance with industry standards and regulations.
  • Qualifications we seek in you :

    • Bachelors or Masters degree in Computer Science, Engineering, or a related field.
    • Strong experience with data engineering technologies, including Vector DB, Graph DB, Airflow, Spark, PySpark, Python, langchain, AWS Functions, Redshift, and SSIS.
    • Strong understanding of data warehousing concepts, ETL processes, and data modeling.
    • Strong understanding of S3 and code-based scripting to move large volumes of data across application storage layers
    • Familiarity with Generative AI concepts and technologies, such as GPT-4, Transformers, and other natural language processing techniques.
    • Excellent problem-solving, analytical, and critical thinking skills.
    • Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams.
    • Preferred Qualifications / skills

    • Knowledge of cloud computing platforms, such as AWS, Azure, or Google Cloud Platform, is a plus.
    • Experience with big data technologies, such as Hadoop, Hive, or Presto, is a plus.
    • Familiarity with machine learning frameworks, such as TensorFlow or PyTorch, is a plus.
    • A continuous learning mindset and a passion for staying up-to-date with the latest advancements in data engineering and Generative AI.
    • Skills Required

      Cloud Computing, Artificial Intelligence, Pytorch, Big Data Technologies, Aws, Tensorflow

    Create a job alert for this search

    Aws Data Engineer • Delhi, Hyderabad / Secunderabad, Telangana

    Related jobs
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    AllnessjobsHyderabad
    Job Role : Data Engineer Experience : 8+ Years Mode : Hybrid &...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    IntraEdgeHyderabad, IN
    Snowflake, AWS (Lambda, Glue), DBT, and SQL.The ideal candidate will be responsible for enabling seamless data integration, transformation, and analytics to support business intelligence and advanc...Show moreLast updated: 30+ days ago
    • Promoted
    Procorp Systems - Senior Data Engineer - AWS / Apache Airflow

    Procorp Systems - Senior Data Engineer - AWS / Apache Airflow

    PROCORP SYSTEMS INDIA PRIVATE LIMITEDHyderabad
    Job Title : Senior Data Engineer AWS Redshift & Airflow Location : Hyderabad Employment Type : Full-Tim...Show moreLast updated: 6 days ago
    • Promoted
    AmpsTek - AWS Data Engineer

    AmpsTek - AWS Data Engineer

    AmpstekHyderabad
    Job Role : AWS Data Location : Chennai / Bangalore / Hyderabad / Pune (Hybrid) Key Skills : A...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    ACL Digitalhyderabad, telangana, in
    Design, develop, and optimize Spark-based data pipelines on Databricks for large-scale data processing.Design, develop, and optimize AWS pipeline as applicable. Implement and manage GitHub asset bun...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    ADPhyderabad, telangana, in
    Below is the JD followed for Data Engineering.Position : Data Engineer with 4 to Y8ears experience in AWS, PySpark, Python, SQL and Data bricks. In this role candidate will be responsible for unders...Show moreLast updated: 26 days ago
    • Promoted
    Cloud Data Platform Engineer - AWS

    Cloud Data Platform Engineer - AWS

    hirezy.aiHyderabad
    Role Description : As a Technical Lead - Cloud Data Platform (AWS) at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platfo...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Innodata Inc.hyderabad, telangana, in
    CI / CD practices, Databricks (Spark), Python, Github and SQL.The ideal candidate should have hands-on expertise in building and automating data pipelines, managing multi-environment deployments, and...Show moreLast updated: 25 days ago
    • Promoted
    Celebal Technologies - AWS Data Engineer

    Celebal Technologies - AWS Data Engineer

    Celebal TechnologiesHyderabad
    DATA ENGINEER (Databricks & AWS) Overview : As a Data Engineer, you will work with multiple teams to deliver solutions on the AWS Cl...Show moreLast updated: 4 days ago
    • Promoted
    AWS Data Engineer - Azure Databricks

    AWS Data Engineer - Azure Databricks

    TECHOAKS IT SOLUTIONS PRIVATE LIMITEDHyderabad
    Client : Disney Job Title : AWS Data Engineer (Databricks Focus) Location : Hyderabad ...Show moreLast updated: 30+ days ago
    • Promoted
    MLOps / AWS Data Engineer

    MLOps / AWS Data Engineer

    People Prime World WideHyderabad
    Key Responsibilities : - Design, build, and maintain scalable MLOps pipelines using AWS tools such as SageMaker, Lambda, Step Fu...Show moreLast updated: 4 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kanerika Incsecunderabad, telangana, in
    Databricks, Microsoft Fabric, or Snowflake.You will also play a key role in enabling effective data visualization and reporting for business stakeholders. Databricks, Fabric, or Snowflake.Power BI d...Show moreLast updated: 18 days ago
    • Promoted
    Data Engineer - AWS

    Data Engineer - AWS

    Digivance SolutionHyderabad
    Job Title : Data Engineer Data Platforms (AWS) Location : Hyderabad Experience Requirements <...Show moreLast updated: 6 days ago
    • Promoted
    Data Engineer Lead

    Data Engineer Lead

    Enterprise MindsHyderabad, India
    We’re Hiring : Data Engineer Lead at Enterprise Minds.Hiring #DataEngineer #Cloud #Azure #AWS #GCP #HyderabadJobs #EnterpriseMinds #Careers). Work Mode : Work From Office (5.At Enterprise Minds , w...Show moreLast updated: 3 days ago
    • Promoted
    AWS Data Engineer - PySpark / Business Intelligence

    AWS Data Engineer - PySpark / Business Intelligence

    NS Global CorporationHyderabad
    Responsibilities : - Design, develop, and maintain scalable data pipelines using Databricks, PySpark, and other relevant technologies.Build and op...Show moreLast updated: 30+ days ago
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    Coforgehyderabad, telangana, in
    As an AWS Data Engineer, you will be responsible for designing, developing, and optimizing robust, scalable, and secure data pipelines and infrastructure on AWS. You’ll work with diverse datasets an...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    ValueLabsHyderabad, Telangana, India
    Data Engineer, AWS Glue, Lambda, ETL, Python, Airflow ,Snowflake.We are seeking a Data Engineer responsible for building, maintaining, and optimizing the infrastructure that enables data collection...Show moreLast updated: 30+ days ago
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    HireIT ConsultantsHyderabad
    Job Title : AWS Data Engineer IoT, ERP / MES, CRM & Generative AI Integration Location : Remote / Onsite / Hybrid Job Type : Full-Time Show moreLast updated: 30+ days ago
    • Promoted
    Accenture - AWS Data Engineer

    Accenture - AWS Data Engineer

    Accenture OperationHyderabad
    Management Level : 9- Consultant Location : Open Must-have skills : Data Engineering, AWS(mandate) Show moreLast updated: 19 days ago
    • Promoted
    AWS Data Engineer - Python / PySpark

    AWS Data Engineer - Python / PySpark

    MLOPS Solutions Private LimitedHyderabad
    Job Description - AWS Data Engineer Location : Hyderabad / Bangalore Job Type : Contract | Work from Offi...Show moreLast updated: 18 days ago