Talent.com
This job offer is not available in your country.
Data Engineer AWS / Power platform Engineer

Data Engineer AWS / Power platform Engineer

ConfidentialKolkata
30+ days ago
Job description
  • Design and develop data pipelines for Generative AI projects by leveraging a combination of technologies, including Vector DB, Graph DB, Airflow, Spark, PySpark, Python, LangChain, AWS Functions, Redshift, and SSIS. This will involve the logical and efficient integration of these tools to create seamless, high-performance data flows that efficiently support the data requirements of our cutting-edge AI initiatives. Collaborate with data scientists, AI researchers, and other stakeholders to understand data requirements and translate them into effective data engineering solutions.
  • User will be managing movement, organization and quality assessments of large set of data to facilitate the creation of Knowledge base for RAG systems and model training
  • Demonstrate familiarity with data integration services such as AWS Glue and Azure Data Factory, showcasing the ability to effectively utilize these platforms for seamless data ingestion, transformation, and orchestration across various sources and destinations.
  • Possess proficiency in constructing data warehouses and data lakes, demonstrating a strong foundation in organizing and consolidating large volumes of structured and unstructured data for efficient storage, retrieval, and analysis.
  • Optimize and maintain data pipelines to ensure high-performance, reliable, and scalable data processing.
  • Develop and implement data validation and quality assurance procedures to ensure the accuracy and consistency of the data used in Generative AI projects.
  • Monitor and troubleshoot data pipeline performance, identify bottlenecks, and implement improvements as necessary.
  • Stay current with emerging trends and technologies in the fields of data engineering, Generative AI, and related areas to ensure the continued success of our projects.
  • Collaborate with team members on documentation, knowledge sharing, and best practices for data engineering within a Generative AI context.
  • Ensure data privacy and security compliance in accordance with industry standards and regulations.
  • Qualifications we seek in you :

    • Bachelors or Masters degree in Computer Science, Engineering, or a related field.
    • Strong experience with data engineering technologies, including Vector DB, Graph DB, Airflow, Spark, PySpark, Python, langchain, AWS Functions, Redshift, and SSIS.
    • Strong understanding of data warehousing concepts, ETL processes, and data modeling.
    • Strong understanding of S3 and code-based scripting to move large volumes of data across application storage layers
    • Familiarity with Generative AI concepts and technologies, such as GPT-4, Transformers, and other natural language processing techniques.
    • Excellent problem-solving, analytical, and critical thinking skills.
    • Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams.
    • Preferred Qualifications / skills

    • Knowledge of cloud computing platforms, such as AWS, Azure, or Google Cloud Platform, is a plus.
    • Experience with big data technologies, such as Hadoop, Hive, or Presto, is a plus.
    • Familiarity with machine learning frameworks, such as TensorFlow or PyTorch, is a plus.
    • A continuous learning mindset and a passion for staying up-to-date with the latest advancements in data engineering and Generative AI.
    • Skills Required

      Cloud Computing, Artificial Intelligence, Pytorch, Big Data Technologies, Aws, Tensorflow

    Create a job alert for this search

    Aws Data Engineer • Kolkata

    Related jobs
    • Promoted
    Data Engineer

    Data Engineer

    Vriba SolutionsKolkata, IN
    Design, develop & maintain ETL / ELT pipelines.Ingest & transform data from APIs, DBs, files, streams.Build real-time & batch processing solutions. Data validation, quality & cleansing.Translate busin...Show moreLast updated: 8 days ago
    • Promoted
    Data Platform Engineer

    Data Platform Engineer

    Intuitive.CloudKolkata, IN
    With the reputation of being a.Digital Transformation challenges across following Intuitive Superpowers : .Application & Database Modernization. Platform Engineering (IaC / EaC, DevSecOps & SRE).Cloud N...Show moreLast updated: 18 days ago
    • Promoted
    Senior Big Data Engineer

    Senior Big Data Engineer

    VeltrisKolkata, IN
    Veltris is a Digital Product Engineering Services partner committed to driving technology-enabled transformation across enterprises, businesses, and industries. We specialize in delivering next-gene...Show moreLast updated: 28 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kumaran SystemsKolkata, IN
    Strong hands-on experience in Databricks.Proven expertise in building and managing data ingestion pipelines.Exposure to Databricks data ingestion jobs along with incident management frameworks.Expe...Show moreLast updated: 21 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Eucloid Data SolutionsKolkata, IN
    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 21 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Searce IncKolkata, IN
    Searce means ‘a fine sieve’ & indicates ‘to refine, to analyze, to improve’.It signifies our way of working : To improve to the finest degree of excellence, ‘solving for better’ every time.Searcians...Show moreLast updated: 8 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Bahwan CyberTekKolkata, IN
    Job Title : Data Engineer – Google Cloud Platform (GCP).We are seeking a skilled and motivated Data Engineer with hands-on experience in building scalable data pipelines and cloud-native data soluti...Show moreLast updated: 30+ days ago
    • Promoted
    Cloud Engineer

    Cloud Engineer

    Strobes Security, Inc.Kolkata, IN
    We are looking for a Mid-level Cloud Engineer with hands-on expertise in designing, automating, and operating production-grade cloud infrastructure. This role requires a strong background in AWS ser...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    SAIVA AIKolkata, IN
    We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 21 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    InfogainKolkata, IN
    Big Data Engineer (Lead) : As a Big Data Engineer (Lead), you will be responsible for leading a team of big data engineers. You will work closely with clients and team members to understand their req...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Canopus Infosystems - A CMMI Level 3 CompanyKolkata, IN
    Python expertise and hands-on experience in handling large datasets, data cleaning, analysis, and visualization.The ideal candidate should be capable of building data pipelines, performing web scra...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    RevXKolkata, IN
    RevX helps app businesses acquire and reengage users via programmatic to retain, monetize, and accelerate revenue.We're all about taking your app businesses to a new growth level.We rely on data sc...Show moreLast updated: 30+ days ago
    • Promoted
    Principal Data Engineer

    Principal Data Engineer

    XebiaKolkata, IN
    We’re Hiring : Principal Data Engineer | Any Xebia Location (Hybrid, 3 days in office per week).Any Xebia Location (Hybrid – 3 days in office per week). Data Engineering with 4+ years team leadership...Show moreLast updated: 30+ days ago
    • Promoted
    AWS Cloud Engineer

    AWS Cloud Engineer

    ProgliteKolkata, IN
    Infrastructure & System Administration : .Deploy, manage, and optimize EC2 instances across dev, test, and production environments. Perform system administration and troubleshooting for Linux and Wind...Show moreLast updated: 21 days ago
    • Promoted
    Senior Cloud Engineer AWS

    Senior Cloud Engineer AWS

    Matrix USAKolkata, IN
    We are seeking an experienced AWS Developer proficient in Python and PySpark to design, develop, and maintain scalable, serverless data processing and workflow automation solutions on AWS.The ideal...Show moreLast updated: 17 days ago
    • Promoted
    Data Engineer

    Data Engineer

    XebiaKolkata, IN
    We’re Hiring : Data Engineer | Xebia.Any Xebia location (Hybrid, 3 days office per week).Immediate to 2 weeks – only apply if you can join early. Databricks, Python, SQL, and Postgres.The ideal candi...Show moreLast updated: 18 days ago
    • Promoted
    Cloud Engineer Lead (AWS)

    Cloud Engineer Lead (AWS)

    Datapel SystemsKolkata, IN
    The Senior Cloud Engineer (AWS) will be responsible for developing, maintaining, optimising and supporting the cloud infrastructure that supports Datapel’s Warehouse Management System (WMS) and rel...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    EveriseKolkata, IN
    Join us on our mission to elevate customer experiences for people around the world.As a member of the Everise family, you will be part of a global experience company that believes in being people-f...Show moreLast updated: 20 days ago
    • Promoted
    Data / ML Engineer

    Data / ML Engineer

    GleantapKolkata, IN
    Gleantap is a customer engagement platform powering fitness, wellness, and service businesses.Python (scikit-learn, LightGBM, XGBoost). Set up retraining and monitoring pipelines (Airflow, MLflow, o...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    IntraEdgeKolkata, IN
    The Senior Data Engineer will design, develop, monitor and maintain a robust and scalable data platform used by other data analyst and engineering teams to deliver powerful insights to both interna...Show moreLast updated: 30+ days ago