Talent.com
Lead Data Integration Engineer
Lead Data Integration EngineerInfoCepts • Pune, Republic Of India, IN
Lead Data Integration Engineer

Lead Data Integration Engineer

InfoCepts • Pune, Republic Of India, IN
22 days ago
Job description

Key Result Areas and Activities :

  • Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements.
  • Should be able to design complex Big Data use cases using AWS Services and Databricks, under guidance of Architect
  • Should be able to assist the architect in reasoning about various architecture choices in terms of cost, performance and durability.
  • Should be able to suggest optimizations in existing designs.
  • Project documentation adheres to quality guidelines & schedules.
  • Works hand in hand with Architect & PM for successful delivery of project and provide estimation, scoping, scheduling assistance.
  • Articulate design decisions very clearly with stakeholders.
  • Perform Proof-of-Concepts and document all observations before proposing a new solution.
  • Conduct design review sessions with other teams and identify scope of improvement.
  • Suggests automation to improve existing processes.

Work and Technical Experience :

Must-Have :

  • In-depth knowledge of the following AWS services : S3, EC2, EMR, Athena, AWS Glue, Lambda
  • Experience with at least one MPP database : AWS Redshift, Snowflake, SingleStore
  • Proficiency in Big Data technologies : Apache Spark, Databricks
  • Must have strong programming skills in Python
  • Responsible for building data pipelines in AWS And Databricks
  • Experience with Big Data table formats, such as Delta Lake (open source)
  • Must have very strong SQL skills
  • Experience with orchestration tools like Apache Airflow
  • Expertise in developing ETL workflows with complex transformations such as SCD, deduplications, aggregations, etc.
  • Should be a quick and self-learner, ready to adapt to new AWS services or Big Data technologies as required
  • Strong understanding of data warehousing concepts
  • Good to have :

  • Cloud Databases – Snowflake, AWS Aurora
  • Big Data – Hadoop, Hive
  • Cloud Databases– AWS Aurora
  • Associate Level or Professional Level AWS Certification or Databricks Certification
  • Qualifications :

  • Overall, 7+ years of IT Exp
  • 5+ years of relevant experience in AWS related project
  • Bachelor’s degree in computer science, engineering, or related field (master's degree is a plus)
  • Demonstrated continued learning through one or more technical certifications or related methods.
  • Qualities :

  • Hold strong technical knowledge and experience
  • Should have the capability to deep dive and research in various technical related fields
  • Self-motivated and focused on delivering outcomes for a fast-growing team and firm
  • Able to communicate persuasively through speaking, writing, and client presentations
  • Able to consult, write, and present persuasively
  • Able to work in a self-organized and cross-functional team
  • Able to iterate based on new information, peer reviews, and feedback
  • Prior experience of working in a large media company would be added advantage
  • Create a job alert for this search

    Data Integration Engineer • Pune, Republic Of India, IN

    Related jobs
    D365 Data Integration Engineer

    D365 Data Integration Engineer

    GyanSys Inc. • Republic Of India, IN
    Need to have databricks experience as well as experience with handling D365 data.Need strong SQL skills, strong analytical skills.Show more
    Last updated: 12 days ago • Promoted
    Cloud Data Integration Engineer

    Cloud Data Integration Engineer

    Coforge • Pune, Republic Of India, IN
    The ideal candidate will be responsible for designing, implementing, and maintaining scalable data solutions that support business intelligence and analytics needs. Design, implement, and maintain d...Show more
    Last updated: 30+ days ago • Promoted
    Data Integration Engineer

    Data Integration Engineer

    Synechron • Republic Of India, IN
    We have immediate opportunity for Data Engineer – 3-7 Years.We began life in 2001 as a small, self-funded team of technology specialists. Since then, we’ve grown our organization to 14,500+ people, ...Show more
    Last updated: 30+ days ago • Promoted
    Cloud Data Integration Engineer

    Cloud Data Integration Engineer

    System Soft Technologies • Republic Of India, IN
    Location : Remote (3–4-hour time zone overlaps with EST if off shore).Experience with next flow is required, as the consultant will make targeted enhancements to existing workflows and pipelines.Whi...Show more
    Last updated: 1 day ago • Promoted
    Integration Engineer

    Integration Engineer

    Bahwan CyberTek • Chennai, Republic Of India, IN
    Mid-Level Integration Engineer.Support, maintain, and enhance existing Java Spring Boot microservices in a distributed environment. Design, develop, and implement ETL processes using tools like Info...Show more
    Last updated: 19 days ago • Promoted
    Data Integration Lead

    Data Integration Lead

    Digile • Republic Of India, IN
    We’re Hiring : Oracle ETL (ODI / OIC) Systems Analyst – Lead Role (8+ Years).This is a high-ownership role covering the full delivery lifecycle — from assessment and design to build, testing, deployme...Show more
    Last updated: 8 days ago • Promoted
    ETL Architect and Data Integration Lead

    ETL Architect and Data Integration Lead

    Acies • Republic Of India, IN
    Highlight of the engagement opportunity.Number of years of experience expected : .And must know pandas and NumPy libraries. Areas of past experience preferred : .ETL documentation / Data mapping / ETL d...Show more
    Last updated: 3 days ago • Promoted
    Lead Data Integration Engineer

    Lead Data Integration Engineer

    Quantiphi • Republic Of India, IN
    Quantiphi is an award-winning Data Science and Machine Learning Software and Services Company focused on helping organizations translate the big promise of Machine Learning technologies into quanti...Show more
    Last updated: 20 days ago • Promoted
    Data Integration Engineer

    Data Integration Engineer

    Kumaran Systems • Republic Of India, IN
    We are seeking a skilled Azure Databricks with strong hands-on experience in Microsoft Azure services such as Data Factory, Storage, Synapse, and Key Vault. The candidate should be proficient in Dat...Show more
    Last updated: 7 days ago • Promoted
    Data Integration Engineer

    Data Integration Engineer

    OpenGov Inc. • Pune, Republic Of India, IN
    OpenGov is the leader in AI and ERP solutions for local and state governments in the U.More than 2,000 cities, counties, state agencies, school districts, and special districts rely on the OpenGov ...Show more
    Last updated: 21 days ago • Promoted
    Data Integration Engineer

    Data Integration Engineer

    TerraGiG • Pune, Republic Of India, IN
    Job Title : Informatica DEI Engineer.Develop and support data integration workflows (batch / streaming).Integrate DEI with Cloudera, optimize performance, and ensure data quality.Maintain documentatio...Show more
    Last updated: 30+ days ago • Promoted
    Lead Oracle Data Integration Engineer

    Lead Oracle Data Integration Engineer

    Strattek Consulting • Republic Of India, IN
    This role centers on architecting robust data pipelines across on-prem and cloud Oracle ecosystems, automating ingestion from varied sources including APIs. Exposure to Oracle FDI (Fusion Data Intel...Show more
    Last updated: 7 days ago • Promoted
    Senior Data Integration Engineer

    Senior Data Integration Engineer

    IntraEdge • Republic Of India, IN
    We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team. You will be responsible for building scalable and reli...Show more
    Last updated: 19 days ago • Promoted
    Principal Data Integration Engineer

    Principal Data Integration Engineer

    ListenFirst • Jaipur, Republic Of India, IN
    We are looking for a skilled and motivated Senior Data Engineer with strong experience in data engineering and ETL processes. The ideal candidate should be comfortable working with any object-orient...Show more
    Last updated: 20 hours ago • Promoted • New!
    Data Integration Engineer

    Data Integration Engineer

    Programmers.io • Jaipur, Republic Of India, IN
    Shift Timings : General Shift (12 : 00 PM IST till 9 : 00 PM or 9 : 30 PM IST).Location : PAN India-Remote Until Office Resume, Work from Home. Experience required : 5 to 8 years.Design, develop, and maintai...Show more
    Last updated: 30+ days ago • Promoted
    Principal Data Integration Engineer

    Principal Data Integration Engineer

    Futran Solutions • Pune, Republic Of India, IN
    Job Title : Senior Data Engineer / Data Engineer Lead.We are seeking a highly skilled and experienced Senior Data Engineer with a strong background in Microsoft Azure data platforms.The ideal candida...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Integration Architect

    Lead Data Integration Architect

    Epsilon Technologies Group • Republic Of India, IN
    About Epsilon Technologies Group.Epsilon is a capital markets technology solutions and services firm helping financial institutions manage everything on their balance sheet with a scalable and inno...Show more
    Last updated: 11 days ago • Promoted
    Lead Full Stack Engineer & Data Integration Specialist

    Lead Full Stack Engineer & Data Integration Specialist

    Black Dog Labs • Republic Of India, IN
    Senior Fullstack Engineer (with Data Engineering Experience).Remote (collaboration across time zones), India or LATAM preferred. Proficient English communication.Full-Stack Engineering / Backend Eng...Show more
    Last updated: 1 day ago • Promoted