Talent.com
Senior Big Data Solutions Engineer
Senior Big Data Solutions EngineerInfoCepts • Pune, Republic Of India, IN
Senior Big Data Solutions Engineer

Senior Big Data Solutions Engineer

InfoCepts • Pune, Republic Of India, IN
16 days ago
Job description

Key Result Areas and Activities :

  • Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements.
  • Should be able to design complex Big Data use cases using AWS Services and Databricks, under guidance of Architect
  • Should be able to assist the architect in reasoning about various architecture choices in terms of cost, performance and durability.
  • Should be able to suggest optimizations in existing designs.
  • Project documentation adheres to quality guidelines & schedules.
  • Works hand in hand with Architect & PM for successful delivery of project and provide estimation, scoping, scheduling assistance.
  • Articulate design decisions very clearly with stakeholders.
  • Perform Proof-of-Concepts and document all observations before proposing a new solution.
  • Conduct design review sessions with other teams and identify scope of improvement.
  • Suggests automation to improve existing processes.

Work and Technical Experience :

Must-Have :

  • In-depth knowledge of the following AWS services : S3, EC2, EMR, Athena, AWS Glue, Lambda
  • Experience with at least one MPP database : AWS Redshift, Snowflake, SingleStore
  • Proficiency in Big Data technologies : Apache Spark, Databricks
  • Must have strong programming skills in Python
  • Responsible for building data pipelines in AWS And Databricks
  • Experience with Big Data table formats, such as Delta Lake (open source)
  • Must have very strong SQL skills
  • Experience with orchestration tools like Apache Airflow
  • Expertise in developing ETL workflows with complex transformations such as SCD, deduplications, aggregations, etc.
  • Should be a quick and self-learner, ready to adapt to new AWS services or Big Data technologies as required
  • Strong understanding of data warehousing concepts
  • Good to have :

  • Cloud Databases – Snowflake, AWS Aurora
  • Big Data – Hadoop, Hive
  • Cloud Databases– AWS Aurora
  • Associate Level or Professional Level AWS Certification or Databricks Certification
  • Qualifications :

  • Overall, 7+ years of IT Exp
  • 5+ years of relevant experience in AWS related project
  • Bachelor’s degree in computer science, engineering, or related field (master's degree is a plus)
  • Demonstrated continued learning through one or more technical certifications or related methods.
  • Qualities :

  • Hold strong technical knowledge and experience
  • Should have the capability to deep dive and research in various technical related fields
  • Self-motivated and focused on delivering outcomes for a fast-growing team and firm
  • Able to communicate persuasively through speaking, writing, and client presentations
  • Able to consult, write, and present persuasively
  • Able to work in a self-organized and cross-functional team
  • Able to iterate based on new information, peer reviews, and feedback
  • Prior experience of working in a large media company would be added advantage
  • Create a job alert for this search

    Senior Data Engineer • Pune, Republic Of India, IN

    Related jobs
    Senior Data Engineer

    Senior Data Engineer

    Intellias • Nagpur, IN
    Apache Flink / Apache Spark (Streaming).Data Engineer or similar role, with hands-on expertise in large-scale, production-grade data pipelines. Kafka + Flink / Spark Streaming).Python for data engin...Show more
    Last updated: 16 days ago • Promoted
    (Senior) Azure Data Engineer - Full remote - contractor in USD

    (Senior) Azure Data Engineer - Full remote - contractor in USD

    All European Careers • Nagpur, IN
    Remote
    For an international project in Chennai, we are urgently looking for a Full Remote Senior Azure Data Engineer, who will build data pipeline for enterprise search applications using ADF and Databric...Show more
    Last updated: 1 day ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Adaptive Technology Insights • India
    We are looking for an experienced.You will be responsible for designing, building, and optimizing data pipelines and architecture to support analytics, reporting, and data-driven decision-making ac...Show more
    Last updated: 1 hour ago • Promoted • New!
    Data Engineer | Bangalore | Leading Big 4 Consulting Firm

    Data Engineer | Bangalore | Leading Big 4 Consulting Firm

    Acme Services • Nagpur, IN
    Design, develop, and maintain scalable and robust data pipelines (ETL / ELT) using PySpark on the Databricks platform.Implement data ingestion strategies to load data from various sources (e.APIs, da...Show more
    Last updated: 1 hour ago • Promoted • New!
    Senior Data Engineer

    Senior Data Engineer

    American Inference • India
    Please apply directly through for this position.We kindly request that candidates refrain from contacting company officials via email or messages regarding this role. We are an AI and Data Consulti...Show more
    Last updated: 13 days ago • Promoted
    Senior AWS Data Engineer

    Senior AWS Data Engineer

    CYAN360 • India, India
    Position : Senior AWS Data Engineer.Work Timings : 2 : 30 PM to 11 : 30 PM IST.Need someone who can join immediately or in 15 days • • •. Design, develop, and deploy end-to-end data pipelines on AWS cloud in...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer - Data Acquisition

    Senior Data Engineer - Data Acquisition

    InfoBeans • Nagpur, IN
    We are seeking a highly skilled.Senior Data Engineer – Data Acquisition (ODS).The ideal candidate will have extensive hands-on experience in building and optimizing data ingestion and transformatio...Show more
    Last updated: 8 days ago • Promoted
    Senior Oracle Data Engineer

    Senior Oracle Data Engineer

    Strattek Consulting • Nagpur, IN
    This role centers on architecting robust data pipelines across on-prem and cloud Oracle ecosystems, automating ingestion from varied sources including APIs. Exposure to Oracle FDI (Fusion Data Intel...Show more
    Last updated: 1 day ago • Promoted
    Data Engineer

    Data Engineer

    IntraEdge • Nagpur, IN
    We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team. You will be responsible for building scalable and reli...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Mindlance • India
    We’re looking for a strong, hands-on Sr Data Engineer who can independently drive client conversations, collaborate with SMEs, and deliver high-quality data solutions in a cloud-native environment....Show more
    Last updated: 1 hour ago • Promoted • New!
    Senior DataOps Engineer (AWS)

    Senior DataOps Engineer (AWS)

    MSBC Group • Nagpur, IN
    Join us as a Senior DataOps Engineer (AWS)—Drive High-Performance Data Systems for Financial Services.Lead the E-Comms data pipeline within Compass’s Application Simplification workstream : design, ...Show more
    Last updated: 1 hour ago • Promoted • New!
    Data Engineer

    Data Engineer

    Alp Consulting Ltd. • Nagpur, IN
    Architect and maintain our Amazon Redshift Serverless data warehouse.Design and implement ETL pipelines from operational Redshift to staging (DSA), landing (DLA), and TDW layers.Model data using st...Show more
    Last updated: 30+ days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    KPI Partners • Nagpur, IN
    We are seeking a highly skilled Principal Data Engineer to join our dynamic team at KPI Partners.Lead data engineering initiatives and projects from conception to delivery.Design and implement scal...Show more
    Last updated: 1 hour ago • Promoted • New!
    Senior Data Architect- Snowflake

    Senior Data Architect- Snowflake

    USEReady • Nagpur, IN
    USEReady is a data and analytics firm that provides the strategies, tools, capability, and capacity that businesses need to turn their data into a competitive advantage. USEReady partners with cloud...Show more
    Last updated: 1 hour ago • Promoted • New!
    Senior Data Engineer

    Senior Data Engineer

    Donyati • Nagpur, IN
    We are seeking a highly skilled Senior Data Engineer to join our team in building a modern data platform on AWS.You will play a key role in transitioning from legacy systems to a scalable, cloud-na...Show more
    Last updated: 1 day ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    TEKsystems Global Services in India • India
    Develop and manage data pipelines using ADF and Azure data bricks.Expert knowledge in data warehousing, data modeling and ETL. Good to have datastage exposure.Expertise in on-prem to azure cloud mig...Show more
    Last updated: 1 hour ago • Promoted • New!
    Senior Data Engineer

    Senior Data Engineer

    SAIVA AI • Nagpur, IN
    We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Platform Engineer

    Senior Data Platform Engineer

    Black Dog Labs • Nagpur, IN
    Remote (collaboration across time zones), India or LATAM preferred.Proficient English communication.Data Engineering / Backend Engineering / DevOps. We’re looking for a hands-on Senior Data Platform...Show more
    Last updated: 30+ days ago • Promoted