Talent.com
This job offer is not available in your country.
Sr Associate Big Data Engineering

Sr Associate Big Data Engineering

AT&Thyderabad, India
12 hours ago
Job description

Job Description

Key Responsibilities :

Data Engineering & Architecture :

  • Design, develop, and maintain high-performance data pipelines for structured and unstructured data using Azure Data Bricks and Apache Spark.
  • Build and manage scalable data ingestion frameworks for batch and real-time data processing.
  • Implement and optimize data lake architecture in Azure Data Lake to support analytics and reporting workloads.
  • Develop and optimize data models and queries in Azure Synapse Analytics to power BI and analytics use cases.

Cloud-Based Data Solutions :

  • Architect and implement modern data lakehouses combining the best of data lakes and data warehouses.
  • Leverage Azure services like Data Factory, Event Hub, and Blob Storage for end-to-end data workflows.
  • Ensure security, compliance, and governance of data through Azure Role-Based Access Control (RBAC) and Data Lake ACLs.
  • ETL / ELT Development :

  • Develop robust ETL / ELT pipelines using Azure Data Factory, Data Bricks notebooks, and PySpark.
  • Perform data transformations, cleansing, and validation to prepare datasets for analysis.
  • Manage and monitor job orchestration, ensuring pipelines run efficiently and reliably.
  • Performance Optimization :

  • Optimize Spark jobs and SQL queries for large-scale data processing.
  • Implement partitioning, caching, and indexing strategies to improve performance and scalability of big data workloads.
  • Conduct capacity planning and recommend infrastructure optimizations for cost-effectiveness.
  • Collaboration & Stakeholder Management :

  • Work closely with business analysts, data scientists, and product teams to understand data requirements and deliver solutions.
  • Participate in cross-functional design sessions to translate business needs into technical specifications.
  • Provide thought leadership on best practices in data engineering and cloud computing.
  • Documentation & Knowledge Sharing :

  • Create detailed documentation for data workflows, pipelines, and architectural decisions.
  • Mentor junior team members and promote a culture of learning and innovation.
  • Required Qualifications :

  • Experience : 7+ years of experience in data engineering, big data, or cloud-based data solutions.Proven expertise with Azure Data Bricks, Azure Data Lake, and Azure Synapse Analytics.
  • Technical Skills : Strong hands-on experience with Apache Spark and distributed data processing frameworks.Advanced proficiency in Python and SQL for data manipulation and pipeline development.Deep understanding of data modeling for OLAP, OLTP, and dimensional data models.Experience with ETL / ELT tools like Azure Data Factory or Informatica.Familiarity with Azure DevOps for CI / CD pipelines and version control.
  • Big Data Ecosystem : Familiarity with Delta Lake for managing big data in Azure.Experience with streaming data frameworks like Kafka, Event Hub, or Spark Streaming.
  • Cloud Expertise : Strong understanding of Azure cloud architecture, including storage, compute, and networking.Knowledge of Azure security best practices, such as encryption and key management.
  • Preferred Skills (Nice to Have) :

  • Experience with machine learning pipelines and frameworks like MLFlow or Azure Machine Learning.
  • Knowledge of data visualization tools such as Power BI for creating dashboards and reports.
  • Familiarity with Terraform or ARM templates for infrastructure as code (IaC).
  • Exposure to NoSQL databases like Cosmos DB or MongoDB.
  • Experience with data governance to

    Weekly Hours : 40

    Time Type : Regular

    Location :

    Hyderabad, Andhra Pradesh, India

    It is the policy of AT&T to provide equal employment opportunity (EEO) to all persons regardless of age, color, national origin, citizenship status, physical or mental disability, race, religion, creed, gender, sex, sexual orientation, gender identity and / or expression, genetic information, marital status, status with regard to public assistance, veteran status, or any other characteristic protected by federal, state or local law. In addition, AT&T will provide reasonable accommodations for qualified individuals with disabilities. AT&T is a fair chance employer and does not initiate a background check until an offer is made.

    Create a job alert for this search

    Data Big Data • hyderabad, India

    Related jobs
    • Promoted
    Associate Architect - Data Engineering

    Associate Architect - Data Engineering

    Response Informaticssecunderabad, telangana, in
    We are seeking an experienced Data Architect to lead the transformation of enterprise data.Alteryx workflows into Azure Databricks. Microsoft Azure ecosystem, including Azure.Data Factory, Databrick...Show moreLast updated: 27 days ago
    • Promoted
    Sr Engineer, Data [T500-20445]

    Sr Engineer, Data [T500-20445]

    ANSRHyderabad, Telangana, India
    ANSR is hiring for one of its clients.NASDAQ : TMUS), headquartered in Bellevue, Washington, is America’s supercharged Un-carrier, connecting millions through its strong nationwide network and flags...Show moreLast updated: 8 days ago
    • Promoted
    • New!
    Sr Data Analytics Engineer – ATCQE

    Sr Data Analytics Engineer – ATCQE

    Micronhyderabad, India
    Our vision is to transform how the world uses information to enrich life for all.Micron Technology is a world leader in innovating memory and storage solutions that accelerate the transformation of...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    Core Data Platform - Software Engineering-Associate-Bengaluru

    Core Data Platform - Software Engineering-Associate-Bengaluru

    Goldman Sachshyderabad, India
    What We Do At Goldman Sachs, our Engineers don’t just make things – we make things possible.Change the world by connecting people and capital with ideas. Solve the most challenging and pressing engi...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    Sr Specialist Data / AI Engineering

    Sr Specialist Data / AI Engineering

    AT&Thyderabad, India
    Data Engineering & Architecture : .Design, develop, and maintain high-performance data pipelines for structured and unstructured data using Azure Data Bricks and Apache Spark.Build and manage scalabl...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    Data and Analytics Senior Associate

    Data and Analytics Senior Associate

    RSMhyderabad, India
    We are the leading provider of professional services to the middle market globally, our purpose is to instill confidence in a world of change, empowering our clients and people to realize their ful...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    Data and Analytics Associate

    Data and Analytics Associate

    RSMhyderabad, India
    We are the leading provider of professional services to the middle market globally, our purpose is to instill confidence in a world of change, empowering our clients and people to realize their ful...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    Data Engineer- Associate

    Data Engineer- Associate

    FedExhyderabad, India
    Responsible for developing, optimize, and maintaining business intelligence and data warehouse systems, ensuring secure, efficient data storage and retrieval, enabling self-service data exploration...Show moreLast updated: 12 hours ago
    • Promoted
    Associate Data Engineer - Collibra / Alation

    Associate Data Engineer - Collibra / Alation

    TeamWare SolutionsHyderabad
    We are seeking an Associate Data Engineer to support data engineering and management initiatives.This role involves developing data solutions, conducting quality checks, and collaborating with team...Show moreLast updated: 6 days ago
    • Promoted
    • New!
    Sr Cloud Big Data Engineer, SMAI

    Sr Cloud Big Data Engineer, SMAI

    Micronhyderabad, India
    Our vision is to transform how the world uses information to enrich life for all.Micron Technology is a world leader in innovating memory and storage solutions that accelerate the transformation of...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    Sr. Systems Engineer (Big Data)

    Sr. Systems Engineer (Big Data)

    Anicalls (Pty) Ltdhyderabad, India
    The person hired will provide administration and platform support to our Development teams as new Cloud applications are deployed. The person would ensure Code and access to data is optimized, minim...Show moreLast updated: 12 hours ago
    • Promoted
    Senior Big Data Engineer

    Senior Big Data Engineer

    Grid DynamicsHyderabad, Telangana, India
    Proven experience leading data engineering teams, including distributed teams across multiple geographies and time zones. Effective in managing cross-team collaboration with architects, product mana...Show moreLast updated: 20 days ago
    • Promoted
    • New!
    Sr. Associate

    Sr. Associate

    Genpacthyderabad, India
    Genpact (NYSE : G) is a global professional services and solutions firm delivering outcomes that shape the future.Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepr...Show moreLast updated: 12 hours ago
    • Promoted
    Associate Manager - Data Solution [T500-20296]

    Associate Manager - Data Solution [T500-20296]

    H.E. Services IndiaHyderabad, Telangana, India
    A legacy of excellence, driving innovation and personalized service to create exceptional customer experiences.Services' vibrant tech Center in Hyderabad, you’ll have the opportunity to contribute ...Show moreLast updated: 8 days ago
    • Promoted
    Associate Director, Generative AI

    Associate Director, Generative AI

    PepsiCohyderabad, telangana, in
    PepsiCo is seeking a strategic and visionary Generative AI Solutions leader to lead transformative AI initiatives across Consumer, Commercial, and Reporting functions. This role will focus on design...Show moreLast updated: 30+ days ago
    • Promoted
    Associate Director / Director - Data Engineering & Modelling

    Associate Director / Director - Data Engineering & Modelling

    Sharpheads RPOHyderabad
    Job Title : Associate Director / Director Data Engineering & Modelling Location Hyderabad, Pune or Bengaluru (Hybrid) We ...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Sr. Big Data Developer

    Sr. Big Data Developer

    Anicalls (Pty) Ltdhyderabad, India
    Superior oral, and written communication skills, as well as the willingness to collaborate across teams of internal and external technical staff, business analysts, software support, and operations...Show moreLast updated: 12 hours ago
    • Promoted
    • New!
    Senior Associate - Data Engineer

    Senior Associate - Data Engineer

    Genpacthyderabad, India
    Ready to shape the future of work?.At Genpact, we don’t just adapt to change—we drive it.AI and digital innovation are redefining industries, and we’re leading the charge.Genpact’s AI Gigafactory, ...Show moreLast updated: 12 hours ago