Data Pipeline Development : Design, build, and maintain efficient, scalable, and reliable data pipelines using Databricks and Python to ingest, transform, and load data from various sources.
Databricks Expertise : Leverage advanced features of Databricks, including Delta Lake, Spark SQL, and Databricks notebooks, for large-scale data processing and analytics.
ETL / ELT Processes : Develop and optimize complex ETL / ELT processes to ensure data quality, consistency, and timely availability.
Data Modeling : Collaborate with data scientists and analysts to understand data requirements and design appropriate data models for optimal storage and retrieval.
Performance Optimization : Identify and resolve performance bottlenecks in data pipelines and Databricks jobs, optimizing data processing efficiency.
Cloud Integration : Work with cloud services (e.g., Azure Data Lake Storage, AWS S3, Google Cloud Storage) for data storage and integration with Databricks.
Automation : Implement automation for data pipeline orchestration, monitoring, and alerting.
Collaboration : Work closely with cross-functional teams, including data scientists, analysts, and software engineers, to deliver comprehensive data solutions.
Documentation : Create and maintain technical documentation for data pipelines, data models, and Skills & Qualifications
Experience : 6 to 10 years of hands-on experience as a Data Engineer.
Data Bricks : Strong, proven experience with Databricks platform, including Spark SQL and Delta Lake.
Python : Expert-level proficiency in Python for data engineering tasks.
SQL : Excellent command of SQL for data manipulation, querying, and optimization.
Data Warehousing : Solid understanding of data warehousing concepts, data modeling (star schema, snowflake schema), and ETL / ELT principles.
Cloud Platforms : Experience with at least one major cloud provider's data services (e.g., Azure Data Factory, AWS Glue, Google Cloud Dataflow).
Version Control : Proficiency with version control systems, especially Git.
Problem-Solving : Strong analytical and problem-solving skills to tackle complex data challenges.
Communication : Excellent verbal and written communication skills.
Education : Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
ref : hirist.tech)
Create a job alert for this search
Data Engineer • Hyderabad
Related jobs
Promoted
Big Data Developer - AWS Glue
RudrakshaHyderabad
Position : Big Data Engineer Location : Noida & Hyderabad (onsite) Type : Full Time Show moreLast updated: 8 days ago
Promoted
AWS Data Engineer - ETL / PySpark / Python
ChangeleadersHyderabad
Position : AWS Data Engineer Experience : 5+ Yrs Location : Bangalore, Hyderabad, Chennai, Pune, Gurgaon <...Show moreLast updated: 15 days ago
Promoted
Data Engineer
CozzeraHyderabad, IN
We are seeking a highly skilled and experienced.The ideal candidate should have.Design, develop, and maintain robust, scalable, and efficient data pipelines and ETL / ELT processes.Integrate data fro...Show moreLast updated: 2 days ago
Data Engineer
Z2DataHyderabad, Telangana, India
We’re building a powerful Data Analytics Engine to transform raw market and historical data into actionable intelligence in the electronics supply chain industry.
We’re now looking to bring on a ful...Show moreLast updated: 11 days ago
Senior Data Engineer with Python & AWS
SEIN CONSULTING PRIVATE LIMITEDHyderabad, Telangana, India
Senior Python Data Engineer AWS Focus - In Hyderabad - Permanent & Onsite Opportunity.We are seeking a highly skilled Python Data Engineer with deep expertise in AWS-based data solutions.Th...Show moreLast updated: 30+ days ago
AWS Databricks
LTIMindtreeHyderabad, Telangana, India
Location-Mumbai / Pune / Bangalore / Hyderabad / Chennai.Candidate must be experienced working in projects involving.Other ideal qualifications include experiences in.
Primarily looking for a data engineer ...Show moreLast updated: 14 days ago
Data Engineer (AWS Glue)
SRIJAN TECHNOLOGIESINDIA
We are seeking a highly skilled and motivated Data Engineer to join our dynamic team.The ideal candidate will have extensive experience with AWS Glue, Apache Airflow, Kafka, SQL, Python, and DataOp...Show moreLast updated: 30+ days ago
Promoted
Data Engineer – ETL
Demand AIHyderabad, IN
ETL development and data engineering.Looking for a Data Engineer specializing in ETL processes to design and implement data pipelines that support data integration and analytics initiatives.Design,...Show moreLast updated: 22 days ago
Technical Lead- ETL with AWS Glue-P...
Anicalls (Pty) LtdHyderabad, India
Design, Build and Maintain and Deliver technical architectures that help our manufacturing, automotive, transportation, engineering customers digitally transform their operations.Experience is writ...Show moreLast updated: 30+ days ago
AWS Glue ETL Developer
Tata Consultancy ServicesHyderabad, Telangana, India
We are seeking a highly skilled AWS Glue ETL Developer with 8 to 12+ years of experience.The ideal candidate will have a strong background in data engineering with extensive experience in ETL proce...Show moreLast updated: 14 days ago
Data Engineer – AWS Glue, Redshift,, PySpark
Cyepro SolutionsHyderabad, Telangana, India
Cyepro Solutions is at the forefront of innovation in customer relationship management, specializing in the automotive industry.
We offer a platform that empowers dealerships, manufacturers, and ser...Show moreLast updated: 20 days ago
Promoted
AWS Data Engineer - ETL / Python
DeqodeHyderabad
Key Responsibilities : - Design and develop robust ETL / ELT pipelines using Python and AWS Glue / Lambda Show moreLast updated: 29 days ago
ETL Developer
Saransh IncHyderabad, Telangana, India
Walk-In Drive | ETL Lead & Senior Developer | Hyderabad | Immediate Joiners Preferred 🚨.ETL Lead Developer (8+ yrs, with 5+ yrs in Lead roles).
ETL Senior Developer (6–8 yrs).AWS (Glue, S3, Athena)...Show moreLast updated: 2 days ago
Promoted
Perficient - Data Engineer - ETL / AWS
Perficient India Private LimitedHyderabad
Key Responsibilities : - Design, build, and maintain scalable ETL pipelines and data workflows using AWS Glue, Spark, a...Show moreLast updated: 29 days ago
AWS Data Engineer RedShift-C2H-TCS
Axiom Software Solutions LimitedHyderabad, TS, IN
Quick Apply
Job Title : AWS Data Engineer - RedShift.Company : Axiom Software Solutions Limited.Location : On-site (as required).Axiom Software Solutions Limited is seeking an AWS Data Engineer with a strong focu...Show moreLast updated: 21 days ago
Lead / Senior Data Engineer
DATAECONOMYHyderabad, TG, in
Quick Apply
We are seeking a highly experienced and hands-on.Design scalable and secure data pipelines using AWS Glue, PySpark, and related AWS services (EMR, S3, Lambda, etc.
Guide junior engineers, conduct co...Show moreLast updated: 1 day ago
Promoted
Cloud Data Engineer
LemongrassHyderabad, IN
Lemongrass is a software-enabled services provider, synonymous with SAP on Cloud, focused on delivering superior, highly automated Managed Services to Enterprise customers.Our customers span multip...Show moreLast updated: 30+ days ago
Promoted
Proclink - AWS Data Engineer - PySpark / ETL
ProclinkHyderabad
Job Description : AWS Data Engineer We are seeking a skilled AWS Data Engineer with expertise in various AWS services.The ideal candidate will have hands-on experienc...Show moreLast updated: 5 days ago