Senior Data Engineer - Databricks

AuxoAI is seeking a skilled and experienced Data Engineer to join our dynamic team. The ideal candidate will have 7-10 years of prior experience in data engineering with a strong background in Databricks. This role offers an exciting opportunity to work on diverse projects collaborating with cross-functional teams to design build and optimize data pipelines and infrastructure.

Responsibilities :

Designing building and operating scalable on-premises or cloud data architecture

Analyzing business requirements and translating them into technical specifications

Design develop and implement data engineering solutions using Databricks on cloud platforms (AWS Azure)

Design develop and maintain scalable data pipelines and ETL processes

Collaborate with data scientists and analysts to understand data requirements and implement solutions that support analytics and machine learning initiatives.

Optimize data storage and retrieval mechanisms to ensure performance reliability and cost-effectiveness

Implement data governance and security best practices to ensure compliance and data integrity

Troubleshoot and debug data pipeline issues providing timely resolution and proactive monitoring

Stay abreast of emerging technologies and industry trends recommending innovative solutions to enhance data engineering capabilities.

Requirements

Bachelors or Masters degree in Computer Science Engineering or a related field.

Overall 7 years of prior experience in data engineering with a focus on designing and building data pipelines

4 years experience in Databricks and Spark in at least two end-to-end data engineering implementations

Comprehensive understanding of the Databricks products and ecosystem

Strong programming skills in languages such as Python Scala or SQL.

Experience with data modeling ETL processes and data warehousing concepts.

Familiarity with containerization technologies orchestration tools is a plus.

Qualifications : 5 + years of experience in developing and deploying the RPA solutions and should have good knowledge on any programming knowledge. Possess advanced certifications in at least one RPA tool, such as UiPath. Expert in orchestrator administrations and UiPath troubleshooting Be proficient in a programming language like Python, .NET, or Java. Demonstrate a strong understanding of RPA best practices and lifecycle management. Exhibit excellent problem-solving skills and the ability to break down complex challenges into manageable tasks. Have strong verbal and written communication skills to effectively collaborate with stakeholders and team members. Thrive in a fast-paced startup environment with a proactive and innovative mindset.

Key Skills

Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

Employment Type : Full Time

Experience : years

Vacancy : 1