Data Engineer - Business Intelligence
AVE QUESTNoida3 days ago
Job descriptionMentor junior software developers and build a strong team Model data and metadata to support discovery, ad-hoc, and pre-built reporting Design and implement data pipelines using Hadoop, Spark, and Azure services such as Blob Storage, SQL Database, Event Hubs, Data Factory, Synapse Analytics, and Databricks Should be able to write Programs & Scripting, Strong in SQL, Proficiency in Python or Scala. Experience with PowerShell or Azure CLI for automation is a plus Partner with security, privacy, and legal teams to deliver solutions that comply with security and privacy policies Own the design, development, and maintenance of datasets our business analytics teams will use to drive key business decisions Develop and promote standard methodologies in data engineering, including scalability, Tune and ensure compute performance by optimizing queries, databases, files, tables, Ensure data and report service level agreements are met Analyze and solve problems at their root, stepping back to understand the broader context Own continuous engineering operational excellence of the datasets that drive key business decisions Learn and understand a broad range of data resources and know when, how, and which to use and which not to use Keep up to date with advances in big data technologies and run pilots to design the data Continually improve ongoing reporting and analysis processes, automating or simplifying Triage many possible courses of action in a high-ambiguity environment, making use of both Bachelors degree in computer science or related technical field 10+ years of experience in data architecture and business intelligence 5+ years of experience in developing solutions in distributed technologies such as Hadoop, Spark Experience in delivering end-to-end solutions using Azure services Blob Storage, SQL Database, Event Hubs, Data Factory, Synapse Analytics, and HDInsight Experience in programming using Python, Java, or Scala Expert in data modeling, metadata management, and data quality SQL performance tuning Strong interpersonal and multitasking skills with the ability to balance competing priorities Excellent communication (verbal and written) and interpersonal skills and an ability to effectively communicate with both business and technical teams An ability to work in a fast-paced ambiguous environment where continuous innovation is occurring Experience with a business intelligence reporting Qualifications / Skills : Experience with Databricks for advanced analytics and data processing Understanding of well-architected data pipeline designs Expertise in monitoring and fault-tolerant pipeline designs Knowledge of cluster configuration for optimal performance Ability to create cost-optimal solutions Experience in exploratory data analysis (dashboarding, plotting) using machine learning technologies and algorithms is desirable Good knowledge of standard machine learning techniques (like regression, classification, anomaly detection, forecasting) by using standard machine learning libraries part of Spark, Python is desirable Prior experience in gen AI and related tools and techniques (such as large language models, prompt engineering) is desirable
Description :
In this role, we are seeking a data engineer to design, implement, and optimize cloud-based
data pipelines using Microsoft Azure services including ADF, Synapse, and :
- Collaborate with our business analytics and data science teams, gathering requirements and
delivering complete business intelligence solutions
reusability, maintainability, and usability
architecture to scale with increased data volume using Azure
self-service support for datasets
quantitative analysis and business we seek in you! : Qualifications / Skills :
(ref : hirist.tech)
Create a job alert for this search
Data Engineer • Noida