The IT Data Engineer plays a critical role in enabling data-driven decision-making across the organization by designing, building, and maintaining robust data infrastructure. This position is responsible for developing scalable data pipelines, managing databases and data warehouses, and ensuring the integrity, accessibility, and quality of enterprise data. Through thoughtful architecture and efficient processing systems, the Data Engineer supports analytical initiatives and empowers teams with reliable, well-organized data assets.
Key Responsibilities
- Design, build, and maintain scalable data pipelines to extract, transform, and load data from diverse sources into centralized repositories (e.g., data warehouses, data lakes)
- Develop and manage databases, data warehouses, and data lakes to support analytics and reporting needs
- Write and maintain efficient code and scripts for data processing, transformation, and manipulation
- Ensure data quality, consistency, and reliability across all systems and pipelines
- Implement and uphold data security, privacy, and access controls in compliance with governance standards
- Collaborate with cross-functional teams to deliver data solutions aligned with business objectives
- Monitor and optimize the performance, scalability, and cost-efficiency of data infrastructure
- Create and maintain clear documentation for data architecture, processes, and workflows
- Stay current with emerging data engineering technologies and recommend improvements to existing systems
- Design and implement data models and schemas to support analytical and machine learning use cases
- Build curated datasets and data marts to enable self-service analytics for business users
- Establish monitoring and alerting systems to proactively detect and resolve data pipeline issues
- Support machine learning operations by collaborating with data scientists on feature engineering and model deployment
- Leverage cloud platforms (e.g., AWS, Azure, GCP) to build and maintain modern, scalable data infrastructure
- Contribute to data governance initiatives by ensuring compliance with internal policies and external regulations
While the job description describes what is anticipated as the requirements of the position, the job requirements are subject to change based upon any changing needs and requirements of the business.
Required Skills
Proficiency in designing, building, and supporting large-scale data warehouses using cloud platforms such as Azure, AWS, or Google CloudStrong experience with Big Data technologies (e.g., Snowflake, Hadoop, Spark), data modeling (OLTP / OLAP), and database design and managementSkilled in writing and debugging complex SQL scripts and Python-based data processing using libraries such as Pandas and PySparkHands-on experience with ETL / ELT processes, orchestration tools (e.g., Apache Airflow, Azure Data Factory), and CI / CD practicesFamiliarity with containerization and deployment tools like Docker and KubernetesExperience with version control systems such as GitKnowledge of streaming data technologies (e.g., Kafka, Spark Streaming) for real-time data processingUnderstanding of data governance, privacy practices, and compliance regulationsExisting technical knowledge of modern systems software, protocols, and standards, including SQL Server, Snowflake / Databricks / Fabric, and Power BIExperience with data visualization tools such as Power BI or TableauAbility to translate business requirements into scalable technical solutionsStrong analytical, problem-solving, and troubleshooting skillsExcellent communication and collaboration abilities across technical and non-technical teamsDetail-oriented with a strong focus on data quality, consistency, and reliabilityComfortable working independently and in a fast-paced, team-oriented environmentFamiliarity with Agile or Scrum methodologies is a plusRequired Experience
Bachelor's degree in Computer Science, Software Engineering or related field.5+ years of experience in data engineering or BI / reporting roles with a strong back-end focusTungsten Automation Corporation, Inc. is an Equal Opportunity Employer M / F / Disability / Vets
Tungsten Automation Corporation, Inc. is an Equal Opportunity Employer M / F / Disability / Vets
Skills Required
Pyspark, Kafka, Tableau, Data Modeling, ELT, Apache Airflow, Docker, snowflake , Python, Aws, Hadoop, Power Bi, Olap, Sql Server, Sql, Google Cloud, Git, Spark Streaming, Azure Data Factory, Pandas, Spark, Databricks, Azure, Kubernetes, fabric , Oltp, Etl