Talent.com
This job offer is not available in your country.
Intermediate Airflow Developer

Intermediate Airflow Developer

iVedha Inc.Delhi, IN
30+ days ago
Job description

INTERMEDIATE AIRFLOW DEVELOPER

About the Role :

We are looking for an experienced Intermediate Airflow Developer with over 2 years of experience to help transition our existing Windows scheduler jobs to Apache Airflow DAGs. In this role, you’ll play a critical part in modernizing and optimizing our task automation processes by converting existing jobs into efficient, manageable, and scalable workflows in Airflow. You will also work on security hardening, implementing data pipelines, and designing ETL processes.

Key Responsibilities :

  • Convert Windows Scheduler Jobs to Airflow : Migrate existing Windows-based scheduled jobs into Airflow DAGs, ensuring smooth execution and reliability.
  • Develop and Optimize DAGs : Author, schedule, and monitor DAGs (Directed Acyclic Graphs) to handle data workflows, ETL tasks, and various automation processes.
  • Programming and Scripting : Use Python as the primary language for Airflow DAGs and task logic, with experience in SQL for data manipulation.
  • Set Up and Configure Airflow : Provide comprehensive instructions and configurations for setting up Airflow environments, including deployment, resource allocation, and high availability.
  • Security Hardening : Implement security best practices in Airflow, including role-based access control, network security, logging, and data protection.
  • Data Pipelines and ETL : Design and implement data pipelines to move, transform, and aggregate data, ensuring data accuracy and performance.
  • Troubleshooting and Optimization : Proactively monitor DAGs for performance issues, manage task dependencies, and optimize for performance and reliability.
  • Documentation and Knowledge Transfer : Document migration steps, Airflow setup instructions, and security configurations for future reference and knowledge sharing within the team.

Requirements :

  • 2+ years of experience in working with scheduling tools, task automation, and job orchestration.
  • Proficiency in Apache Airflow : Experience authoring and managing DAGs, with a solid understanding of Airflow architecture and best practices.
  • Strong Python Skills : Ability to write clear, maintainable Python code for data workflows and ETL processes.
  • SQL Knowledge : Proficiency in SQL for data manipulation within ETL tasks.
  • Experience with Windows Scheduler : Familiarity with converting Windows-scheduled jobs to Airflow and handling dependencies.
  • Security Awareness : Knowledge of security hardening principles within Airflow, including user roles, network configurations, and logging.
  • Problem-Solving Skills : Strong analytical skills to debug and troubleshoot workflow issues and optimize DAGs for performance.
  • Communication Skills : Ability to document and communicate setup processes, configurations, and migration steps clearly.
  • Preferred Qualifications :

  • Experience with Airflow in cloud environments such as AWS, GCP, or Azure.
  • Exposure to Docker and containerization for Airflow deployments.
  • Familiarity with DevOps practices and CI / CD tools.
  • Knowledge of additional programming languages like Bash, PowerShell, or JavaScript.
  • Create a job alert for this search

    Developer • Delhi, IN

    Related jobs
    • Promoted
    MicroGenesis Techsoft - Python Developer - Apache Airflow

    MicroGenesis Techsoft - Python Developer - Apache Airflow

    MicroGenesis TechsoftBangalore
    About the job Job Title : Python Developer Location : Bangalore, India Company : MicroGenesis TechSoft ...Show moreLast updated: 25 days ago
    Python, Airflow Data Engineer

    Python, Airflow Data Engineer

    Infosys LimitedBengaluru, Karnataka, India
    Python + Airflow Data Engineer.We are seeking a highly skilled Python / Airflow Data Engineer with 3 to 6 years of experience, specifically with a strong background in AWS technologies.The ideal ca...Show moreLast updated: 12 days ago
    • Promoted
    • New!
    Airflow Engineer - Python / Apache

    Airflow Engineer - Python / Apache

    Hire AlphaIndia
    Role : Airflow Engineer (Python, Apache) Grade : C1 Location : Netherlands / Pan India &...Show moreLast updated: 13 hours ago
    • Promoted
    Senior Data Engineer - Snowflake DB / Airflow

    Senior Data Engineer - Snowflake DB / Airflow

    Neerinfo SolutionsDelhi, IN
    Remote
    About the job : Job Description : We are looking for a talented and passionate Senior Data Engineer to join our growing data...Show moreLast updated: 30+ days ago
    Senior Java Software Engineer

    Senior Java Software Engineer

    L&T Technology ServicesIndia
    L&T Technology Services is seeking a Java Developer (Experience range - 5+ years) of experience, proficient in : .We are seeking a skilled Java Developer with strong expertise in Spring Boot, Microse...Show moreLast updated: 23 days ago
    • Promoted
    Intermediate Airflow Developer

    Intermediate Airflow Developer

    iVedha Inc.Dindigul, IN
    INTERMEDIATE AIRFLOW DEVELOPER.We are looking for an experienced Intermediate Airflow Developer with over 2 years of experience to help transition our existing Windows scheduler jobs to Apache Airf...Show moreLast updated: 30+ days ago
    • Promoted
    Airflow Engineer (Python, Apache)

    Airflow Engineer (Python, Apache)

    HireAlphaJaipur, IN
    Role : Airflow Engineer (Python, Apache).Location : Netherlands / Pan India.We are seeking a Python Developer to join our IGA (Identity Governance and Administration) team as an Airflow Engineer.This r...Show moreLast updated: 1 day ago
    Java Developer (Intermediate Level)

    Java Developer (Intermediate Level)

    SYNECHRONINDIA
    We are seeking an Intermediate Java Developer to contribute to the development of innovative applications using modern technologies. The ideal candidate will have a strong background in Java develop...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer - ETL / Apache Airflow

    Data Engineer - ETL / Apache Airflow

    HyrEzy Talent SolutionsPune
    Job Description and Responsibilities : Solicit business, functional, non-functional and technical requirements through interviewing and requiremen...Show moreLast updated: 30+ days ago
    • Promoted
    GCP Data Engineer - Python / SQL

    GCP Data Engineer - Python / SQL

    QUARKS TECHNOSOFT PRIVATE LIMITEDJaipur
    Location : : 510 years Job Type : Full-time Domain : Data Engineering / GCP / Cloud Analytic...Show moreLast updated: 25 days ago
    Airflow Lead

    Airflow Lead

    QuEST Global Services Pte. LtdIndia
    Quest Global is an organization at the forefront of innovation and one of the world’s fastest growing engineering services firms with deep domain knowledge and recognized expertise in the top OEMs ...Show moreLast updated: 30+ days ago
    • Promoted
    Sr. Data Engineer-Airflow, PLSQL

    Sr. Data Engineer-Airflow, PLSQL

    RelantoGreater Bengaluru Area, India
    Position : Senior Data Engineer - Airflow, PLSQL.Location : Hyderabad / Bangalore / Pune.Seeking a Senior Data Engineer with strong expertise in Apache Airflow and Oracle PL / SQL, along with working exper...Show moreLast updated: 25 days ago
    Intermediate Mobile Developer

    Intermediate Mobile Developer

    myBlueprintBangalore, Karnataka, IN
    Quick Apply
    About myBlueprint myBlueprint is a leading developer of K-12 EdTech tools used by over 1 million students across Canada.We create best-in-class digital portfolios and career / life planning tools to ...Show moreLast updated: 30+ days ago
    Python Airflow

    Python Airflow

    virtusa consulting services pvt. ltd.INDIA
    Seeking a developer who has good experience in Athena, Python code, Glue, Lambda, DMS, RDS, Redshift Cloud Formation, and other AWS serverless resources. Can optimize data models for performance and...Show moreLast updated: 30+ days ago
    Intermediate Software Developer, Cloud Backend (AWS)

    Intermediate Software Developer, Cloud Backend (AWS)

    Evertz Microsystems LimitedBengaluru, Karnataka, IN
    Quick Apply
    We’re looking for a motivated Intermediate Software Developer with a strong background in cloud-native backend development on AWS to join our fast-paced, collaborative team.You’ll work ...Show moreLast updated: 30+ days ago
    Intermediate Apps Developer

    Intermediate Apps Developer

    UPS IndiaChennai, Tamil Nadu, India
    Explore your next opportunity at a Fortune Global 500 organization.Envision innovative possibilities, experience our rewarding culture, and work with talented teams that help you become better ever...Show moreLast updated: 30+ days ago
    • Promoted
    ExxonMobil - Data Engineer - ETL / Airflow

    ExxonMobil - Data Engineer - ETL / Airflow

    Exxonmobil Lubricants Pvt. LtdBangalore
    About us : At ExxonMobil, our vision is to lead in energy innovations that advance modern living and a net-zero future.As one of the worlds largest publicly traded en...Show moreLast updated: 26 days ago
    Intermediate Full Stack Developer

    Intermediate Full Stack Developer

    ExxonMobil CorporationBengaluru, KA, IN
    Closely work with a team of developers, leading every initiative.Close collaboration with Product Owner, Product Manager, Scrum Master and Architects. Work closely with Lead Devs to the design, arch...Show moreLast updated: 17 days ago
    Machine Learning Engineer | GCP + Airflow

    Machine Learning Engineer | GCP + Airflow

    VAYUZ TechnologiesIndia
    Must have : Strong on programming languages like Python, Java One cloud hands-on experience (GCP preferred) Experience working with Dockers Environments managing (e. Experience with orchestrators lik...Show moreLast updated: 1 day ago
    • Promoted
    Data Engineer (GCP)

    Data Engineer (GCP)

    SourcebaeBengaluru, Karnataka, India
    GCP) EDW Experience with GCP cloud data related services.Google Big Query from SQL sources as well as APIs.Familiar with authentications,. Key Skills : Google Cloud Storage, Google Big Query, Google ...Show moreLast updated: 1 day ago