ETL Specialist

We are seeking a highly skilled ETL Specialist with expertise in building, maintaining, and optimizing data pipelines using Python scripting. The ideal candidate will have experience working in a Linux environment, managing large-scale data ingestion, processing files in S3, and balancing disk space and warehouse storage efficiently. This role will be responsible for ensuring seamless data movement across systems while maintaining performance, scalability, and reliability.

Key Responsibilities :

• ETL Pipeline Development : Design, develop, and maintain efficient ETL workflows using Python to extract, transform, and load data into structured data warehouses.
• Data Pipeline Optimization : Monitor and optimize data pipeline performance, ensuring scalability and reliability in handling large data volumes.
• Linux Server Management : Work in a Linux-based environment, executing command-line operations, managing processes, and troubleshooting system performance issues.
• File Handling & Storage Management : Efficiently manage data files in Amazon S3, ensuring proper storage organization, retrieval, and archiving of data.
• Disk Space & Warehouse Balancing : Proactively monitor and manage disk space usage, preventing storage bottlenecks and ensuring warehouse efficiency.
• Error Handling & Logging : Implement robust error-handling mechanisms and logging systems to monitor data pipeline health.
• Automation & Scheduling : Automate ETL processes using cron jobs, Airflow, or other workflow orchestration tools.
• Data Quality & Validation : Ensure data integrity and consistency by implementing validation checks and reconciliation processes.
• Security & Compliance : Follow best practices in data security, access control, and compliance while handling sensitive data.
• Collaboration with Teams : Work closely with data engineers, analysts, and product teams to align data processing with business needs.

Skills Required :

Nice to Have :

Qualifications :