Senior Data Engineer

Job SummaryKogta Financial Ltd. is seeking an experienced and highly skilled ETL & Data Warehouse Developer with strong expertise in AWS data services . As a key member of our data engineering team, you will be responsible for designing, developing, and optimizing ETL pipelines and scalable data warehouse solutions on the AWS platform. The ideal candidate will have a solid background in data modeling, SQL performance tuning, and AWS-based data architectures .Key Responsibilities1. ETL DevelopmentDesign, develop, and implement robust ETL processes using AWS EMR, AWS Data Pipeline, or custom scripts .Ensure efficient extraction, transformation, and loading of data from multiple sources into the data warehouse.2. Data WarehousingDesign and maintain scalable, high-performance data warehouse solutions on AWS.Implement optimized data models and structures for AWS Redshift or equivalent environments.3. AWS Service UtilizationLeverage AWS services such as S3, Lambda, Redshift , EMR , Step Functions and others to build end-to-end data solutions.Stay updated with AWS advancements and recommend new tools or services to enhance data infrastructure.4. SQL ExpertiseDevelop and optimize complex SQL queries, stored procedures, and views for analytics and reporting.Troubleshoot and fine-tune SQL performance to ensure efficient data retrieval.5. Pyspark ExpertiseDesign, Develop and optimize ETL pipelines using Pyspark .Leverage optimization and data modelling capabilities to design data pipelines.6. Performance OptimizationContinuously monitor and enhance ETL and query performance.Identify and resolve processing bottlenecks across data pipelines and warehouse layers.7. Data Integration & CollaborationCollaborate with cross-functional teams to integrate data from multiple systems.Partner with business stakeholders to understand and deliver on data requirements.8. Security & ComplianceImplement strong data security and governance measures.Ensure compliance with relevant industry standards and organizational policies.9. DocumentationMaintain detailed documentation for ETL processes, data models, and configurations .Ensure proper handover and knowledge