Data Pipeline Architect

We are looking for a Sr Data Engineer who is passionate about managing large volumes of data, eager to learn and take on challenges, and committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then we want to hear from you

Roles and Responsibilities :

• Experience working with AWS service for data ETL
• 3-5 yrs years of Experience in developing data ingestion, data processing, and analytical pipelines for big data, relational databases, NoSQL, and data warehouse solutions
• Experience in designing data models effectively to manage optimal performance.
• Designing, developing, monitoring, and maintaining end-to-end data pipelines
• Should have an extensive working knowledge of handling around 100GB of data in data ingestion, data processing and pipeline creation.
• Experience with designing and implementing complex high-throughput distributed batch and / or real-time solutions using one of AWS Glue, Apache Kafka, AWS Kinesis, Spark, Flink and Apache Airflow
• Extensive hands-on experience implementing data migration, data processing, ETL, or ELT pipeline development and monitoring.
• Strong programming skills in Python, SQL skills for data analysis and reporting
• Should know about any Data Warehouse like Redshift, Snowflake, and BigQuery
• Proven ability to work independently, mentor peers and meet tight deadlines
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases like Postgres, SQL Server, MySQL
• Should know about NoSQL Database like MongoDB, Cassandra, Neptune
• Experience in writing complex queries, functions, store procedures, execution plans, Performance Tuning and Query Optimization, Indexing, Partitioning, and Denormalization
• Should be well versed with migration strategies by manual and automated processes
• Excellent interpersonal, communication skills (both verbal and written), and ability to interact effectively
• Should attend client calls independently
• AWS certification would be a big plus