Principal Data Pipeline Architect

Sonata SoftwareBengaluru, Republic Of India, IN

24 days ago

Job description

Mandatory

Experience building data pipelines
Highly proficient in python, Data frames
Experience with AWS Lambda, Events (SQS), API calls, Kinesis, DynamoDB
Highly proficient in SQL Query
Snowflake or experience with similar analytics database
Experience extracting data from SQL and NoSQL databases.
Experience with CI / CD pipeline to release the code in different environment

Nice to have

AWS CDK

Experience with upgrading Python version and all the linked connector in AWS environment using CDK

Experience with other modern ETL platforms

Snowpipe - to move data from S3 bucket to Snowflake

Job Description

A Typical Day

The Senior Software Engineer supports functions which require automation / systems development, including initial development and ongoing support of the suite of applications that would help manage loan, pricing, enterprise data pipeline, or other functions depending on the department. As the Application Developer, you must be proficient in Python, Data structures and ETL skill sets that support the IT development path.

The Senior Application Developer will :

Data Integration : Design, develop, and maintain integrations between pennymac loan origination systems, using AWS Fargate, Lambda, Kinesis, SQS, Snowpipe etc.

Data Quality : Implement data quality rules, matching algorithms, and cleansing routines to ensure the accuracy and completeness of loan production data

Performance Optimization : Monitor and optimize the performance of all data pipelines and their integrations to ensure efficient data processing and access, especially in relation to real-time analytic consumers.

Collaboration : Work closely with data engineers, architects, business stakeholders, and teams managing or consuming production data to gather requirements, design solutions, and troubleshoot production incidents and issues.

Business Continuity : Remain available to help diagnose and troubleshoot issues. On-call shifts rotate on a regular basis.

What You’ll Bring

Bachelor’s degree or equivalent work experience

5+ years of relevant work experience

Knowledge of Agile Methodologies, such as Scrum is helpful

Experience with relational databases (Snowflake, SQL Server)

Experience with NoSQL databases (DynamoDb)

. Experience with API calls

. Experience with software upgrade like Python, Py odbc, snowflake connector etc

Experience designing and implementing with Data warehouse solutions leveraging Snowflake.

Knowledge of design patterns, refactoring and unit testing

Financial services experience, especially related to mortgages and mortgage backed securities, is desirable

Create a job alert for this search

Data Pipeline Architect • Bengaluru, Republic Of India, IN

Related jobs

Promoted

Enterprise Data Architect

DBiz.aihosur, tamil nadu, in

You will ensure that data is modeled, structured and integrated across the organization to support analytics and AI / ML initiatives at scale. You will work closely with business stakeholders and tech...Show moreLast updated: 15 days ago

Promoted

AI Data Architect

Tata Consultancy Serviceshosur, tamil nadu, in

We are seeking an inventive Data Architect for AI with 8–15 years of experience to lead the strategic design and implementation of enterprise-scale AI solutions. ETL / ELT, Big Data cloud data solutio...Show moreLast updated: 21 days ago

Promoted

Senior Azure Data Engineer (Architect) - Pan India

Genpacthosur, tamil nadu, in

We are seeking an experienced Azure Data & Analytics Architect with a strong background in delivery and excellent communication skills. The ideal candidate will have experience and a proven track re...Show moreLast updated: 18 days ago

Promoted

Data Architect - Snowflake (Job Location : Kerala)

Reflections Info Systemshosur, tamil nadu, in

We are looking for 10 + year experienced Data Architect with strong background in Snowflake, demonstrating leadership in technical design, architecture, and implementation of complex data solutions...Show moreLast updated: 17 days ago

Promoted

Data Architect

BrillioBengaluru, Karnataka, India

Architecting, designing and implementing Modern Data Platforms and Solutions using Microsoft Azure Cloud and understands reference architectures. Deep understanding Data Migration Strategies with 2+...Show moreLast updated: 4 days ago

Promoted

Principal Azure Cloud Architect / Principal Engineer

First American (India)hosur, tamil nadu, in

Job Title : Principal Azure Cloud Architect / Principal Engineer (20+ Years Experience).We are seeking a technical visionary Principal Azure Cloud Architect with 20+ years of experience to lead the tr...Show moreLast updated: 30+ days ago

Promoted

Azure Data Engineer

LTIMindtreehosur, tamil nadu, in

Role Senior Data Engineer 8 years of experience.Build reusable utilities templates and automation pipelines.Design scalable data engineering frameworks standards and best practices.Provide architec...Show moreLast updated: 30+ days ago

Promoted

Data Architect

Tata Consultancy Serviceshosur, tamil nadu, in

Role- Snowflake Data Architect.Cloud data warehouse technologies on Snowflake and AWS, Azure or GCP.At least one End-to-end Snowflake implementation is a must covering all aspects including archite...Show moreLast updated: 30+ days ago

Promoted

Backend and Data Pipeline Engineer

JRD Systemshosur, tamil nadu, in

Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago

Promoted

Principal / Senior Data Architect

AaysBengaluru, IN

Position : Principal / Senior Data Architect.You will act as a key member of the consulting team helping Clients to re-invent their corporate finance function by leveraging advanced analytics.You wil...Show moreLast updated: 30+ days ago

Promoted

Senior Architect - Data

Quantiphihosur, tamil nadu, in

Quantiphi is an award-winning Data Science and Machine Learning Software and Services Company focused on helping organizations translate the big promise of Machine Learning technologies into quanti...Show moreLast updated: 30+ days ago

Promoted

Python Tech Lead / Sr Architect

G10Xhosur, tamil nadu, in

Python Developer & Architect (AWS | Kafka | Integrations).We are seeking an experienced Technical Lead / Senior Python Developer & Architect with strong expertise in AWS, Kafka, and large-scale sys...Show moreLast updated: 2 days ago

Promoted

GCP Data Architect

Tata Consultancy Servicesbangalore, karnataka, in

Design and Implement Data Architectures : Architect and build scalable, end-to-end data solutions on GCP, encompassing data ingestion, transformation, storage, and consumption.Develop Data Pipelines...Show moreLast updated: 26 days ago

Promoted

Senior Data Architect

MacroHirehosur, tamil nadu, in

Job Profile- Sr Data Architect.Type of job- Contract for 1 year (extendable).Data Architect will focus on efficiency, security, and performance. Responsibilities include monitoring, optimization, tr...Show moreLast updated: 1 day ago

Promoted

Senior DAC Architect

Mulya TechnologiesGreater Bengaluru Area, India

About Omni Design Technologies.Omni Design Technologies is a leading provider of high-performance, ultra-low power IP cores, from 28nm down through advanced FinFET nodes, which enable differentiate...Show moreLast updated: 30+ days ago

Promoted

Data Architect

Persistent SystemsBengaluru, Karnataka, India

We are looking for an experienced and talented Data Architect to join our growing data engineering team.The ideal candidate will have a strong background in working with Python, Pyspark, AWS / Databr...Show moreLast updated: 30+ days ago

Promoted

Data Architect

CareerUS Solutionshosur, tamil nadu, in

The Data Architect is responsible for.This includes ensuring that data is.The role focuses on developing.Collaborate with software engineers, analysts, and stakeholders to understand data needs and...Show moreLast updated: 3 days ago

Promoted

Azure Data Architect

Programmers.iohosur, tamil nadu, in

We offer a vibrant and collaborative work environment, cutting-edge tools and technologies, and ample opportunities for professional growth. Job Title : Azure Data Architect.Experience required : 15+ ...Show moreLast updated: 1 day ago