No longer accepting applications
Data Engineer
OWOWamritsar, punjab, in8 days ago
Job descriptionBuild data collection pipelines for LLM fine-tuning (conversation logs, tool executions) Create feature stores for GNN training (customer interactions, engagement signals) Implement data versioning and lineage tracking for ML experiments Design A / B testing data infrastructure with CUPED variance reduction Build real-time feature computation pipelines for contextual bandits Design BigQuery schemas for marketing analytics and performance tracking Create materialized views and aggregation pipelines for real-time dashboards Implement data quality monitoring and anomaly detection Build observability infrastructure (Prometheus metrics, Grafana dashboards) Develop cost optimization strategies for cloud data warehousing MongoDB (conversation state, active sessions) Redis (caching, rate limiting, real-time data) Milvus (vector embeddings, semantic search) Neo4j (customer journey graphs, persona networks) BigQuery (analytics warehouse, historical data) Apache Airflow or Prefect (workflow orchestration) Pandas , Polars (data transformation) Apache Spark (optional - for large-scale processing) dbt (data transformation and modeling) vLLM (LLM inference serving) MLflow (model registry, experiment tracking) Sentence Transformers (embedding generation) PyTorch , TensorFlow (ML model training) Google Cloud Platform (BigQuery, Cloud Storage, Compute) Docker & Kubernetes (containerization, orchestration) Terraform (infrastructure as code) GitHub Actions or GitLab CI (CI / CD pipelines) Python 3.10+ (primary language) SQL (complex queries, query optimization) Shell scripting (Bash / Zsh) Git (version control) 5+ years of data engineering experience with production systems Expert-level SQL and database design skills Strong Python programming (async / await, type hints, testing) Experience with at least 3 different database technologies (SQL, NoSQL, Vector, Graph) Proven track record building high-scale data pipelines (> Deep understanding of data modeling (dimensional, normalized, denormalized) Experience with cloud data warehouses (BigQuery, Redshift, or Snowflake) Strong knowledge of data quality, validation, and governance Excellent debugging and optimization skills Experience with vector databases (Milvus, Pinecone, Weaviate, Qdrant) Experience with graph databases (Neo4j, ArangoDB, Neptune) Knowledge of embedding models and semantic search Experience with ML data pipelines (feature stores, model training data) Understanding of A / B testing and experimental design Experience with real-time streaming (Kafka, Pub / Sub, Kinesis) Knowledge of LLMs and conversational AI systems Experience with data migration projects (especially large-scale) Background in marketing technology or customer data platforms Experience with PyTorch Geometric or graph neural networks Knowledge of marketing analytics (attribution, segmentation, personalization) Familiarity with LangChain , LangGraph , or agent frameworks Experience with cost optimization in cloud environments Contributions to open-source data engineering projects Experience with data compliance (GDPR, CCPA) Migrate 10M+ conversation vectors from Pinecone to Milvus Design and implement MongoDB schemas for real-time agent state Set up Neo4j graph database with customer journey models Create BigQuery data warehouse with partitioned tables Build automated data quality monitoring system Implement caching strategies (Redis) for 10x latency reduction Optimize vector search queries (target : Create real-time analytics dashboards (Grafana) Build LLM fine-tuning data pipeline Implement feature store for GNN training Create A / B testing data infrastructure Design multi-armed bandit state management Collaborative team : Work with ML engineers, backend developers, and data scientists Modern stack : Latest technologies and tools Impact : Your work directly affects millions of marketing interactions Autonomy : Own your projects end-to-end Growth : Clear path to Senior / Lead / Principal roles
What You'll Build
Core Responsibilities
Data Architecture & Infrastructure (40%)
- Design and implement a multi-database architecture (MongoDB, Redis, Milvus, Neo4j, BigQuery)
- Build scalable data pipelines for real-time conversation processing and personalization
- Architect ETL / ELT workflows for data migration from legacy systems
- Implement data partitioning, sharding, and optimization strategies for high-throughput systems
- Create data governance frameworks ensuring quality, security, and compliance Vector & Graph Database Systems (25%)
- Design and optimize Milvus vector collections for semantic search (1024-dim embeddings)
- Build graph schemas in Neo4j for customer journey mapping and persona relationships
- Implement HNSW indexing strategies and similarity search optimization
- Create hybrid search systems combining vector, full-text, and graph queries
- Monitor and tune database performance (query latency, throughput, resource utilization)
ML Data Infrastructure (20%)
Analytics & Monitoring (15%)
Technical Stack You'll Work With
Databases & Storage
Data Processing & Orchestration
ML / AI Data Pipeline
Cloud & Infrastructure
Programming & Tools
Requirements
Must-Have Skills
1M records / day)
Highly Desirable
Nice-to-Have
Key Projects You'll Own
Phase 1 : Foundation
Phase 2 : Optimization
Phase 3 : ML Infrastructure
Work Environment
Create a job alert for this search
Data Engineer • amritsar, punjab, in
Related jobs
We’re Hiring – Freelancer Bidder (Remote / Mohali) HR Digital Company is expanding our digital network! We’re looking for a Freelancer Bidder / Business Development Executive who can bring in pro...Show moreLast updated: 5 hours ago
Data & AI Engineer – Cyber Risk Intelligence Platform – India.Quantara AI is a next-generation.Cyber Risk Intelligence and Governance.
CISOs, Boards, and executive teams.Our AI-powered solution comb...Show moreLast updated: 5 hours ago
![Tool room engineer [3 Days Left]](https://cdn-logos.talent.com/v1/logo/image?feedcode=attb-in&company_name=Fine%20Finish)
- Promoted
Senior Data Engineer
USEReadyamritsar, punjab, in Job Title : Senior Databricks Engineer.As a Senior Databricks Engineer, you will be responsible for designing, developing, and optimizing our data architecture and pipelines on the Databricks Lakeho...Show moreLast updated: 23 days ago
- Promoted
- New!
Urgent Search : Desktop Support Engineer - Kapurthala
ITC InfotechKapurthala Town, Punjab, India Desktop Support Engineer Location : Kapurthala, Punjab Mode : Work from Office - Installation and configuration of Windows 10 / 11 OS.
Troubleshooting and resolving OS-related issues - User support a...Show moreLast updated: 5 hours ago
- Promoted
Data Engineer
Response Informaticsamritsar, punjab, in AWS services : Must be proficient in building scalable data pipelines and managing cloud-native ETL workflows.Snowflake : Moderate understanding of Snowflake architecture.
CICD - Terraform or CloudFo...Show moreLast updated: 23 days ago
- Promoted
- New!
▷ (High Salary) Freelancer Bidder (Remote / Mohali)
HR Digital CompanyBatala, Punjab, IndiaRemote
- Promoted
Senior Software Engineer
Programmers.ioamritsar, punjab, in We are seeking a highly skilled and experienced Senior Azure Data Engineer to join our team.The ideal candidate will have deep expertise in Microsoft Azure data services, cloud-based data engineeri...Show moreLast updated: 13 days ago
- Promoted
- New!
Ai Video Editor (Urgent)
Manvi CreationBatala, Punjab, India Job Opening – New Office in Dera Bassi, Punjab! We are excited to announce the opening of our new office in Dera Bassi, Punjab, and we’re looking for passionate individuals to join our growing tea...Show moreLast updated: 5 hours ago
- Promoted
- New!
▷ [15h Left] Graphic designer and Video editor
HR Digital CompanyBatala, Punjab, India We’re Hiring — Graphic Designer & Video Editor Location : Bestech, Mohali, Punjab Company : HR Digital Company Contact : +91 9988777561 Experience : Minimum 6 months – 1 year preferred (Freshers we...Show moreLast updated: 5 hours ago
- Promoted
- New!
▷ (Urgent Search) Deputy Manager Purchase
Job Join HR ConsultantsBatala, Punjab, India Job Description Position : Asst Manager – Purchase Industry : Sheet metal / Automotive Experience : Minimum 10 Years Budgeted CTC : Up to ₹7 lacs Job Purpose The Deputy Manager – Purchase will be...Show moreLast updated: 5 hours ago
- Promoted
Desktop Support Engineer - Kapurthala
ITC InfotechKapurthala, Punjab, India Installation and configuration of Windows 10 / 11 OS.Troubleshooting and resolving OS-related issues.User support and issue resolution.
Provide first-level support for end-users experiencing issues wi...Show moreLast updated: 13 days ago
- Promoted
- New!
AWS Data Engineer
Tata Consultancy Servicesamritsar, punjab, in Aws data engineer having strong experience of Python.Technical / Behavioral Competency.Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from di...Show moreLast updated: 5 hours ago
- Promoted
- New!
Data & AI Engineer – Cyber Risk Intelligence Platform – India / Remote
Quantara AIamritsar, punjab, inRemote
- Promoted
Azure Cloud - AI ML Python Backend Engineer
Serenoamritsar, punjab, in You're someone who’s already shipped GenAI stuff—even if it was small : a chatbot, a RAG tool, or an agent prototype.You live in Python, LangChain, LlamaIndex, Hugging Face, and vector DBs like FAIS...Show moreLast updated: 8 days ago
- Promoted
Desktop Support Specialist
ITC InfotechKapurthala Town, Punjab, India Job Opportunity Desktop Support Specialist at ITC Infotech Location : KAPURTHALA / Trichy / Mallur Experience Required : 1.
Years Job Type : Full-Time Budget : Max 3.Shift allowance only Notice per...Show moreLast updated: 4 days ago
- Promoted
- New!
Human Resources Executive
HR Digital CompanyBatala, Punjab, India We’re Hiring – HR (Human Resources) for Spa Industry! ♀️ Join our growing spa team and take charge of recruitment, coordination, and staff management in a professional yet relaxing environment.Loc...Show moreLast updated: 11 hours ago
- Promoted
- New!
Apply in 3 Minutes! Analytical / Conjugation - Assistant Manager
Panacea BiotecBatala, Punjab, India Roles and Responsibility : Analytical / Conjugation Analytical Characterization - Development of assays for Protein and Carbohydrate characterization.
Standardization and optimization of various Bio...Show moreLast updated: 5 hours ago
- Promoted
- New!
▷ (15h Left) Process Engineering Manager
Job Join HR ConsultantsBatala, Punjab, India Process & Manufacturing Engineering - Develop, evaluate, and optimize manufacturing processes for fabrication, tubular components, and paint shop operations.
Troubleshoot process issues and impleme...Show moreLast updated: 5 hours ago
- Promoted
Area Sales Manager | Prepaid Sales
ConfidentialAmritsar, Ludhiana, Kapurthala Location - Amritsar / Kapurthala.Key Result Areas / Accountabilities.Review and evaluate Channel partners with focus on distributor 3i infrastructure (office, DSE, computer), investment (working capi...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
People Prime Worldwideamritsar, punjab, in Job Title : Senior Data Engineer.Skills : Data Engineering, AgenticAI, Python, SQL, Api’s, Azure cloud services, SQL and NoSQL ,Docker, Kubernetes.
We are seeking a highly skilled and experienced Seni...Show moreLast updated: 30+ days ago
- Promoted
- New!
Tool room engineer [3 Days Left]
Fine FinishBatala, Punjab, India Role Description This is a full-time on-site role for a Tool Room Engineer located in Dera Bassi.The Tool Room Engineer will be responsible for designing and developing tooling, die for press, mec...Show moreLast updated: 5 hours ago
- Promoted
Data Entry
ConfidentialHoshiarpur, Ferozepur, Gurdaspur Job Description : Data Entry / Back Office.Please call on the given number to apply 08375858125.Accurately enter, update, and manage financial data in banking systems.
Ensure data accuracy, integrity...Show moreLast updated: 30+ days ago