Job Description :
We are intending to hire Data engineer to handle day-to-day activities involving data ingestion from multiple source locations, help identify data sources, to troubleshoot issues, and engage with a third-party vendor to meet stakeholders needs.
Work Location : Chennai or Hyderabad or Pune / WFO.
Shift hours : 2.00pm to 11.00pm IST.
- Required Immediate Joiners.
Required Skills : - Python
- Processing of large quantities of text documents
- Extraction of text from Office and PDF documents
- Input json to an API, output json to an API
- Nifi (or similar technology compatible with current EMIT practices)
- Basic understanding of AI / ML concepts
- Database / Search engine / SOLR skills
- SQL build queries to analyze, create and update databases
- Understands the basics of hybrid search
- Experience working with terabytes (TB) of data
- Basic OpenML / Python / Azure knowledge
- Scripting knowledge / experience in an Azure environment to automate
- Cloud systems experience related to search and :
- DataBricks
- Snowflake
- ESRI ArcGIS / SDE
- New GenAI app being developed
Scope of work :
Ingest TB of data from multiple sources identified by the Ingestion LeadOptimize data pipelines to improve on data processing, speed, and data availabilityMake data available for end users from several hundred LAN and SharePoint areasMonitor data pipelines daily and fix issues related to scripts, platforms, and ingestionWork closely with the Ingestion Lead & Vendor on issues related to data Skills demonstrated :SOLR Backend databaseNifi Data movementPyspark Data ProcessingHive & Oozie For jobs monitoringQuerying SQL, HQl and SOLR queryingPythonBehavioral Skills demonstrated :
Excellent communication skillsAbility to receive direction from a Lead and implementPrior experience working in an Agile setup, preferredExperience troubleshooting technical issues and quality control checking of workExperience working with a globally distributed team in differentref : hirist.tech)