Talent.com
This job offer is not available in your country.
Big Data Engineer - Python / PySpark

Big Data Engineer - Python / PySpark

DexianChennai
15 days ago
Job description

Job Description :

We are intending to hire Data engineer to handle day-to-day activities involving data ingestion from multiple source locations, help identify data sources, to troubleshoot issues, and engage with a third-party vendor to meet stakeholders needs.

Work Location : Chennai or Hyderabad or Pune / WFO.

Shift hours : 2.00pm to 11.00pm IST.

  • Required Immediate Joiners.
  • Required Skills :
  • Python
  • Processing of large quantities of text documents
  • Extraction of text from Office and PDF documents
  • Input json to an API, output json to an API
  • Nifi (or similar technology compatible with current EMIT practices)
  • Basic understanding of AI / ML concepts
  • Database / Search engine / SOLR skills
  • SQL build queries to analyze, create and update databases
  • Understands the basics of hybrid search
  • Experience working with terabytes (TB) of data
  • Basic OpenML / Python / Azure knowledge
  • Scripting knowledge / experience in an Azure environment to automate
  • Cloud systems experience related to search and :
  • DataBricks
  • Snowflake
  • ESRI ArcGIS / SDE
  • New GenAI app being developed

Scope of work :

  • Ingest TB of data from multiple sources identified by the Ingestion Lead
  • Optimize data pipelines to improve on data processing, speed, and data availability
  • Make data available for end users from several hundred LAN and SharePoint areas
  • Monitor data pipelines daily and fix issues related to scripts, platforms, and ingestion
  • Work closely with the Ingestion Lead & Vendor on issues related to data Skills demonstrated :
  • SOLR Backend database
  • Nifi Data movement
  • Pyspark Data Processing
  • Hive & Oozie For jobs monitoring
  • Querying SQL, HQl and SOLR querying
  • Python
  • Behavioral Skills demonstrated :

  • Excellent communication skills
  • Ability to receive direction from a Lead and implement
  • Prior experience working in an Agile setup, preferred
  • Experience troubleshooting technical issues and quality control checking of work
  • Experience working with a globally distributed team in different
  • ref : hirist.tech)

    Create a job alert for this search

    Big Data Engineer • Chennai

    Related jobs
    • Promoted
    Big Data Developer - Python / Hadoop

    Big Data Developer - Python / Hadoop

    Risk Resources LLPChennai
    Required Technical Skill Set : Hadoop, Python, PySpark, HIVE Must Have : Hands-on experience of Had...Show moreLast updated: 30+ days ago
    Big Data Engineer

    Big Data Engineer

    Saaki Argus & Averil ConsultingChennai, Tamil Nadu, India
    Quick Apply
    Notice Period : Immediate Joiner or 30 Day.Hands-on experience on technologies like.Hive, Scala, Spark, AWS, HBase.Proficient in Hadoop distributions such as. Good working experience on technologies...Show moreLast updated: 30+ days ago
    • Promoted
    AWS Data Engineer - ETL / PySpark / Python

    AWS Data Engineer - ETL / PySpark / Python

    ChangeleadersChennai
    Position : AWS Data Engineer Experience : 5+ Yrs Location : Bangalore, Hyderabad, Chennai, Pune, Gurgaon <...Show moreLast updated: 15 days ago
    • Promoted
    Data Engineer - Typescript / Pyspark

    Data Engineer - Typescript / Pyspark

    NP GroupChennai, IN
    Data Engineer - Python, Pyspark & Typescript.Rate : $20 - $25 per hour ($3,000 - $4,000 per month).We have an immediate requirement for an experienced Data Engineer to join the global engineering te...Show moreLast updated: 11 days ago
    • Promoted
    Azure Data Engineer - Python / PySpark

    Azure Data Engineer - Python / PySpark

    AceNetChennai
    About Us : AceNet Consulting is a fast-growing global business and technology consulting firm specializing in business strategy, digital transformation, technology co...Show moreLast updated: 29 days ago
    • Promoted
    Colan Infotech - Azure Data Engineer - Python / PySpark

    Colan Infotech - Azure Data Engineer - Python / PySpark

    Colan Infotech Pvt LtdChennai
    Azure Data Engineer Location : Chennai / Bangalore (Work from Office only) Experience : 3-5 Years Job Type : Full-time Show moreLast updated: 12 days ago
    • Promoted
    Azure Databricks Engineer - Python / PySpark

    Azure Databricks Engineer - Python / PySpark

    optimum solutionsChennai
    We are seeking a skilled and experienced Azure Databricks Engineer to join our growing data engineering team.The ideal candidate will have deep hands-on expertise in building scalable data pipeline...Show moreLast updated: 4 days ago
    • Promoted
    Azure Data Engineer

    Azure Data Engineer

    RandomTreesChennai, IN
    We are seeking a skilled and experienced.The ideal candidate will have a strong background in building and optimizing data pipelines and data sets, utilizing Azure Data Factory, Databricks, PySpark...Show moreLast updated: 26 days ago
    • Promoted
    Data Engineer - ETL / PySpark

    Data Engineer - ETL / PySpark

    Intelliserve Konsulting Pvt. Ltd.Chennai
    Job Description : One should be responsible for maintaining, expanding, and optimizing our data warehouse, data pipeline, and data products.The dat...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer - Python / PySpark

    Senior Data Engineer - Python / PySpark

    Vtricks TechnologiesChennai
    Job Title : Senior Data Engineer - Azure | ADF | Databricks | PySpark | AWS Location : Bangalore, Hyderabad, Chennai (Hybrid Mode) Experience Required : 5+...Show moreLast updated: 15 days ago
    • Promoted
    CDM Smith - Senior Data Engineer - Python / PySpark

    CDM Smith - Senior Data Engineer - Python / PySpark

    CDM SMITH GLOBAL SERVICES INDIA LLPChennai
    Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer with deep expertise in Databricks to join our Digital Capital tea...Show moreLast updated: 17 days ago
    • Promoted
    Data Engineer - PySpark / Hive

    Data Engineer - PySpark / Hive

    Pan Asia HR SolutionsChennai
    Job Title : Data Engineer PySpark & Hive Location : Chennai / Pune / Bengaluru / Noida / Hyderabad Work Mode : ...Show moreLast updated: 7 days ago
    • Promoted
    Senior Data Engineer - ETL / PySpark

    Senior Data Engineer - ETL / PySpark

    Renovision Automation Services Pvt.Ltd.Chennai
    Essential Duties and Responsibilities : - This section contains a list of five to eight primary responsibilities of this role th...Show moreLast updated: 19 days ago
    • Promoted
    Data Engineer - ETL / PySpark

    Data Engineer - ETL / PySpark

    Axim DigitechChennai
    Key Skill : Pyspark, Cloudera Data Platfrorm, Big data Hadoop, Hive, Kafka Responsibilities : <...Show moreLast updated: 17 days ago
    • Promoted
    Data Engineer - Python / PySpark

    Data Engineer - Python / PySpark

    sphere ITChennai
    Responsibilities : - Ability to write clean, maintainable, and robust code in Python - Understanding and expertise of software engineeri...Show moreLast updated: 21 days ago
    • Promoted
    Data Engineering Manager - Python / PySpark / Kafka

    Data Engineering Manager - Python / PySpark / Kafka

    Risk Resources IndiaChennai
    Responsibilities : - Ability to design and develop a high-performance data pipeline framework from scratch ...Show moreLast updated: 28 days ago
    • Promoted
    Big Data Engineer

    Big Data Engineer

    Kresta Softech Private LimitedChennai, IN
    We’re looking for a skilled Big Data Engineer.Notice Period : Immediate / 15 days joiners Mandatory.Big Data, AWS, CI / CD Pipelines, Scala.Show moreLast updated: 21 days ago
    Data Engineer

    Data Engineer

    Tata Consultancy ServicesChennai, Tamil Nadu, India
    TCS present an excellent opportunity for Data Engineer.Experience required : 5-12 yrs.Skills : Apache Spark, Python, PySpark, Java, GCP BQ,BIG Data, Python, SQL, PySpark GCP.Walk in Interview date : ...Show moreLast updated: 13 days ago