Talent.com
This job offer is not available in your country.
Data Engineer (Big Data, Hadoop, Spark) - Renewable Contract

Data Engineer (Big Data, Hadoop, Spark) - Renewable Contract

DautomMalappuram, IN
1 day ago
Job description

Data Engineer (Big Data, Hadoop, Spark)

Location : Offshore - India (Remote)

Contract Duration : 6 Months (Renewable)

Client Industry : Banking / Financial Services

Employment Type : Contract (Payroll : Dautom IT Consultants LLC)

Notice Period : 30 days or less

Experience : 6-10+ years of proven experience as a Big Data Engineer.

Application Note :

Independent contractors with their own firms are not encouraged to apply for this role.

About the Role :

We are seeking a highly experienced Senior Data Engineer with 7-10 years of deep expertise in the Big Data ecosystem. The ideal candidate will be a hands-on technical leader, proficient in designing, building, and tuning large-scale distributed data systems. This role involves leading a team of data engineers while owning key deliverables and ensuring the integrity and performance of our data platforms.

Key Responsibilities :

  • Technical Leadership & Hands-On Development :
  • Lead and mentor a backend / distributed data systems team while remaining hands-on with code and architecture.
  • Own end-to-end deliverables for the Big Data team from a delivery perspective.
  • Write, tune, and optimize complex Java, MapReduce, Hive, and Spark jobs.
  • Design and build scalable dimensional data models to improve data accessibility, efficiency, and quality.
  • Big Data Engineering :
  • Utilize the Hadoop stack extensively (HDFS, MapReduce, Pig, Hive, HBase, Flume, Sqoop).
  • Develop and maintain solutions using NoSQL stores like Cassandra and HBase.
  • Build and manage data pipelines using stream-processing frameworks such as Spark-Streaming.
  • Data Management & BI :
  • Manage the business intelligence team and vendor partners, prioritizing projects based on business needs.
  • Develop high-quality dashboards using industry best practices.
  • Analyze and confirm the integrity of source data.
  • Lead the deployment and auditing of data models for accuracy.
  • Team & Vendor Management :
  • Manage a team of data engineers, including full-time associates and third-party resources.
  • Foster a cooperative work environment and assist team members in achieving group goals.

Required Skills & Qualifications :

  • Experience : 6-10+ years of proven experience as a Big Data Engineer.
  • Big Data Frameworks : In-depth knowledge of Hadoop (Cloudera distribution preferred), Spark, and similar frameworks.
  • Big Data Tools : Proficiency with Big Data querying tools like Pig, Hive, and Impala.
  • Programming Languages : Strong skills in Java and object-oriented / function scripting languages like Python or Scala.
  • Database Expertise :
  • Advanced SQL skills with expertise in performance tuning.
  • Experience with relational SQL (e.g., Postgres) and NoSQL databases (e.g., Cassandra).
  • Data Modeling : Experience in designing and building dimensional data models.
  • Stream Processing : Hands-on experience with stream-processing systems (e.g., Spark-Streaming, Storm).

    • Desired Skills (Pluses) :

  • Experience with Azure cloud services .
  • Knowledge of the Banking / Financial Services industry.
  • Familiarity with data science and machine learning tools and technologies.
  • Knowledge of other scripting languages like C++, Linux, Ruby, PHP, or R.

    • Personal Attributes :

  • Excellent problem-solving skills for complex networking, data, and software issues.
  • Ability to effectively plan, organize, and prioritize work.
  • Strong interpersonal and communication skills.
  • A collaborative team player who builds and maintains positive work relationships.
    • Create a job alert for this search

    Big Data Engineer • Malappuram, IN