Job Description : Data Engineer (Big Data / Kafka)
We are seeking a highly experienced Senior Data Engineer with a deep background in Big Data technologies to join our team. This is a contract role for a major project in the Banking sector.
Key Details :
- Role : Data Engineer
- Industry : Banking (Financial Services)
- Work Location : Remote (India)
- Contract Duration : 6 months initial contract (with a strong likelihood of extension to long-term)
- Notice Period : 30 days or less
- Note : This role is for professionals seeking a contract engagement, not for freelancers or independent contractors.
Key Responsibilities :
Technical Execution : Design, write, and tune complex data processing jobs using Java, MapReduce, Hive, and Spark.Team Leadership & Management : Lead and mentor a team of data engineers (including full-time associates and vendor partners), ensuring project prioritization aligns with business needs. Remain hands-on with code and architecture.Big Data Ecosystem Management : Work extensively across the Hadoop stack (HDFS, Pig, Hive, HBase, Flume, Sqoop) and NoSQL stores (e.g., Cassandra).Data Modeling & BI : Design and build dimensional data models. Manage the business intelligence function, developing high-quality dashboards using industry best practices.Data Integrity & Deployment : Analyze source data integrity and lead the deployment and auditing of data models and attributes for accuracy.Stream Processing : Develop and maintain solutions using stream-processing systems like Spark-Streaming.SQL & Performance Tuning : Demonstrate proficiency in writing and performance-tuning advanced SQL queries.Required Skills & Qualifications :
Experience : 6-10+ years of proven experience as a Big Data Engineer.Big Data Frameworks : In-depth knowledge of Hadoop (Cloudera distribution preferred), Spark, and similar frameworks.Programming & Scripting : Strong expertise in Java. Experience with object-oriented / object function scripting languages like Python and Scala.Querying Tools : Good knowledge of Big Data querying tools such as Pig, Hive, and Impala.Databases : Experience with both relational SQL (e.g., Postgres) and NoSQL databases (e.g., Cassandra).Streaming Tech : Hands-on experience with stream-processing systems like Spark-Streaming or Storm.Data Modeling : Proven experience in designing and building dimensional data models.Leadership : Demonstrated experience leading a Backend / Distributed Data Systems team while remaining technically hands-on.Preferred Skills (Pluses) :
Experience with Azure cloud services.Knowledge of data science and machine learning tools and technologies.Experience in the Financial Services / Banking industry.Personal Attributes :
Able to effectively plan, organize, and own deliverables.Strong interpersonal and communication skills.Ability to solve complex networking, data, and software issues.A collaborative team player who builds and maintains cooperative work relationships.