Freelance Apache NiFi + GCP Data Engineer

Company Description

ThreatXIntel is a startup cybersecurity company specializing in protecting businesses and organizations from cyber threats. We offer services including cloud security, web and mobile security testing, DevSecOps, and cloud security assessments to deliver customized, affordable solutions tailored to client needs. Committed to proactive security, we continuously monitor and test digital environments to mitigate vulnerabilities before they can be exploited. Our mission is to empower businesses of all sizes with high-quality cybersecurity services to protect their digital assets and enable business growth.

Role Description

We are seeking an experienced Freelance Data Engineer with strong expertise in Apache NiFi and the Google Cloud Platform (GCP) ecosystem. The consultant will design and implement real-time data ingestion pipelines, integrate with GCP services, and ensure secure, scalable, and high-performance data delivery for analytics and downstream platforms.

Key Responsibilities

• Design and build real-time ingestion pipelines using Apache NiFi for structured and unstructured datasets.
• Integrate NiFi pipelines with the GCP ecosystem , including BigQuery, Pub / Sub, Dataflow, Dataproc, Cloud Storage, and Composer (Airflow) .
• Implement low-latency, high-throughput data movement across ingestion, storage, and processing layers.
• Develop and optimize SQL queries for BigQuery and relational systems.
• Build Python scripts for data transformations, automation tasks, and custom NiFi processors.
• Apply data quality , validation, deduplication, and error-handling frameworks within ingestion flows.
• Work with analysts, data scientists, and platform teams to deliver clean, reliable, analytics-ready datasets.
• Enforce GCP IAM , VPC Service Controls , encryption policies, and secure-by-design practices.
• Monitor, troubleshoot, and tune NiFi processor performance and GCP data pipelines.
• Document data flows, architecture standards, and operational best practices.

Required Skills & Qualifications