Develop data pipelines supporting Cigna digital experiences and analytics systems.Collaborate on architecture decisions and develop solutions using tools in AWS and Databricks.Production data pipeline development in Spark using Python and SQL on DatabricksAssemble large, complex data sets that meet functional and analytics business requirements.Develop both batch data processing and real time streaming technologies.Identify and address bottlenecks in data pipelines to improve performance.Improve data reliability, efficiency, and quality of data.Work with stakeholders including Analysts, Product, and Engineering teams to assist with data-related technical issues and support their data infrastructure needs.Required Skills & Experience :
- 5 - 8 years of experience in data systems and analytics development.
- Expert in advanced SQL, Spark and Python.
- 3+ years of experience with Databricks - Unity Catalog, Workflows and Autoloader.
- Experience developing Big Data pipelines in AWS cloud.
- bachelors Degree in Computer Science , Information Systems, Data Analytics or related.
- Experience with Git repository and CI / CD pipeline.
Desired Experience :
- Understanding of web and mobile analytics.
- Experience with Data Security and managing PII / PHI in production environments.
- Understanding of common Big Data file formats : Databricks Delta, CSV, JSON, Parquet, etc
- Understanding of Adobe Analytics or Customer Journey Analytics (CJA).
- Understanding of the medical insurance industry.
- Experience with Terraform code for Databricks job deployment.
Skills Required
Spark, Databricks, Sql, Python, Aws