AWS Data Lake Administrator

AWS Data Lake Administrator

LOCATION – BLR / HYD

Job Description :

We are seeking a highly skilled AWS Data Lake Administrator to manage and optimize our data lake architecture on AWS. The ideal candidate will have strong experience with AWS data services such as S3, Lake Formation, Glue Catalog, Redshift Spectrum, and extensive SQL knowledge. The role involves managing data ingestion, security, and performance of the data lake, ensuring efficient and secure access to data for analytics and reporting.

Key Responsibilities :

• Administer and manage the AWS Data Lake infrastructure, ensuring high availability, security, and performance.
• Configure and manage AWS S3, Lake Formation, and Glue Data Catalog to organize, secure, and catalog data within the data lake.
• Set up and manage Redshift Spectrum for querying and analyzing data stored in S3 using SQL and Redshift.
• Implement and manage data ingestion pipelines to ingest structured and unstructured data from various sources into the data lake using AWS services like Glue ETL, Lambda, and other orchestration tools.
• Define and enforce data governance policies, access control, and security measures using AWS Lake Formation, ensuring compliance with organizational and regulatory requirements.
• Optimize data storage in S3 through partitioning, compression, and appropriate data formats like Parquet, Avro, or ORC to improve query performance.
• Monitor and manage the Glue Catalog for maintaining metadata, table definitions, and data lineage within the data lake.
• Collaborate with data engineers, architects, and analysts to ensure seamless access to data for analytics and machine learning.
• Troubleshoot and optimize query performance in Redshift Spectrum, and manage workload performance in the data lake environment.
• Implement backup, recovery, and disaster recovery strategies for data lake assets, ensuring data integrity and availability.
• Maintain detailed documentation of data lake architecture, security policies, and data governance procedures.