Senior databricks administrator -awsSonatype • Hyderabad, Andhra Pradesh, India
Senior databricks administrator -aws
Sonatype • Hyderabad, Andhra Pradesh, India
3 days ago
Job descriptionUser Provisioning and Management : Onboard and offboard users, groups, and service principals within Databricks, including integration with identity providers (Id Ps) like Azure Active Directory or Okta via SCIM. Manage user roles and entitlements at both the account and workspace levels (Account Admins, Workspace Admins, Metastore Admins, etc.). Implement and maintain role-based access control (RBAC) and attribute-based access control (ABAC) to ensure appropriate data and resource access. Data Lake Governance (Unity Catalog focus) : Configure and manage Unity Catalog metastores, catalogs, schemas, and tables. Define and enforce data access policies (e.g., table-level, column-level, row-level security) using Unity Catalog. Manage data lineage and auditing capabilities to track data flow and usage. Collaborate with data owners and stakeholders to define data quality standards and ensure data integrity. Implement data retention and lifecycle management policies. Aligning Data Sensitivity Policy to Enforceable Data Governance : Translate organizational data classification and sensitivity policies into technical controls within Databricks. Utilize features like data masking and encryption to protect sensitive information. Ensure compliance with regulatory requirements (e.g., GDPR, HIPAA, CCPA) by implementing appropriate security measures. Conduct regular security audits and vulnerability assessments. Managing Cluster and Budget Policies : Define and implement compute policies to control cluster creation, configuration, and resource usage, ensuring cost optimization. Monitor and manage serverless budget policies to attribute usage to specific teams or projects. Optimize cluster configurations for performance and cost-effectiveness, leveraging features like auto-scaling and auto-termination. Manage cluster pools to reduce startup times and improve resource allocation. Reporting and Addressing Discrepancies : Monitor Databricks platform health, performance, and resource utilization. Identify and troubleshoot issues related to user access, data availability, cluster performance, and policy violations. Generate reports on platform usage, costs, security incidents, and compliance. Investigate and resolve discrepancies in data, reports, or system behavior in collaboration with data engineers, data scientists, and other teams. Develop and maintain comprehensive documentation of configurations, procedures, and best practices. Collaboration and Support : Provide technical support and guidance to Databricks users, data engineers, and data scientists. Collaborate with cloud infrastructure teams (AWS, Azure, GCP) to manage underlying cloud resources. Stay up-to-date with the latest Databricks features, best practices, and industry trends. Databricks Platform Expertise : Deep understanding of Databricks architecture, workspaces, and key components (Unity Catalog, Delta Lake, Spark, SQL Analytics). Proficiency in Databricks administration console and APIs. Experience with Databricks Workflows, Jobs, and Delta Live Tables (DLT) for orchestration and pipeline management. Cloud Platform Knowledge : Strong experience with AWS and its relevant services. Data Governance & Security : Solid understanding of data governance principles, data classification, and data lifecycle management. Experience implementing security controls, access policies (RBAC), and encryption. Familiarity with compliance standards (GDPR, HIPAA, CCPA) and auditing practices. Programming & Scripting : Proficiency in SQL for data querying and access control. Deep expertise in Terraform is essential, extending beyond basic knowledge to managing complex, multi-project infrastructure. This includes hands-on experience with custom Terraform modules crucial for Data Mesh orchestration. Scripting skills (e.g., Python, Terraform) for automation and administrative tasks. Familiarity with Spark and Py Spark concepts for troubleshooting and optimization. Identity and Access Management (IAM) : Experience with enterprise identity providers (e.g., Azure AD, Okta, Active Directory) and SCIM provisioning. Networking Concepts : Understanding of network security, VPNs, VPCS, private links, VPC peering, and connectivity within cloud environments. Monitoring & Logging Tools : Experience with monitoring tools (e.g., Datadog, Observe, cloud-native monitoring) for platform health and performance. Problem-Solving and Troubleshooting : Ability to diagnose and resolve complex technical issues efficiently. Communication : Excellent verbal and written communication skills to interact with technical and non-technical stakeholders. Attention to Detail : Meticulous in configuring policies, managing access, and ensuring data integrity. Proactive and Self-Driven : Ability to anticipate issues, recommend solutions, and continuously improve the platform. Collaboration : Work effectively with cross-functional teams (data engineers, data scientists, security teams). Analytical Thinking : Ability to analyze data and system logs to identify trends and discrepancies.
Role Summary
- The Databricks Administrator will be responsible for the overall health, security, and performance of the Databricks platform. This includes managing user access, implementing and enforcing data governance policies, optimizing cluster resources, and ensuring data sensitivity policies are effectively applied across the data lakehouse. The administrator will also be crucial in identifying, reporting, and resolving discrepancies within the platform's operation and configuration.
Key Responsibilities
Technical Skills :
Soft Skills
Create a job alert for this search
Aws Administrator • Hyderabad, Andhra Pradesh, India
Related jobs
Senior Databricks Administrator -Aws
Sonatype • Hyderabad, Republic Of India, IN
The Databricks Administrator will be responsible for the overall health, security, and performance of the Databricks platform.
This includes managing user access, implementing and enforcing data gov...Show more
Last updated: 8 days ago • Promoted
Senior data engineer
Talentoj • Hyderabad, Andhra Pradesh, India
Location : (Open Location) | Type : Full-Time.We are looking for a highly experienced Senior Data Engineer with strong expertise in Python , Advanced SQL , Snowflake , Amazon Redshift , Databricks , ...Show more
Last updated: 7 hours ago • Promoted • New!
Senior azure data engineer
Programmers.io • Hyderabad, Andhra Pradesh, India
Job Title : Senior Azure Data Engineer.Skills Required : Azure Synapse, Azure Data Factory, Azure Databricks, and ADLS Gen2, Py Spark, SQL, Microsoft Fabrics.
Experience Range : 8 Years - 12 years.We a...Show more
Last updated: 30+ days ago • Promoted
Data engineer / databricks
ValueMomentum • Hyderabad, Andhra Pradesh, India
Databricks Engineer-Databricks.Primary skills : Databricks, Py Spark, SQL.Secondary skills : Advanced SQL, Azure Data Factory, and Azure Data lake.
Design and develop ETL pipelines using ADF for data ...Show more
Last updated: 7 hours ago • Promoted • New!
Senior Databricks Administrator -AWS
Sonatype • Hyderabad, Telangana, India
The Databricks Administrator will be responsible for the overall health, security, and performance of the Databricks platform.
This includes managing user access, implementing and enforcing data gov...Show more
Last updated: 8 days ago • Promoted
Azure databricks
Tata Consultancy Services • Hyderabad, Andhra Pradesh, India
TCS has been a great pioneer in feeding the fire of young Techies like you.We are a global leader in the technology arena and there’s nothing that can stop us from growing together.Your role is of ...Show more
Last updated: 3 days ago • Promoted
Aws-redshift database administrator
Tata Consultancy Services • Hyderabad, Andhra Pradesh, India
Role : AWS-Redshift Database Administrator.LOCATION : Bengaluru / Hyderabad / Pune / Bhubaneshwar / Chennai / Kochi.Good in designing, implementing, performance tuning and maintaining database systems by ...Show more
Last updated: 7 hours ago • Promoted • New!
Senior data engineer
Creditsafe Technology • Hyderabad, Andhra Pradesh, India
About Creditsafe : With the largest owned database supported by a partner network, Creditsafe provides data on 320 million companies in over 100 countries to support its customers in this regard.Cre...Show more
Last updated: 7 hours ago • Promoted • New!
Senior Data Engineer
VRIZE • Hyderabad, IN
Design, architect, administer, and manage large database systems, particularly on Microsoft platforms such as SQL Server and Azure Database.
Develop and optimize complex SQL queries for data extract...Show more
Last updated: 4 days ago • Promoted
Senior SQL Server Database Administrator – Azure
Maitsys • Hyderabad, IN
Senior (Minimum 10 years of professional experience).We are looking for an experienced.Senior SQL Server Database Administrator (DBA).
The ideal candidate will have a proven track record in managing...Show more
Last updated: 4 days ago • Promoted
Cloud database administrator
ValueMomentum • Hyderabad, Andhra Pradesh, India
Job Title : Cloud Database Administrator.We are looking for a skilled Cloud Database Administrator (DBA) with 5+years strong experience in SQL Server on AWS platform.
The role includes database devel...Show more
Last updated: 3 days ago • Promoted
Oracle exadata administrator
Oracle • Hyderabad, Andhra Pradesh, India
Job title : Principal / Senior Consultant (Oracle Exadata).Location : Bengaluru / Chennai / Hyderabad / Mumbai / Pune / Noida / Gurugram.
This position is for a Principal / Senior Consultant (Oracle Database) for t...Show more
Last updated: 7 hours ago • Promoted • New!
Databricks SME
DataE2E Technologies • Hyderabad, IN
Lead the architecture and implementation of.Collaborate with business stakeholders, data scientists, and analysts to gather requirements and deliver actionable data solutions.Integrate data from mu...Show more
Last updated: 12 days ago • Promoted
Senior sql server database administrator – azure
Maitsys • Hyderabad, Andhra Pradesh, India
Experience Level : Senior (Minimum 10 years of professional experience) Job Summary : We are looking for an experienced Senior SQL Server Database Administrator (DBA) with strong expertise in Azure...Show more
Last updated: 7 hours ago • Promoted • New!
Database administrator
LTIMindtree • Hyderabad, Andhra Pradesh, India
We are seeking an experienced Snowflake Database Administrator DBA to join our team The ideal candidate will have a deep understanding of Data Warehousing fundamentals and concepts with a proven tr...Show more
Last updated: 7 hours ago • Promoted • New!
Database administrator -mssql
Cloud4C Services • Hyderabad, Andhra Pradesh, India
Cloud4 C is a global leader in AI-powered, automation-driven, application-centric managed services and cloud infrastructure, delivering Sovereign and Secure Industry Hybrid Cloud solutions across 2...Show more
Last updated: 30+ days ago • Promoted
Senior Database Administrator
MasterControl • Hyderabad, IN
Our mission is the same as that of our customers to bring life-changing products to more people sooner.The MasterControl Platform helps organizations digitize, automate and connect quality and comp...Show more
Last updated: 10 days ago • Promoted
AWS Data Architect
ACL Digital • Hyderabad, IN
AWS (S3, Redshift, Glue, Lake Formation, IAM).Proficient in data modeling, performance tuning, and security best practices.
.AWS Certified Solutions Architect preferred.Show more
Last updated: 16 days ago • Promoted