Databricks Platform Engineer (AWS)

Role - Databricks Platform Engineer / Manager (AWS)

Exp - 5 + Yrs

Location - Hybrid - Noida, Bengaluru, Chennai, Pune, Kochi, Mumbai, Jaipur, Gurugram

Note:

1. BGV Mandatory

2.Avoid AP and HYD profiles

Job Description:

Own the design, administration, security, governance, performance, automation, and cost optimization of the enterprise Databricks platform on AWS. Enable data engineering, analytics, and AI teams through scalable, secure, and reliable platform operations.

Key Responsibilities

Manage Databricks workspaces across environments (Dev, Test, UAT, Prod).

Maintain platform availability, upgrades, performance, and reliability.

Define cluster policies, instance pools, and standardized configurations.

Integrate and manage AWS IAM, SSO, authentication, authorization, PATs, service principals, and secret scopes.

Administer Unity Catalog, RBAC, governance, compliance, and audit controls.

Support Git integration, CI/CD pipelines, Terraform-based IaC, and automated deployments.

Monitor platform health, troubleshoot incidents, and support production operations.

Optimize Databricks usage, cluster sizing, auto-scaling, and cloud costs.

Maintain audit logging, security compliance, and operational documentation.

Required Skills

Databricks: Workspace administration, clusters, cluster policies, instance pools, workflows,

Repos, Unity Catalog, DBR management.

AWS: IAM, VPC, EC2, S3, CloudWatch, KMS, networking, security groups.

DevOps: Git, CI/CD, Terraform, IaC, Azure DevOps/GitHub Actions/Jenkins.

Operations: Monitoring, incident management, troubleshooting, performance tuning, cost optimization.

Strong cloud architecture, networking, security, scalability, HA, and production support experience.

Required Experience

Enterprise Databricks administration across multiple workspaces.

Unity Catalog and AWS IAM integration.

Cluster governance, runtime upgrades, CI/CD, Git integration, monitoring, audit logging, and secrets management.

Terraform, DevOps, production support, and cost optimization initiatives.

Preferred Qualifications

Large-scale Databricks environments (100+ users).

Terraform-based provisioning.

Lakehouse architecture, Delta Lake, FinOps, governance, and security frameworks.

Nice to Have

AWS Organizations, Control Tower, PrivateLink, VPC endpoints.

CloudWatch, Datadog, Splunk.

Apache Spark internals and AI/ML workloads on Databricks.

Education & Certifications

Bachelor's degree in a relevant field.

Preferred: Databricks certifications, AWS Solutions Architect, AWS SysOps Administrator, Terraform Associate.

Ideal Candidate

A platform engineering professional with strong AWS and Databricks expertise, capable of owning platform operations, security, governance, automation, and cost optimization while collaborating with engineering, DevOps, and security teams.

Compensation: 70,[hidden],000.00 per month

Experience:

Databricks administration: 5 years (Required)

Work Location: Hybrid remote in Noida, Uttar Pradesh .