Databricks Solution Architect - Insurance Domain

Job Overview:We are looking for an accomplished Databricks Solution Architect / Data Architect to lead the design, modernization, and implementation of enterprise-scale data platforms on Azure. In this senior role, you will define target-state data architecture, establish scalable Lakehouse design patterns, and guide the development of high-performance data pipelines using Databricks, Delta Lake, and related cloud-native services.Key Responsibilities:- Lead the end-to-end architecture and solution design for enterprise data platforms on Azure, with strong focus on Databricks Lakehouse, Delta Lake, and scalable cloud-native data ecosystems.- Define target-state data architecture, ingestion patterns, transformation frameworks, and serving layers to support reporting, advanced analytics, ML, and business-critical decisioning use cases.- Design and implement robust ETL/ELT pipelines using PySpark, SQL, Databricks Workflows, Auto Loader, and Delta Live Tables for batch and near real-time processing.- Own architecture standards for data modeling, medallion design, reusable engineering patterns, CI/CD, code quality, environment strategy, and release management across Databricks solutions.- Drive platform governance and security using Unity Catalog, RBAC/ABAC controls, lineage, auditability, and integration with enterprise governance services such as Purview.- Optimize solution performance by tuning Spark workloads, cluster policies, partitioning strategy, file sizing, caching, and compute cost management for large-scale data processing.- Collaborate with business stakeholders, product owners, analysts, architects, and downstream consumers to translate functional and non-functional requirements into scalable technical designs.- Provide technical leadership to engineering teams by reviewing designs, guiding implementation, resolving architectural bottlenecks, and establishing best practices for Databricks-based delivery.- Evaluate and recommend Databricks capabilities such as Photon, serverless compute, Lakehouse Federation, and streaming patterns to improve scalability, maintainability, and time to value.- Ensure strong delivery governance through estimation, technical planning, dependency management, risk mitigation, and Agile execution including sprint planning, backlog refinement, and design reviews.Candidate Profile:- 10 - 15 years of experience in data engineering, cloud data platform design, or enterprise data architecture, with at least 5+ years of strong hands-on experience on Databricks and Azure.- Bachelors degree in computer science, Information Technology, Engineering, or a related discipline; masters degree is preferred.- Strong expertise in designing modern data architectures, including Lakehouse, medallion architecture, data modeling, data warehousing, and scalable ingestion and transformation frameworks.- Deep technical proficiency in Databricks, PySpark, Python, SQL, Delta Lake, Databricks Workflows, Auto Loader, and Delta Live Tables.- Strong experience with Azure cloud services such as Azure Data Factory, Azure Data Lake Storage, Azure Key Vault, Azure DevOps, and integration of Databricks with broader enterprise cloud ecosystems.- Hands-on experience in defining architecture standards, reusable design patterns, CI/CD strategy, environment management, and delivery best practices for enterprise-scale data platforms.- Experience with data governance, lineage, and security frameworks including Unity Catalog, role-based access controls, and integration with enterprise governance tools such as Azure Purview.- Proven ability to optimize large-scale Spark and Databricks workloads, including performance tuning, cluster sizing, workload management, and cost optimization.- Experience engaging with business stakeholders, product owners, architects, and engineering teams to convert business requirements into scalable solution designs and implementation roadmaps.- Strong communication, leadership, and problem-solving skills with the ability to mentor teams, review technical designs, and drive architecture decisions across cross-functional programs.- Preferred: Knowledge of insurance domain data models, regulatory considerations, and analytics use cases relevant to underwriting, claims, pricing, or risk functions. (ref:hirist.tech) .