Databricks Practice Lead
We are seeking an experienced Databricks Practice Lead to drive the technical vision, growth, and delivery excellence of our Databricks practice. This role combines deep hands-on expertise in modern data engineering with leadership responsibilities, including defining technical standards, mentoring engineering teams, supporting business development, and delivering scalable cloud-based data solutions for enterprise clients.
You will work closely with clients, engineering teams, solution architects, delivery leaders, and business stakeholders to design innovative data platforms, guide technical decision-making, and expand the organization's Databricks capabilities. The ideal candidate is both a strong technical leader and a trusted advisor who can bridge business objectives with modern data engineering solutions.
Responsibilities
-
Lead the technical direction and growth of the Databricks Practice, establishing standards, best practices, and reusable solution accelerators.
-
Design and review enterprise-scale data platform architectures using Databricks, Apache Spark, and Microsoft Azure.
-
Provide technical leadership and architectural guidance across multiple client engagements.
-
Partner with clients to understand business challenges and recommend scalable data and analytics solutions.
-
Lead technical discovery sessions, solution workshops, architecture reviews, and design discussions with clients.
-
Mentor and coach Data Engineers, Technical Leads, and Solution Architects, fostering technical excellence and career development.
-
Define best practices for data engineering, software engineering, DevOps, security, governance, and operational excellence.
-
Establish standards for ETL/ELT development, data quality, monitoring, performance optimization, and production support.
-
Drive adoption of modern data architectures, including Lakehouse, Delta Lake, and event-driven data processing.
-
Collaborate with Delivery Managers and Engineering Managers to support project planning, staffing, technical risk management, and solution quality.
-
Support pre-sales activities, including solution design, technical proposals, effort estimation, client presentations, and proof-of-concept initiatives.
-
Stay current with Databricks platform capabilities and emerging technologies, identifying opportunities to improve services and expand practice offerings.
-
Promote the adoption of AI-assisted software development and engineering best practices across the practice.
-
8+ years of professional experience in Data Engineering, Data Platform Engineering, or related disciplines.
-
3+ years of experience leading technical teams, architecture initiatives, or data engineering practices.
-
Deep expertise with Databricks, including workspace administration, notebooks, jobs, workflows, Unity Catalog, Delta Lake, and Lakehouse architecture.
-
Strong experience developing scalable data pipelines using Apache Spark (PySpark).
-
Extensive experience with Microsoft Azure, including services such as Azure Data Lake Storage (ADLS), Azure Data Factory, Azure Event Hubs, Azure Key Vault, and Azure DevOps.
-
Strong programming skills in Python, including clean coding practices and object-oriented programming principles.
-
Experience with orchestration tools such as Apache Airflow or Dagster.
-
Experience working with streaming technologies such as Apache Kafka or Azure Event Hubs.
-
Strong understanding of data architecture, dimensional data modeling, metadata management, and enterprise data platform design.
-
Experience implementing data governance, data quality, security, and compliance practices.
-
Experience building and optimizing enterprise ETL/ELT solutions.
-
Strong knowledge of SQL and relational database concepts.
-
Experience deploying cloud-native data solutions, preferably on Microsoft Azure.
-
Excellent consulting, presentation, and stakeholder management skills with experience interacting directl