Senior Data Platform Architect

Job Summary
We are looking for a Senior Data Platform Architect with strong end-to-end data and platform skills to design, operate, and secure the Azure Databricks that powers finance, revenue management, marketing, and booking and reservation across the business. This architect will report to Director, Customer Data and Identity Engineering.
Beyond design, this role will cover the platform administration in North America including Unity Catalog governance, access and credentials, pipeline reliability, and cost — and turns it into something documented, automated. You will also set the standard for how AI-assisted coding development is used across the data team, and define the tooling that makes it safe.

Job Responsibilities:

End-to-end platform architecture and implementation

- Design the medallion architecture: catalog and schema design, layer contracts, and the promotion path from development through QA to production.

- Design robust, scalable data platform architectures aligned with enterprise security and networking standards.

- Act as a bridge between Data Solutions and Infrastructure, ensuring the platform decisions you make are the ones BI, AI, data science, and activation teams can actually build on.

Databricks & Unity Catalog administration

- Administer Unity Catalog: catalog bindings, external locations, storage credentials, grants, and the service-principal model behind them.

- Define workspace and compute standards — cluster policies, runtime versions, pool lifecycle, job compute versus all-purpose, and serverless where it pays for itself.

- Manage platform identity end to end: users, groups, SCIM provisioning from Entra ID, secret scopes, and token governance.

- Keep the estate current: runtime upgrades, deprecation tracking, and retiring the pools and jobs that quietly stop working.

Ingestion, orchestration & integration

- Design pipeline architecture across Azure Data Factory and Databricks Workflows: dependency design, idempotency, watermarking, and retry and backfill semantics that survive a bad source day.

- Keep source integrations healthy — Dataverse and Synapse Link exports, finance and booking systems, third-party feeds — including the runtime, connector, and credential lifecycle they silently depend on.

- Establish data contracts with upstream owners so schema changes reach the platform before they reach production.

- Land a modernization decision for SQL Server to Databricks without breaking the reporting built on it.

AI-assisted engineering standards & harness enablement

- Establish and enforce the AI coding standard for the data team: what assistants may generate, what must be human-reviewed before merge, and what may never reach production without a test behind it.

- Build and maintain the AI coding harness — repository instruction files, project context, MCP servers, tool permissions, and sandboxed environments — so generated code arrives already matching our conventions.

- Establish review practice for generated code and SQL: correctness against the data model, cost and performance impact, and provenance recorded in the pull request.

- Continuously evaluate and improve output quality, refining instructions and context based on what the team actually gets wrong — treating the harness as a product with users, not a config file.

AI/ML platform administration

- Administer ML assets in Unity Catalog — registered models, aliases, feature tables, and MLflow experiments — including the grants and promotion path from development through QA to production.

- Define GPU and ML compute standards: node-type allowlists, Azure quota management, idle termination, spot strategy, and ML Runtime version lifecycle, so expensive compute cannot be left running unnoticed.

- Govern foundation model and external LLM access through the AI Gateway — rate limits, usage tracking, payload logging, guardrails, and provider credentials held in secret scopes rather than notebooks.

- Extend network