Senior Data Engineer

About Us

At Cast & Crew, we’ve empowered creativity and supported the global entertainment industry for decades. Together with our family of brands - Backstage, CAPS, Checks & Balances, Final Draft, Media Services, Sargent-Disc, and The TEAM Companies – we operate as a combined entertainment technology and services provider offering industry standard screenwriting accounting software, digital payroll products, data & reporting, and a host of creative tools. The industry continues to move faster than ever, and the need for our expertise, our technology, and our people has never been greater. We are a production’s best ally every step of the way. #OneCastOneCrew

At Cast & Crew, engineers own the systems they build and operate. Our platforms support mission-critical workflows across the entertainment industry, where reliability, scalability, and accountability matter. We hire engineers who take ownership, solve problems end-to-end, automate wherever possible, and continuously raise the bar for engineering excellence.
Summary

Cast & Crew is seeking an experienced Senior Data Engineer to design, build, and operate scalable data platforms that power mission-critical applications, analytics, reporting, and AI-driven solutions across the entertainment production ecosystem.

This role focuses on building reliable, cloud-native data pipelines that ingest, transform, secure, and serve operational data for internal engineering teams, business intelligence, and customer-facing applications. You will work across structured and semi-structured data, enabling modern analytics, AI workloads, and real-time event processing.

You will collaborate closely with software engineers, platform engineers, product teams, and data consumers while leveraging AI-assisted development tools to accelerate delivery and improve engineering productivity.
Required Qualifications

-
Bachelor's degree in Computer Science, Software Engineering, Information Systems, or related field

-
5+ years of experience building production data platforms and pipelines

- Strong Python development experience

-
Strong SQL development and query optimization skills

-
Experience designing and maintaining ETL/ELT pipelines

-
Experience working with relational databases (PostgreSQL, MySQL, Aurora)

-
Experience working with NoSQL databases (DynamoDB, MongoDB, document databases)

-
Experience building data lakes and cloud-native data architectures

-
Experience with Snowflake, BigQuery, Redshift, or similar cloud data warehouses

-
Experience with AWS cloud services including S3, Lambda, EventBridge, DynamoDB, ECS/EKS, and IAM

-
Experience building event-driven and streaming data solutions

-
Experience working with Kafka, SNS/SQS, Kinesis, or similar messaging technologies

-
Experience with data orchestration tools such as Airflow, Prefect, or Dagster

-
Experience with Git and modern CI/CD pipelines (Azure DevOps, GitHub Actions, or similar)

-
Experience building highly available, scalable production systems

-
Strong understanding of data modeling, partitioning, indexing, and performance optimization

-
Experience monitoring and operating production data platforms

Nice to Have

-
Experience with Apache Spark or distributed processing frameworks

-
Experience with Debezium, CDC, or change data capture architectures

-
Experience with Iceberg, Delta Lake, or modern lakehouse technologies

-
Experience with OpenSearch or Elasticsearch

-
Experience supporting AI and machine learning data platforms

-
Experience building feature stores or vector databases

-
Experience with document processing (OCR, Textract, PDF extraction)

-
Experience in media, payroll, financial systems, or enterprise SaaS platforms

-
Experience using AI-assisted development tools such as Claude Code or GitHub Copilot

What You Will Build

-
Enterprise-scale data ingestion and transformation pipelines

-
Cloud-native data platforms supporting analytics and operational workl