Sr Analyst, Data Integration & Workflows
About the Role:
Grade Level (for internal use):
11 The Team:
The Senior Analyst, Data Integration & Workflows plays a critical role within the Data AI & Enablement organization, serving as a senior technical leader responsible for designing, implementing, and operationalizing production-grade data pipelines and workflow automation that power SPDJI's index and analytical solutions. This role combines hands-on technical expertise with leadership capabilities to drive delivery excellence, mentor technical talent, and ensure all data integration solutions meet enterprise standards for quality, reliability, and maintainability.
Responsibilities and Impact:
Technical Leadership & Solution Delivery
- Lead complex data integration initiatives from design through production deployment, ensuring solutions are scalable, observable, and aligned with enterprise architecture standards
- Design and implement production-grade data pipelines (batch and streaming) that transform raw inputs into trusted curated outputs, incorporating robust error handling, validation, and reconciliation controls
- Establish and evangelize engineering best practices for ETL/ELT patterns, workflow orchestration, data quality controls, and operational observability across the team and value streams
- Drive technical decision-making for pipeline architecture, technology selection, and design patterns, balancing business requirements with technical feasibility and long-term maintainability
- Partner with PPD on technical planning and feasibility, providing realistic estimates, identifying technical dependencies, and shaping scope to ensure achievable delivery commitments
Enablement & Co-Development
- Lead hands-on enablement with value stream SMEs through pair programming, structured guidance, and co-development sessions—adapting approach based on SME technical capability
- Assess SME technical readiness and recommend appropriate engagement models (SME-led with review, co-development, or led build with validation)
- Build reusable automation components and templates (frameworks for ingestion, validation, transformation, publishing, backfills) that accelerate consistent delivery across domains
- Develop SME technical capabilities through targeted coaching, code reviews, and knowledge transfer, fostering a culture of engineering excellence and continuous learning
- Create and maintain technical documentation, including reference architectures, design patterns, coding standards, and implementation guides
Quality Assurance & Production Readiness
- Conduct comprehensive code reviews for SME-built and team-developed pipelines, ensuring adherence to standards for maintainability, testing, logging, data validation, and documentation
- Implement data reliability controls including validation rules, reconciliation checks, anomaly detection, and completeness/timeliness monitoring that protect downstream index processes
- Engineer observability and monitoring solutions by implementing logging standards, metrics, alerts, and runbooks that enable effective production support
- Prepare IT-ready handover artifacts including technical documentation, test evidence, operational procedures, and clear support boundaries
- Partner with IT during QA and deployment, resolving issues quickly and ensuring solutions meet enterprise standards for security, supportability, and operational excellence
Operational Excellence & Continuous Improvement
- Provide L3 support for production business-logic issues, collaborating with value stream SMEs to drive root-cause analysis and implement permanent fixes for recurring failures
- Optimize pipeline performance and cost through appropriate partitioning strategies, caching, incremental processing patterns, and compute resource tuning
- Implement workflow orchestration patterns (scheduling, dependency management, retries, idempotency, parameterization) ensuring pipelines are resilient to upstream variability
- Capture and share