Senior Data Engineer

Senior Data Engineer

Experience: 6+ Years

Location: WFH / Remote (India)

Open Positions: Multiple

Qualification: B.Tech / M.Tech / MCA orequivalent

Work Timings: 1:30 PM IST – 10:30 PM IST

Functional Area: Data Engineering
About the Role

Weare seeking an experienced Senior Data Engineer to design, develop, and maintain scalable, reliable, andhigh-performance data platforms and pipelines that support businessintelligence, analytics, and data-driven decision-making.

Theideal candidate will have strong hands-on expertise in Python,Apache Spark, Apache Airflow, Amazon Redshift, SQL, and modern data engineeringpractices, along with mandatory experience in the Healthcare domain. The candidate should haveexperience building end-to-end ETL/ELT pipelines, processing large-scaledatasets, implementing data quality frameworks, and supporting enterpriseanalytics and reporting needs.

This rolerequires a technically strong professional who can collaborate with businessstakeholders, data analysts, architects, and application teams to deliverrobust and scalable data solutions.
Key Responsibilities

- Design, develop, and maintain scalable data pipelines and ETL/ELT workflows for structured and unstructured data.

- Build and optimize data ingestion, transformation, and data integration processes using Python and Apache Spark.

- Develop, schedule, and monitor workflows using Apache Airflow.

- Design and manage enterprise data warehouse solutions using Amazon Redshift.

- Implement data models, data marts, and reporting datasets to support analytics and business intelligence requirements.

- Optimize data processing jobs and database performance for scalability and efficiency.

- Work with large healthcare datasets, ensuring data accuracy, consistency, and compliance with industry standards.

- Collaborate with business stakeholders to understand reporting and data requirements.

- Implement data quality checks, validation frameworks, and monitoring processes.

- Troubleshoot data pipeline issues and perform root cause analysis.

- Ensure data security, governance, and compliance requirements are adhered to.

- Participate in code reviews, development best practices, and continuous improvement initiatives.

- Create and maintain technical documentation, data flow diagrams, and data mapping documents.

- Work closely with cross-functional teams including Data Science, Analytics, Product, and Engineering teams.

Skills, Knowledge & Experience
Mandatory Skills

- 6+ years of experience in Data Engineering, Big Data Engineering, or related roles.

- Mandatory experience working in the Healthcare domain.

- Strong hands-on experience in Python development for data engineering solutions.

- Strong hands-on experience with Apache Spark (PySpark) for large-scale data processing.

- Hands-on experience with Apache Airflow for workflow orchestration and scheduling.

- Strong experience designing, developing, and optimizing solutions on Amazon Redshift.

- Advanced SQL skills with experience in query tuning and performance optimization.

- Experience building ETL/ELT pipelines and modern data integration frameworks.

- Strong understanding of data warehousing concepts, dimensional modelling, and data architecture.

- Experience handling large-scale datasets and complex data transformation requirements.

- Strong debugging, troubleshooting, and performance tuning skills.

- Experience working with cloud-based data platforms and services (AWS or Azure) is preferred.

- Excellent analytical and problem-solving skills.

- Strong communication and stakeholder management capabilities.

Preferred Skills

- Experience with AWS data services such as S3, Glue, EMR, Athena, Lambda, or Kinesis

- Experience with Azure Data Factory, Azure Databricks, ADLS, Synapse Analytics, or other Azure data services

- Exposure to Databricks, Snowflake, or modern Lakehouse architectures

- Experience with CI/CD for Data Engineering pipelines

- Knowledge of He