Spark Developer

About the Role

We are looking for a Senior Software Engineer with strong expertise in Apache Spark and distributed systems. The role involves working on Spark internals, contributing to the Apache Spark open-source community, and building scalable data processing solutions aligned with the platform requirements.
Key Responsibilities

- Contribute fixes and enhancements to the Apache Spark open-source project.

- Upgrade and maintain compatibility with newer Spark releases.

- Optimize Spark internals for performance, scalability, and reliability.

- Design and develop distributed data processing solutions.

- Debug complex Spark-related performance and stability issues.

- Collaborate with product and platform teams on architecture and technical initiatives.

Required Skills

- Strong experience with Apache Spark internals and distributed systems.

- Hands-on experience contributing to open-source projects.

- Strong Java and/or Scala programming skills.

- Expertise in query execution, distributed processing, and performance optimization.

- Strong debugging and problem-solving abilities.

- Ability to work independently and drive technical initiatives.

Good to Have

- Experience with Kubernetes and cloud-native environments.

- Knowledge of JVM tuning and system optimization.

- Contributions to Apache or other open-source communities.

- Experience with large-scale analytics or data platforms.

Originally posted on Himalayas