Databricks Developer (Java & Apache Spark) with IRS MBI Clearance required

This is a remote position.
Job Title: Databricks Developer (Java & Apache Spark) Location: Remote Duration: Full-Time NEED IRS MBI Clearance. We are looking for a Databricks Developer who can design, build, and support large-scale data processing solutions. This role focuses on working with complex, high-volume data in a secure environment. You should be comfortable writing clean Java code, building Spark jobs, and improving performance in distributed systems. You’ll work closely with engineers, analysts, and business teams to deliver reliable data pipelines that scale well and meet security and compliance requirements.

What You’ll Do Build and maintain data pipelines using Apache Spark on Databricks Write clean and efficient Java (Java 8 or higher) following best coding practices Work with large structured and semi-structured data sets Tune Spark jobs to improve performance, stability, and cost Partner with other teams to understand requirements and deliver solutions Make sure data handling follows security and governance standards Investigate and fix issues in production environments What We’re Looking For Active federal background clearance (client-issued laptop is a plus) Bachelor’s degree in Computer Science or a related field 8+ years of hands-on experience in data engineering or big data development Technical Skills Data & Systems Experience working with large enterprise or government data systems Good understanding of data structures, security, and compliance Java Strong experience with Java 8 or newer Comfortable with Streams, Lambdas, and object-oriented design Apache Spark Hands-on experience with Spark Core, Spark SQL, and DataFrames Understanding of RDDs and when to use them Experience with batch processing and streaming Strong skills in Spark performance tuning Able to debug jobs using Spark UI and logs Big Data Tools Experience with tools like HDFS, Hive, or HBase Familiar with Kafka, S3, or cloud-based data lakes Experience with Parquet, Avro, or ORC formats ETL & Data Processing Experience building batch and real-time ETL pipelines Skilled in data cleansing, transformation, and enrichment DevOps & Deployment Experience running Spark on YARN, Kubernetes, or cloud platforms Familiar with CI/CD tools like Jenkins or GitHub Actions Experience with monitoring tools and production logs Testing & Code Quality Unit testing using JUnit or TestNG Experience with Mockito or similar tools Comfortable validating data and testing Spark jobs Ways You’ll Succeed You’re used to working in Agile teams You document your work clearly You stay calm and focused when troubleshooting production issues Nice to Have: Experience with Scala or Python in Spark Hands-on experience with Databricks or Google Dataproc Knowledge of Delta Lake or Apache Iceberg Background in data modeling and performance design

Originally posted on Himalayas