Lead Databricks Engineer

Job Description

We are looking for a Lead Databricks Engineer to design, build, and scale our cloud-based lakehouse platform. You will own the end-to-end architecture of our data ecosystem on Databricks. Collaborate with data science and analytics teams to productionize ML and analytical workloads. Set the technical direction for ingestion, transformation, governance, and performance optimization across petabyte-scale datasets. You will also be responsible for writing production code and mentoring engineers, shaping standards that the broader data organization will adopt.

Qualifications

1. 8+ years of data engineering experience, with 4+ years building production workloads on Databricks. 2. Deep expertise in Apache Spark (PySpark and Spark SQL) — including performance tuning, partitioning strategy, and the Catalyst/Photon execution model. 3. Strong hands-on experience with Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables. 4. Production experience on at least one major cloud (AWS, Azure, or GCP), including networking, IAM, storage (S3/ADLS/GCS), and compute primitives. 5. Proficiency in Python and SQL; comfort with Scala is a plus. 6. Experience designing medallion (bronze/silver/gold) architectures and dimensional models for analytics. 7. Strong CI/CD and DevOps practice: Git, Terraform, Databricks Asset Bundles or dbx, automated testing of data pipelines. 8. Track record of leading technical projects end-to-end and mentoring engineers. 9. Excellent written and verbal communication; able to drive alignment with both engineering and business stakeholders.

Benefits

- Bounteous is proud to be an equal opportunity employer. - Bounteous does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, physical or mental disability, national origin, veteran status, or any other status protected under federal, state, or local law. - Bounteous is willing to sponsor eligible candidates for employment visas.