Databricks Engineer

Posted 28 October 2025
LocationNew York
Job type Permanent
Discipline Software DevelopmentData & Analytics and Data Science
Reference34571

Job description

Job Title: Databricks Data Engineer 
Location: New York, NY – Hybrid 

Role Description: As part of our Data Engineering team, you will be joining a tight knit team of motivated technology professionals where ongoing learning and development are central to our ethos and internal culture. We are seeking a Databricks Data Engineer to design, build, and optimize data pipelines and models on the Databricks Lakehouse Platform. This is a hands-on engineering role where you will develop secure, scalable solutions that deliver high-quality, analytics-ready data to power business insights and AI initiatives.

Key Responsibilities: 
• Build and maintain ETL/ELT pipelines to ingest, transform, and deliver data using Databricks, Delta Lake, and PySpark.
• Design and implement efficient, scalable data models within the Lakehouse architecture.
• Write performant Spark SQL and Python code optimized for distributed compute environments.
• Develop and manage Databricks workflows, jobs, and notebooks for end-to-end data processing.
• Implement robust data quality, governance, and monitoring frameworks leveraging tools like Unity Catalog and Delta Live Tables.
• Collaborate with data engineers, analysts, and business stakeholders to deliver clean, curated datasets and analytical models.
• Follow engineering best practices including version control (Git), code reviews, and CI/CD integration.
• Troubleshoot data pipeline issues, performance bottlenecks, and optimize cluster configurations.
• Stay current with Databricks platform enhancements, applying new features to improve data delivery, performance, and reliability.

Key Requirements & Technical Experience:
• 2+ years of experience as a Data Engineer with hands-on expertise in Databricks.
• Strong knowledge of Spark, PySpark, SQL, and Delta Lake.
• Experience designing and orchestrating data pipelines using tools such as Airflow, Azure 
Data Factory, dbt, or Databricks Workflows.
• Familiarity with at least one cloud platform (AWS, Azure, or GCP) and associated services 
• Proficiency in Python for data engineering tasks and automation.
• Understanding of data governance, security, and access control best practices (Unity 
Catalog experience a plus).
• Strong collaboration skills and ability to operate within agile, cross-functional teams