Lead Databricks Engineer

At NCS Australia, we believe in doing technology services better. Our commitment to quality, focus on people, and willingness to challenge traditional thinking set us apart. Our team brings this belief to life by partnering with our clients and communities to make tomorrow together. We are committed to creating an environment that prioritises innovation, collaboration, and purposeful work. Our diverse team is empowered to make a meaningful impact with curiosity, creativity and resilience to shape better outcomes. Join us and accept the challenge of creating a better tomorrow. As a Lead Databricks Engineer, you will play a critical role in delivering large-scale data transformation programs. You will lead the design and implementation of modern Lakehouse platforms on Databricks, working closely with client stakeholders, architects, and cross-functional teams. This is a client-facing leadership role requiring strong consulting capabilities, technical depth, and delivery ownership. The Lead Databricks Engineer is responsible for designing and implementing scalable data platforms using Databricks, Apache Spark, and Delta Lake, including building robust batch and real-time data pipelines and developing ingestion frameworks for both structured and unstructured data while optimising performance, cost, and reliability. The role involves leading and mentoring a team of data engineers, driving best practices across coding, CI/CD, and DevOps, and collaborating with cross-functional stakeholders to translate business needs into effective data solutions, while owning end-to-end project delivery. Additionally, the engineer will implement data solutions across cloud platforms—primarily Azure, but also AWS or GCP—integrating with services such as Azure Data Factory, AWS Glue, and modern data warehouses like Snowflake or Synapse, while managing infrastructure through Infrastructure-as-Code tools. A strong focus is placed on data governance, including implementing quality checks, mon