Senior Data Engineer (Databricks Migration)

Participate in the migration of a large-scale analytical platform from BigQuery to Databricks Design and implement scalable Lakehouse architectures using Databricks and Delta Lake Analyze existing ETL / ELT workloads and define migration approaches Develop and optimize data pipelines processing large volumes of retail and analytical data Implement incremental processing strategies and scalable transformation frameworks Build and maintain Spark-based data processing solutions using PySpark Design and maintain medallion architecture layers including Bronze, Silver, and Gold Implement data governance and security best practices using Unity Catalog Collaborate with Data Science, Analytics, Product, and Customer Engineering teams Participate in architecture discussions and technical solution design Develop reusable data platform components and engineering standards Conduct code reviews and contribute to platform reliability and maintainability Troubleshoot and optimize complex SQL and Spark workloads Support production deployments and platform modernization activities 5+ years of professional experience as a Data Engineer Strong programming skills in Python and advanced SQL Hands-on commercial experience with Databricks Strong knowledge of Apache Spark, primarily PySpark Experience designing and building modern cloud-based data platforms Experience developing ETL / ELT pipelines and large-scale data processing solutions Hands-on experience with Delta Lake Experience with Spark Declarative Pipelines Experience with cluster monitoring, metrics analysis, and performance optimization Strong understanding of distributed data processing architectures Solid understanding of data warehousing concepts and dimensional modeling Experience with Airflow or similar orchestration tools Experience optimizing complex analytical SQL workloads Experience implementing CI / CD practices for data engineering platforms Strong troubleshooting and performance optimization skills Ability to work