Senior Data Engineer Databricks

Job Description

. We are looking for an experienced Senior Data Engineer - Databricks to design, develop, and maintain scalable data pipelines on the Databricks platform.

. The role requires strong expertise in PySpark, Databricks, and modern data engineering practices, along with experience modernizing legacy data pipelines and building high-performance data solutions in cloud environments.

Key Responsibilities

. Design, build, and maintain scalable data pipelines using Databricks and PySpark.

. Develop end-to-end data workflows including data ingestion, transformation, and consumption.

. Optimize Spark jobs, cluster configurations, and pipeline performance.

. Manage and orchestrate workflows using Databricks Jobs and notebooks.

. Refactor legacy ETL pipelines into modern PySpark-based ELT frameworks.

. Implement data quality checks, monitoring, and error handling mechanisms. <...