We are seeking an experienced Databricks Data Engineer to help establish a data-driven culture by building scalable data solutions and collaborating with various stakeholders. The ideal candidate will have hands-on expertise in Databricks environments, data pipelines, and big data technologies, as well as the ability to bridge technical and business needs.

Work Location & Engagement

• Full-time with partial remote work available.
• Location: Gent
• Start Date: As soon as possible.

Key Responsibilities

• Build and maintain scalable data architectures and data lakes on the Databricks platform.
• Design and optimize data pipelines for ingestion, transformation, and data loading from multiple sources.
• Implement monitoring and alerting systems to address data processing issues proactively.
• Troubleshoot and resolve performance and data quality challenges in Databricks environments.
• Collaborate with data engineers to ensure scalability, reliability, and security of Databricks clusters.
• Develop optimized SQL queries and Spark jobs for efficient data processing.
• Provide guidance and technical support to data scientists and analysts using Databricks.
• Document solutions, architectures, and processes related to Databricks implementations.

Requirements

Hard Skills:

• Strong knowledge of big data technologies like Apache Spark and distributed computing.
• Experience with cloud platforms, particularly AWS Databricks.
• Proficiency in data ingestion, ETL processes, streaming, and batch data management.
• Expertise in building data pipelines using Databricks notebooks, Delta Lake, and Databricks Jobs.
• Proficiency in SQL and programming languages such as Python or Scala.
• Understanding of data governance, security, and compliance requirements.

Soft Skills:

• Fluent in English (speaking, writing, and understanding).
• Excellent communication, problem-solving, and analytical skills.
• Team-oriented with the ability to take initiative in a dynamic, fast-paced environment.

Additional Advantage

• Experience with the installation and configuration of Databricks clusters, including cluster settings, network connectivity, and access controls.

This is a unique opportunity to contribute to the success of a growing platform by leveraging your Databricks expertise in a collaborative and innovative environment.