For a client, we are seeking an experienced Kafka Expert.

Role Overview

We are looking for a Kafka Expert to join our team, focusing on data infrastructure and application development with expertise in Apache Kafka and AWS technologies (MSK).

Expected Responsibilities

• Design, develop, and maintain data pipelines using Apache Kafka and AWS services.
• Implement and manage Kafka Connect connectors for data integration.
• Configure and maintain Kafka Schema Registry for data governance.
• Monitor and optimize the performance of Kafka clusters on MSK.
• Develop or assist with developing Java applications that interact with the Kafka ecosystem.
• Write Infrastructure as Code using Terraform for AWS resources (optional, but nice to have).
• Collaborate with team members on data architecture decisions.
• Implement data security and compliance measures.
• Troubleshoot and resolve production issues.

Required Technical Skills

• Strong experience with the Apache Kafka ecosystem:
  • Kafka message brokers and topics
  • Kafka Connect framework
  • Schema Registry and Avro serialization
• Proficient in Java development or able to translate Kafka topics towards Java implementation.
• Experience with AWS services:
  • Amazon MSK (Managed Streaming for Kafka)
  • S3, RDS, and DynamoDB
  • Lambda, SNS, and SQS
  • IAM and security best practices
• Infrastructure as Code using Terraform.
• Version control with Git.

Optional Additional Skills

• Experience with:
  • AWS Glue for ETL processes
  • Databricks platform
  • Snowflake data warehouse
  • AWS Kinesis Firehose
• Knowledge of data modeling and design patterns.
• Understanding of event-driven architectures.
• Experience with CI/CD pipelines.

Soft Skills

• Strong problem-solving abilities.
• Excellent communication skills.
• Team player with a collaborative mindset.
• Self-motivated and proactive approach.
• Ability to work in an agile environment.

Experience Level

Minimum 3-5 years of relevant experience in software development with a specific focus on data infrastructure and the Kafka ecosystem.