Data Engineer Kafka - Freelance

Data Engineer Kafka – Freelance 

We are looking for a Data Engineer with extensive Kafka experience. The project is about implementing an ETL Framework for reading data from Kafka and loading into Oracle and HDFS. The ETL Framework needs to be a structure that over parameters (metainformation) you can launch hundreds of jobs every 15 minutes. The ETL framework will be implemented mainly with Spark Scala in micro-batch mode. 

Must have skill:

  1. Experience with Kafka > 2 years (ideally with many topics in the 100 to 1000 area)
  2. Experience with Spark > 2 years
  3. Experience with Software architecture in Java
  4. Experience with Scala 1 year
  5. English

Nice to have

  • German
  • CI/CD experience
  • SCRUM

If this project looks interesting to you feel free to send an email to info@hibernian.de