StreamThoughts
coming soon

Kafka Connect: Integrate your data as a real-time event stream

Price : 650 € HT - per attendee

For more information about this training course, please free to contact :
training@streamthoughts.io

Description

During this one-day workshop you will have the opportunity to learn in detail how the Apache Kafka Connect Framework works. We will present and discuss the uses, architecture concepts and Java APIs to develop your own data integration connectors.

Course Objectives

You will learn how to use, configure and deploy the Kafka Connect solution to integrate your data as a real-time event stream. You will learn how to implement your own data integration connectors. You will also discuss best practices for developing and deploying Kafka connectors.

Pedagogy

50% theory, 50% practise

Who Should Attend ?

This workshop is designed for Developers, Architects and Data Engineers who need to create data integration pipelines, in real time, through Apache Kafka.

Course Duration

1 Day

Course Prerequisites

Participants must be familiar with Java development. Participants should also be familiar with the basic concepts of Apache Kafka.

Course Content

1) Introduction

Motivations
What is Kafka Connect?
What is it used for?
The ecosystem, Confluent Connect Hub
Advantages & Disadvantages

2) Concepts and Architectures

Types of connector: Source & Sink
Kafka Connect Cluster: Workers & Tasks
Message formats: Converters
Data transformation: Single Message Transforms (SMTs)
Plugins
Delivery guarantees

3) Managing and deploying connectors

Deployment Models: Standalone & Distributed
The REST API
Configuring connectors
Installing new plugins
Strategies to deployed Kafka Connect: Dedicated vs Mutualized

4) Data Integration

Integrate data from filesystem with SpoolDir and FilePulse
Capturing Database Changes: Data Sourcing vs Change Data Capture
Introduction to Kafka Connect JDBC
Introduction to Debezium

5) Developing connectors

The main JAVA interfaces
The model and data schemas
Managing Source and Sink Offsets
Developing Transformers
Developing RESTs extensions
Best practices for development

6) Handling Errors

Dead Letter Queues

7) Security

Authentications
ACLs
Externalizing Configurations

8) Monitoring and Tools