Robin is a Developer Advocate at Confluent, the company founded by the original creators of Apache Kafka, as well as an Oracle Groundbreaker Ambassador and ACE Director (Alumnus). His career has always involved data, from the old worlds of COBOL and DB2, through the worlds of Oracle and Hadoop, and into the current world with Kafka. His particular interests are analytics, systems architecture, performance testing and optimization. He blogs at http://cnfl.io/rmoff and http://rmoff.net/ (and previously http://ritt.md/rmoff) and can be found tweeting grumpy geek thoughts as @rmoff. Outside of work he enjoys drinking good beer and eating fried breakfasts, although generally not at the same time.
Integrating Apache Kafka with other systems in a reliable and scalable way is often a key part of a streaming platform. Fortunately, Apache Kafka includes the Connect API that enables streaming integration both in and out of Kafka. Like any technology, understanding its architecture and deployment patterns is key to successful use, as is knowing where to go looking when things aren’t working.
This talk will discuss the key design concepts within Kafka Connect and the pros and cons of standalone vs distributed deployment modes. We’ll do a live demo of building pipelines with Kafka Connect for streaming data in from databases, and out to targets including Elasticsearch. With some gremlins along the way, we’ll go hands-on in methodically diagnosing and resolving common issues encountered with Kafka Connect. The talk will finish off by discussing more advanced topics including Single Message Transforms, and deployment of Kafka Connect in containers.
|Real-time SQL stream processing at scale with Apache Kafka and KSQL||Strata Data Conference, London||April 2019|
|The Changing Face of ETL: Event-Driven Architectures for Data Engineers||Strata Data Conference, London||April 2019|
|ATM Fraud Detection with Apache Kafka and KSQL||London Kafka Meetup||April 2019|
|No More Silos: Integrating Databases and Apache Kafka||Kafka Summit New York 2019||April 2019|
|The Changing Face of ETL: Event-Driven Architectures for Data Engineers||Paris Apache Kafka Meetup||March 2019|
|QCon Workshop: Apache Kafka and KSQL in Action : Let’s Build a Streaming Data Pipeline!||QCon London||March 2019|
|The Changing Face of ETL: Event-Driven Architectures for Data Engineers||Big Data Tech Warsaw||February 2019|
|ATM Fraud detection with Kafka and KSQL||IT Tage||December 2018|
|No More Silos: Integrating Databases and Apache Kafka||UKOUG 2018||December 2018|
|Apache Kafka and KSQL in Action : Let’s Build a Streaming Data Pipeline!||LISA 18||October 2018|
|Embrace the Anarchy : Apache Kafka's Role in Modern Data Architectures||Munich Microservices Meetup||August 2018|
|Apache Kafka and KSQL in Action : Let’s Build a Streaming Data Pipeline!||Berlin Kafka Meetup||July 2018|