I don’t always test my streams, but when I do I do it in production

I don’t always test my streams, but when I do I do it in production August 2020 / Kafka Summit @gamussa @gamussa || @jconf.dev @jconf.dev || @confluentinc @confluentinc

@gamussa | @jconf.dev | @confluentinc

Special thanks @inponomarev @gamussa @bsideup | @jconf.dev | @confluentinc

https://gamov.dev/jdevconf2020 @gamussa | @jconf.dev | @confluentinc

Agenda 01 04 Kafka Apps Testing Streams 02 05 Testing basics Testing ksqlDB 03 06 Can we better? Conclusion What are those? TopologyTestDriver vs TestContainers Run Kafka and CP locally Ksql-test-runner vs TestContainers Always Be Testing Test with TestContainers @gamussa | @jconf.dev | @confluentinc

Preface @gamussa | @jconf.dev | @confluentinc

Senior Developer: - Where did you get this code? Junior Developer: - From Stackoverflow Senior Developer: - from question or answer? @gamussa | @jconf.dev | @confluentinc

An example from my life: Kafka Tutorials website @gamussa | @jconf.dev | @confluentinc

Literate Programming… Do you speak it… https://en.wikipedia.org/wiki/Literate_programming @gamussa | @jconf.dev | @confluentinc

@gamussa | @jconf.dev | @confluentinc

@gamussa | @jconf.dev | @confluentinc

@gamussa | @jconf.dev | @confluentinc

@gamussa | @jconf.dev | @confluentinc

@gamussa | @jconf.dev | @confluentinc

Kafka Apps aka stream processing @gamussa | @jconf.dev | @confluentinc

Stream Processing is the toolset for dealing with events as they move! @gamussa | @jconf.dev | @confluentinc

Producers Kafka Consumers @gamussa | @jconf.dev | @confluentinc

What are Kafka Apps? My App @gamussa | @jconf.dev | @confluentinc

Schema, WUT? @gamussa | @jconf.dev | @confluentinc

Let’s see some code @gamussa | @jconf.dev | @confluentinc

Kafka Streams for stream processing @gamussa | @jconf.dev | @confluentinc

the KAFKA STREAMS API is a JAVA API to BUILD REAL-TIME APPLICATIONS @gamussa | @jconf.dev | @confluentinc

https://kafka.apache.org/documentation/streams/developer-guide/testing.html @gamussa | @jconf.dev | @confluentinc

Topology Test driver @gamussa | @jconf.dev | @confluentinc

• TopologyTestDriver • Unit test your Kafka Streams Topology • Doesn’t Require Kafka infra to run • Convenience methods to define input and output • Convenience methods to provide fixtures and validate the results @gamussa | @jconf.dev | @confluentinc

Let’s see some code @gamussa | @jconf.dev | @confluentinc

@gamussa | @jconf.dev | @confluentinc

@gamussa | @jconf.dev | @confluentinc

When TTD doesn’t help? https://gamov.dev/livestreams-s01e05 @gamussa | @jconf.dev | @confluentinc

Ksqldb for Stream processing @gamussa | @jconf.dev | @confluentinc

@gamussa | @jconf.dev | @confluentinc

@gamussa | @jconf.dev | @confluentinc

Let’s see some code @gamussa | @jconf.dev | @confluentinc

Conclusion @gamussa | @jconf.dev | @confluentinc

What did you learn today? • What’s Kafka apps apps and how to test them • Producer / Consumer API • Kafka Streams • ksqlDB • How containers and TestContainer help a lot @gamussa | @jconf.dev | @confluentinc

@gamussa | @jconf.dev | @confluentinc

Want to learn more? @gamussa | #springone | @confluentinc

Learn Kafka. Start building with Apache Kafka at Confluent Developer. developer.confluent.io Watch full version https://gamov.dev/developer

Confluent Community Slack A vibrant community of over 16,000 members Come along and discuss Apache Kafka and Confluent Platform on dedicated channels including #ksqlDB, #connect, #clients, and more http://cnfl.io/slack @gamussa | @jconf.dev | @confluentinc

@gamussa | @jconf.dev | @confluentinc