A presentation at SpringOne 2020 by Viktor Gamov
The Tales of Event-Driven Told by a Spring Developer September 2020 / SpringOne @gamussa | #springone | @confluentinc
@gamussa | #springone | @confluentinc
Preface @gamussa | #springone | @confluentinc
Photo by Khai Sze Ong on Unsplash
Photo by Rainier Ridao on Unsplash \
Photo by Rohit Tandon on Unsplash
Photo by Theodore Moore on Unsplash
Photo by Cristian Grecu on Unsplash
@gamussa | #springone | @confluentinc
Refactoring to microservices
It used to be so simple…
@gamussa | #springone | @confluentinc
Normal Applications (i.e., monoliths) @gamussa | #springone | @confluentinc
Monoliths are hard to think about @gamussa | #springone | @confluentinc
Monoliths are hard to change @gamussa | #springone | @confluentinc
Re-integration? @gamussa | #springone | @confluentinc
Re-integration @gamussa | #springone | @confluentinc
Photo by Deva Darshan from Pexels
There are no good ways to integrate microservices @gamussa | #springone | @confluentinc
There are no-good ways to integrate microservices @gamussa | #springone | @confluentinc
Filesystem @gamussa | #springone | @confluentinc
@gamussa | #springone | @confluentinc
Database @gamussa | #springone | @confluentinc
@gamussa | #springone | @confluentinc
Integrating Microservices through the database • “I have a database and I know how to use it.” • Eventually causes services to co-mingle. • Violation the «bounded context» • Great to use inside a service boundary! • Terrible for sharing data or negotiating change. @gamussa | #springone | @confluentinc
RPC @gamussa | #springone | @confluentinc
@gamussa | #springone | @confluentinc
Integrating microservices via RPC • Avoids problems of database integration • Feels natural • Aligns with the request/ response paradigm • Problem: cascading failures • Question: how do you debug this system? 🤔 • Answer: you build a log. @gamussa | #springone | @confluentinc
Log4j type of log? @gamussa | #springone | @confluentinc 🤔
Events? @gamussa | #springone | @confluentinc
What’s an event? @gamussa | #springone | @confluentinc
an occurrence, especially one of some importance. @gamussa | #springone | @confluentinc
A combination of: Notification State transfer @gamussa | #springone | @confluentinc
Also, events are immutable. @gamussa | #springone | @confluentinc
Core abstraction @gamussa • DB - table • Hadoop file • Kafka - ? | #springone | @confluentinc
LOG @gamussa | #springone | @confluentinc
A Richer Microservices Application @gamussa | #springone | @confluentinc
@gamussa | #springone | @confluentinc
Ratings Service operating on the event stream Browser ratings Ratings Service Webserver Order rawRequested rating ratings ratings counts KStreams API used to count and sum ratings as they arrive ratings sums KAFKA @gamussa | @ #springone | @confluentinc
Schema Management Browser ratings Ratings Service Webserver Order rawRequested rating ratings ratings counts KStreams API used to count and sum ratings as they arrive ratings sums KAFKA @gamussa | @ #springone | @confluentinc Schema Registry
Access legacy relation data Browser ratings Ratings Service Webserver Order rawRequested rating Connect ratings ratings counts KStreams API used to count and sum ratings as they arrive ratings sums KAFKA @gamussa | @ #springone | @confluentinc Movies Schema Registry
Materialize tables inside the app Browser ratings Ratings Service Movies Lookup table created inside the Rating Service Webserver Order rawRequested rating Connect ratings ratings counts ratings sums KAFKA @gamussa | @ #springone | @confluentinc Movies Schema Registry
Create a new table, persist it to the log Browser ratings rated-movies Ratings Service Writable table created for rated movies Movies Webserver Order rawRequested rating Connect ratings ratings counts ratings sums KAFKA @gamussa | @ #springone | @confluentinc Movies rated-movies Schema Registry
Create a new table, persist it to the log Browser ratings rated-movies Ratings Service Movies TRANSACTION Webserver Order rawRequested rating Connect ratings ratings counts ratings sums KAFKA @gamussa | @ #springone | @confluentinc Movies rated-movies Schema Registry
Hydrate a materialized view Browser ratings Ratings Service an embedded database CDC Webserver Order rawRequested rating Connect ratings ratings counts ratings sums Movies rated-movies KAFKA @gamussa | @ #springone | @confluentinc Schema Registry
What did you learn today? • Streaming services improve on the traditional microservices pattern in a number of ways: • Decouple ecosystems for easier change • Evolve existing databases into event streams with Kafka Connect. • Build on asynchronicity first. Add synchronicity where needed. @gamussa | #springone | @confluentinc
What did you learn today? • Remove the need for circuit breakers or other “overload” patterns. • Quickly combine and operate on data from many different services. • Embrace an immutable log that can be replayed. • Keep many databases, in many independent services, in sync. • Safely manage the evolution of data with the schema registry. @gamussa | #springone | @confluentinc
Want to learn more? @gamussa | #springone | @confluentinc
https://gamov.dev/spring20
Learn Kafka. Start building with Apache Kafka at Confluent Developer. developer.confluent.io Watch full version https://gamov.dev/developer
@gamussa | #springone | @confluentinc
Developers have long employed message queues to decouple subsystems and provide an approximation of asynchronous processing. However, these queuing systems don’t adequately deliver on the promise of event-driven architectures and often lead to the usage of anti-patterns. The events are carrying both notification and state, which enables developers and data engineers to design and develop to event-driven systems. Developers benefit from the asynchronous communication that events enable between services, and data engineers benefit from the integration capabilities.
In this talk, Viktor will explain the concepts of events, their relevance to software and data engineers, as well as their power for effectively unifying architectures. You’ll learn how stream processing makes sense in microservices and data integration projects. The talk concludes with a hands-on demonstration of these concepts in practice, using a modern toolchain: Kotlin, Spring Boot, Ktor, and Apache Kafka!
The following code examples from the presentation can be tried out live.
Here’s what was said about this presentation on social media.
