The Tales of Event-Driven Told by a Spring Developer

The Tales of Event-Driven Told by a Spring Developer September 2020 / SpringOne @gamussa | #springone | @confluentinc

@gamussa | #springone | @confluentinc

Preface @gamussa | #springone | @confluentinc

Photo by Khai Sze Ong on Unsplash

Photo by Rainier Ridao on Unsplash \

Photo by Rohit Tandon on Unsplash

Photo by Theodore Moore on Unsplash

Photo by Cristian Grecu on Unsplash

@gamussa | #springone | @confluentinc

Refactoring to microservices

It used to be so simple…

@gamussa | #springone | @confluentinc

Normal Applications (i.e., monoliths) @gamussa | #springone | @confluentinc

Monoliths are hard to think about @gamussa | #springone | @confluentinc

Monoliths are hard to change @gamussa | #springone | @confluentinc

Re-integration? @gamussa | #springone | @confluentinc

Re-integration @gamussa | #springone | @confluentinc

Photo by Deva Darshan from Pexels

There are no good ways to integrate microservices @gamussa | #springone | @confluentinc

There are no-good ways to integrate microservices @gamussa | #springone | @confluentinc

Filesystem @gamussa | #springone | @confluentinc

@gamussa | #springone | @confluentinc

Database @gamussa | #springone | @confluentinc

@gamussa | #springone | @confluentinc

Integrating Microservices through the database • “I have a database and I know how to use it.” • Eventually causes services to co-mingle. • Violation the «bounded context» • Great to use inside a service boundary! • Terrible for sharing data or negotiating change. @gamussa | #springone | @confluentinc

RPC @gamussa | #springone | @confluentinc

@gamussa | #springone | @confluentinc

Integrating microservices via RPC • Avoids problems of database integration • Feels natural • Aligns with the request/ response paradigm • Problem: cascading failures • Question: how do you debug this system? 🤔 • Answer: you build a log. @gamussa | #springone | @confluentinc

Log4j type of log? @gamussa | #springone | @confluentinc 🤔

Events? @gamussa | #springone | @confluentinc

What’s an event? @gamussa | #springone | @confluentinc

an occurrence, especially one of some importance. @gamussa | #springone | @confluentinc

A combination of: Notification State transfer @gamussa | #springone | @confluentinc

Also, events are immutable. @gamussa | #springone | @confluentinc

Core abstraction @gamussa • DB - table • Hadoop file • Kafka - ? | #springone | @confluentinc

LOG @gamussa | #springone | @confluentinc

A Richer Microservices Application @gamussa | #springone | @confluentinc

@gamussa | #springone | @confluentinc

Ratings Service operating on the event stream Browser ratings Ratings Service Webserver Order rawRequested rating ratings ratings counts KStreams API used to count and sum ratings as they arrive ratings sums KAFKA @gamussa | @ #springone | @confluentinc

Schema Management Browser ratings Ratings Service Webserver Order rawRequested rating ratings ratings counts KStreams API used to count and sum ratings as they arrive ratings sums KAFKA @gamussa | @ #springone | @confluentinc Schema Registry

Access legacy relation data Browser ratings Ratings Service Webserver Order rawRequested rating Connect ratings ratings counts KStreams API used to count and sum ratings as they arrive ratings sums KAFKA @gamussa | @ #springone | @confluentinc Movies Schema Registry

Materialize tables inside the app Browser ratings Ratings Service Movies Lookup table created inside the Rating Service Webserver Order rawRequested rating Connect ratings ratings counts ratings sums KAFKA @gamussa | @ #springone | @confluentinc Movies Schema Registry

Create a new table, persist it to the log Browser ratings rated-movies Ratings Service Writable table created for rated movies Movies Webserver Order rawRequested rating Connect ratings ratings counts ratings sums KAFKA @gamussa | @ #springone | @confluentinc Movies rated-movies Schema Registry

Create a new table, persist it to the log Browser ratings rated-movies Ratings Service Movies TRANSACTION Webserver Order rawRequested rating Connect ratings ratings counts ratings sums KAFKA @gamussa | @ #springone | @confluentinc Movies rated-movies Schema Registry

Hydrate a materialized view Browser ratings Ratings Service an embedded database CDC Webserver Order rawRequested rating Connect ratings ratings counts ratings sums Movies rated-movies KAFKA @gamussa | @ #springone | @confluentinc Schema Registry

What did you learn today? • Streaming services improve on the traditional microservices pattern in a number of ways: • Decouple ecosystems for easier change • Evolve existing databases into event streams with Kafka Connect. • Build on asynchronicity first. Add synchronicity where needed. @gamussa | #springone | @confluentinc

What did you learn today? • Remove the need for circuit breakers or other “overload” patterns. • Quickly combine and operate on data from many different services. • Embrace an immutable log that can be replayed. • Keep many databases, in many independent services, in sync. • Safely manage the evolution of data with the schema registry. @gamussa | #springone | @confluentinc

Want to learn more? @gamussa | #springone | @confluentinc

https://gamov.dev/spring20

Learn Kafka. Start building with Apache Kafka at Confluent Developer. developer.confluent.io Watch full version https://gamov.dev/developer

@gamussa | #springone | @confluentinc