A presentation at DACH Virtual MongoDB Community by Hans-Peter Grahsl
Event Streaming with Apache Kafka & MongoDB
Hans-Peter Grahsl • working & living in Graz • technical trainer @ NETCONOMY • engineer / consultant / speaker • original sink connector author • Confluent Community Catalyst @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 2
Event Streaming is on the rise… @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 3
Diminishing Value of Data @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 4
Events @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 5
events… …events everywhere! @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 6
Streaming @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 7
“… data processing that is designed with infinite data sets in mind.” — Tyler Akidau @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 8
☞ messaging ☞ integration ☞ processing plus storage @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 9
@hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 10
Operational Data Store ? @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 11
Operational Data Store ? @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 12
Operational data stores provide… ☞ richer data models ☞ arbitrary CRUD + convenient aggregations ☞ powerful secondary indexes ☞ fast + flexible analytics ☞ direct end-user serving @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 13
Why Apache Kafka and MongoDB ? @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 14
Apache Kafka + MongoDB = ✓ scalability & high availability ✓ embrace data in motion ✓ data retention support ✓ seamless cold storage integration ✓ fully managed cloud offerings @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 15
Kafka Connectors @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 16
Source Connectors @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 17
Sink Connectors @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 18
@hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 19
@hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 20
Exemplary Use Cases @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 21
Single Customer View @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 22
Synchronization across Services @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 23
Real-Time Recommendations @hpgrahsl | #MongoDB DACH #Community, 22nd October 2020, Virtual 24
reach out to me @hpgrahsl
!!! slides are here https://bit.ly/35kplwp
Kurzzusammenfassung:
Quer durch viele Branchen erleben wir seit einigen Jahren eine sehr starke Tendenz in Richtung Datenstromverarbeitung. Unternehmen wollen und müssen zum Teil ihre bestehenden Datenarchitekturen neu denken. Dies erfordet abseits traditionaller Datenbanken sowie batch-getriebener Tools einen verstärkten Einsatz modernerer Technologien, welche robuste und skalierbare Event Streaming Architekturen ermöglichen. Oftmals bildet dazu Apache Kafka das Herzstück unternehmensweiter Datenarchitekturen, jedoch benötigen die meisten praktischen Szenarien nach wie vor eine geeignete Datenbank die flexible genug ist, um der Vielfalt an geschäftsbedingten Anforderungen gerecht werden zu können und gleichzeitig zentrale Aspekte wie Hochverfügbarkeit und Skalierbarkeit abdecken kann. Dieser Kurzvortrag soll Impulsgeber sein und das Potential für echtzeitnahe Event Streaming Zwecke aufzeigen, welches aus dem Zusammenspiel des “Power-Duos” MongoDB & Apache Kafka entsteht.
The following resources were mentioned during the presentation or are useful additional information.
for free. You
can too.