Awesome Monitoring Infrastructure Using the Elastic Stack

A presentation at Linux.conf.au 2019 in in Christchurch, New Zealand by Mark Walkom

IoT is great, until something breaks or you want to see how things perform at scale! To handle the volumes of data that any IoT project can generate, many developers default to Open Source tool chains to capture and analyse logs and metrics, and then pinpoint errors, faults or inefficiencies in their systems.

One of the most widely used Open Source tools for this is the Elastic Stack:

  • Beats as a lightweight agent/forwarder of data.
  • Logstash for further parsing and enrichment.
  • Elasticsearch doing the hard work of storing and analysis.
  • Kibana for powerful visualisations.

This workshop gives you an overview of the four technologies, how they work together and how, from microservice logs to IoT metrics to containerised apps, it has you covered.

At the end of the workshop, you’ll walk away with a set of connected Docker containers capturing logs and metrics from your laptop, a head start on dashboards for analysis, and ideas to deploy for your next project.

https://linux.conf.au/schedule/presentation/161/

Resources

The following resources were mentioned during the presentation or are useful additional information.

Code

The following code examples from the presentation can be tried out live.