A presentation at EMFCamp in in Eastnor, Ledbury HR8, UK by Lorna Jane Mitchell
Landscape of Open Source Databases Lorna Mitchell, Aiven
Keeping up with Databases • We need more databases, because we have more data • The right technology choice is important • Open source is secureable and future-proof @aiven_io ~ @lornajane
Data Sources Two main sources of data • my own Opinion • https://db-engines.com/en/ranking @aiven_io ~ @lornajane
Relational Databases Traditional databases • pre-defined tables with columns • relations between tables, e.g. book has an author @aiven_io ~ @lornajane
MySQL License: GPLv2 World’s most-used open source database • part of LAMP stack (Linux Apache MySQL PHP/Python/Perl) • proprietary Enterprise Server version also available @aiven_io ~ @lornajane
MariaDB License: GPLv2 • drop-in replacement for MySQL • support for additional storage engines • proprietary Enterprise Server version also available @aiven_io ~ @lornajane
PostgreSQL License: PostgreSQL license (MIT-ish) • powerful and performant relational database • many contributors, healthy community • lots of extensions @aiven_io ~ @lornajane
PostGIS License: GPLv2 Spatial database, as an extension to PostgreSQL. • support for geographical object data types • functions for working with area, distance, etc • specialist indexes to support spatial queries @aiven_io ~ @lornajane
TimescaleDB License: Apache2, some features TSL Extension for PostgreSQL • table types for timeseries data • additional SQL functions @aiven_io ~ @lornajane
Time Series Data Time series data: • a timestamp • a measurement @aiven_io ~ @lornajane
InfluxDB License: MIT • time series database • IoT, metrics, energy • clustered version has proprietary license @aiven_io ~ @lornajane
Re-use wire protocols Build a new database, use an existing wire protocol to get clients and integrations Examples: • CrateDB uses PostgreSQL protocol • VictoriaMetrics and M3DB use Influx and Prometheus protocols @aiven_io ~ @lornajane
SQLite License: public domain • file based, no server • embeddable • ideal edge model database @aiven_io ~ @lornajane
Redis License: BSD Speedy in-memory key value store • used for caching, queueing • supports many data types (lists, sets, hashes, etc) • 3rd most popular open source database @aiven_io ~ @lornajane
Key/Value Stores Other key value stores worth a mention: • Memcached • etcd • ArangoDB @aiven_io ~ @lornajane
Apache Cassandra License: Apache2 • distributed database for commodity hardware • designed for very large volumes of data • use denormalised data storage (no joins) @aiven_io ~ @lornajane
Distributed Databases Horizontally scalable for writes, spread across multiple nodes • data organised into shards or partitions • usually also replicated for redundancy • complexity handled by database @aiven_io ~ @lornajane
OpenSearch License: Apache2 Open source fork of Elasticsearch • powerful search and aggregation features • flexible data structure, but defined indexes • Opensearch Dashboards is the fork of Kibana @aiven_io ~ @lornajane
Open Source Databases Best technology around, whatever your data needs @aiven_io ~ @lornajane
Resources • https://aiven.io - DBaaS • https://uptime.aiven.io - Open source data event • https://lornajane.net - my websitb/blog • 7 Databases in 7 Weeks (2nd edition) @aiven_io ~ @lornajane
Every year we collect more data than before, and the tools we use to manage that data are evolving to accommodate our changing needs - but it can be difficult to keep up with all the innovations! This session will give you a tour of what’s happening in open source databases, from someone who lives the adventures of open source data in her day job at Aiven! You will travel from the well-trodden paths of relational databases, through the leafy glades of time series, to the landmarks of search and document databases. This session is recommended for people with an interest in software who want to learn about the overall trends, license changes, rising stars, and which database technologies are here to stay.
