A presentation at HackConf 2021 in in Sofia, Bulgaria by Elena Georgieva
@geoellena On Data: Sky above, sand below, peace within October 2021 Elena Georgieva
@geoellena ● How did we get here? ● Modern data paradigms ● FT’s approach
@geoellena 1988 2010 2020
@geoellena
Data warehouse Photo: https://www.pexels.com/@chanaka-318741 @geoellena
Data warehouse @geoellena
At the same time… Source: bnr.bg @geoellena
At the same time… @geoellena
At the same time… @geoellena
At the same time… @geoellena
At the same time… Source: CoderDojo Bulgaria @geoellena
@geoellena 1988 2010 2020
@geoellena Source: wikipedia.org
@geoellena Source: ft.com
Data Lake @geoellena
Data Lake @geoellena Rila lakes, Bulgaria Photo: https://www.pexels.com/@bkrustev
Data Lake Source: https://vdocument.in/making-sense-of-big-data.html @geoellena
@geoellena 1988 2010 2020
@geoellena ● How did we get here? ● Modern data paradigms ● FT’s approach
@geoellena Best of both worlds
@geoellena https://www.pexels.com/@riciardus
@geoellena https://www.pexels.com/@rakicevic-nenad-233369
Data Lakehouse @geoellena Breitenwang, Austria https://www.pexels.com/@lucasallmann
@geoellena Source: https://databricks.com/ Article: What is a Lakehouse? by Ben Lorica, Michael Armbrust, Ali Ghodsi, Reynold Xin and Matei Zaharia
@geoellena Criteria Schema enforcement and data governance Support for diverse data types Support for diverse workloads Low latency Transaction support Separated storage and compute Openness Easy access to data Data Warehouse Data Lake Lakehouse
@geoellena Data Democratisation Photos: https://www.pexels.com/@anna-nekrashevich
@geoellena How we can achieve Data Democratisation? ● Self-service dashboarding ● Central data stores in the Cloud ● Data federation ● Data virtualization
Data Science Photos: https://www.pexels.com/@thisisengineering @geoellena
Why it is not that new as a concept? Photos: https://www.pexels.com/@pixabay @geoellena
@geoellena Source: ft.com
@geoellena Source: btvnovinite.bg
Hacker, Statistician, Domain expert Source: Drew Conway @geoellena
@geoellena ● How did we get here? ● Modern data paradigms ● FT’s approach
Now… @geoellena
@geoellena 2008 2014 2016 2019 The Modest start Warehouse in the Clouds Real-time data Warehouse with Lake
@geoellena The FT Data Platform 2019
@geoellena Serving Layer
@geoellena The challenges with Data Lake ● Slowing down data delivery ● Management overhead ● Governance and slow backfills
@geoellena 2008 2014 2016 2019 2020+ The Modest start Warehouse in the Clouds Real-time data Warehouse with a Lake Lakehouse
@geoellena
@geoellena
@geoellena
@geoellena
@geoellena
@geoellena
@geoellena
Active Projects ● Self-service ● Separating storage from compute ● MLOps and data time travels @geoellena
@geoellena
Sky above Photo: https://www.pexels.com/@szaboviktor @geoellena
Sand Below Photo: https://www.pexels.com/@negativespace @geoellena
Peace within Photo: https://www.pexels.com/@pixabay @geoellena
twitter @geoellena THANK YOU bit.ly/ftcareers
An exciting technology time travel with bites from reality, arts, and music. Hear from a data veteran how data architecture, data science, and data culture evolved over time to reach 2021 with the latest paradigms of Data Lake House and Data Democratisation.
for free. You
can too.