Data Engineering and the Death of Visual ETL

A presentation at Bored in Quarantine Virtual Series by Stewart Bryson

A funny thing happened on the way to modern, data-driven analytics: we ditched the GUI for ETL development. There are drawbacks to moving data using code instead of clicks, especially for organizations heavily invested in visual tools. But writing code has an upside as well.

In this presentation, we’ll explore the data engineering discipline, and how it compares to conventional ETL. Using Databricks and Snowflake together, we’ll look at classic batch and modern streaming use cases, and consider DataOps requirements around testing and delivery where code really shines. And we’ll do it all inside of an interactive notebook.


The following resources were mentioned during the presentation or are useful additional information.