Rediscover the known Universe with NASA dataset Horacio Gonzalez @LostInBrittany Emmanuel Feller @moyowi
A presentation at SnowCamp 2019 in January 2019 in Grenoble, France by Horacio Gonzalez
Rediscover the known Universe with NASA dataset Horacio Gonzalez @LostInBrittany Emmanuel Feller @moyowi
Horacio Gonzalez @LostInBrittany Spaniard lost in Brittany, developer, dreamer and all-around geek #HelloExoWorld @LostInBrittany @moyowi
Emmanuel Feller @moyowi Développeur passionné #HelloExoWorld @LostInBrittany @moyowi
Pierre & Aurélien we ♥ you #HelloExoWorld @LostInBrittany @moyowi
HelloExoWorld Looking for exoplanets in NASA datasets #HelloExoWorld @LostInBrittany @moyowi
HelloExoWorld Once upon a time… #HelloExoWorld @LostInBrittany @moyowi
What not to do if you love astronomy To live in Brest #HelloExoWorld @LostInBrittany @moyowi
Looking for solutions Computer stuff Astronomy Mixing passions #HelloExoWorld @LostInBrittany @moyowi
Google is your friend… Let’s find a project #HelloExoWorld @LostInBrittany @moyowi
Exoplanets? Planets orbiting stars far away #HelloExoWorld @LostInBrittany @moyowi
How do we find them? The transit method seems the best #HelloExoWorld @LostInBrittany @moyowi
Exoplanets detection From theory to practice #HelloExoWorld @LostInBrittany @moyowi
The transit method Credits: NASA’s Goddard Space Flight Center #HelloExoWorld @LostInBrittany @moyowi
How do we look for transits? Image credits : NASA Image credits : NASA Kepler Tess #HelloExoWorld @LostInBrittany @moyowi
#HelloExoWorld @LostInBrittany @moyowi
Watching the sky By Carter Roberts [Public domain], via Wikimedia Commons #HelloExoWorld @LostInBrittany @moyowi
Kepler image A star : 12*12px #HelloExoWorld @LostInBrittany @moyowi
And what kind of data we get? Pleiades By NASA, ESA, AURA/Caltech, Palomar Observatory. Via Wikimedia Common #HelloExoWorld @LostInBrittany @moyowi
Well, that’s the problem Seven stars, seven different profiles #HelloExoWorld @LostInBrittany @moyowi
Kinda big data Over 40 million light curves #HelloExoWorld @LostInBrittany @moyowi
Big AND open data Lots of datasets in #opendata #HelloExoWorld @LostInBrittany @moyowi
And we can help with that! Let’s use our tools to analyse the data #HelloExoWorld @LostInBrittany @moyowi
Time Series To analyse Kepler datasets #HelloExoWorld @LostInBrittany @moyowi
Kepler: spatial Time Series Definition of Time Series: A series of data points indexed in time order #HelloExoWorld @LostInBrittany @moyowi
Time Series ● ● ● ● ● ● ● Stock Market Analysis Economic Forecasting Budgetary Analysis Process and Quality Control Workload Projections Census Analysis … #HelloExoWorld @LostInBrittany @moyowi
Time Series Applications: ▪ Understanding the data ▪ Fit a model – – Monitoring Forecasting #HelloExoWorld @LostInBrittany @moyowi
Time Series Stock market Analytics Economic Forecasting $$$ Study & Research #HelloExoWorld @LostInBrittany @moyowi
Time Series Many specific analytical tools: ● Moving average ● ARMA (AutoRegressive Moving Average) ● Multivariate ARMA models ● ARCH (AutoRegressive Conditional Heteroscedasticity) ● Dynamic time warping ● … #HelloExoWorld @LostInBrittany @moyowi
Time Series Specific application of general tools ● Artificial neural networks ● Hidden Markov model ● Fourier & Wavelets transforms ● Entropy encoding ● … #HelloExoWorld @LostInBrittany @moyowi
Dealing with Time Series The 3 ‘v’ #HelloExoWorld @LostInBrittany @moyowi
Monitoring OVH with Time Series #HelloExoWorld @LostInBrittany @moyowi
OVH Metrics A metrics data platform #HelloExoWorld @LostInBrittany @moyowi
Tools to deal with Time Series Many options #HelloExoWorld @LostInBrittany @moyowi
Metrics Data Platform #HelloExoWorld @LostInBrittany @moyowi
Metrics’ metrics ● 1.5M datapoints/s, 24/7 ● Peaks at ~10M datapoints/s ● 500M unique series #HelloExoWorld @LostInBrittany @moyowi
Metrics Data Platform + + #HelloExoWorld @LostInBrittany @moyowi
Why Warp 10? Warp 10 is a software platform that ● Ingests and stores time series ● Manipulates and analyzes time series #HelloExoWorld @LostInBrittany @moyowi
Analytics is the key to success Fetching data is only the tip of the iceberg #HelloExoWorld @LostInBrittany @moyowi
Manipulating Time Series with Warp 10 A true Time Series analysis toolbox ● ● ● Hundreds of functions Manipulation frameworks Analysis workflow #HelloExoWorld @LostInBrittany @moyowi
Anatomy of a time series Each time series is composed of: org.nasa.kepler.starlight { keplerId: 52163778 } Metadata ▪ ▪ Class name Labels Datapoints ▪ ▪ Timestamp Value #HelloExoWorld @LostInBrittany @moyowi
Class names and labels Class names define the kind of measure ▪ Starlight, heart rate, speed… org.nasa.kepler.starlight { keplerId: 52163778 } Labels define particular traits of a TS ▪ Device Id, Device Type, Private User Id… #HelloExoWorld @LostInBrittany @moyowi
A match made in heaven Warp 10, OVH Metrics and HelloExoWorld #HelloExoWorld @LostInBrittany @moyowi
What we have done ● ● ● ● Downloaded and parsed 40 millions of FITS files Pushed it to OVH Metrics Select a cool subset as training set Verified we could find the same planets as NASA #HelloExoWorld @LostInBrittany @moyowi
From kepler-11 raw data #HelloExoWorld @LostInBrittany @moyowi
To (candidates) exoplanets #HelloExoWorld @LostInBrittany @moyowi
Your job #HelloExoWorld @LostInBrittany @moyowi
Let’s get started!
What’s next? Where do we go from here? #HelloExoWorld @LostInBrittany @moyowi
Only the beginning Better detection New import method Explorer Deep learning satellite/star location #HelloExoWorld Yours? @LostInBrittany @moyowi
A growing team #HelloExoWorld @LostInBrittany @moyowi
And you! Join us! https://helloexo.world https://xkcd.com/1371/ #HelloExoWorld @LostInBrittany @moyowi
OVH Platform Come speak with us about your kubernetes, time-series or observability projects and OVH Platform #HelloExoWorld @LostInBrittany @moyowi
Thank you, dear sponsors! #HelloExoWorld @LostInBrittany @moyowi
Thank you! #HelloExoWorld @LostInBrittany @moyowi
