Rediscover the known Universe with NASA dataset

Rediscover the known Universe with NASA dataset @LostInBritanny @AurrelH95 @PierreZ @moyowi @jugsummercamp @helloexoworld

Pierre Zemb @PierreZ Infrastructure Engineer working on distributed systems @jugsummercamp @helloexoworld

Aurélien Hébert @AurrelH95 Software Engineer and data lover @jugsummercamp @helloexoworld

Horacio Gonzalez @LostInBrittany Spaniard lost in Brittany, developer, dreamer and all-around geek @jugsummercamp @helloexoworld

Emmanuel we ♥ you @jugsummercamp @helloexoworld

HelloExoWorld Looking for exoplanets in NASA datasets @jugsummercamp @helloexoworld

HelloExoWorld Once upon a time... @jugsummercamp @helloexoworld

What not to do if you love astronomy Live in Brest @jugsummercamp @helloexoworld

Looking for solutions Computer stuff Astronomy Mixing passions @jugsummercamp @helloexoworld

Google is your friend... Let's find a project @jugsummercamp @helloexoworld

Exoplanets? Planets orbiting stars far away @jugsummercamp @helloexoworld

How do we find them? The transit method seems the best @jugsummercamp @helloexoworld

Exoplanets detection From theory to practice @jugsummercamp @helloexoworld

The transit method Credits: NASA’s Goddard Space Flight Center @jugsummercamp @helloexoworld

How do we look for transits? Image credits : NASA Image credits : NASA Kepler Tess @jugsummercamp @helloexoworld

@jugsummercamp @helloexoworld

Watching the sky By Carter Roberts [Public domain], via Wikimedia Commons @jugsummercamp @helloexoworld

Kepler image A star : 12*12px @jugsummercamp @helloexoworld

And what kind of data we get? Pleiades By NASA, ESA, AURA/Caltech, Palomar Observatory. Via Wikimedia Common @jugsummercamp @helloexoworld

Well, that's the problem Seven stars, seven different profiles @jugsummercamp @helloexoworld

Kinda big data Over 40 million light curves @jugsummercamp @helloexoworld

Big AND open data Lots of datasets in #opendata @jugsummercamp @helloexoworld

And we can help with that! Let's use our tools to analyse the data @jugsummercamp @helloexoworld

Time Series To analyse Kepler datasets @jugsummercamp @helloexoworld

Kepler: spatial Time Series Definition of Time Series: A series of data points indexed in time order @jugsummercamp @helloexoworld

Time Series ● ● ● ● ● ● ● Stock Market Analysis Economic Forecasting Budgetary Analysis Process and Quality Control Workload Projections Census Analysis ... @jugsummercamp @helloexoworld

Time Series Applications: ● Understanding the data ● Fit a model ○ Monitoring ○ Forecasting @jugsummercamp @helloexoworld

Time Series Stock market Analytics Economic Forecasting $$$ Study & Research @jugsummercamp @helloexoworld

Time Series Many specific analytical tools: ● ● ● ● ● ● Moving average ARMA (AutoRegressive Moving Average) Multivariate ARMA models ARCH (AutoRegressive Conditional Heteroscedasticity) Dynamic time warping ... @jugsummercamp @helloexoworld

Time Series Specific application of general tools ● ● ● ● ● Artificial neural networks Hidden Markov model Fourier & Wavelets transforms Entropy encoding ... @jugsummercamp @helloexoworld

Dealing with Time Series The 3 'v' @jugsummercamp @helloexoworld

Monitoring OVH with Time Series @jugsummercamp @helloexoworld

OVH Metrics A metrics data platform @jugsummercamp @helloexoworld

Tools to deal with Time Series Many options @jugsummercamp @helloexoworld

Metrics Data Platform @jugsummercamp @helloexoworld

Metrics’ metrics ● 1.5M datapoints/s, 24/7 ● Peaks at ~10M datapoints/s ● 500M unique series @jugsummercamp @helloexoworld

Metrics Data Platform + + @jugsummercamp @helloexoworld

Why Warp 10? Warp 10 is a software platform that ● Ingests and stores time series ● Manipulates and analyzes time series @jugsummercamp @helloexoworld

Analytics is the key to success Fetching data is only the tip of the iceberg @jugsummercamp @helloexoworld

Manipulating Time Series with Warp 10 A true Time Series analysis toolbox ○ Hundreds of functions ○ Manipulation frameworks ○ Analysis workflow @jugsummercamp @helloexoworld

Anatomy of a time series Each time series is composed of: ● Metadata ○ ○ org.nasa.kepler.starlight { keplerId: 52163778 } Class name Labels ● Datapoints ○ ○ Timestamp Value @jugsummercamp @helloexoworld

Class names and labels ● Class names define the kind of measure ○ Starlight, heart rate, speed… ● Labels define particular traits of a TS ○ org.nasa.kepler.starlight { keplerId: 52163778 } Device Id, Device Type, Private User Id... @jugsummercamp @helloexoworld

A match made in heaven Warp 10, OVH Metrics and HelloExoWorld @jugsummercamp @helloexoworld

What we have done ● ● ● ● Downloaded and parsed 40 millions of FITS files Pushed it to OVH Metrics Select a cool subset as training set Verified we could find the same planets as NASA @jugsummercamp @helloexoworld

From kepler-11 raw data @jugsummercamp @helloexoworld

To (candidates) exoplanets @jugsummercamp @helloexoworld

Your job @jugsummercamp @helloexoworld

Let's get started! 1. Connect to https://bit.ly/2H7Z5b3 or Connect to WIFI HEW-5G (or HEW) 2. Password is helloexoworld 3. Click on cancel on user password window 4. Open chrome/chromium on 192.168.1.2 Reach step 3.2 and enjoy! @jugsummercamp @helloexoworld

What's next? Where do we go from here? @jugsummercamp @helloexoworld

Only the beginning Better detection New import method Explorer Deep learning satellite/star location @jugsummercamp Yours? @helloexoworld

A growing team @jugsummercamp @helloexoworld

And you! Join us! https://helloexo.world https://xkcd.com/1371/ @jugsummercamp @helloexoworld

OVH Metrics Come speak with us about your time-series or monitoring projects and OVH Metrics @jugsummercamp @helloexoworld

Thank you, dear sponsors! @jugsummercamp @helloexoworld

Thank you! @jugsummercamp @helloexoworld

Rediscover the known Universe with NASA dataset @LostInBritanny @AurrelH95 @PierreZ @moyowi @jugsummercamp @helloexoworld