A presentation at Very Tech Trip in in Paris, France by Eléa PETTON
Claire Gallot & Eléa Petton Data, AI… what if they were the solution to understand sign language?
Claire Gallot SRE Dataconvergence Team 2
Eléa Petton Machine Learning Engineer AI Solutions Team @EleaPetton eleapttn Eléa PETTON 3
Objectives ASL dataset detection So what are we doing today? First solution test Data augmentation AI model training AI app deployment Second solution test 4
We have a wonderful idea! Detecting sign language with an AI to be able to understand it easily… Objectives 5
American Sign Language Letters Dataset v1 1728 images ASL dataset 26 classes David Lee Roboflow C 0.96 6
ASL dataset A B C D E F G H I J K L M N O P Q R S T U V W X Y Z 7
Object Detection 8
You Only Look Once YOLOv7 Family of algorithms Recognition and detection of various objects in pictures or videos Real-time Fast + accurate 9
Detect ASL letters with YOLOv7 W TRAIN custom model DETECT in real time 0.89 DISPLAY results 10
First solution test So should we test this app? 11
What’s wrong? Misidentification of letters 12
What’s wrong? Misidentification of letters Low precisions 12
What’s wrong? Misidentification of letters Low precisions False objects recognition 12
Why do we have poor results? Too small database - 1728 images only Lack of diversity in the dataset - poorly varied backgrounds - similar brightness and exposures - same type of hands (same skin tone, no tattoos, no rings, …) - not enough diversity in the foregrounds 13
Basic ASL dataset A B C D E F G H I J K L M N O P Q R S T U V W X Y Z 14
AI model EVALUATION AI OB TEBOOO O N TEBOOO O N KS model EXPORTATION model CREATION KS GE CT STORA E J AI data PROCESSING ASL LETTERS dataset TEBOOO O N KS K NOTEBO R A OK SP A trip to the best model… model TRAINING EPLOY AI D REAL-TIME DETECTION app DEVELOPMENT app DEPLOYMENT 15
How to solve the problem? n o i t a t n e m g u a data original dataset original dataset + new custom dataset resulting dataset with augmented data 16
Data augmentation How can I expand my dataset quickly? 17
Get more data I have some ideas to solve our problem! Asking my friends for new pictures Becoming a photoshop professional Doing some data augmentation with dedicated a tool 18
Data augmentation original horizontal flip shift brightness saturation noise blur contrast 19
Data processing tools Use already existing tools Albumentation Distributed image processing 20
Distributed processing 21
Distributed processing original dataset distributed image processing resulting dataset with augmented data 22
After data augmentation From 1500 images to 4500 2 generated images for each existing one Albumentation => 2 to 3 hours pySpark => 2 to 3 minutes 23
AI model training To train an AI model, I need GPU power! 24
AI tools NOTEBOOK AI Notebooks AI Training AI Deploy 25
AI Notebooks Access to a Jupyter Notebook in the Cloud Jupyter Miniconda 26
Models comparaison - metrics Precision & Recall 27
Models comparaison - metrics mAPs 28
Models comparaison - losses Training set 29
Models comparaison - losses Validation set 30
Models comparaison - confusion matrix 31
AI app deployment Time for deployment guys! Docker lovers in the room? 32
Deploy an AI app AI model Python script Docker image AI Deploy 33
Second solution test I’m sure this time it’s right! 34
Conclusion What did we learn today? 35
What did we learn today? Improve a dataset and an AI model thanks to Data Augmentation Train a model with GPU power Compare the models with Weights & Biases Build an AI app using the trained AI model Deploy an AI Deploy app with security and high availability 36
So did you enjoy the trip? Yes! Thanks for coming with us 37
References OVHcloud Data Processing documentation: https://docs.ovh.com/gb/en/data-processing/ OVHcloud AI examples repository: https://github.com/ovh/ai-training-examples OVHcloud AI documentation: https://docs.ovh.com/gb/en/publiccloud/ai/ YOLOv7 repository: https://github.com/WongKinYiu/yolov7 ASL alphabet dataset: https://public.roboflow.com/object-detection/american-sign-language-letters Full code: https://github.com/eleapttn/conference-asl-detection-yolov7-streamlit Public cloud access: https://www.ovhcloud.com/fr/public-cloud/ 38
Dans le domaine de l’Intelligence Artificielle, nous parlons souvent de Machine Learning ou de Deep Learning, mais au final que signifient ces termes et surtout comment peuvent-ils nous aider dans la vie courante ?
Durant ce talk nous allons tout d’abord tenter de comprendre la place qu’occupe l’IA dans le vaste domaine de la santé. Nous montrerons ensuite l’importance de la donnée et nous développerons une app d’IA capable de détecter et d’interpréter la langue des signes.
Ce talk permettra de montrer comment il est possible de traiter la donnée brute rapidement pour l’adapter un algorithme d’IA grâce à Apache Spark. Nous verrons aussi comment utiliser le traitement d’image distribué comme moyen d’enrichir ses données d’entraînement.
La seconde étape consistera à entraîner un modèle de détection d’objets à reconnaître les différents signes de l’alphabet ASL (American Sign Language) au travers d’un notebook Jupyter. .
Entraîner un modèle c’est bien, l’utiliser c’est mieux ! Les développeurs et développeuses apprendront à créer une app d’intelligence artificielle à partir du modèle entraîné pour de la détection de signes sur images ou flux vidéos. Pour finir, nous verrons comment déployer cette application avec Docker pour pouvoir la rendre accessible. Cette application permettra à quiconque de comprendre la langue des signes avec une détection et une retranscription écrite.
for free. You
can too.