Using Retrieval Augmented Generation (RAG) with Sitecore Search to ground GPT queries

Using Retrieval Augmented Generation (RAG) with Sitecore Search to ground GPT queries

Rob Earlam Head of Developer Advocacy, Sitecore Developer advocate lover music listener software developer movie pizza eater meat smoker living in rob-earlam @rob@robearlam.com RobEarlam @RobEarlam.com https://robearlam.com/

THANK YOU to our sponsors P L AT I N U M GOLD COMMUNITY PLUS COMMUNITY

Large Language Model (LLM) Hallucinations • LLM Hallucination are grammatically correct but factually inaccurate, or nonsensical. • Sounds convincing, but don’t align with reality • Can lead to confusion & mistrust • Addressing hallucination is essential for building trust in AI-generated content

How can you get better results? Train your own model Pretrain model based on your domainspecific data. Slow Expensive (10’s of millions $$) Still point-in-time

How can you get better results? Fine-tune existing model Adapt an existing model Fine-tuned models often forget or lose capabilities Reliant on the quantity and quality of training data Lacks external knowledge Still point-in-time

How can you get better results? Retrieval Augmented Generation (RAG) “Grounding” queries using your domain information with existing Models. Significantly cheaper. Retains the capabilities of existing model. Ability to change knowledge sources. No need to retrain when data changes.

Retrieval Augmented Generation (RAG)

Sitecore Search on the Developer Portal

Sitecore Search on the Developer Portal Sitecore.com Discover Sitecore YouTube channel Helix Documentation Sitecore GitHub Repositories Sitecore Stack Exchange Sitecore Community Blogs Sitecore Developer Portal Sitecore Documentation Sitecore Knowledge Base Sitecore PowerShell Documentation OrderCloud Documentation Sitecore Blok Sitecore Changelog

Hello Sitecore ChatBot! • Part of internal Sitecore AI Hackathon • Today’s focus on RAG portions of the project, but much more built into this project • Personalise – Persona based tailoring of results • CDP – tracking and governance

What should we ask? https://forms.office.com/e/G3SmLq3jgz

RAG with Sitecore Search Sitecore.com Sitecore Changelog Sitecore Developer Portal Sitecore Developer Portal Azure OpenAI

Summary • With LLM’s you need to think “probabilistically” not “deterministically” • 1 + 1 doesn’t always equal 2 • Using RAG allows you to ground queries without the need for expensive training or fine-tuning • Sitecore Search can be a great option for RAG as customers already have their data stored there.

Thank you