A presentation at SUGCON India 2024 in in Bengaluru, Karnataka, India by Rob Earlam
Using Retrieval Augmented Generation (RAG) with Sitecore Search to ground GPT queries
Rob Earlam Head of Developer Advocacy, Sitecore Developer advocate lover music listener software developer movie pizza eater meat smoker living in rob-earlam @rob@robearlam.com RobEarlam @RobEarlam.com https://robearlam.com/
THANK YOU to our sponsors P L AT I N U M GOLD COMMUNITY PLUS COMMUNITY
Large Language Model (LLM) Hallucinations • LLM Hallucination are grammatically correct but factually inaccurate, or nonsensical. • Sounds convincing, but don’t align with reality • Can lead to confusion & mistrust • Addressing hallucination is essential for building trust in AI-generated content
How can you get better results? Train your own model Pretrain model based on your domainspecific data. Slow Expensive (10’s of millions $$) Still point-in-time
How can you get better results? Fine-tune existing model Adapt an existing model Fine-tuned models often forget or lose capabilities Reliant on the quantity and quality of training data Lacks external knowledge Still point-in-time
How can you get better results? Retrieval Augmented Generation (RAG) “Grounding” queries using your domain information with existing Models. Significantly cheaper. Retains the capabilities of existing model. Ability to change knowledge sources. No need to retrain when data changes.
Retrieval Augmented Generation (RAG)
Sitecore Search on the Developer Portal
Sitecore Search on the Developer Portal Sitecore.com Discover Sitecore YouTube channel Helix Documentation Sitecore GitHub Repositories Sitecore Stack Exchange Sitecore Community Blogs Sitecore Developer Portal Sitecore Documentation Sitecore Knowledge Base Sitecore PowerShell Documentation OrderCloud Documentation Sitecore Blok Sitecore Changelog
Hello Sitecore ChatBot! • Part of internal Sitecore AI Hackathon • Today’s focus on RAG portions of the project, but much more built into this project • Personalise – Persona based tailoring of results • CDP – tracking and governance
What should we ask? https://forms.office.com/e/G3SmLq3jgz
RAG with Sitecore Search Sitecore.com Sitecore Changelog Sitecore Developer Portal Sitecore Developer Portal Azure OpenAI
Summary • With LLM’s you need to think “probabilistically” not “deterministically” • 1 + 1 doesn’t always equal 2 • Using RAG allows you to ground queries without the need for expensive training or fine-tuning • Sitecore Search can be a great option for RAG as customers already have their data stored there.
Thank you
In this session Rob will show how you can use Retrieval Augmented Generation (RAG) to pull data from Sitecore Search and ground queries against Large Language Models (LLMs) like GPT-4. This ensures the LLM only uses contextually relevant information when generating responses, giving greater control over what is returned to the end user.
Rob will show how you can use data indexed in Sitecore Search to achieve this quickly and easily in a ChatBot implementation – it will leverage queries against Sitecore Search to ground requests made into an LLM. This will be implemented on Sitecore’s Developer Portal, using the data indexed from all Sitecore’s different web properties to provide a contextually aware ChatBot experience. We will show how you can control what data is being used to generate responses, and as such provide a much more tailored experience to the users.
The Developer Portal is an open-source project, and all of the source code shown in the demo will be publicly available to the attendees.
for free. You
can too.