Private AI at Home: Secure, Open-Source, and Fully Yours

Private AI at Home Secure, OpenSource And Fully Yours Paul Czarkowski DevOpsDays Chicago 2025 [jarvis]

3 Who am I ? Paul Czarkowski Senior Principal Cloud Services Black Belt Red Hat [Jarvis] Technologist | DevOp | SysAdmin | BBQ Enthusiast

What is Generative AI ?

(Multidimensional Vector Database) [napkin.ai]

[napkin.ai]

Proprietary LLMs

$$$$$$ $$$$$$ [chatgpt]

The AI Broligarchy

Google / Facebook / Twitter / etc [napkin.ai]

The platform you see The actual platform [napkin.ai]

[chatgpt]

[napkin.ai]

[jarvis]

Generative AI [jarvis]

[chatgpt]

[chatgpt] [jarvis]

Open Source GenAI [jarvis]

[jarvis]

OSS* LLMs

claims of OSS may be exaggerated

Hardware [jarvis]

Hardware

Hardware CPU Intel AMD RPi GPU Nvidia AMD [jarvis] Unified Memory M-series Mac

Start with what you have… [jarvis]

Gaming PCs [jarvis]

Old DataCenter GPUs https://www.overclock.net/threads/g aming-on-a-tesla-m40-gtx-titan-x-p erformance-for-a-third-the-price.177 7611/ https://ingoeichhorst.medium.com/b uilding-a-wall-mounted-and-wallet-fr iendly-ml-rig-0683a7094704

Breaking news! https://wccftech.com/nvidia-rtx-4090-with-96gb-vram-reportedly-exists/

Macs [jarvis]

Breaking News https://www.apple.com/newsroom/2025/03/apple-unveils-new-mac-studio-the-most-powerful-mac-ever/

Operating System ? [jarvis]

Bazzite [jarvis] https://bazzite.gg/

Local AI stack [jarvis]

Local AI Architecture Diagram https://github.com/coleam00/local-ai-packaged

Open WebUI

Stable Diffusion + SwarmUI

WAN 2.1 + SwarmUI

Jarvis > an epic movie poster showing a strong woman standing victoriously on a battlefield littered with broken robots a american flag can be seen waving in the wind in the background. She is holding up a robot head in her hand and is making a peace sign with her other hand.

Writing Code Assistant “Vibe Coding”

Writing Code with Roo Code

Discuss and Edit Code with Continue.dev

Voice Assistant [jarvis]

Voice Assistant

Other Things… No/Low Code Agentic Talk to your notes… CLI Magic

Conclusion

THANKS! [jarvis] https://tech.paulcz.net/local-ai/