Private AI at Home: Secure, Open-Source, and Fully Yours

Slide 1

Private AI at Home Secure, OpenSource And Fully Yours Paul Czarkowski DevOpsDays Chicago 2025 [jarvis]

Slide 2

3 Who am I ? Paul Czarkowski Senior Principal Cloud Services Black Belt Red Hat [Jarvis] Technologist | DevOp | SysAdmin | BBQ Enthusiast

Slide 3

Slide 4

What is Generative AI ?

Slide 5

(Multidimensional Vector Database) [napkin.ai]

Slide 6

[napkin.ai]

Slide 7

[napkin.ai]

Slide 8

Proprietary LLMs

Slide 9

$$$$$$ $$$$$$ [chatgpt]

Slide 10

The AI Broligarchy

Slide 11

Google / Facebook / Twitter / etc [napkin.ai]

Slide 12

The platform you see The actual platform [napkin.ai]

Slide 13

Slide 14

[chatgpt]

Slide 15

[napkin.ai]

Slide 16

Slide 17

Slide 18

[jarvis]

Slide 19

Generative AI [jarvis]

Slide 20

[chatgpt]

Slide 21

[chatgpt] [jarvis]

Slide 22

Open Source GenAI [jarvis]

Slide 23

[jarvis]

Slide 24

[jarvis]

Slide 25

[jarvis]

Slide 26

Slide 27

OSS* LLMs

claims of OSS may be exaggerated

Slide 28

Hardware [jarvis]

Slide 29

Hardware

Slide 30

Hardware CPU Intel AMD RPi GPU Nvidia AMD [jarvis] Unified Memory M-series Mac

Slide 31

Start with what you have… [jarvis]

Slide 32

Gaming PCs [jarvis]

Slide 33

Old DataCenter GPUs https://www.overclock.net/threads/g aming-on-a-tesla-m40-gtx-titan-x-p erformance-for-a-third-the-price.177 7611/ https://ingoeichhorst.medium.com/b uilding-a-wall-mounted-and-wallet-fr iendly-ml-rig-0683a7094704

Slide 34

Breaking news! https://wccftech.com/nvidia-rtx-4090-with-96gb-vram-reportedly-exists/

Slide 35

Macs [jarvis]

Slide 36

Breaking News https://www.apple.com/newsroom/2025/03/apple-unveils-new-mac-studio-the-most-powerful-mac-ever/

Slide 37

Operating System ? [jarvis]

Slide 38

Bazzite [jarvis] https://bazzite.gg/

Slide 39

Local AI stack [jarvis]

Slide 40

Local AI Architecture Diagram https://github.com/coleam00/local-ai-packaged

Slide 41

Open WebUI

Slide 42

Open WebUI

Slide 43

Open WebUI

Slide 44

Stable Diffusion + SwarmUI

Slide 45

WAN 2.1 + SwarmUI

Slide 46

Jarvis > an epic movie poster showing a strong woman standing victoriously on a battlefield littered with broken robots a american flag can be seen waving in the wind in the background. She is holding up a robot head in her hand and is making a peace sign with her other hand.