Local Ai Server Setup Guides Proxmox 9 - Llama.cpp in LXC w/ GPU Passthrough

In this Local Ai setup guide I show you how to build llama.cpp in an LXC with quad 3090s, download LLMs from unsloth hosted off Huggingface, and run both CPU and GPU inference on Qwen. I show you how to connect to your new Llama.cpp server from our OpenWEBUI container providing a nice looking interface.

*IF YOU HAVE BLACKWELL 50x0 series Nvidia GPU's select the MIT and not Proprietary version during install.*

These guides are meant to be followed in this order:
▶️ Ollama Openwebui Video    • Local Ai Server Setup Guides Proxmox 9 - O...  
📝 Ollama Openwebui Article https://digitalspaceport.com/how-to-s...
▶️ 📍YOU ARE HERE📍 Llamacpp Unsloth Video
📝 Llamacpp Unsloth Article https://digitalspaceport.com/how-to-s...
(Optional Guides)
▶️ vLLM Video    • Local Ai Server Setup Guides Proxmox 9 - L...  
📝 vLLM Article https://digitalspaceport.com/how-to-s...
▶️ VibeVoice 7b TTS Video    • Microsoft Ai VibeVoice 7b TTS Voice Genera...  
📝 VibeVoice Article https://digitalspaceport.com/how-to-s...

Quad 3090 Ryzen AI Rig Build 2025 Video w/cheaper components vs EPYC Build    • ULTIMATE Local AI Quad 3090 Build  
Written Build Guide with all the Updated AI Rig Component Options and Benchmarks https://digitalspaceport.com/local-ai...

⚙️ QUAD 3090 AI HOME SERVER BUILD
GPU Rack Frame https://geni.us/GPU_Rack_Frame
Supermicro H12ssl-i MOBO (better option vs mz32-ar0) https://geni.us/MBD_H12SSL-I-O
Gigabyte MZ32-AR0 MOBO https://geni.us/mz32-ar0_motherboard
AMD 7V13 (newer, faster vs 7702) https://geni.us/EPYC_7V13_CPU
RTX 3090 24GB GPU (x4) https://geni.us/GPU3090
256GB (8x32GB) DDR4 2400 RAM https://geni.us/256GB_DDR4_RAM
PCIe4 Risers (x4) https://geni.us/PCIe4_Riser_Cable
AMD SP3 Air Cooler (easier vs water cooler) https://geni.us/EPYC_SP3_COOLER
iCUE H170i water cooler https://geni.us/iCUE_H170i_Capellix
(sTRX4 fits SP3 and retention kit comes with the CAPELLIX)
CORSAIR HX1500i PSU https://geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4, not needed for H12SSL-i) https://geni.us/SFF8654_to_SFF8654
ARCTIC MX4 Thermal Paste https://geni.us/Arctic_ThermalPaste
Thermal GPU Pads https://geni.us/Kritical-Thermal-Pads
HDD Rack Screws for Fans https://geni.us/HDD_RackScrews

▶️ Local Ai Server Builds:
Quad 3090 Ai Server Build    • INSANE Home AI Server - Quad 3090 Build  
Playlist    • Local Ai Server Builds  

Ways to Support:
🚀 Join as a member for members-only content and extra perks    / digitalspaceport  
☕ Buy Me a Coffee https://www.buymeacoffee.com/digitals...
🔳 Patreon   / digitalspaceport  
👍 Subscribe youtube.com/c/digitalspaceport?sub_co...
🌐 Check out the Website https://digitalspaceport.com

Chapters
0:00 Llama.cpp Complete Build Guide
1:22 Install NVIDIA Toolkit on Proxmox 9 HOST
4:31 Install Llama.cpp LXC Container
6:53 Install NVIDIA Driver in Llama.cpp LXC
8:27 Install NVIDIA TOOLKIT in LXC
10:27 How to take a Backup in Proxmox 9
12:45 Build Llama.cpp in a LXC
15:30 Download a LLM from Huggingface Unsloth for Llama.cpp

*****
As an Amazon Associate I earn from qualifying purchases.
When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network.
*****