Setting up the SCRY host today. I rarely buy high-end hardware but in this case I made an exception. ChatGPT says this should handle two concurrent Ollama users on an 8B LLM. I might run a 3B LLM (5 concurrent users) at first and I should be able to add a second NVIDIA card if necessary.
Features:
AMD Ryzen 9 9900X (12-core) Processor
NVIDIA GeForce RTX 5070, 12GB Graphics
32GB DDR5 6000MHz RGB RAM
1000 Watts 80 Plus Gold Power Supply
