• 0 posts
  • 15 comments
Joined 10 months ago
Cake day: January 18th, 2026
  • Depends on your hardware. Apples unified memory system can give near complete access to the ram for their GPU/npu.

    Apart from that you don’t need a single bit of vram/gpu to host a model. Then can be run completely on CPU+ram. But that comes with a significant processing penalty.

    On top of that there’s new “engines” which can run a LLM from disk storage +system ram/vram, yet again with lots of speed penalities, since you chunk the model into small pieces and load them into ram accordingly. This will also cripple the lifetime of ssds with the sheer amount of read/write cycles.

    Then there’s also dedicated systems or extension cards which can be used as accelerator.

    None the less we speak about a minimum of 5000€'s for a system which can run llms and (lightweight) games simultaneously.

  • Thanks for the warning!

    Also a 27B LLM, next to games? To have it halfway functional that’s minimum 16gb of ram for a quant that’s okay, and more like 20gb+ for a quant where it doesn’t influence the models output as much. And then you’ll need an ai accelerator or the response speed will be in the 10’s of minutes. Essentially a 5k$ or more PC with current prices if you want to game on the gaming OS, lol.