You can load giant models onto normal RAM such as on an Epyc system but they're ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

aurareturn on Jan 30, 2025 | parent | context | favorite | on: Mistral Small 3

You can load giant models onto normal RAM such as on an Epyc system but they're still mostly bottlenecked by low memory bandwidth.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact