KV quantization has long been available in llama.cpp | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		zozbot234 61 days ago \| parent \| context \| favorite \| on: What if AI doesn't need more RAM but better math? KV quantization has long been available in llama.cpp

chr15m 61 days ago [–]

Yes but the optimisation described has not right?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact