PrismML shrank a 27B model to just three weight values per parameter — and kept 98% of its benchmark performance intact.
Tag: research
AutoArk’s Edge0 runs a 35B model on a 24GB Mac mini by predicting which experts it will need from SSD before it needs them.