@jdaviescoates yes, it's an associative machine with lots of repeated data which uses probabilities to pick what the next best association is.
Just like your memory, it's faulty and fades unless you do very responsible things to make sure it isn't faulty. Validate correctness.
Why they'd choose to do this in binary compute systems is beyond stupid. Like asking your storage system to retrieve a specific file and it picks another probabilistic one instead. You'd be furious! Just find the right file!
The algos need to change, and be used in the appropriate places.
Lossy is ok in certain applications, but not where precision is needed. So don't expect it in LLMs as they're designed that way.
We have too many limiting beliefs that need correcting about all sorts of things.
Pushing for precision in a lossy system is upside down.
