dRAE: Representation Autoencoder with Hyper-Spherical Codes
This paper develops a new method for quantizing high-dimensional visual representations into discrete codes that can be used with language models, allowing for more efficient and scalable training. Practitioners might care because it could enable the use of large language models on devices with limited memory.