FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders
This paper improves the performance of representation autoencoders (RAEs) by training a decoder to work with different layers of a pre-trained visual encoder, rather than relying on a fixed layer fusion. A practitioner might care about this because it allows for more flexible and robust image generation.