A paper accepted to International Conference on Learning Representations (ICLR), 2024