How last-layer features and classifier weights organize, and what breaks when idealized assumptions fail.
Neural Collapse describes how, at the end of training, last-layer features of each class concentrate around their class mean, and the class means together with the classifier weights form a simplex equiangular tight frame (ETF). My Ph.D. work asks what happens to this geometry when idealized training assumptions are broken, and how to repair it.
Angle-only representations. The Spherization Layer (Kim & Kim, 2022) represents features using only angles so that norms carry no information, making angular similarity the sole carrier of information.
Non-simplex fixed classifiers. The Fixed Non-negative Orthogonal Classifier (Kim & Kim, 2024) induces zero-mean Neural Collapse and feature dimension separation, which helps continual and imbalanced learning.
Mixed labels and minority collapse. Mixup under class imbalance causes minority collapse not only through imbalanced singleton labels but also through mixed labels; balancing mixed labels alleviates it (Kim & Kim, 2026).
My approach follows three steps: identify a model’s failure from the perspective of representation learning, verify it theoretically and empirically, and propose a theory-grounded remedy.
References
2026
NeurIPS
Does Mixed Label Imbalance Matter to Minority Collapse in Imbalanced Learning?
Hoyong Kim and Kangil Kim
In Advances in Neural Information Processing Systems (NeurIPS), 2026
Minority collapse, where minority classes become indistinguishable, is a significant challenge in imbalanced learning. This challenge is addressed by methods such as Mixup with class-balanced sampling. Although minority collapse has been mathematically analyzed using the layer-peeled model alongside Neural Collapse, no prior work has analyzed minority collapse under Mixup, particularly from the perspective of mixed labels. We investigate this overlooked factor and raise the question: Is mixed label balance important for alleviating minority collapse? Our analysis reveals that (i) mixed labels should be balanced, and (ii) in this setting, interpreting mixed labels as singletons is beneficial. Motivated by this analysis, we propose a Balanced Mixed Label Sampler and a Mixed-Singleton classifier, which balance mixed labels and treat them as singleton labels. Through theoretical analysis and experimental results, we highlight the importance of balancing mixed labels in imbalanced learning.
@inproceedings{kim2026mixedlabel,title={Does Mixed Label Imbalance Matter to Minority Collapse in Imbalanced Learning?},author={Kim, Hoyong and Kim, Kangil},booktitle={Advances in Neural Information Processing Systems (NeurIPS)},year={2026},}
Fixed classifiers in deep networks induce neural collapse toward a simplex equiangular tight frame. We study what happens when the fixed classifier is not a simplex and introduce the fixed non-negative orthogonal classifier, which achieves global optimality in the layer-peeled model while inducing zero-mean neural collapse and feature dimension separation, with benefits for continual and imbalanced learning.
@inproceedings{kim2024fixed,title={Fixed Non-negative Orthogonal Classifier: Inducing Zero-mean Neural Collapse with Feature Dimension Separation},author={Kim, Hoyong and Kim, Kangil},booktitle={International Conference on Learning Representations (ICLR)},year={2024},}
2022
NeurIPS
Spherization Layer: Representation Using Only Angles
Hoyong Kim and Kangil Kim
In Advances in Neural Information Processing Systems (NeurIPS), 2022
We propose the spherization layer, which represents features using only angles on a hypersphere so that the norm carries no information. The layer preserves the expressiveness of conventional layers while making angular similarity the sole carrier of information, and we show its benefits across vision and language tasks.
@inproceedings{kim2022spherization,title={Spherization Layer: Representation Using Only Angles},author={Kim, Hoyong and Kim, Kangil},booktitle={Advances in Neural Information Processing Systems (NeurIPS)},volume={35},pages={16398--16410},year={2022},}
