Imbalanced and continual learning

Margin balancing, stable replay, and class-wise interference in the last layer.

Two settings where representation collapse shows up in practice:

Imbalanced learning. Minority classes suffer from collapsed margins. Asymptotic Midpoint Mixup (Kim et al., 2024) balances margins while only moderately broadening the feature distribution, and the mixed-label analysis in (Kim & Kim, 2026) explains why Mixup itself can hurt minority classes.
Continual learning. Replay-based methods are unstable when masked logits leak gradients. Revisiting softmax masking with stop-gradient (Kim et al., 2024) reduces class-wise interference in the last-layer features and classifier, mitigating catastrophic forgetting.

Earlier, I also worked on out-of-distribution detection with distance metric learning (Yang et al., 2020) and dynamic activation functions (Mai Ngoc et al., 2020).

References

2026

NeurIPS
Does Mixed Label Imbalance Matter to Minority Collapse in Imbalanced Learning?

Hoyong Kim and Kangil Kim

In Advances in Neural Information Processing Systems (NeurIPS), 2026

Abs Bib HTML Website

Minority collapse, where minority classes become indistinguishable, is a significant challenge in imbalanced learning. This challenge is addressed by methods such as Mixup with class-balanced sampling. Although minority collapse has been mathematically analyzed using the layer-peeled model alongside Neural Collapse, no prior work has analyzed minority collapse under Mixup, particularly from the perspective of mixed labels. We investigate this overlooked factor and raise the question: Is mixed label balance important for alleviating minority collapse? Our analysis reveals that (i) mixed labels should be balanced, and (ii) in this setting, interpreting mixed labels as singletons is beneficial. Motivated by this analysis, we propose a Balanced Mixed Label Sampler and a Mixed-Singleton classifier, which balance mixed labels and treat them as singleton labels. Through theoretical analysis and experimental results, we highlight the importance of balancing mixed labels in imbalanced learning.
@inproceedings{kim2026mixedlabel, title = {Does Mixed Label Imbalance Matter to Minority Collapse in Imbalanced Learning?}, author = {Kim, Hoyong and Kim, Kangil}, booktitle = {Advances in Neural Information Processing Systems (NeurIPS)}, year = {2026}, }

2024

CoLLAs
Asymptotic Midpoint Mixup for Margin Balancing and Moderate Broadening

Hoyong Kim, Semi Lee, and Kangil Kim

In Conference on Lifelong Learning Agents (CoLLAs), Workshop Track, 2024

Abs arXiv Bib PDF Video

We analyze how mixup-style augmentation shapes the margins between classes in feature space and propose asymptotic midpoint mixup, which generates samples that approach the midpoint between classes to balance margins while only moderately broadening the feature distribution.
@inproceedings{kim2024asymptotic, title = {Asymptotic Midpoint Mixup for Margin Balancing and Moderate Broadening}, author = {Kim, Hoyong and Lee, Semi and Kim, Kangil}, booktitle = {Conference on Lifelong Learning Agents (CoLLAs), Workshop Track}, year = {2024}, }
CoLLAs
Revisiting Softmax Masking: Stop Gradient for Enhancing Stability in Replay-based Continual Learning

Hoyong Kim, Minchan Kwon, and Kangil Kim

In Conference on Lifelong Learning Agents (CoLLAs), Workshop Track, 2024

Abs arXiv Bib PDF Video

In replay-based continual learning, softmax masking can destabilize training because of gradients from masked logits. We revisit softmax masking and show that applying stop-gradient to the masked terms enhances stability and reduces class-wise interference in the last-layer features and classifier.
@inproceedings{kim2024revisiting, title = {Revisiting Softmax Masking: Stop Gradient for Enhancing Stability in Replay-based Continual Learning}, author = {Kim, Hoyong and Kwon, Minchan and Kim, Kangil}, booktitle = {Conference on Lifelong Learning Agents (CoLLAs), Workshop Track}, year = {2024}, }

2020

SMA

Out-of-Distribution Detection Based on Distance Metric Learning

Donghun Yang, Iksoo Shin, Kien Mai Ngoc, Hoyong Kim, and 2 more authors

In International Conference on Smart Media and Applications (SMA), 2020

DOI Bib PDF

@inproceedings{yang2020ood,
  title = {Out-of-Distribution Detection Based on Distance Metric Learning},
  author = {Yang, Donghun and Shin, Iksoo and Mai Ngoc, Kien and Kim, Hoyong and Yu, Chanhee and Hwang, Myunggwon},
  booktitle = {International Conference on Smart Media and Applications (SMA)},
  pages = {214--218},
  year = {2020},
  doi = {10.1145/3426020.3426076},
}

SMA

DPReLU: Dynamic Parametric Rectified Linear Unit

Kien Mai Ngoc, Donghun Yang, Iksoo Shin, Hoyong Kim, and 1 more author

In International Conference on Smart Media and Applications (SMA), 2020

DOI Bib PDF

@inproceedings{maingoc2020dprelu,
  title = {DPReLU: Dynamic Parametric Rectified Linear Unit},
  author = {Mai Ngoc, Kien and Yang, Donghun and Shin, Iksoo and Kim, Hoyong and Hwang, Myunggwon},
  booktitle = {International Conference on Smart Media and Applications (SMA)},
  pages = {121--125},
  year = {2020},
  doi = {10.1145/3426020.3426049},
}