Openness-aware multi-prototype learning for open set medical diagnosis

Unlike the prevalent image classification paradigm that assumes all samples belong to pre-defined classes, Open set recognition (OSR) indicates that new classes unobserved during training could appear in testing. It mandates a model to not only categorize known classes but also recognize unknowns. Existing prototype-based solutions model each class using a single prototype and recognize samples that are distant from these prototypes as unknowns. However, single-prototype modeling overlooks intra-class variance, leading to large open space risk. Additionally, open space regularization is ignored, allowing unknown samples to remain in their initial positions that overlap with the known space, thus impeding unknown discrimination. To address these limitations, we propose Openness-Aware Multi-Prototype Learning (OAMPL) with two novel designs: (1) Adaptive Open Multi-Prototype Formulation (AOMF) extends single-prototype modeling to a novel multi-prototype formulation. It reduces open space risk by simultaneously avoiding class underrepresentation and anticipating unknown occurrences. Additionally, AOMF incorporates a balancing term, a marginal factor, and a learnable scalar to flexibly fit intricate open environments. (2) Difficulty Aware Openness Simulator (DAOS) dynamically synthesizes fake features at varying difficulties to represent open classes. By punishing the adjacency between the fake and the known, the known-unknown discrimination could be enhanced. DAOS is distinguished by its joint optimization with AOMF, allowing it to cooperate with the classifier to produce samples with appropriate difficulties for effective learning. As OSR is a nascent topic in medical fields, we contribute three benchmark datasets. Compared with state-of-the-art models, our OAMPL maintains closed set accuracy and achieves improvements in OSR at about 1.5 % and 1.2 % measured by AUROC and OSCR, respectively. Extensive ablation experiments demonstrate the effectiveness of each design.