A Multi-modal Deep Learning Framework for Psoriasis Diagnosis by Integrating Unaffected Fingernail Morphology and Clinical Indicators

Xiaoyan Li

Abstract:

Psoriatic arthritis (PsA) is a complex inflamma-
tory disease and a more severe condition of psoriasis (PsO),
frequently accompanied by skin and nail lesions. Early differ-
entiation between these conditions is crucial for appropriate
treatment selection and improved patient outcomes. Therefore,
this study assesses the contribution of the unaffected fingernail
indicators and presents PsACasNet, a multi-modal cascaded
fusion neural network for effective classification of PsA out
of PsO. The framework integrates the clinical phenotypes,
protein structural features, and morphology of the unaffected
fingernails of patients. PsACasNet implements a biology-driven
cascade fusion pathway from molecular (spectral) to morpho-
logical (SEM image), and then to clinical indicators. Through
comprehensive ablation studies evaluating seven different modal
combinations, we demonstrate the effectiveness of our model,
which follows a biology-driven molecular-to-morphological-to-
clinical fusion pathway. Experimental results show that dual-
modal combinations of Clinical+Spectral and Spectral+Image
achieve optimal performance (92.0% accuracy), while spectral
features alone reach 84.0% accuracy, highlighting the discrimi-
native power of molecular characteristics. The comparison with
other traditional machine-learning methods demonstrates the
efficiency of our multi-modal fusion strategies. Moreover, through
the interpretability analysis of the model, we found that the
morphological and protein characteristics of the uninvolved nails
have made outstanding contributions to the diagnosis of PsA and
can serve as effective biomarkers to promote diagnosis.