DistShield: Distribution Preserving Model Obfuscation for Real-Time TEE-Shielded Secure Inference on IoT devices

While on-device inference avoids network latency and private data uploading in IoT, the risk of model thefts has raised serious concern. As a solution, state-of-the-art approach obfuscates critical parameters and shields de-obfuscate keys in TEE, ensuring model confidentiality with minimal overhead. However, we reveal that existing methods lead to anomalous clustering in distribution, undermining the anonymity of protected parameters. Based on this vulnerability, we demonstrate a model stealing attack which could recover over 97% of the model performance without any queries or training, severely compromising the model security. Additionally, existing methods struggle to scale for large language models. To address these challenges, we propose DistShield, which leverages distribution preserving obfuscation to generate obfuscated parameters with no anomaly. Moreover, to minimize the inference latency and adapt to large language models, iterative weight pruning is tailored to precisely narrow down the range of critical parameters. Experimental results demonstrate that our approach achieves robust model security by protecting only 0.013% of the parameters, leading to a 10× reduction in model stealing attack accuracy, with only 11% additional TEE computation latency introduced. DistShield provides promising obfuscation scheme against model thefts on edge, while maintaining real-time inference capabilities.