Empirical Study Reveals Risks in 5 Machine Learning Model Hosting Ecosystems

Researchers have uncovered significant security vulnerabilities within the rapidly expanding ecosystems of machine learning model-sharing platforms! Mohammed Latif Siddiq, Tanzim Hossain Romel, Natalie Sekerak, Beatrice Casey, Joanna C. S. Santos, and colleagues from the University of Notre Dame and IQVIA Inc, present the first large-scale empirical study into the risks associated with remote code execution during model loading, a practice enabled by features like trust_remote_code, across platforms including Hugging Face and ModelScope! Their investigation reveals a widespread reliance on potentially dangerous defaults, inconsistent security measures between platforms, and a concerning lack of awareness amongst developers regarding the implications of running untrusted code, highlighting a critical need to bolster security without sacrificing usability in these increasingly vital machine learning infrastructures.

The study reveals a widespread reliance on potentially unsafe defaults and uneven security enforcement across these platforms, creating a significant challenge for developers and users alike. This work directly addresses the growing concern that the flexibility of these ecosystems, while enabling rapid innovation, introduces a critical attack surface through the execution of arbitrary Python files during model loading, often enabled by flags like trust_remote_code or trust_repo.

The team achieved a comprehensive understanding of this threat landscape by quantifying the frequency with which models require custom code to function, identifying those that execute arbitrary Python files during the loading process. They then employed three complementary static analysis tools, Bandit, CodeQL, and Semgrep, to detect security smells and potential vulnerabilities, categorizing findings using Common Weakness Enumeration (CWE) identifiers to establish a standardized risk taxonomy. Furthermore, the researchers utilized YARA to identify malicious patterns and payload signatures within the models, providing a multi-layered approach to vulnerability detection. This rigorous analysis was coupled with a systematic examination of each platform’s documentation, API design, and safety mechanisms to evaluate their mitigation strategies and enforcement levels.

This breakthrough reveals that as of January 2026, Hugging Face alone hosts approximately 2.1 million models, highlighting the scale of the potential risk. The research establishes that current Large Language Models (LLMs), often comprising billions or even trillions of parameters, frequently introduce non-standard architectural components requiring executable code for proper function. Unlike earlier neural network architectures relying on standardized layers, these complex models necessitate the distribution of accompanying Python code, creating a dependency on potentially untrusted sources. The study unveils that this reliance on custom code extends beyond serialized data, introducing vulnerabilities from unverified source code and expanding the attack surface for malicious actors.

Experiments show that while model hubs facilitate open collaboration, they also create implicit trust relationships between users, model authors, and platforms, a trust that is often misplaced. The researchers supplemented their technical analysis with a qualitative analysis of over 600 developer discussions from platforms like GitHub, Hugging Face, PyTorch Hub, and Stack Overflow, capturing community concerns and misconceptions regarding security and usability. This holistic approach provides actionable recommendations for designing safer model-sharing infrastructures and balancing usability with robust security measures in future AI ecosystems, ultimately aiming to mitigate the risks associated with remote code execution in machine learning model hosting. . The research team conducted the first large-scale empirical study, analysing five major platforms to quantify the frequency of custom code requirements and identify models executing arbitrary Python files.

Experiments revealed that a significant number of models necessitate custom code to function, creating a pathway for malicious actors to embed harmful payloads. Data shows that these platforms often rely on unsafe defaults and exhibit uneven security enforcement, leaving developers vulnerable to remote code execution. The study meticulously applied three static analysis tools, Bandit, CodeQL, and Semgrep, detecting security smells and potential vulnerabilities categorised by Common Weakness Enumeration (CWE) identifiers. Measurements confirm the presence of numerous security smells, including instances of unhandled exceptions and the use of outdated cryptographic algorithms like MD5 (associated with CWE-327).

Furthermore, the team employed YARA to identify malicious patterns and payload signatures within the models, strengthening the assessment of potential threats. Tests proved that enabling features like trust_remote_code or trust_repo effectively grants remote repositories the ability to run arbitrary Python code on a user’s machine, a critical security concern. Researchers systematically analysed platform documentation, API design, and enforcement mechanisms, discovering inconsistencies in security measures across different ecosystems. The work identified over 600 developer discussions from platforms like GitHub, Hugging Face, and Stack Overflow, revealing persistent confusion regarding the implications of executing remote code.

Results demonstrate that over 2.1 million models are hosted on platforms like Hugging Face, with thousands added daily, making manual review impractical and necessitating automated scanning solutions. The breakthrough delivers a detailed understanding of the threat model involving model creators, platform maintainers, and model consumers, highlighting the complex trust boundaries at play. Specifically, the team observed that custom configuration classes, such as DeepseekV3Config, stored within model repositories contain Python code that executes during model loading when the trust_remote_code flag is enabled. Measurements confirm that this mechanism introduces a serious security risk, potentially allowing attackers to deliver payloads like reverse shells, keyloggers, or data exfiltration scripts. The study’s findings underscore the need for safer model-sharing infrastructures and a balanced approach between usability and security in future machine learning ecosystems. Researchers conducted a large-scale study across five major platforms, employing static analysis tools, Bandit, CodeQL, and Semgrep, to identify security flaws categorised using Common Weakness Enumeration (CWE) identifiers, alongside YARA for malicious pattern detection! The findings highlight widespread reliance on insecure defaults and inconsistent security enforcement between platforms, with many lacking robust sandboxing or verification mechanisms, a critical issue given the convergence of threats like prompt injection and supply chain attacks.

Qualitative analysis of developer discussions on platforms like GitHub and Stack Overflow revealed considerable confusion regarding the implications of executing remote code and limited adoption of safer serialization formats such as SafeTensors. Although Hugging Face has implemented automated malware scanning, the study concludes that this alone is insufficient to mitigate the risks! Acknowledging limitations, the authors note that their analysis relies on static analysis, which may not capture all runtime vulnerabilities. Future work will concentrate on developing an automated enforcement framework integrating cryptographic integrity verification with runtime isolation, alongside exploring safer alternatives to custom model loading, a crucial step towards balancing usability and security in these increasingly vital ecosystems. This research underscores the need for a holistic approach to security in model-sharing, addressing multiple threat vectors rather than focusing on isolated attack types.