Securing AI Models in Enterprise: A Sociotechnical Framework

Abstract

Artificial intelligence (AI) systems are becoming integral to enterprise operations, yet they expose organizations to novel security threats—especially when open-source models are adopted without rigorous vetting. This paper presents a sociotechnical framework that integrates technical defenses with governance, aligning NIST’s Cybersecurity Framework (CSF) and AI Risk-Management Framework (AI RMF) with ISO/IEC 27001 controls and Zero-Trust principles. Drawing on recent industry surveys, vendor tool analyses, and documented incidents of backdoored models, the framework prescribes multilayered safeguards across the AI lifecycle: secure development and supply-chain validation, adversarially robust deployment, continuous monitoring and incident response, and human-centered policies and training. The result is a defense-in-depth strategy that enables enterprises to leverage AI confidently while mitigating risks such as data poisoning, prompt injection, model theft, and the leakage of sensitive data.

Introduction

Enterprises now deploy hundreds of machine-learning (ML) and large-language models (LLMs) across critical workflows. Surveys show that nearly three-quarters of organizations expect generative-AI–driven deepfakes and other AI threats to have a “high impact” on their business (Plumb, 2024). At the same time, open-source ecosystems—while accelerating innovation—have enabled adversaries to distribute backdoored models that execute malicious code when loaded (JFrog Security Research Team, 2023). These factors expand the enterprise attack surface, demanding a holistic security strategy that integrates people, processes, and technology.

Method

A multi-pronged qualitative review was conducted:

1. Standards analysis. Core functions from NIST CSF, NIST AI RMF, and ISO/IEC 27001 were mapped to the AI model lifecycle (Identify/Govern, Protect/Map, Detect/Measure, Respond/Manage, Recover). 

2. Industry-incident synthesis. Case studies of malicious Pickle payloads on public repositories (JFrog) and escalating supply-chain breaches, as documented by Dark Reading, were analyzed to ground threat scenarios. 

3. Vendor-tool survey. Security controls recommended by Protect AI (ModelScan / Rebuff / NB Defense), Noma Security’s AI-SPM, and HiddenLayer’s runtime detection platform were reviewed to identify state-of-practice technical mitigations. 

Findings

Governance & Risk Management

Establish an AI governance board that inventories all models (first- and third-party), assigns risk classifications, and enforces Zero-Trust policies—treating every dataset, model, and API call as untrusted until validated (NIST, 2023). 

Secure Development & Supply Chain

• Model vetting: Scan all serialized artifacts with tools such as ModelScan before use; reject unsafe Pickle files or convert to SafeTensor. 

• Data provenance: Hash and version datasets; deploy anomaly detection to flag poisoning attempts.

• DevSecOps gates: Integrate notebook-scanning (NB Defense) and software-composition analysis into CI/CD pipelines. 

Technical Defenses in Production

• Adversarial robustness: Incorporate adversarial training and input-sanitization layers.

• Inference-time protections: Employ runtime monitoring (HiddenLayer) to detect model evasion and implement request-rate limiting plus output watermarking to deter model extraction. 

• Prompt-injection shields: Layer heuristic filters and secondary LLM analysis (e.g., Rebuff) on all LLM inputs/outputs. 

Monitoring & Incident Response

Extend SIEM/SOC playbooks to include AI telemetry (e.g., drift alerts, anomalous query patterns). Red-team models periodically and rehearse AI-specific incident-response scenarios, such as prompt leakage containment. 

Human Factors & Training

Update security-awareness programs for data scientists and general users: secure coding in notebooks, acceptable-use rules for generative AII services, and bias & ethics education. Clear role definitions (e.g., “model owner,” “AI incident commander”) embed accountability across teams.

Discussion

The framework illustrates that technical controls cannot succeed without sociotechnical scaffolding. For instance, a Protect AI scan is only effective if governance mandates its use; conversely, strong policy fails without automated enforcement. Integrating Zero Trust with AI security broadens the principle of “never trust, always verify” to encompass data, models, and outputs. Emerging vendor platforms (e.g., Noma’s three-tiered suite) signal ecosystem maturation, yet organizations must remain agile as attack techniques evolve (Lemos, 2025). 

Conclusion

AI delivers transformative value, but open-source models and novel attack surfaces introduce systemic risk. By embedding AI-specific controls into established frameworks (NIST CSF, AI RMF, ISO 27001) and adopting a Zero-Trust approach across the entire lifecycle, enterprises can operationalize a robust, multilayered defense. Future work should track regulatory developments (e.g., ISO/IEC 42001, EU AI Act) and advance automated red-teaming to keep pace with adversarial innovation.

Blog link https://securingintelligence.blogspot.com/2025/06/securing-ai-models-in-enterprise.html

References

Business Wire. (2023, October 5). Protect AI open sources three tools to help organizations secure AI/ML environments from threats. 

Business Wire. (2024, January 24). Protect AI announces Guardian, a secure gateway to enforce ML model security.

 

Franzen, C. (2024, October 31). Noma arrives to provide security from data storage to deployment for enterprise AI solutions. 

JFrog Security Research Team. (2023). Examining malicious Hugging Face ML models with silent backdoor. 

Lemos, R. (2025, February 14). Open source AI models: Perfect storm for malicious code, vulnerabilities.

 

National Institute of Standards and Technology. (2023). Artificial intelligence risk management framework (AI RMF 1.0). NIST Special Publication AI-100-1. 

Plumb, T. (2024, August 14). iProov: 70% of organizations will be greatly impacted by gen-AI deepfakes. VentureBeat. 

HiddenLayer. (n.d.). Security for AI: Platform overview.