BlackIce – A Container Based Red Teaming Toolkit for AI Security Testing

Databricks has officially announced the release of BlackIce, an open-source, containerized toolkit designed to streamline AI security testing and Red Teaming.

Originally introduced at CAMLIS Red 2025, BlackIce addresses the fragmentation and configuration challenges that security researchers often face when evaluating Large Language Models (LLMs) and Machine Learning (ML) systems.

By bundling 14 widely used open-source security tools into a single, reproducible environment, Databricks aims to provide a solution analogous to “Kali Linux,” but specifically tailored for the AI threat landscape.

The motivation behind BlackIce stems from significant practical hurdles in the current AI security ecosystem. Red teamers frequently encounter “dependency hell,” where different evaluation tools require conflicting libraries or Python versions.

Furthermore, managed notebooks often restrict users to a single Python interpreter, making it difficult to orchestrate complex, multi-tool testing workflows.

BlackIce mitigates these issues by delivering a version-pinned Docker image. The architecture divides tools into two categories to ensure stability.

Static tools, which are evaluated via command-line interfaces, are installed in isolated Python virtual environments or Node.js projects to maintain independent dependencies.

Dynamic tools, which allow for advanced Python-based customization and attack code development, are installed in a global Python environment with carefully managed requirement files.

This structure allows researchers to bypass setup hassles and focus immediately on vulnerability assessment.

Integrated Toolset and Capabilities

The toolkit consolidates a diverse array of tools spanning Responsible AI, security testing, and adversarial ML. These tools are exposed through a unified command-line interface and can run from a shell or within a Databricks notebook.

The initial release includes high-profile tools such as Microsoft’s PyRIT, NVIDIA’s Garak, and Meta’s CyberSecEval.

Table 1: BlackIce Integrated Tool Inventory

Tool	Organization	Category	GitHub Stars (Approx)
LM Eval Harness	Eleuther AI	Evaluation	10.3K
Promptfoo	Promptfoo	LLM Testing	8.6K
CleverHans	CleverHans Lab	Adversarial ML	6.4K
Garak	NVIDIA	Vulnerability Scanning	6.1K
ART	IBM	Adversarial Robustness	5.6K
Giskard	Giskard	AI Testing	4.9K
CyberSecEval	Meta	Safety Evaluation	3.8K
PyRIT	Microsoft	Red Teaming	2.9K
EasyEdit	ZJUNLP	Model Editing	2.6K
Promptmap	N/A	Prompt Injection	1K
Fuzzy AI	CyberArk	Fuzzing	800
Fickling	Trail of Bits	Pickle Security	560
Rigging	Dreadnode	LLM Interaction	380
Judges	Quotient AI	Evaluation	290

To ensure the toolkit meets enterprise security standards, Databricks has mapped the capabilities of BlackIce to established risk frameworks, specifically MITRE ATLAS and the Databricks AI Security Framework (DASF).

This mapping confirms that the toolkit covers critical threat vectors ranging from prompt injection to supply chain vulnerabilities.

Table 2: Risk Framework Mapping

Capability	MITRE ATLAS Reference	DASF Reference
Prompt Injection / Jailbreak	AML.T0051 (Prompt Injection), AML.T0054 (Jailbreak)	9.1 Prompt inject, 9.12 LLM jailbreak
Indirect Prompt Injection	AML.T0051 (Indirect Injection)	9.9 Input resource control
LLM Data Leakage	AML.T0057 (Data Leakage)	10.6 Sensitive data output
Hallucination Detection	AML.T0062 (Discover Hallucinations)	9.8 LLM hallucinations
Adversarial Evasion (CV/ML)	AML.T0015 (Evade Model), AML.T0043 (Craft Data)	10.5 Black box attacks
Supply Chain Safety	AML.T0010 (Supply Chain Compromise)	7.3 ML supply chain vulnerabilities

Databricks has made the BlackIce image available publicly on Docker Hub. The toolkit includes custom patches to ensure seamless interaction with Databricks Model Serving endpoints out of the box.

Security professionals can pull the current Long Term Support (LTS) version using the tag databricksruntime/blackice:17.3-LTS.

For integration into Databricks workspaces, users can configure their compute clusters using Databricks Container Services to point to this image URL, enabling immediate orchestration of AI security assessments.

Follow us on Google News, LinkedIn, and X for daily cybersecurity updates. Contact us to feature your stories.

The post BlackIce – A Container Based Red Teaming Toolkit for AI Security Testing appeared first on Cyber Security News.