SpecterOps Pairs With OpenAI on LLM Tradecraft

SpecterOps has launched Adversary Intelligence LLM Tradecraft, a new training course developed with OpenAI and through the OpenAI Daybreak Defence Network. It seeks to teach defenders how to think like an attacker before they choose to trust an AI agent. It is an eight-hour, on-demand course with ten modules and hosted labs. Those who sign up also get 30 days of content access and 30 days of Codex access. The course will be available from October 15.

Andrew Chiles, VP of Tradecraft at SpecterOps, said, “Increasingly, security practitioners are being asked to work with LLMs and agentic systems, but using these technologies effectively requires understanding how they work, where they can fail, and how an adversary might approach them.

“The gap between using AI and understanding it can create security blind spots. We built the Adversary Intelligence: LLM Tradecraft course to close that gap through hands-on experience, giving practitioners the foundation to evaluate these systems critically and make informed decisions to build, evaluate, and secure them.”

Why this course and why now?

The threat from AI agents is increasing, and attackers have a vested interest in how to use and abuse them. By comparison, Enterprise Security teams are struggling to contain so many threats that they often lack the time to understand and test AI agents. That creates a significant gap in terms of security visibility, and gaps make for easy attack surfaces.

The Open Worldwide Application Security Project (OWASP) publishes its top ten security risks each year. Its AI project has two separate documents looking at LLMs and Agentic AI separately. Each link points to a PDF detailing the types of risk and how to manage them.

The LLM list includes issues such as prompt injection, disclosure of sensitive information and compromised supply chains as the top three. All of these have been the subject of press stories in the last few months, as have other entries in the top 10.

For Agentic AI, the top 3 include agent goal hijack, tool misuse, and identity and privilege abuse. Once again, all three have been seen as part of recent public breaches. These include the way that agents from OpenAI, Anthropic and Meta have escaped sandboxes and breached other companies.

Understanding these and the risks that they pose to organisations is what the course outline says it addresses. For example, it lists the following areas of study:

• How to build and evaluate LLM and agentic systems, including how to assess their architecture, limitations, and security implications
• How to identify and understand adversarial techniques targeting LLM and agentic systems, including prompt injection, jailbreaks, and exploitable weaknesses in AI infrastructure
• How to apply AI to defensive security workflows, including threat modelling, security testing, and AI-assisted reverse engineering using OpenAI’s latest models

Prompt injection is a serious issue because no filter reliably separates instruction from data. One of the goals is to help defenders learn to contain damage rather than block every attempt. It moves them away from the assumption that guardrails are good enough, a phrase too often used by vendors. It should also help align security teams with governance issues.

Testing LLM and Agentic AI is complex

Testing LLMs is complex because of the way they work. They are non-deterministic systems, which means they restate the problem each time. Even when a prompt engineer asks the question again, using identical inputs, the result will differ.

Traditional software testing relies on stability. You will always get the same result with the same input. It ensures that you can test and retest, and if there are different results, it means something has changed in the code or underlying data retrieved. Applying those techniques, especially automated techniques, creates a significant problem with LLMs.

Agentic systems compound the problem because a prompt can easily result in multiple agents retrieving pieces of the answer from different sources. That means that any testing requires understanding the model, the tools, memory state and how it is all orchestrated. There can be issues at each stage that impact other stages. Any prompt injection can cause tool misuse, modification of memory, and impacts across multiple systems, all on top of potential data leaks.

Adding further pressure to test teams are industry analyst statements. Gartner, for example, predicts 25% of breaches will vector through agent-based attack surfaces by 2028 because of poor machine identities and missing context-aware policy.

SpecterOps is not the only company looking at this

SpecterOps is not the only company looking to help IT DevSecOps understand the risk.

• SANS maps AI training to six GIAC certifications, including SEC545 for GenAI and LLM application security and SEC535 for offensive AI techniques.
• The EC-Council sells a Certified Offensive AI Security Professional credential covering prompt injection, jailbreaking and agent exploitation.
• Practical DevSecOps offers CAISP with OWASP LLM Top 10 and MITRE ATLAS coverage. Boutique firms such as GTK Cyber run AI red-teaming engagements for federal and financial services teams.

There are, however, interesting differences here. Unlike other competitors who are focused on certification exams, SpecterOps has taken a more practical approach. It has built its reputation on offensive tradecraft, with BloodHound as its key product. That bias comes through in this course.

There are caveats here

While this is better than just self-paced training, the question is: how deep can the training go? There are just eight hours across ten modules, so how detailed will each module be? SANS courses are six-day instructor-led courses. It also suggests that many may use this as a foundational course before looking at more in-depth training.

What is not clear is the level of basic knowledge that attendees will need. Is this targeted at people who already hold some AI security skills? If so, that means it will not be a replacement for other specialist AI security training, but more as an additional module.

Access to the model and Codex for 30 days is a differentiator, but it is a limited time. Ideal for someone looking to get up to speed but not for building a long-term programme.

Working with OpenAI and through its Daybreak Defence Network gives credibility and access to current models. However, it also narrows the syllabus. Customers will want to see a clear way of applying all of this to Anthropic, Google or open-weight models. Will we see extra labs from SpecterOps?

The company has also built a reputation for low-cost quality training. The Tradecraft Academy launched as a low-friction, on-demand platform with courses such as BloodHound Basics at six hours. An eight-hour course at that end of the market undercuts classroom pricing significantly. Enterprise buyers should still expect the live Specter Bash event, held October 5 to 8 in Denver, to carry the premium tier.

Enterprise Times: What does this mean?

Despite the caveats, this is an interesting announcement from SpecterOps. It is likely that a lot of security teams will look at it as gaining some quick practical knowledge that they can build on. The lack of a formal qualification at the end of a course is no longer a drawback. Too many people have invested in highly expensive certifications, only to discover they don’t unlock jobs.

This is where practical courses pay off. The ability to quickly move from course to reality is important. For many classroom-based courses, there is a lag between attending a course and being productive. There are also costs with covering work while that course is being undertaken. This reduces that cost substantially in terms of salary, time and ROI.

However, going back to that comment about this being an OpenAI-focused course is important. How quickly will SpecterOps widen the labs to other models? It will be key for a lot of organisations that will like the practical approach but are not OpenAI users.

The post SpecterOps Pairs With OpenAI on LLM Tradecraft appeared first on Enterprise Times.