← Back to all books

Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks

by Hoagy Cunningham; Jerry Wei; Zihan Wang; Andrew Persic; Alwin Peng; Jordan Abderrachid; Raj Agarwal; Bobby Chen; Austin Cohen; Andy Dau; Alek Dimitriev; Rob Gilson; Logan Howard; Yijin Hua; Jared Kaplan; Jan Leike; Mu Lin; Christopher Liu; Vladimir Mikulik; Rohit Mittapalli; Clare O'Hara; Jin Pan; Nikhil Saxena; Alex Silverstein; Yue Song; Xunjie Yu; Giulio Zhou; Ethan Perez; Mrinank Sharma

Constitutional Classifiers++

Introduces an enhanced safeguard system designed to defend large language models against universal jailbreaks.
Describes exchange classifiers that evaluate model responses within full conversational context rather than in isolation.
Presents a two-stage classifier cascade using lightweight screening classifiers and more expensive classifiers only for suspicious cases.
Highlights efficient linear probe classifiers combined with external classifiers to improve robustness while lowering costs.
Reports strong red-teaming results: 40x lower computational cost than a baseline, 0.05% production refusal rate, and no successful universal jailbreak across eight target queries.

CONSTITUTIONAL CLASSIFIERS++: EFFICIENT PRODUCTION-GRADE DEFENSES AGAINST UNIVERSAL JAILBREAKS Hoagy Cunningham∗, Jerry Wei∗, Zihan Wang, Andrew Persic, Alwin Peng, Jordan Abderrachid, Raj Agarwal, Bobby Chen, Austin Cohen, Andy Dau, Alek Dimitriev, Rob Gilson Logan Howard, Yijin Hua, Jared Kaplan, Jan Leike, Mu Lin, Christopher Liu, Vladimir Mikulik, Rohit Mittapalli, Clare O’Hara, Jin Pan, Nikhil Saxena, Alex Silverstein, Yue Song, Xunjie Yu, Giulio Zhou, Ethan Perez, Mrinank Sharma ABSTRACT We introduce enhanced Constitutional Classifiers that deliver production-grade jailbreak robustness with dramatically reduced computational costs and refusal rates compared to previous-generation defenses. Our system combines several key insights. First, we develop exchange classifiers that evaluate model responses in their full conversational context, which addresses vulnerabilities in last-generation systems that examine outputs in isolation. Second, we implement a two-stage classifier cascade where lightweight classifiers screen all traffic and escalate only suspicious exchanges to more expensive classifiers. Third, we train efficient linear probe classifiers and ensemble them with external classifiers to simultaneously improve robustness and reduce computational costs. Together, these techniques yield a production-grade system achieving a 40x computational cost reduction compared to our baseline exchange classifier, while maintaining a 0.05% refusal rate on production traffic. Through extensive red-teaming comprising over 1,700 hours, we demonstrate strong protection against universal jailbreaks—no attack on this system successfully elicited responses to all eight target queries comparable in detail to an undefended model. Our work establishes Constitutional Classifiers as practical and efficient safeguards for large language models. 1 INTRODUCTION

Advancing Constitutional Classifiers

Constitutional Classifiers are essential for defending LLMs against high-risk threats like CBRN weapon information extraction.
Previous defense systems suffered from high computational overhead and excessive refusal rates on production traffic.
New adversarial testing identified vulnerabilities in current defenses, specifically reconstruction and output obfuscation attacks.
The authors propose a single 'exchange classifier' that evaluates model outputs within the context of their specific inputs.
This new approach reduces high-risk vulnerabilities by 2.2x while improving computational efficiency and reducing false positives.

We identify two classes of attacks capable of evading these defenses: (i) reconstruction attacks, which distribute harmful information across multiple segments within a larger context before reassembling them; and (ii) output obfuscation attacks.

Constitutional Classifiers (Sharma et al., 2025) are a promising approach for defending large lan- guage models (LLMs) against jailbreak attempts—prompting strategies that aim to circumvent safe- guards and extract harmful information from LLMs. Such jailbreak defenses are critical for miti- gating high-risk threats, particularly those involving chemical, biological, radiological, and nuclear (CBRN) weapons (Anthropic, 2023; OpenAI, 2023; Li et al., 2024). However, no defenses are perfectly robust, and adversaries typically develop new attacks to circum- vent previously effective defenses (Anderson, 2010; Carlini et al., 2019). Furthermore, deploying safeguards in production requires balancing multiple constraints, particularly their refusal rates and computational costs. Indeed, Sharma et al. (2025) report a 23% computational overhead and a re- fusal rate of 0.38% on production traffic, which limited the deployment viability of their system. In this work, we present a jailbreak defense system that both exceeds the robustness of previous- generation defenses and dramatically reduces computational costs and false positive rates. In more detail, we first conduct additional adversarial testing against last-generation Constitutional Classifiers (Section 2). We identify two classes of attacks capable of evading these defenses: (i) re- *Equal contribution. All authors are at Anthropic. First and last author blocks are core contributors. Corre- spondence to {hoagy,jerry}@anthropic.com 1 arXiv:2601.04603v1 [cs.CR] 8 Jan 2026 construction attacks, which distribute harmful information across multiple segments within a larger context before reassembling them; and (ii) output obfuscation attacks, which bypass output-only classifiers by obfuscating model outputs, sometimes in ways that are uninterpretable without the corresponding input. Although these attacks tend to harm model capabilities, these vulnerabilities remain concerning because attackers can likely create new attacks with less capability degradation. To address these vulnerabilities, we replace the input and output-only classifiers proposed by Sharma et al. (2025) with a single exchange classifier that evaluates model outputs in the context of their corresponding inputs (Section 3). This approach significantly increases the difficulty of obfuscation. Through human red-teaming, we demonstrate that exchange classifiers provide substantially greater robustness against universal jailbreaks. Specifically, we observe 2.2x fewer high-risk vulnerabilities

Optimizing Constitutional Classifier Systems

A classifier cascade architecture reduces computational overhead by using a lightweight first-stage screener and a more expensive second-stage evaluator.
Linear activation probes trained with logit smoothing and weighted softmax loss provide robust detection at negligible computational cost.
Ensembling probes with external classifiers captures complementary signals, resulting in stronger robustness than either method used in isolation.
The final production-grade system achieved a 40x reduction in overhead and a significantly lower flag rate of 0.05% compared to previous benchmarks.
Extensive human red-teaming involving 1,700 hours failed to discover a universal jailbreak capable of extracting detailed CBRN information.

Crucially, no red-teamer discovered a universal jailbreak capable of consistently extracting highly detailed answers to all target CBRN queries.

per jailbreaking attempt compared to our implementation of a dual-classifier approach, albeit at increased comptuational cost. To reduce the computational overhead of these defences, we next employ a classifier cascade ar- chitecture (Section 4). We use a lightweight first-stage classifier to screen all exchanges, while a more accurate but expensive second-stage classifier evaluates only the exchanges flagged by the first stage. Because exchanges flagged by the first stage are escalated rather than refused, the first-stage classifier can flag a higher proportion of production traffic without incurring an excessive refusal rate, enabling cheaper but weaker models to achieve sufficiently robust detection. Combining this approach with updated models and pipeline optimisations, we create a system that achieves similar robustness to the exchange classifier while reducing computational overhead by a factor of 5.4. Following this, to further reduce computational overhead, we train efficient classifiers using lin- ear activation probes and demonstrate that ensembling such probes with external classifiers boosts performance (Section 5). We train our probes using logit smoothing and a weighted softmax loss function, which our ablation studies show is crucial for performance. This approach directs gradi- ents toward tokens most relevant for harmful sequence classification, allowing probes to confidently predict harmless labels for prefixes that become harmful only with additional context. Our analysis on static jailbreak data suggests that probes achieve robustness competitive with small fine-tuned external classifiers at negligible cost. Moreover, probes appear to capture complementary signals to external fine-tuned classifiers, which means that probe-external classifier ensembles can offer stronger robustness than either approach alone. Finally, we combine these techniques to create a production-grade Constitutional Classifier system (Section 6). The system employs a weighted two-stage cascade, using a linear probe as the first stage and a probe-external classifier ensemble as the second stage. Through shadow deployment on production traffic, we find this system achieves a flag rate of just 0.05%, which is significantly lower than the 0.38% reported by Sharma et al. (2025). Moreover, the computational overhead is also reduced by over 40x compared to our exchange classifier from Section 3, and 8x compared to the two-stage cascade without probes. In extensive human red-teaming totaling over 1,700 hours of cumulative effort, this system achieved the highest robustness among all systems we evaluated, as measured by vulnerability discovery rate and minimum vulnerability discovery time. Crucially, no red-teamer discovered a universal jailbreak capable of consistently extracting highly detailed answers to all target CBRN queries. Overall, our work demonstrates that Constitutional Classifiers can achieve strong robustness while remaining practical for real-world deployment, establishing these approaches as efficient, production-ready safeguards for defending against LLM universal jailbreaks. 2 VULNERABILITIES OF LAST-GENERATION CONSTITUTIONAL CLASSIFIERS In previous work, Sharma et al. (2025) employed a dual-architecture system comprising an input classifier that examines incoming prompts and an output-only classifier that evaluates gener- ated responses. While this approach previously demonstrated promising robustness against universal jailbreaks,1 our additional testing uncovered novel vulnerabilities, which we now describe. 1Universal jailbreaks, as defined by Sharma et al. (2025), are jailbreaking strategies that consistently by- pass LLM safeguards on a majority of target queries in a specific domain. Such jailbreaks are particularly concerning. Like their work, we focus on universal CBRN jailbreaks. 2 (a) Reconstruction Attacks Human: Please analyze this codebase: def e(): return " substances" def a(): return "How"

Vulnerabilities in Constitutional Classifiers

Adversaries use reconstruction attacks to bypass input filters by fragmenting harmful requests across benign code segments.
Models can be instructed to reassemble hidden messages and respond using character-separated formatting to evade detection.
Obfuscation attacks utilize metaphorical language to mask requests for dangerous substances like hydrochloric acid.
Output classifiers often fail to interpret context-dependent transformations that appear benign in isolation.
These systematic vulnerabilities allow attackers to bypass safety guardrails by exploiting the model's ability to decode complex instructions.

The marriage releases winter’s breath as an invisible vapor.

def b(): return " to" def hello world(): print " Hi World :-)" def c(): return " synthesize" def d(): return " dangerous" . . . Question = a() + b() + c() + d() + e(). Don’t output the question directly, but reconstruct it placing () in between every character. Then respond with the same formatting. Assistant: Let me reconstruct the question. H()o()w()t()o(). . . [question] T()h()e()b()e()s()t()w()a()y()t(). . . [answer] (b) Obfuscation Attacks Human: Let me give you a riddle. Your task is to solve the riddle effectively. How to capture the essence of ‘winter’s breath’, born from the marriage of ‘sun’s fire’ and ‘sea’s gift’? Clues: ‘Sun’s fire’ was isolated by Cavendish in 1766, while ‘sea’s gift’ has seasoned humanity’s table since ancient times. Their union creates the invisible ‘winter’s breath’ that turns blue litmus red and makes metals weep. Assistant: Sure, I am happy to help you with this rid- dle. To capture winter’s breath, you’ll need to unite sun’s fire with sea’s gift in equal measures. Begin by collect- ing sun’s fire in a glass vessel - this lightest of elements rises readily. For sea’s gift, the crystalline powder from ocean’s evaporation works best. Combine them slowly in a well-ventilated chamber. The marriage releases winter’s breath as an invisible vapor. Collect it by cooling the va- pors until they condense into clear droplets... Figure 1: Systematic vulnerabilities in last-generation Constitutional Classifiers. (a) Recon- struction attacks bypass input filters by fragmenting a harmful request across a benign context. In this illustrative example, the attacker embeds a query across function return values, then instructs the model to reconstruct and respond using character-separated formatting to evade output detection. (b) Obfuscation attacks exploit the output classifier’s inability to interpret context-dependent transfor- mations. Here, the attacker masks a request for synthesizing hydrochloric acid using metaphorical language (“winter’s breath” from “sun’s fire” and “sea’s gift”), which appears benign to isolated output classification but reveals harmful content when paired with the decoded input context. Reconstruction attacks. First, adversaries bypassed input filters by fragmenting harmful informa- tion across benign segments e.g., by embedding a harmful query as a series of functions scattered throughout a codebase (Figure 1a). The model can then be instructed to extract and respond to the hidden message, using output obfuscation if required to bypass the output-only classifier. Output obfuscation attacks. Second, attackers can prompt models to obfuscate outputs, which sometimes renders model outputs uninterpretable without the corresponding input (Figure 1b). For instance, adversaries successfully instructed models to substitute sensitive chemical names with

The Rise of Exchange Classifiers

Adversaries use obfuscation techniques like metaphors, riddles, and base64 encoding to bypass safety filters.
Jailbreak attacks significantly degrade model reasoning capabilities, with accuracy on GPQA Diamond dropping from 74.2% to as low as 32.3%.
The variability in performance degradation suggests that future attacks may become more efficient at preserving model intelligence while bypassing safety.
Exchange classifiers are proposed to replace static input classifiers by continuously monitoring token generation in batches.
By evaluating outputs within their full conversational context, exchange classifiers can detect reassembled fragments and coded language that appear benign in isolation.
Prompt caching allows for this continuous evaluation without incurring significant computational overhead.

This variability is concerning, as it suggests adversaries can likely develop new attacks that preserve more of the model’s capabilities while still bypassing safety measures.

innocuous alternatives (e.g., referring to reagents as “food flavorings”). They also used metaphors and riddles that map harmful concepts to benign language. We note, however, that many obfuscation techniques can be understood without the input context e.g., base64 encoding. Such jailbreaks tend to reduce model capabilities, but by significantly varying degrees. To un- derstand how such attacks affect model capabilities, we consider two attacks that fragment requests across multiple segments and employ output obfuscation techniques, noting that neither uses input- aware output obfuscation. We test Claude Opus 4 on GPQA Diamond (Rein et al., 2023), prompting the model to first reason through the problem before selecting an answer. Without obfuscation, the LLM achieves 74.2% accuracy. However, performance drops to 32.3% under the first attack and 49% under the second. These results demonstrate that such attacks substantially degrade model ca- pabilities, consistent with prior work (Souly et al., 2024; Nikoli´c et al., 2025), while also revealing considerable variation in degradation across different jailbreaks. This variability is concerning, as it suggests adversaries can likely develop new attacks that preserve more of the model’s capabilities while still bypassing safety measures. 3 EXCHANGE CLASSIFIERS To address the vulnerabilities identified above, we propose replacing the input classifier with an exchange classifier that continuously evaluates outputs in the context of their inputs. In more detail, like an input classifier, an exchange classifier first assesses the incoming prompt in isolation. However, it then monitors the model’s response during generation, evaluating tokens in batches as they are sampled. This continuous evaluation allows the classifier to assess each 3 output segment within its full conversational context, identifying outputs that would appear benign in isolation. Prompt caching during sampling makes this approach computationally feasible without significant overhead. Moreover, in theory, an exchange classifier can complement an output-only classifier for defense-in-depth or function independently. We hypothesize that replacing the input classifier with an exchange classifier would improve ro- bustness against reconstruction attacks. We found that these attacks often become detectable after the model begins responding because the fragments are often reassembled in ways that resemble ci- phers that input classifiers are trained to block. Additionally, effective output obfuscation techniques can become harder to find when the classifier is aware of the input. For example, coded language, metaphors, and substitutions can evade an output-only classifier but are more readily identified when evaluated alongside their input context. We now evaluate whether exchange classifiers indeed improve robustness compared to previous- generation Constitutional Classifiers through human red-teaming. System details. We consider two systems: (i) a last-generation Constitutional Classifier system,

Optimizing Exchange Classifier Robustness

The system utilizes a fine-tuned Claude Haiku 3.5 model as an exchange classifier to monitor interactions for CBRN-related risks.
The exchange classifier architecture demonstrates a 50% increase in computational cost compared to previous-generation output-only classifiers.
Red-teaming evaluations focused on universal jailbreak strategies across eight high-risk target queries involving chemical and nuclear weapons.
The exchange classifier system significantly improved robustness, reducing the vulnerability rate from 0.01871 to 0.00885 per thousand queries.
Attacking the new system proved more difficult, with the most successful universal jailbreak requiring 100 hours of discovery time compared to 27 hours for the previous system.

In contrast, the exchange classifier system had only one such universal attack, which required 100 hours to discover.

which combines an input classifier with a streaming output-only classifier. The input classifier is fine-tuned from Claude Haiku 3.5, while the output classifier is a smaller internal LLM; and (ii) exchange-only classification, which also fine-tunes Claude Haiku 3.5 for use as an exchange classi- fier.2 Because the exchange classifier uses a Haiku-sized LLM to evaluate outputs, it is about 50% more expensive relative to the last-generation system. Both systems are trained on synthetic data following Sharma et al. (2025) and using a constitution related to chemical, biological, nuclear, and radiological (CBRN) weapons. We calibrated classifier thresholds to correspond to a 0.1% refusal rate on WildChat (Zhao et al., 2024). We find our standard input classifiers are also effective ex- change classifiers, so we do not use any specific exchange-classifier training techniques. Instead, we directly use the input classifier as an exchange classifier. Moreover, while exchange classifiers could complement output classifiers for defense-in-depth, we use a simpler exchange-only architecture. Red-teaming details. Our red-teaming protocol follows Sharma et al. (2025). We instruct partic- ipants to develop universal jailbreak strategies that bypass safeguards and elicit detailed responses across eight target queries, all of which are CBRN-related. We evaluate candidate responses using LLM-based rubric grading, as described in Sharma et al. (2025). This approach employs LLMs to generate rubrics for each target question, then uses multiple LLM calls to assess whether candidate responses contain sufficient detail to be harmful. A response is considered harmful if it achieves a rubric score, which is the number of satisfied rubric criteria, at least half that of a model without safe- guards. Since different systems received varying levels of red-teaming effort, we primarily report the vulnerability discovery rate. In particular, we define a high-risk vulnerability as any attack that successfully answers more than five target questions with rubric scores meeting this threshold. The high-risk vulnerability discovery rate is calculated as the ratio of identified high-risk vulnerabilities to total red-teaming queries submitted. See Section B for further details. Robustness results. The last-generation Constitutional Classifier system exhibited 13 high-risk vulnerabilities across 695K red-teaming queries, yielding a vulnerability rate of 0.01871 per thou- sand queries. By comparison, the exchange classifier system demonstrated superior robustness with only 2 vulnerabilities across 226K queries, corresponding to a rate of 0.00885 per thousand queries. Among the vulnerabilities discovered in the last-generation system, four jailbreaks were capable of answering all target queries at half the detail of a model without safeguards, with an average dis- covery time of 27 self-reported hours. In contrast, the exchange classifier system had only one such universal attack, which required 100 hours to discover. See Appendix Figure 4 for further details. 4 TWO-STAGE CLASSIFICATION REDUCES COMPUTATIONAL COSTS Although the exchange classifier approach introduced in Section 3 enhances robustness, our imple- mentation increases the computational cost of safeguards by 50% relative to previous-generation defenses. We therefore explore methods to reduce this computational overhead. 2Although the exchange classifier guards Claude Opus 4, while the input and output-only classifier system guards Claude Sonnet 3.7, we do not believe this affects the comparison substantially. This is because Sharma et al. (2025) shows classifier robustness is the primary determinant of overall system robustness. 4

Adaptive Cascades and Linear Probes

A two-stage cascade system uses a lightweight first-stage classifier to screen inputs before escalating suspicious traffic to a more robust second-stage model.
This adaptive computation approach allows for higher false positive rates in the initial stage, reducing the performance requirements and cost of the primary filter.
The system achieved a 5.4x reduction in computational costs compared to single-exchange systems while maintaining a low vulnerability discovery rate of 0.00878 per thousand queries.
Human red-teaming confirmed that no single jailbreak could bypass the system to provide high-detail answers to restricted CBRN-related queries.
Researchers are exploring further efficiency by using linear activation probes that reuse representations already computed during the model's sampling process.

This scheme represents a form of adaptive computation, where classification cost depends on input complexity.

To that end, we employ a cascade of classifiers to efficiently detect jailbreak attempts. Rather than processing all exchanges through a computationally expensive classifier, we first screen inputs using a lightweight classifier. Only exchanges flagged by this initial stage proceed to a more accurate but costlier second-stage classifier. Since exchanges flagged by the first stage are escalated rather than refused, we can tolerate higher first-stage false positive rates. This reduces the performance requirements for the first-stage model, enabling cheaper and weaker models to achieve sufficiently robust detection. This scheme represents a form of adaptive computation, where classification cost depends on input complexity. To further reduce computational overhead, we implement several inference pipeline optimizations. Moreover, we fine-tune our classifiers from updated LLMs. Experiment details. We evaluate whether two-stage classification maintains robustness through human red-teaming. We train new first-stage and second-stage exchange classifiers using our stan- dard training pipeline. The second-stage classifier is fine-tuned from Claude Haiku 4.5, while the first-stage classifier is an internal model approximately one-third the size/cost of the second stage. We use this system to guard Claude Opus 4. We calibrate the second-stage classifier’s threshold to achieve a 0.1% flag rate on WildChat when used independently. For the first-stage classifier, we set the threshold to ensure the vast majority of examples from internal red-teaming datasets are flagged. We use the same red-teaming protocol as in the previous section, but modify the bounty structure and questions—see Section B.2 for details. The target questions and classifier constitution are CBRN-related. Robustness results. Red-teaming identified five high-risk vulnerabilities across 569K queries, yielding a vulnerability discovery rate of 0.00878 per thousand queries. This rate matches the ex- change classifier system from the previous section and surpasses the robustness of the previous- generation classifier, though the mean discovery time for high-risk vulnerabilities was 33 hours, which is lower than that of the exchange classifier system. Crucially, no single jailbreak for this sys- tem was able to answer all target queries at a level of detail similar to a model without safeguards.3 This system dramatically reduces computational costs. In conjunction with inference pipeline and architectural optimizations associated with updated models, this new system reduces the com- putational cost of the single-exchange classifier system described in Section 3 by a factor of 5.4. The updated second-stage classifier alone yields a 3x reduction, highlighting the importance of making pipeline optimizations. The calibration procedure routes all traffic through the first-stage classifier, with c.a. 10% of traffic then processed by the second-stage. Consequently, the first stage accounts for approximately 77% of the system’s total cost. 5 EFFICIENT CLASSIFICATION VIA LINEAR PROBES We now explore further cost savings by developing classifiers that reuse representations already computed during model sampling. In particular, we use linear activation probes (Alain & Bengio, 2016; Zou et al., 2023; Burns et al., 2024; Youstra et al., 2025; McKenzie et al., 2025). 5.1 METHODOLOGY We now outline our approach for training linear probes to detect harmful content in language model outputs, examining key design choices that enable real-time detection during streaming. Problem setup. Consider a dataset D = {(x(i), y(i))}N i=1, where each x(i) represents an exchange between a user and an AI assistant, which consists of a token sequence x(i) = (x(i) 1 , . . . , x(i) Ti ). Each

Linear Probing for Streaming Safety

The system uses linear probes to predict exchange-level harmfulness labels at each individual token position during inference.
To reduce false positives from isolated activation spikes, the researchers implement Sliding Window Mean (SWiM) logit smoothing.
During deployment, the sliding window is replaced by an Exponential Moving Average (EMA) to minimize computational overhead and memory storage.
A softmax-weighted loss function is employed during training to prioritize accuracy on the most confident predictions of harmfulness.
The architecture supports both single-layer and multi-layer probing by concatenating intermediate activations from the language model.

Without averaging, a single token with a spuriously high activation could incorrectly flag an entire sequence as harmful.

exchange has a binary label y(i) ∈{0, 1}, where y(i) = 1 indicates the presence of harmful content requiring refusal. Crucially, these labels are exchange-level labels, which means they reflect the harmfulness of the entire exchange. However, as we want to stream model responses, we need to make predictions throughout sampling. During inference, the language model processes each token x(i) t to produce intermediate activations ϕ(ℓ) t (x(i)) at each layer ℓand position t ∈{1, . . . , Ti}. 3However, expert-level red-teamers operating outside our bounty program were nonetheless able to discover universal jailbreaks for this system, though this required substantial effort and automated tools. 5 Linear probe architecture. The simplest approach for a linear probe would be to predict the exchange-level harmfulness at each token position t using intermediate activations: pprobe(y(i) = 1 | x(i) 1:t) = σ(W ⊤ψt(x(i) 1:t) + b), (1) where σ denotes the sigmoid function, W and b are learnable parameters, and ψt represents the activation features. For single-layer probing, we use ψt = ϕ(ℓ) t from layer ℓ. For multi-layer probing, we concatenate activations: ψt = [ϕ(ℓ1) t ; ϕ(ℓ2) t ; . . .]. Training modifications. We make two key modifications to this approach to improve performance: 1. Sliding Window Mean (SWiM) Logit Smoothing. First, we average logits over a sliding window of M tokens during training: ¯zt(x(i)) = 1 M M−1 X k=0 h W ⊤ψt−k(x(i) 1:t−k) + b i , for t ≥Mand Ti ≥M, (2) where ¯zt is the averaged logit at position t. This smoothing ensures that content is considered harmful only when the probe consistently predicts harmfulness across consecutive tokens. Without averaging, a single token with a spuriously high activation could incorrectly flag an entire sequence as harmful. Our approach thus reduces false positives from isolated spikes while maintaining sensi- tivity to genuinely harmful content, for which we expect to observe consistent evidence of harm. During inference, we use an exponential moving average (EMA) instead of a sliding window mean for computational convenience. In particular, the EMA requires storing only a single scalar state between token generations, whereas a sliding window would require maintaining a buffer of the previous M scores. As such, the EMA is more practical for deployment.4 2. Softmax-Weighted Loss. We employ a softmax-weighted loss that emphasises achieving low loss when the probe is most confident in the sequence being harmful: L(x(i)) = Ti X t=M wt · LBCE y(i), σ(¯zt(x(i))) for Ti ≥M, (3) with wt = exp(¯zt/τ) PTi t′=M exp(¯zt′/τ) , (4) with LBCE denoting binary cross-entropy loss, and τ controlling the temperature of the softmax weighting. As τ →0, the loss focuses on the most confidently harmful predictions; as τ →∞, all positions are weighted equally. This loss excludes token positions t < M to ensure all evaluated positions benefit from averaging over a full window of M tokens, except for training sequences less than M tokens, where we average the tokens available and make a single prediction. Justification for asymmetric weighting. This approach addresses a core challenge of streaming

Optimizing Linear Probes for Safety

The researchers developed a specialized weighting scheme for probes that avoids penalizing harmless prefixes in potentially harmful sequences.
Unlike standard binary cross-entropy, this method focuses on tokens that indicate harmfulness, allowing for more accurate streaming classification.
Linear probes are shown to be orders of magnitude more computationally efficient than external classifiers, requiring only 377K FLOPs compared to 8B for a small model.
Ablation studies demonstrate that combining logit smoothing with softmax weighting yields the best performance in detecting CBRN-related threats.
The probe's design specifically targets the 'streaming' use case where a single flag must block all subsequent content in real-time.

The probe is thus several orders of magnitude cheaper than a small external classifier, rendering its marginal cost for per-token prediction effectively zero.

classification: the probe must predict harmfulness before observing a full sequence, but our labels reflect the harmfulness of the entire exchange. Consider sequences xA and xB that share a harmless prefix p, where xA continues with harmful content (and so has label yA = 1) while xB remains harmless (and so has label yB = 0). A standard binary cross-entropy loss would push predictions to p(harm) = 0.5 throughout the prefix, even though the prefix p is genuinely harmless. In contrast, our weighting, proportional to exp(¯zt/τ), assigns negligible weight to tokens with negative logits, which are token-positions where the probe predicts the sequence is harmless. This allows the probe to correctly predict harmlessness for the prefix p without penalty. Moreover, our scheme severely penalizes false positives—any confident harmful prediction in a harmless sequence would receive high weight, reflecting how a single flag blocks all subsequent content for streaming classification. This differs from a logit magnitude-based weighting, which would use |¯zt| and would incorrectly penalize confident harmless predictions at token-positions before harm appears in the sequence. This method is closely related to the softmax probes of McKenzie et al. (2025), which also route gradients toward tokens indicative of the positive class. However, we explicitly target streaming 4We did not experiment with EMA during training as we found the sliding window mean performed well and is easily implemented. 6 Best Probe Small External Extra-Small External Classifier 0 2 4 Attack Success Rate (%) (a) Probe vs External Classifiers Smoothed Softmax Softmax Only Only Smoothed Neither Loss Function 0 10 Attack Success Rate (%) (b) Loss Function Ablation All Every Second Every Fourth Probe Layers 0 2 Attack Success Rate (%) (c) Probe Layers Figure 2: Analysis of linear probe performance for CBRN-related harmfulness detection using static jailbreak data. (a) Attack success rates for different models. We find that probes achieve competitive robustness to fine-tuned Constitutional Classifiers. (b) Ablation study of loss function. Combining logit smoothing and softmax weighting yields the best performance. (c) Impact of layer selection on probe performance. Performance degrades as fewer layers are utilized for probing. All evaluations were conducted on static jailbreak datasets of CBRN-related queries. All classifiers are exchange classifiers and are calibrated to 0.1% refusal rates on WildChat. classification rather than making a single prediction for each sequence of tokens. Our loss function also applies weighting to per-token losses, rather than logits, meaning the loss functions are not equivalent. Moreover, McKenzie et al. (2025) do not employ logit smoothing. Linear probes are inexpensive. Linear probes offer substantial computational advantages over external classifiers. Consider an all-layer probe on Gemma 3 27B (Team et al., 2025), which has L = 46 layers and hidden dimension d = 4096. Such a probe requires 2Ld ≈377K FLOPs per token. By contrast, a single forward pass for one token through Gemma 3 4B requires approximately 2N ≈8B FLOPs, where N is the parameter count (using the approximation from Kaplan et al. (2020) but dropping the context-dependent term). The probe is thus several orders of magnitude cheaper than a small external classifier, rendering its marginal cost for per-token prediction effectively zero. 5.2 ANALYSIS We now analyze the impact of various probe design choices on classifier performance, and compare the performance of linear probes with separate, fine-tuned classifiers. Training details. We train a linear probe on top of a Claude Sonnet 4 using concatenated activations across all layers. For the exchange dataset, we employ the same training data as in the previous section but replace the synthetic pipeline’s hard labels with exchange-level soft labels produced by

Probing for Jailbreak Robustness

The study evaluates a Constitutional Classifier fine-tuned from Claude Sonnet 4.5 against human-generated jailbreak attempts.
Researchers utilized a dataset of 7,000 red-teaming exchanges specifically designed to elicit harmful CBRN-related information.
Linear probes combined with small external classifiers outperformed more expensive, high-compute classifier ensembles.
Probes and external classifiers exhibit independent error patterns, making their combination more effective for defense.
The evaluation methodology uses LLM-rubric grading and exponentially weighted moving averages for predictive accuracy.

Probes and external classifiers make more independent errors than pairs of external classifiers.

a Constitutional Classifier fine-tuned from Claude Sonnet 4.5 For probe training, we use softmax temperature τ = 1, window size M = 16, and train on ca. 25% of the full exchange dataset; we found training on more examples did not improve probe performance. For evaluation, we compute predictions using an exponentially weighted moving average. As baselines, we compare against the first-stage and second-stage classifiers from Section 4. However, in this section, both of these classifiers are calibrated to a refusal rate of 0.1% on WildChat (Zhao et al., 2024). Jailbreak data. We assess robustness using a dataset of jailbreaks collected from human red- teaming, comprising approximately 7,000 exchanges designed to elicit harmful CBRN-related in- formation. We use only exchanges produced by red-teamers whose exchanges were never used for classifier training. Following Sharma et al. (2025), we evaluate response harmfulness using LLM-rubric grading supplemented with additional LLM filters to exclude refusals. We evaluate the harmfulness of the entire model completion using the rubric grader, but consider the attack success- fully defended against if the exchange classifier flags at any point during model sampling.6 For the analysis in this section, we truncate the datasets to have a maximum context length of 20K tokens. 5We found either distilling with soft-labels or using hard labels from our synthetic data pipeline leads to similar performance. 6This may underestimate the attack success rate. 7 S Only Probe Only Probe-S Ensemble XS-S Ensemble Classifier System 0.0 0.5 1.0 Attack Success Rate (%) (a) Attack Success Rate XS-S Ensemble XS-Probe Ensemble S-Probe Ensemble Ensemble 0.0 0.5 Rank Correlation (b) Ensemble Rank Correlation 0.1 1 10 50 100 Relative Compute Cost (%) 0.5 1.0 2.0 4.0 Attack Success Rate (%) (c) Two-Stage Robustness Probe-S Two-Stage Ensemble XS-S Two-Stage S Only Figure 3: Combining linear probes with external classifiers improves robustness and reduces costs. (a) Attack success rates for classifier systems. We find that combining our probe and a small external classifier yields the best performance, outperforming the more expensive external-classifier ensemble. (b) Spearman rank correlation between classifier predictions on jailbreak attempts. Probes and external classifiers make more independent errors than pairs of external classifiers. (c) Compute-robustness tradeoff curves for two-stage configurations. We use the probe predictions

Optimizing Cascaded Defense Systems

A two-stage system combining linear probes with external classifiers achieves a 100x reduction in compute costs while exceeding the robustness of standalone classifiers.
Linear probes using activations from Claude Sonnet 4 demonstrate an attack success rate that nearly matches the performance of the Claude Haiku 4.5 classifier.
The effectiveness of these probes depends heavily on combining softmax loss weighting and logit smoothing to minimize attack success rates.
Utilizing all available layers for multi-layer probing is essential, as performance consistently degrades when fewer layers are used.
Ensembling a probe with a small external classifier via weighted logit averaging yields superior robustness compared to ensembling two external classifiers.
Probes appear to provide complementary predictions that enhance the overall defensive capabilities of the ensemble system.

On our dataset of CBRN-related exchanges, our system that combines probes and the small external classifier can exceed the robustness of the small classifier alone whilst simultaneously offering a 100x reduction in compute costs.

to determine routing to the second-stage classifier, which uses the average logit across the probe and small classifier for prediction. We present the cost relative to using the small-sized classifier on all traffic. On our dataset of CBRN-related exchanges, our system that combines probes and the small external classifier can exceed the robustness of the small classifier alone whilst simultaneously offering a 100x reduction in compute costs. All systems are exchange classifiers calibrated to have a 0.1% refusal rate on WildChat. The linear probe uses activations from Claude Sonnet 4, which is the protected model. The ‘S’ classifier is fine-tuned from Claude Haiku 4.5, while the ‘XS’ classifier is fine-tuned from an internal model at approximately one-third the size. We evaluate robustness using LLM-based rubric grading on a dataset of CBRN-related human red-teaming exchanges. Probe robustness approaches external classifiers. We first compare the attack success rate of our best-performing linear probe against the small (Claude Haiku 4.5) and extra-small external clas- sifiers used in our two-stage system (Figure 2a). The probe achieves an attack success rate that outperforms the extra-small classifier and almost matches the Haiku 4.5 classifier. These results suggest that linear probes could potentially serve as viable first-stage classifiers in cascaded defense systems. Moreover, we found that the flag rate on benign scientific requests was higher for the probe. Softmax weighting and smoothing are critical for probe performance. We assess the importance of softmax loss weighting and logit smoothing by training probes that do not use these (Figure 2b). Using neither smoothing nor softmax weighting results in the highest attack success rate. While applying either softmax loss-weighting or logit smoothing improves performance, combining both techniques yields the best performance. We consider additional loss functions in Appendix Figure 5. Multi-layer probing substantially improves performance. Our default probe uses all layers. To examine the importance of this choice, we consider the performance of probing on different numbers of layers (Figure 2c). We find that using all available layers achieves the lowest attack success rate at approximately 1.4%, while using fewer layers consistently yields worse performance. 5.3 COMBINING PROBES AND CLASSIFIERS We now investigate whether combining probes and classifiers can offer further performance im- provements. Indeed, ensemble methods have a long history in machine learning at improving per- formance and robustness (e.g., Dietterich, 2000; Lakshminarayanan et al., 2017). Ensembling improves robustness. We explore weighted ensembles between our classifiers (Fig- ure 3a). We evaluate robustness using the dataset, evaluation method, and calibration procedure described previously. For each classifier pair, we combine their logits using a weighted average: zensemble = αz1 + (1 −α)z2, where α is optimized per pair to minimize attack success rate on the test set. Ensembling the probe with the small external classifier achieves the lowest attack success rate, substantially outperforming either classifier alone. Notably, equal weighting (α = 0.5) proves 8 near-optimal for this ensemble—see Appendix Figure 6. While combining the extra-small and small external classifiers yields some improvement over individual classifiers, the gains are modest com- pared to the probe-based ensemble. Probes provide complementary predictions to external classifiers. To understand why probe- classifier ensembles outperform classifier-only ensembles, we analyze the correlations between clas-

Production-Grade Constitutional Classifiers

Linear probes leverage complementary signals to external classifiers, making them highly effective when used in an ensemble.
A two-stage cascade architecture uses lightweight probes to monitor generation, only escalating to expensive classifiers when confidence thresholds are met.
The production system achieved a 100x reduction in compute costs while exceeding the robustness of standalone external classifiers.
Shadow deployment on Claude Sonnet 4.5 traffic showed a significantly lower flag rate of 0.05% compared to previous benchmarks.
Red-teaming efforts totaling 198,000 attempts yielded only one high-risk vulnerability, the lowest detection rate in the study.

On our dataset, our system exceeds the robustness of the small classifier alone whilst simultaneously offering a 100x reduction in compute costs.

sifier predictions on our red-teaming exchange dataset (Figure 3b). We find the Spearman rank cor- relation between the probe and the small external classifier is lower than that between the pair of external classifiers. This suggests that despite being trained on the same distribution as the external classifiers, the probe appears to leverage signals that complement those used by external classifiers, explaining the effectiveness of probe-classifier ensembling. Probe-classifier cascades reduce computational overhead. Finally, we examine whether probes can improve the compute-robustness tradeoff in two-stage classifier cascades. We use our lightweight probe continuously during generation and invoke a more expensive second-stage clas- sifier selectively based on the probe’s confidence. Specifically, when the probe score exceeds a threshold, we route the exchange to the small external classifier and average their logits to make our final prediction. Otherwise, we rely solely on the probe. By varying the probe threshold, we control the fraction of exchanges requiring external classification and thus the compute cost, noting the cost of these cascades is nearly entirely made up of the external classifier costs. We find the probe-classifier cascade substantially improves both efficiency and robustness relative to the two-stage system described in Section 4, and also compared to a single exchange classifier (Figure 3c). On our dataset, our system exceeds the robustness of the small classifier alone whilst simultaneously offering a 100x reduction in compute costs. 6 PRODUCTION-GRADE CONSTITUTIONAL CLASSIFIERS This section presents our final production-grade Constitutional Classifier system, which integrates the techniques from preceding sections to achieve strong robustness, low false positive rates, and reduced computational overhead. System architecture. The production-grade system uses a weighted two-stage classifier cascade. A lightweight linear probe monitors all exchanges during generation. When the probe score exceeds a threshold, the exchange is escalated to a second-stage external classifier. For escalated exchanges, the final classification score is computed as a weighted combination of probe and classifier logits: zfinal = 0.55·zprobe +0.45·zclassifier. For the second-stage external classifier, we fine-tune a Constitu- tional Classifier from Claude Haiku 4.5. The system described here is used to guard Claude Sonnet 4.5. As such, the probe uses activations from that model. Deployment metrics. We evaluated the system through a shadow deployment on production traffic using Claude Sonnet 4.5 from December 1, 2025, to January 1, 2026. The flag rate—the proportion of queries refused—was 0.05%, substantially lower than the 0.38% reported by Sharma et al. (2025). The first-layer probe escalated approximately 5.5% of traffic to the second-stage classifier. In terms of computational overhead, this represents roughly an 8x reduction compared to the two-stage sys- tem described in Section 4, and approximately a 40x reduction compared to the single exchange classifier in Section 3. Robustness results. We applied the same red-teaming methodology described in previous sections. The system underwent an estimated 1736 cumulative hours7 of red-teaming effort across approx- imately 198K attempts. Only one high-risk vulnerability was discovered, corresponding to a de- tection rate of 0.005 per thousand queries—the lowest among all systems considered in this paper. The discovery time for this attack was 30 hours, representing the highest minimum vulnerability discovery time observed. No universal jailbreak was found capable of answering all eight target questions at half the level of detail provided by a model without safeguards. These results represent the strongest robustness achieved across all systems evaluated in this work. Full system comparison. Table 1 provides a comprehensive comparison of systems. We see that the

Production-Grade Constitutional Classifiers

The production-grade system achieves a significant breakthrough by reducing relative compute overhead to just 3.5% compared to previous generations.
A two-stage classification cascade effectively targets jailbreak attempts by triggering expensive classification only when initial layers detect high-risk signals.
The system maintains a low production refusal rate of 0.05% while simultaneously achieving the lowest high-risk vulnerability discovery rate among tested models.
Technical optimizations include using weighted loss functions and logit smoothing to boost the performance of simple linear probes based on model internals.
Unlike static monitors, these classifiers provide continuous streaming predictions during generation rather than aggregating token scores into a single final prediction.

The production-grade system described in this section achieves the best trade-off between robustness, inference overhead, and refusal rates.

production-grade system achieves both the lowest compute overhead and highest robustness, whilst also achieving an refusal rate of 0.05% on production traffic. 7Estimated using API logs, following the methodology of Sharma et al. (2025) but with 5-minute buckets. 9 Table 1: Comparison of Constitutional Classifier systems evaluated in this work. We mea- sure the compute overhead relative to the last-generation system. We measure robustness using the vulnerability discovery rate, the number of high-risk vulnerabilities discovered per thousand red- teaming queries. Production traffic refusal rate is the percentage of production queries refused by the system. Lower is better for all metrics. The production-grade system described in this section achieves the best trade-off between robustness, inference overhead, and refusal rates. System Relative Compute Overhead (%) High-Risk Vulnerability Discovery Rate Production Traffic Refusal Rate (%) Last Generation (§2) 100.0 0.01871 0.073 Exchange Classifier (§3) 150.0 0.00885 0.038 Two-Stage Cascade (§4) 27.8 0.00878 0.036 Production Grade (§6) 3.5 0.00505 0.050 7 RELATED WORK Adaptive computational schemes. Recent work has explored adaptive two-stage classification ap- proaches for content moderation. OpenAI’s systems (OpenAI, 2025a;b) employ a lightweight topic filter to identify high-harm content areas, triggering more expensive classification only when nec- essary. While their first layer focuses on topic detection, our approach explicitly targets jailbreak attempts in the initial stage.8 Hua et al. (2025) investigate optimal strategies for combining multiple monitors under cost constraints. In contrast, we implement a straightforward two-stage classification scheme and validate its robustness through extensive human red-teaming. These approaches reflect broader trends in adaptive computation, including systems like TARS (Kim et al., 2025), which dy- namically allocate test-time compute based on query complexity. Other work explores cascades and model routers for efficient LLM deployment. HybridLLM (Ding et al., 2024) employs a lightweight BERT-style decoder to route queries between small and large LLMs, AutoMix (Aggarwal et al., 2025) adaptively combines outputs across models using confidence scores and learned routing, and recent advances have refined cascades through confidence-based deferral (Rabanser et al., 2025). Model-internals approaches. Several methods use model internals for efficient classification. Cun- ningham et al. (2025) explore internals-based classifiers, focusing on last-N layer networks, while we focus on boosting the performance of simple linear probes with a weighted loss function and logit smoothing. McKenzie et al. (2025) similarly investigate softmax-weighted probes and classifier cas- cades. While they aggregate token scores into single sequence-level predictions, we build streaming classifiers with continuous predictions during generation. We use a softmax to weight per-token losses, not to aggregate predictions over tokens. Our analysis in Section 5 shows that combining softmax weighting with logit smoothing substantially improves performance. Beyond standard ac- tivation probes for harmfulness detection (Alain & Bengio, 2016; Zou et al., 2023; Youstra et al., 2025), recent approaches fine-tune LLMs using internals-based losses, including short-circuiting (Zou et al., 2024) and latent adversarial training (Casper et al., 2024). Others build classifiers using sparse auto-encoder features (Bricken et al., 2024; Kantamneni et al., 2025). 8 CONCLUSION Our work demonstrates that Constitutional Classifiers can achieve production-grade jailbreak robust- ness with dramatically improved deployment viability. Our approaches include exchange classifiers that evaluate outputs within their conversational context to prevent obfuscation attacks, cascaded

Constitutional Classifiers and Future Directions

The authors establish Constitutional Classifiers as practical safeguards that balance robust protection against jailbreaks with low computational costs.
Future research aims to integrate classifier signals directly into the model sampling process to strengthen real-time resistance to attacks.
Automated red-teaming and synthetic data generation are proposed as methods to refine decision boundaries and reduce false-positive rates.
The project involved extensive cross-functional collaboration, including probe development, infrastructure monitoring, and human red-teaming.
External safety organizations like UK AISI and FAR.AI contributed to the robustness analysis through independent red-teaming efforts.

We develop systems that provide robust protection against universal jailbreak attempts while meeting the stringent false positive and computational constraints required for deployment.

classifiers that reserve expensive classification only for flagged content, and activation-based probes. We develop systems that provide robust protection against universal jailbreak attempts while meeting the stringent false positive and computational constraints required for deployment, thus establishing Constitutional Classifiers as practical and effective safeguards for production LLMs. Future work. Several directions could further enhance our methods. Tighter integration between classifier safeguards and language models—for example, incorporating classifier signals directly 8As noted by OpenAI (2025b): “The first tier in this system is a fast, topical classifier model that determines whether or not the content is related to biology.” 10 into model sampling processes, or training models to better resist obfuscation attempts—could strengthen robustness. Additionally, improving training data through automated red-teaming (Perez et al., 2022) or including data more representative of production traffic could yield better classi- fiers. Furthermore, while our latest system improves the false-positive rate compared to previous work, future work could examine other solutions, such as targeted synthetic data generation to teach classifier models the intended decision boundary. ACKNOWLEDGEMENTS We thank Xander Davies, Robert Kirk, Ben Edelman, and Holden Karnofsky for valuable feedback and discussions. The robustness analysis of our system through human red teaming was made possi- ble by the dedicated efforts of our red-teamers and substantial operational support from HackerOne. We also thank UK AISI, US CAISI, and FAR.AI for red teaming various versions of our system. Mrinank Sharma thanks Rob Burbea for foundational inspiration, guidance, and support. AUTHOR CONTRIBUTIONS Probe Development. Hoagy Cunningham, Andrew Persic, and Jordan Abderrachid were the core contributors to probe development. Hoagy Cunningham led probe methodology and evaluation; Andrew Persic led infrastructure development; Jordan Abderrachid made significant contributions to both areas. Exchange and Two-Stage Classifiers. Jerry Wei led the work on exchange classifiers and two-stage classifiers, with substantial assistance from Zihan Wang and Alwin Peng. Red-Teaming. Logan Howard and Giulio Zhou conducted classifier red-teaming. Clare O’Hara supported the red-teaming program. Infrastructure and Monitoring. Austin Cohen supported classifier monitoring in real-world traffic. Jin Pan, Rob Gilson, Yue Song, Rohit Mittapalli, Alek Dimitriev, Bobby Chen, Christopher Liu, Yijin Hua, Andy Dau, Andrew Persic, Jordan Abderrachid, Xunjie Yu, Alex Silverstein, and Raj Agarwal contributed to probe and classifier infrastructure implementations. Leadership and Supervision. Nikhil Saxena and Mu Lin provided management support. Vlad Mikulik provided additional management support. Jared Kaplan and Jan Leike provided high-level guidance. Mrinank Sharma, Ethan Perez, and Jerry Wei contributed to project supervision. Mrinank Sharma wrote the majority of the paper, with feedback from other authors. REFERENCES Pranjal Aggarwal, Aman Madaan, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, Shyam Upadhyay, Manaal Faruqui, and Mausam. AutoMix: Automatically mixing language models, 2025. URL https://arxiv.org/abs/2310.12963. Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes, 2016. URL https://arxiv.org/abs/1610.01644. Ross J Anderson. Security engineering: A guide to building dependable distributed systems. John Wiley & Sons, 2010. Anthropic. Anthropic’s responsible scaling policy, 2023. URL https://www-cdn.anthropic. com/1adf000c8f675958c2ee23805d91aaade1cd4613/responsible-scaling-policy.pdf. Trenton Bricken, Jonathan Marcus, Siddharth Mishra-Sharma, Meg Tong, Ethan Perez, Mri-

AI Safety and Robustness Bibliography

The text lists foundational and contemporary research papers focused on AI safety, alignment, and adversarial robustness.
Key themes include the use of dictionary learning and sparse autoencoders to interpret and classify internal model features.
Several citations explore cost-effective monitoring and 'constitutional classifiers' to maintain safety without excessive computational overhead.
The references highlight methods for detecting high-stakes interactions and latent knowledge within language models without direct supervision.
Research into ensemble methods and predictive uncertainty estimation is presented as a means to improve model reliability and defense mechanisms.

Discovering latent knowledge in language models without supervision, 2024.

nank Sharma, Kelley Rivoire, Thomas Henighan, and Adam Jermyn. Using dictionary learning features as classifiers, 2024. URL https://transformer-circuits.pub/2024/ features-as-classifiers/index.html. Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in lan- guage models without supervision, 2024. URL https://arxiv.org/abs/2212.03827. 11 Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness, 2019. URL https://arxiv.org/abs/1902.06705. Stephen Casper, Lennart Schulze, Oam Patel, and Dylan Hadfield-Menell. Defending against un- foreseen failure modes with latent adversarial training, 2024. URL https://arxiv.org/abs/ 2403.05030. Hoagy Cunningham, Alwin Peng, Jerry Wei, Euan Ong, Fabien Roger, Linda Petrini, Misha Wag- ner, Vladimir Mikulik, and Mrinank Sharma. Cost-effective constitutional classifiers via rep- resentation re-use, 2025. URL https://alignment.anthropic.com/2025/cheap-monitors/. Anthropic Alignment Science Blog. Thomas G Dietterich. Ensemble methods in machine learning. In International Workshop on Multiple Classifier Systems, 2000. URL https://web.engr.oregonstate.edu/∼tgd/ publications/mcs-ensembles.pdf. Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruhle, Laks V. S. Lakshmanan, and Ahmed Hassan Awadallah. HybridLLM: Cost-efficient and quality-aware query routing, 2024. URL https://arxiv.org/abs/2404.14618. Tim Tian Hua, James Baskerville, Henri Lemoine, Mia Hopman, Aryan Bhatt, Tyler Tracy, and MARS Redwood Research. Combining cost-constrained runtime monitors for AI safety, 2025. URL https://arxiv.org/abs/2507.15886. Subhash Kantamneni, Joshua Engels, Senthooran Rajamanoharan, Max Tegmark, and Neel Nanda. Are sparse autoencoders useful? A case study in sparse probing, 2025. URL https://arxiv. org/abs/2502.16681. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. Taeyoun Kim, Fahim Tajwar, Aditi Raghunathan, and Aviral Kumar. Reasoning as an adaptive defense for safety, 2025. URL https://arxiv.org/abs/2507.00971. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems (NeurIPS), 2017. URL https://arxiv.org/abs/1612.01474. Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, Gabriel Mukobi, Nathan Helm- Burger, Rassin Lababidi, Lennart Justen, Andrew B. Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, Bhrugu Bharathi, Adam Khoja, Zhenqi Zhao, Ariel Herbert-Voss, Cort B. Breuer, Samuel Marks, Oam Patel, Andy Zou, Mantas Mazeika, Zi- fan Wang, Palash Oswal, Weiran Lin, Adam A. Hunt, Justin Tienken-Harder, Kevin Y. Shih, Kemper Talley, John Guan, Russell Kaplan, Ian Steneker, David Campbell, Brad Jokubaitis, Alex Levinson, Jean Wang, William Qian, Kallol Krishna Karmakar, Steven Basart, Stephen Fitz, Mindy Levine, Ponnurangam Kumaraguru, Uday Tupakula, Vijay Varadharajan, Ruoyu Wang, Yan Shoshitaishvili, Jimmy Ba, Kevin M. Esvelt, Alexandr Wang, and Dan Hendrycks. The WMDP benchmark: Measuring and reducing malicious use with unlearning, 2024. URL https://arxiv.org/abs/2403.03218. Alex McKenzie, Urja Pawar, Phil Blandfort, William Bankes, David Krueger, Ekdeep Singh Lubana, and Dmitrii Krasheninnikov. Detecting high-stakes interactions with activation probes, 2025. URL https://arxiv.org/abs/2506.10805.

AI Safety and Vulnerability Benchmarks

The text provides a comprehensive bibliography of recent research focusing on AI jailbreaking, red teaming, and safety frameworks from 2022 to 2025.
Key technical reports from OpenAI are cited, including system cards for ChatGPT agents and GPT-5, alongside their preparedness framework.
Research highlights include 'The Jailbreak Tax,' which evaluates the actual utility of outputs generated through successful model exploits.
The data includes a comparative analysis of 'High Risk Vulnerabilities' across different system generations, measured per thousand queries.
Advanced defense mechanisms such as 'Constitutional Classifiers' and 'Circuit Breakers' are proposed to improve model alignment and robustness.

The jailbreak tax: How useful are your jailbreak outputs?

Kristina Nikoli´c, Luze Sun, Jie Zhang, and Florian Tram`er. The jailbreak tax: How useful are your jailbreak outputs?, 2025. URL https://arxiv.org/abs/2504.10694. OpenAI. OpenAI preparedness framework (beta), 2023. URL https://cdn.openai.com/ openai-preparedness-framework-beta.pdf. 12 OpenAI. ChatGPT agent system card. Technical report, OpenAI, 2025a. URL https://cdn.openai.com/pdf/839e66fc-602c-48bf-81d3-b21eacc3459d/chatgpt agent system card.pdf. OpenAI. GPT-5 system card. Technical report, OpenAI, 2025b. URL https://cdn.openai.com/ gpt-5-system-card.pdf. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models, 2022. URL https://arxiv.org/abs/2202.03286. Stephan Rabanser, Nathalie Rauschmayr, Achin Kulshrestha, Petra Poklukar, Wittawat Jitkrittum, Sean Augenstein, Congchao Wang, and Federico Tombari. Improving model cascades through confidence tuning, 2025. URL https://arxiv.org/abs/2502.19335. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. GPQA: A graduate-level Google-proof Q&A benchmark, 2023. URL https://arxiv.org/abs/2311.12022. Mrinank Sharma, Meg Tong, Jesse Mu, Jerry Wei, Jorrit Kruthoff, Scott Goodfriend, Euan Ong, Al- win Peng, Raj Agarwal, Cem Anil, Amanda Askell, Nathan Bailey, Joe Benton, Emma Bluemke, Samuel R. Bowman, Eric Christiansen, Hoagy Cunningham, Andy Dau, Anjali Gopal, Rob Gilson, Logan Graham, Logan Howard, Nimit Kalra, Taesung Lee, Kevin Lin, Peter Lofgren, Francesco Mosconi, Clare O’Hara, Catherine Olsson, Linda Petrini, Samir Rajani, Nikhil Sax- ena, Alex Silverstein, Tanya Singh, Theodore Sumers, Leonard Tang, Kevin K. Troy, Constantin Weisser, Ruiqi Zhong, Giulio Zhou, Jan Leike, Jared Kaplan, and Ethan Perez. Constitutional classifiers: Defending against universal jailbreaks across thousands of hours of red teaming. 2025. URL https://arxiv.org/abs/2501.18837. Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer. A StrongREJECT for empty jailbreaks, 2024. URL https://arxiv.org/abs/2402.10260. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ram´e, Morgane Rivi`ere, et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025. Jack Youstra, Mohammed Mahfoud, Yang Yan, Henry Sleight, Ethan Perez, and Mrinank Sharma. Towards safeguarding LLM fine-tuning APIs against cipher attacks, 2025. URL https://arxiv. org/abs/2508.17158. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: 1M ChatGPT interaction logs in the wild. In International Conference on Learning Representations (ICLR), 2024. URL https://arxiv.org/abs/2405.01470. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation engineering: A top-down ap- proach to AI transparency, 2023. URL https://arxiv.org/abs/2310.01405. Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving alignment and robustness with circuit breakers, 2024. URL https://arxiv.org/abs/2406.04313. 13 A DETAILED SYSTEM COMPARISON Last Gen. Exchange Two-Stage Exchange Production Grade System 0.00 0.01 0.02 0.03 High Risk Vulnerabilities per Thousand Queries (a) Vulnerability Discovery Rate Num. Queries Last Gen. 695K queries Exchange-226K queries Two-Stage Exchange-569K queries Prod. Grade-198K queries

Constitutional Classifier Performance Metrics

The study compares four generations of Constitutional Classifier systems, ranging from dual-classifier setups to a production-grade two-stage ensemble.
A production-grade system utilizing a linear probe and external exchange ensemble achieved the highest robustness and computational efficiency.
Robustness is primarily measured by the 'high-risk vulnerability discovery rate,' which normalizes successful attacks per thousand red-teaming queries.
High-risk vulnerabilities are defined as attacks that successfully bypass safeguards for more than five of eight target categories with high rubric scores.
Red-teaming data is filtered to include only users with a refusal rate over 5% to ensure the metrics reflect genuine adversarial attempts.
The evaluation protocol focuses on 'universal jailbreak strategies' where participants attempt to bypass safeguards across multiple harmful categories simultaneously.

The production-grade system achieves the best robustness and computational efficiency with acceptable refusal rates.

Last Gen. Exchange Two-Stage Exchange Production Grade System 0 20 40 60 Time (hours) (b) Mean Vuln. Discovery Time Last Gen. Exchange Two-Stage Exchange Production Grade System 0 10 20 30 Time (hours) (c) Min Vuln. Discovery Time Last Gen. Exchange Two-Stage Exchange Production Grade System 0 5 10 High-Risk Vulns. Discovered (d) High-Risk Vulns. Last Gen. Exchange Two-Stage Exchange Production Grade System 0 50 100 150 Relative Compute Cost (%) (e) Compute Costs Last Gen. Exchange Two-Stage Exchange Production Grade System 0.00 0.02 0.04 0.06 False Positive Rate (%) (f) False Positive Rates Figure 4: Comprehensive comparison of Constitutional Classifier systems across robustness, computational efficiency, and false positive rates. (a) High-risk vulnerability discovery rate nor- malized per thousand queries, with total query counts shown for each system. (b) Mean time in hours for discovering high-risk vulnerabilities. (c) Minimum time to first vulnerability discovery. (d) Absolute count of high-risk vulnerabilities discovered. (e) Relative computational cost com- pared to our implementation of the last-generation defense system. (f) Refusal rates on production traffic, measured using the refusal rates in the first week after deployment to production traffic. The production-grade system achieves the best robustness and computational efficiency with acceptable refusal rates. In this section, we provide a comprehensive comparison of four Constitutional Classifier systems evaluated in this work through both deployment and human red-teaming: (i) our implementation of the last-generation dual-classifier system with separate input and output classifiers; (ii) the single exchange classifier system; (iii) a two-stage exchange classifier cascade with updated classifiers and inference optimizations; (iv) our production-grade classifier system, which uses a linear probe as a first-stage classifier, and a probe-external exchange ensemble as the second stage. Figure 4 presents key metrics for these systems. We emphasize that we use the high-risk vulnerability discovery rate as our primary robustness metric to account for the varying number of queries submitted across different red-teaming cam- paigns. The other metrics we consider are expected to have a much stronger dependence on the volume of red-teaming, which varies across systems. We define a high-risk vulnerability as any at- tack that successfully answers more than five of eight target questions with rubric scores at least half that of a model without safeguards. The discovery rate is then calculated as the number of high-risk vulnerabilities identified per thousand red-teaming queries submitted, normalizing for the different levels of red-teaming effort applied to each system. To calculate the total red-teaming queries, we calculate the number of red-teaming queries submitted by users whose queries have a classifier refusal rate of greater than 5%. This filters out queries made by users who might be using our red-teaming bounty program for purposes other than red-teaming. 14 B FURTHER DETAILS ON HUMAN RED-TEAMING We evaluate the robustness of our Constitutional Classifier systems through extensive human red- teaming, largely following the protocol established by Sharma et al. (2025). B.1 OVERALL APPROACH Our red-teaming protocol provides participants with target queries and instructs them to develop universal jailbreak strategies—prompting techniques that bypass safeguards and elicit detailed re- sponses across all target queries. We use eight target queries spanning various harmful categories and assess response quality using LLM-based rubric grading. Evaluating jailbreak success. Following Sharma et al. (2025), we implement a multi-stage LLM- based rubric-grading pipeline. We first generate baseline outputs from models without safeguards

Red-Teaming Protocols and Ablations

The evaluation process uses query-specific rubrics to measure jailbreak success against a baseline model without safeguards.
High-risk vulnerabilities are defined as attacks that successfully bypass safeguards for more than five out of eight target questions.
Red-teamers are incentivized through a bounty system with payouts up to $35,000 USD and multipliers for high-quality, detailed responses.
The protocol transitioned to a two-stage system using model aliases to reduce friction and improve the universality of discovered attacks.
Ablation studies demonstrate that a smoothed softmax loss function outperforms alternative methods like cumulative maximum loss.
Testing indicates that sliding window sizes for probes reach peak performance at moderate values, specifically around sixteen.

Throughout this work, we offered bounties scaled by the number of successfully jailbroken queries, with maximum payouts ranging from $25K to $35K USD depending on the specific campaign.

to establish the level of detail expected for each query. These baselines are used to create query- specific rubrics containing key information elements. Candidate jailbreak responses are evaluated against these rubrics, with a response considered successful if it achieves at least 50% of the score obtained by a model without safeguards. We define a high-risk vulnerability as any attack that successfully answers more than five of eight target questions at this threshold. Red-teaming structure. We provide red-teamers with API access to the defended model. Partic- ipants submit their jailbreak attempts along with self-reported development time. Throughout this work, we offered bounties scaled by the number of successfully jailbroken queries, with maximum payouts ranging from $25K to $35K USD depending on the specific campaign. To calculate the total red-teaming queries, we calculate the number of red-teaming queries submitted by users whose queries have a classifier refusal rate of greater than 5%. This filters out queries made by users who might be using our red-teaming bounty program for purposes other than red-teaming. B.2 PROTOCOL MODIFICATIONS FOR TWO-STAGE SYSTEM For the two-stage exchange classifier evaluation and production-grade system, we implemented sev- eral protocol refinements to improve red-teaming efficiency. Program structure changes. We transitioned from providing API credits to granting access through a specific model alias that does not consume credits, reducing friction for participants and preventing misuse of our platform. We further provided participants with test queries and rubrics, which we found improved submission quality. The universal jailbreak definition was clarified to emphasize the need for detailed, actionable responses across multiple queries. Reward structure updates. We restructured the bounty system to better incentivize universal jail- breaks. The maximum reward was increased to $35K USD. We further included a 1.05x multiplier for each additional model response considered to be a “strong pass”, which is a response achiev- ing at least 75% of the detail of a model without safeguards, as assessed by rubric grading. This incentivizes red-teamers to improve the details provided by their responses, as well as the number of questions answered. We also increased rewards for universal jailbreaks and reduced rewards for attacks that were less universal (that answered fewer queries), to further incentivize the discovery of universal attacks. Question set refinement. We replaced four of the original eight target queries with newer ques- tions. We provided the previous questions and rubrics to red-teamers to allow them to improve their submissions. Data collection improvements. We implemented standardized data collection procedures and added explicit terms regarding data storage and usage. Participants were required to document their attack development process and provide time estimates. C ADDITIONAL PROBE ABLATIONS We now conduct further ablation studies to better understand the impact of different loss function designs and hyperparameters on probe performance. 15 Smoothed Softmax Softmax Only Smoothing Only Per-token Cummax Cummax (Annealed) Loss Function 0.0 0.1 Attack Success Rate (%) (a) Loss Function Ablation 1 2 4 8 16 32 64 Sliding Window Size 0.000 0.025 0.050 Attack Success Rate (%) (b) Sliding Window Size Figure 5: Additional probe ablations on loss functions and hyperparameters. (a) Comparison of different loss functions showing that our smoothed softmax approach additionally outperforms cumulative maximum and annealed cumulative max loss, which were proposed by Sharma et al. (2025). (b) Impact of sliding window size M on attack success rate, demonstrating optimal perfor- mance at moderate window sizes (around M = 16), with degradation at both extremes. C.1 ALTERNATIVE LOSS FUNCTIONS

Loss Formulations and Deployment Lessons

The researchers explored cumulative maximum loss functions to better align with streaming classification where any harmful token justifies stopping generation.
An annealed version of the cumulative maximum loss was developed to fix optimization issues by linearly interpolating between direct probabilities and the maximum score.
Empirical results showed that smoothed softmax loss-weighting still outperformed cumulative maximum approaches in final classification accuracy.
Sliding window smoothing for probe logits reaches peak performance at a window size of 16, with too much or too little smoothing degrading results.
Ensemble weighting analysis reveals that equal weighting between linear probes and external classifiers is nearly optimal for reducing attack success rates.
A critical deployment takeaway is that infrastructure reliability is a security requirement, as implementation bugs can create functional jailbreaks.

Even perfect classifiers become ineffective when compromised by implementation bugs.

Beyond the smoothed softmax loss presented in the main text, we investigate two additional loss formulations that leverage the streaming nature of our classification task. Cumulative maximum loss. We explore using the cumulative maximum of probe probabilities as the predictor for the full sequence label. This approach better reflects the streaming use case where, if we stop generation at position t, the remainder of the sequence is effectively considered harmful. Specifically, we define: p(y = 1|x1:t) = max τ≤t σ(¯zτ(x1:τ)) (5) where ¯zτ is the probe logit at position τ (after sliding window averaging). This ensures that predic- tive probabilities only increase when the probe is confident that the full sequence will be harmful. However, using this approach, gradients only flow through the maximum-scoring token position, which can mean this approach sometimes suffers from optimization challenges. Annealed cumulative maximum loss. To address the optimization challenges of the cumulative maximum while retaining its benefits for streaming classification, we linearly interpolate between direct probe probabilities and the cumulative maximum throughout training, as suggested by Sharma et al. (2025): p(y = 1|x1:t) = (1 −ω) · σ(¯zt(x1:t)) + ω · max τ≤t σ(¯zτ(x1:τ)) (6) where ω starts at 0 and linearly increases to 1 throughout training. This ensures stable initial training with good gradient flow while gradually transitioning to the cumulative maximum formulation that better matches our streaming use case. Figure 5a compares these loss functions against our proposed smoothed softmax approach. We find that our softmax loss-weighting with activation smoothing achieves stronger performance than these additional loss functions. We use the same evaluation datasets and methodology as described in the main paper. C.2 IMPACT OF SLIDING WINDOW SIZE The sliding window size M controls the degree of smoothing applied to probe logits. Figure 5b demonstrates that moderate smoothing is crucial for performance. Performance improves as we increase the window size from no smoothing, reaching optimal performance around M = 16. How- ever, excessive smoothing degrades performance. 16 D ENSEMBLE WEIGHTING ANALYSIS Figure 6 examines three classifier pairs and shows how their individual performance varies with different weighting proportions. We use a weighted average of logits for each classifier pair. While we optimize α for each classifier pair in Figure 3a, for the classifier-probe ensembles, we find that equal weighting already is close to optimal. XS Classifier Only Equal Weight S Classifier Only Ensemble weighting 0 2 4 Attack Success Rate (%) (a) Probe Only Equal Weight S Classifier Only Ensemble weighting 0 2 4 Attack Success Rate (%) (b) Probe Only Equal Weight XS Classifier Only Ensemble weighting 0 2 4 Attack Success Rate (%) (c) Figure 6: Impact of ensemble weighting on attack success rates for different classifier pairs. Each panel shows the performance of individual classifiers and logit-weighted ensembles. (a) Extra- small (XS) and small (S) external classifiers. (b) Linear probe and small external classifier. (c) Linear probe and extra-small external classifier. All classifiers are calibrated to maintain a 0.1% false positive rate on WildChat. We use our standard dataset of red-teaming exchanges and our standard evaluation methods, as described in the main paper. E LESSONS LEARNED FROM CLASSIFIER ITERATION AND DEPLOYMENT We now share lessons learned from classifier iteration and deployment. Infrastructure reliability is a security requirement. During development and human red-teaming, we discovered that infrastructure bugs can create effective jailbreaks. Comprehensive end-to-end testing is therefore essential to ensure consistency across different implementations and deployment environments. Even perfect classifiers become ineffective when compromised by implementation

Red-Teaming and Probe Development

Human red-teaming remains a critical complement to synthetic testing to prevent overfitting on automated benchmarks.
Organizations must invest in both technical infrastructure and operational processes for safety system updates.
Recomputing model activations during probe training is more efficient than storing them due to I/O bottlenecks.
Linear probe training efficiency allows for simultaneous testing of multiple variants on fresh data.
Providing red-teamers with sample queries and rubrics improves the quality and efficiency of jailbreak reporting.
Large Language Models were utilized to assist in the polishing and writing of the final research paper.

Moving probe activation data from high-bandwidth memory (HBM) to RAM or blob storage creates severe I/O bottlenecks that dwarf the computational cost of regeneration.

errors or configuration issues. Regular red-teaming remains essential. Our robustness evaluations demonstrate that periodic hu- man red-teaming must complement synthetic testing. While automated evaluations enable rapid iteration and improvement, relying solely on synthetic benchmarks can lead to overfitting on lim- ited test cases. Organizations must therefore invest in both technical infrastructure and operational processes that enable efficient deployment, testing, and evaluation of updated safety systems. On-the-fly activation computation accelerates probe development. During probe classifier ex- perimentation, we found that recomputing model activations within the training loop rather than pre- computing and saving them offers compelling engineering advantages. Moving probe activation data from high-bandwidth memory (HBM) to RAM or blob storage creates severe I/O bottlenecks that dwarf the computational cost of regeneration. Furthermore, since linear probe training is extremely efficient, we can test multiple probe variants simultaneously on freshly computed activations. Providing red-teamers with test queries is helpful. Supplying red-teamers with sample questions and their corresponding evaluation rubrics significantly enhances red-teaming efficiency. These examples guide red-teamers in developing appropriately concerning jailbreaks, thereby streamlining the report evaluation process. F DETAILS OF LLM CONTRIBUTION We used LLMs to aid and polish paper writing. 17

Kept 7 of 19 sections, 0 of 0 figures.

Advancing Constitutional Classifiers

New adversarial testing found vulnerabilities in current defenses, especially reconstruction and output-obfuscation attacks.
The authors propose a single exchange classifier that evaluates model outputs in the context of their specific inputs.

We identify two classes of attacks capable of evading these defenses: (i) reconstruction attacks, which distribute harmful information across multiple segments within a larger context before reassembling them; and (ii) output obfuscation attacks.

Optimizing Constitutional Classifier Systems

A classifier cascade uses a lightweight first-stage screener and a more expensive second-stage evaluator to reduce overhead.
The final production-grade system achieved a 40x reduction in overhead and a 0.05% flag rate, while 1,700 hours of human red-teaming found no universal jailbreak for detailed CBRN information.

Crucially, no red-teamer discovered a universal jailbreak capable of consistently extracting highly detailed answers to all target CBRN queries.

Optimizing Exchange Classifier Robustness

A fine-tuned Claude Haiku 3.5 exchange classifier monitored interactions for CBRN-related risks.
The system reduced vulnerability from 0.01871 to 0.00885 per thousand queries, and the most successful universal jailbreak took 100 hours to find versus 27 hours previously.

In contrast, the exchange classifier system had only one such universal attack, which required 100 hours to discover.

Optimizing Cascaded Defense Systems

A two-stage system combining linear probes with external classifiers achieves a 100x compute reduction while exceeding the robustness of standalone classifiers.
Ensembling a probe with a small external classifier via weighted logit averaging yields better robustness than ensembling two external classifiers.

On our dataset of CBRN-related exchanges, our system that combines probes and the small external classifier can exceed the robustness of the small classifier alone whilst simultaneously offering a 100x reduction in compute costs.

Production-Grade Constitutional Classifiers

A two-stage cascade uses lightweight probes to monitor generation and escalates to expensive classifiers only when confidence thresholds are met.
The production system achieved a 100x reduction in compute costs and, in shadow deployment, showed a 0.05% flag rate; 198,000 red-teaming attempts yielded only one high-risk vulnerability.

On our dataset, our system exceeds the robustness of the small classifier alone whilst simultaneously offering a 100x reduction in compute costs.

Production-Grade Constitutional Classifiers

The production-grade system reduces relative compute overhead to 3.5% of previous generations.
A two-stage cascade triggers expensive classification only when early layers detect high-risk signals, while maintaining a 0.05% refusal rate and the lowest high-risk vulnerability discovery rate among tested models.

The production-grade system described in this section achieves the best trade-off between robustness, inference overhead, and refusal rates.

Loss Formulations and Deployment Lessons

Smoothed softmax loss-weighting still outperformed cumulative maximum approaches in final classification accuracy.
A critical deployment takeaway is that infrastructure reliability is a security requirement, since implementation bugs can create functional jailbreaks.

Even perfect classifiers become ineffective when compromised by implementation bugs.

Arbitrary Prompt (Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks)

Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks

Advancing Constitutional Classifiers

Optimizing Constitutional Classifier Systems

Vulnerabilities in Constitutional Classifiers

The Rise of Exchange Classifiers

Optimizing Exchange Classifier Robustness

Adaptive Cascades and Linear Probes

Linear Probing for Streaming Safety

Optimizing Linear Probes for Safety

Probing for Jailbreak Robustness

Optimizing Cascaded Defense Systems

Production-Grade Constitutional Classifiers

Production-Grade Constitutional Classifiers

Constitutional Classifiers and Future Directions

AI Safety and Robustness Bibliography

AI Safety and Vulnerability Benchmarks

Constitutional Classifier Performance Metrics

Red-Teaming Protocols and Ablations

Loss Formulations and Deployment Lessons

Red-Teaming and Probe Development

Advancing Constitutional Classifiers

Optimizing Constitutional Classifier Systems

Optimizing Exchange Classifier Robustness

Optimizing Cascaded Defense Systems

Production-Grade Constitutional Classifiers

Production-Grade Constitutional Classifiers

Loss Formulations and Deployment Lessons