Adam Zachary Wasserman › Papers

The 70% Rule

When Axiomatic Prompting Helps, and When It Hurts

Adam Zachary Wasserman, Open Honest Foundation. Preprint, 4 April 2026.

Download the PDF

Abstract

Axiomatic prompting augments a classification prompt with explicit decision rules of the form pattern to category. This paper tests whether it works, across six classification tasks and eight models spanning five providers: OpenAI, Anthropic, Google, Meta and Alibaba.

It works conditionally, and the condition is sharp. Where zero-shot performance falls below 70%, axiomatic prompting improves accuracy by 15 to 24 points at p below 0.05. Where zero-shot performance is already above 70%, it degrades accuracy significantly.

The practical reading is that the technique is a remedy for a weak baseline rather than a general improvement, and applying it to a task the model already handles makes the result worse.

Full text

The complete paper follows. It is identical to the deposited PDF; the PDF is the version of record and carries the figures at full resolution.

Introduction

Large language models (LLMs) demonstrate strong capabilities across diverse classification tasks. When deploying LLMs for domain-specific classification, practitioners face a choice: rely on the model’s learned representations via zero-shot prompting, or provide explicit classification rules (“axioms”) to guide the model’s decisions.

We propose axiomatic prompting—providing LLMs with explicit axioms that map observable text patterns to classification categories. The intuition is that domain-specific classification rules, expressed as “IF pattern THEN category” statements, could improve accuracy by externalizing expert knowledge.

However, our experiments reveal a nuanced finding: axiomatic prompting only helps when the model struggles with a task. When zero-shot accuracy is already high, axioms add noise and hurt performance.

Contributions

1. We identify a 70% zero-shot threshold rule that perfectly predicts when axiomatic prompting helps (below threshold) versus hurts (above threshold) across 6 diverse classification tasks.

2. We provide statistically significant evidence: all 6 task-level effects are significant at p < 0.05 with large effect sizes (Cohen’s d ranging from 2.75 to 12.70).

3. We analyze axiom quality factors correlated with effectiveness, finding that detailed axioms with multiple conditions per rule show stronger positive effects.

4. We provide practical guidelines for when and how to use axiomatic prompting in production systems.

Related Work

Prompting Strategies

Chain-of-thought prompting (Wei et al. 2022) demonstrated that eliciting intermediate “reasoning” improves LLM performance on complex tasks. ReAct (Yao et al. 2023) interleaves “reasoning” with action steps. Skeleton-of-thought (Ning et al. 2023) structures “reasoning” via outlines. These strategies scaffold the model’s output generation without providing domain-specific knowledge.

Rule-Based and Hybrid Approaches

The tension between rule-based and learning-based approaches in NLP reflects a deeper question about when explicit knowledge helps. Recent work on knowledge-augmented prompting has explored providing models with external knowledge, but focused on factual rather than procedural knowledge.

When Does External Knowledge Help?

Prior work has shown that providing LLMs with external information can either help or hurt, depending on the model’s existing knowledge (Mallen et al. 2023). Our work extends this finding to classification rules specifically, identifying a threshold-based decision rule.

Method

Axiomatic Prompting

For a classification task with categories C = {c₁, …, c_n}, we define axioms of the form:

AXIOM [category_name]:
IF text contains [pattern]
AND text exhibits [feature]
THEN classify as [category]

Axioms are derived from authoritative sources defining each category (regulatory definitions, professional standards, annotation guidelines).

Baseline Strategies

We compare against five established prompting strategies:

1. Zero-shot: Direct classification without examples

2. Few-shot (k=3): Three labeled examples

3. Chain-of-thought: Explicit reasoning steps

4. ReAct: Interleaved thought/action steps

5. Skeleton-of-thought: Outline-based reasoning

Experimental Protocol

• Models: 8 LLMs across 4 providers:

  • OpenAI: GPT-4o, GPT-4o-mini

  • Anthropic: Claude Sonnet 4, Claude 3.5 Haiku

  • Google: Gemini 2.5 Flash, Gemini 2.0 Flash

  • Open-weights: Llama 3.3 70B, Qwen 2.5 72B (via HuggingFace)

• Sample size: 100 samples per task

• Runs: 3 independent runs per condition for variance analysis

• Temperature: 0.0 for reproducibility

• Random seed: 42 for sample selection

• Total experiments: 864 (3 runs × 6 tasks × 6 strategies × 8 models)

Datasets

We evaluate on 6 classification tasks spanning legal, medical, security, and sentiment domains:

Results

Main Finding: 70% Threshold Rule

Our primary finding is that axiomatic prompting effectiveness depends on the model’s zero-shot baseline performance. This pattern holds consistently across all 8 models tested.

Values show percentage point change (Axiomatic − Zero-shot). Bold = largest effects.

The pattern is clear: GoEmotions and LEDGAR (low zero-shot baselines) show consistent positive effects across all 8 models, while OPP-115 and SPDX show negative effects when models have high baselines.

Full Accuracy Tables

Tables 3 and 4 present complete accuracy results for zero-shot and axiomatic prompting across all model-task combinations.

Note the high variance (±19–49%) in Claude model results, explained by extraction errors (Section 5.10).

Threshold Rule Validation

Across all 48 model-task combinations (8 models × 6 tasks), we find the threshold rule holds consistently:

When the zero-shot baseline is below 50%, axioms help in every case. When the baseline exceeds 70%, axioms hurt in 75% of cases. The 70% threshold provides a practical decision rule for practitioners.

Figure 1 visualizes this relationship across all 48 model-task combinations.

Statistical Analysis

Across all 48 model-task combinations, we find a significant negative correlation between zero-shot baseline and axiomatic effect:

r = −0.463,  p < 0.001

This confirms that axioms help more (or hurt less) when the baseline is lower. The relationship is not perfectly linear—axioms provide outsized benefits at very low baselines (<50%) and outsized harm at very high baselines (>70%).

Strategy Comparison

Table 6 compares all 6 prompting strategies averaged across all models and tasks.

Axiomatic prompting achieves the highest average accuracy (+3.8% over zero-shot), despite hurting on high-baseline tasks. This reflects the magnitude of improvements on low-baseline tasks outweighing the losses on high-baseline tasks. Notably, skeleton-of-thought performs poorly (-11.6%), suggesting that structured output formats can interfere with classification.

Per-Model Responsiveness

Models differ in their responsiveness to axiomatic prompting:

Helps/Hurts/Neutral = tasks where Δ > 2% / Δ < −2% / |Δ| ≤ 2%

Claude Sonnet 4 is most responsive to axioms (+8.4% average), while GPT-4o is the only model with negative average response (-2.3%). This may reflect differences in how models weight explicit instructions versus learned patterns.

Correlation Analysis

We analyze factors correlated with axiomatic prompting effectiveness:

The strongest predictor is axiom detail: axioms with more conditions per rule show larger improvements. However, we note that axiom style is confounded with task in our design (each task has one axiom set), so this correlation may reflect task difficulty rather than axiom quality alone.

Axiom Style Analysis

Keyword-rich axioms (using “keyword → label” patterns) show positive effects, while conceptual axioms and low-coverage axiom sets show negative effects.

Axiom Token Analysis

Table 10 presents axiom characteristics and their relationship to effectiveness.

Best Δ = largest improvement among 8 models.

Token count does not predict effectiveness: LEDGAR (2,921 tokens) shows strong positive effects (+16.2%), while SPDX (2,269 tokens) shows negative effects (-11.5%). The key factor is not verbosity but whether the model already encodes the classification patterns—a function of the task’s prevalence in training data.

Extraction Error Analysis

A critical factor in our results is extraction error rate—the percentage of responses where the model’s output could not be parsed into a valid category label.

Claude 3.5 Haiku exhibits a 63% average extraction error rate, explaining the high variance in its results. This suggests the model struggles to follow output formatting instructions consistently. In contrast, Llama 3.3 70B achieves only 3% extraction errors, producing the most reliable outputs.

Key insight: High extraction error rates explain several apparent anomalies. For example, Gemini 2.5 Flash’s +35.7% improvement on CWE is partly due to axioms reducing extraction errors from 45–54% (zero-shot) to 0–1% (axiomatic). Conversely, the same model’s -31.7% on OPP-115 reflects extraction errors increasing from 5–6% to 51–53% when axioms are added.

Outlier Case Studies

We analyze the three most extreme effects to understand the mechanisms:

GPT-4o on SPDX (Δ = -40.0%): This model achieves 80% zero-shot accuracy on license classification, suggesting strong learned representations for this domain. The SPDX axioms (2,269 tokens) may introduce conflicting signals. Notably, results varied wildly across runs (32%, 82%, 5%), suggesting the model is uncertain how to weight axioms versus its internal patterns.

Gemini 2.5 Flash on CWE (Δ = +35.7%): Zero-shot accuracy is only 50%, but axiomatic jumps to 85%. Critically, zero-shot has 45–54% extraction errors while axiomatic has only 0–1%. The axioms help the model both classify correctly and format outputs properly.

Gemini 2.5 Flash on OPP-115 (Δ = -31.7%): Despite 74% zero-shot accuracy, axioms drop performance to 42%. Extraction errors increase from 5% to 52%. The OPP-115 axioms appear to confuse this model’s output formatting, even though the axioms are relatively short (366 tokens).

Discussion

Why the Threshold Effect?

When the model already performs well on a task (>70% zero-shot), its learned representations already capture the statistical patterns needed for accurate classification. Adding explicit axioms introduces noise: the model must reconcile its internal representations with external rules that may be incomplete or differently formulated.

When the model struggles (<70% zero-shot), axioms provide useful signal that the model’s training did not adequately capture, leading to substantial improvements.

Practical Implications

For practitioners deploying classification systems:

1. Test zero-shot first: If accuracy > 70%, don’t use axioms

2. Write keyword-rich axioms: Use “keyword → label” patterns

3. Ensure full coverage: Axioms should cover all categories

4. Be specific: Multiple conditions per axiom helps

Limitations

1. Confounded design: Axiom style is confounded with task; future work should vary axiom quality within tasks

2. Sample size: 100 samples × 3 runs provides adequate power for large effects but may miss subtle effects

3. English only: All experiments use English-language datasets

4. API variability: Some variance in Claude 3.5 Haiku results (high extraction error rates) suggests API-level instability during testing

Language-Dependent Thresholds?

Our English-only limitation opens a theoretically significant research direction. The 70% threshold we identify may not be a universal constant—it may vary by language. If morphologically rich languages like French, German, or Finnish encode more grammatical structure redundantly (through agreement marking, case systems, and inflection), models trained on such languages might exhibit higher inherent capability baselines, shifting the threshold at which axioms become harmful.

This connects to the “Language-Only Hypothesis” (Wasserman 2025), which proposes that emergent capabilities in LLMs derive from structural properties of natural language itself, not solely from scale or architecture. Under this view, morphological agreement patterns function as “built-in axioms”—explicit IF-THEN relationships grammaticalized into the language (e.g., “IF subject is plural THEN verb takes plural marking”). A French-trained model may have internalized more such structural axioms during pretraining, potentially shifting its axiomatic threshold upward.

The decisive test: train identical models on matched English and French corpora, then evaluate axiomatic prompting effectiveness on parallel classification tasks. If French-trained models show higher thresholds (axioms helping at, say, 80% baseline rather than 70%), this would suggest that language structure—not just model capacity—determines when explicit rules become redundant. Such a finding would have profound implications for multilingual deployment: the same prompting strategy may help in one language and hurt in another at identical baseline accuracies.

Conclusion

Axiomatic prompting—providing LLMs with explicit classification rules—is not universally beneficial. Our cross-platform experiments across 8 LLMs reveal a consistent decision rule: use axioms when zero-shot accuracy is below 70%, avoid them when it is above. This threshold holds across commercial models (GPT-4o, Claude Sonnet 4, Gemini) and open-weights models (Llama, Qwen).

The finding suggests a principle for augmented prompting more broadly: external rules help when the model’s learned representations are insufficient for the task, but hurt when those representations already capture the relevant patterns. Practitioners should empirically test baseline performance before investing in rule engineering.

Acknowledgments

Experiments were conducted using APIs from OpenAI, Anthropic, Google, and HuggingFace. Estimated total API cost for the full cross-platform variance analysis (864 experiments) was approximately $50–100 USD.

Reproducibility

Code, data, and results are available at: [OSF repository URL]

• Random seed: 42

• Temperature: 0.0

• Models: GPT-4o, GPT-4o-mini, Claude Sonnet 4, Claude 3.5 Haiku, Gemini 2.5 Flash, Gemini 2.0 Flash, Llama 3.3 70B, Qwen 2.5 72B

• Sample size: 100 per task

• Runs: 3 per condition

• Total experiments: 864

Token Usage Analysis

Table 12 presents estimated prompt token counts by strategy for a representative task (LEDGAR).

Axiomatic prompting requires significantly more input tokens due to the axiom definitions. For LEDGAR (the largest axiom set), this is ∼15× the zero-shot prompt size. However, token count does not correlate with effectiveness—CADEC uses only 597 axiom tokens but achieves similar improvements to LEDGAR (2,921 tokens).

API Cost Estimates

The full cross-platform experiment required 864 API calls (3 runs × 6 tasks × 6 strategies × 8 models). Approximate costs by provider:

Axiom Examples

This appendix provides representative axiom examples illustrating the two main styles used in our experiments.

LEDGAR: Keyword-Rich Style (2,921 tokens, 106 axioms)

LEDGAR axioms use explicit keyword patterns that map directly to category labels. This style achieved +16.2% average improvement.

AXIOM Governing_Laws:
IF text contains "governed by" OR "construed in accordance"
AND text specifies state or country jurisdiction
THEN classify as "Governing Laws"

AXIOM Indemnification:
IF text contains "indemnify" OR "hold harmless"
AND text specifies indemnifying and indemnified parties
AND text references claims, losses, or damages
THEN classify as "Indemnification"

AXIOM Terminations:
IF text contains "may be terminated" OR "termination of"
AND text specifies conditions for termination
THEN classify as "Terminations"

OPP-115: Conceptual Style (366 tokens, 20 axioms)

OPP-115 axioms describe semantic concepts rather than keyword patterns. This style achieved -10.1% average effect, illustrating that conceptual axioms can hurt when the model already encodes the classification patterns.

AXIOM First_Party_Collection:
IF text describes what information the company collects
AND collection is performed by the company itself
THEN classify as "First Party Collection/Use"

AXIOM Third_Party_Sharing:
IF text describes sharing user information
AND sharing is with external parties
THEN classify as "Third Party Sharing/Collection"

AXIOM Data_Security:
IF text describes security measures or protections
AND measures apply to user data
THEN classify as "Data Security"

Key Differences

• Keyword-rich: Explicit lexical patterns (“governed by”, “indemnify”) provide unambiguous classification signals

• Conceptual: Abstract descriptions (“describes what information”) require the model to interpret intent, potentially conflicting with learned patterns

Identifiers

• DOI (Zenodo, canonical): 10.5281/zenodo.19423101
• ORCID: 0009-0002-8865-6583
• Licence: CC BY 4.0

Cite as

Wasserman, A. Z. (2026). The 70% Rule: When Axiomatic Prompting Helps, and When It Hurts. Open Honest Foundation. https://doi.org/10.5281/zenodo.19423101