When the exception is the answer

Two passages arrive at a retrieval-augmented model. The first says international transactions incur a 3% fee. The second says fees are waived for premium cardholders.

Conflict-aware RAG sees a contradiction and does what it was designed to do: it weighs credibility, picks a winner and answers from it. Whichever passage loses, the answer is now wrong for someone. Worse, every standard answer-correctness metric scores it as fine, because the surviving sentence is true.

These two passages were never in conflict. They are a rule and its exception, each true within its own scope. ScopeCompose is a research build about that difference.

Naming the loss

What cannot be measured cannot be argued about, so the first contribution is a measurement. Suppression Rate is a metric suite that registers when a valid, separately scoped branch of the truth has been dropped from an answer: exactly the failure that correctness metrics are blind to.

Four ways a rule meets its exception

Detection runs in two stages, a DeBERTa-v3 natural-language-inference filter first and a language model only when the cheap filter cannot decide. A five-class classifier, compared across three architectures, then labels each disagreement, and a scope analysis decides how the exception relates to the rule:

• Refinement. The exception narrows the rule. Compose them.
• Disjoint. Each governs a different case. Compose them.
• Redundant. They say the same thing. Keep one.
• Opposed. A genuine contradiction. Only here does choosing a source make sense.

One check guards the whole analysis: the decision must not change when the two passages swap places. A scope judgement that depends on reading order is not a judgement at all.

Built by two, joined by a contract

Detection and composition are built by two researchers working independently. They meet at a frozen, versioned JSON record that neither side can change alone, so each half can move quickly without breaking the other. The study runs at zero spend: Groq handles the judgement calls, and a local Qwen 2.5 on Ollama takes over offline through the same code path.

Where it stands

A 54-case feasibility probe, verified by hand, has passed its gate. A benchmark of naturally occurring rule-and-exception pairs is under construction, kept strictly apart from synthetically split pairs in any headline result. The first annotation pilot failed on one axis; the failure was diagnosed, written up and folded back into the manual, which is the only useful thing a failed pilot can become.

Answer the rule, keep the exception, and say which is which.

The project room is here.