Low-Cost Inference-Time Signals Outperform NLI Models and LLM Judges at Detecting Unsupported Retrieval-Augmented Generation Answers

Retrieval-augmented generation (RAG) is widely adopted to ground large language models in external documents, yet RAG systems still produce fluent claims that the retrieved context does not support. Most detection methods answer this with a second, larger judge model, which is impractical for teams without that budget. We ask how far one can get using only signals already available at inference time on modest hardware. Over 150 question–context pairs built from 50 arXiv abstracts, generated with a 4-bit Qwen2.5-7B on a single 6 GB consumer graphics card, we compare three such signals (token log-probability, self- consistency across sampled generations, and natural language inference (NLI) entailment against the retrieved context) against a near-free lexical baseline and a 7B large language model used as a judge. Ground truth is a two-annotator consensus (Cohen's kappa = 0.78). The ordering we found inverts the one the literature assumes. Mean token log-probability, which is free, leads on area under the receiver operating characteristic curve (AUROC) at 0.849 and is statistically indistinguishable from a lexical baseline costing 41 ms, while the NLI cross- encoder reaches only 0.669 at 2.6 s per query and the judge attains a lower F1 score than the free signal at 5.0 s. A stratified analysis shows the log-probability result is not reducible to answer length. We also find the judge systematically violates an explicit instruction in its own rubric, and that our strongest manipulation elicits appropriate abstention more often than fabrication. The practical recommendation is to exhaust free signals before paying for expensive ones.