What does a confusing prompt cost?
Models are graded on what they answer and almost never on what answering burns. A laboratory for the energy price of ambiguity, and the discipline of labelling every estimate.
- research
- energy
- llms
Every conversation with a language model has a meter running that nobody reads. Accuracy is benchmarked to the decimal; latency has its dashboards. The electricity spent answering a muddled question, compared with a clear one, goes unrecorded.
The Computational Entropy Lab exists to read that meter. The question is simple to state and hard to answer honestly: does a vague, contradictory or bloated prompt make a model spend more energy per token?
Controlled damage
A mutation engine takes a clean prompt and damages it under nine seeded, reproducible conditions: typo noise, verbosity, ambiguity, contradiction, negation, reordering, formality, code-switching, and an untouched baseline. Each runs at a tunable intensity.
Length is the trap. A verbose prompt is longer, and longer prompts cost more for reasons that have nothing to do with confusion. So every mutation has a length-matched control, which separates a prompt that is longer from one that is harder to understand.
Instruments that admit what they are
The energy figures are the whole point, so they are the part held to the strictest standard. GPU draw comes from real NVIDIA NVML wattage. Where the hardware cannot be read directly, CPU and memory use a labelled TDP estimate, cross-checked with CodeCarbon. Every number carries its provenance, measured, proxied or simulated, and the three are never blended in a chart or a table.
An estimate presented as a measurement is not a small error. It is the end of the study.
From signal to coefficient
A statistics suite runs Pearson, Spearman and Kendall correlations with confidence intervals, alongside ANOVA and Kruskal-Wallis, and reports effect sizes rather than bare p-values. The result it works towards is the Prompt Entropy Coefficient: how the Semantic Instability Index of a prompt relates to the energy each token costs.
Built to be checked
Inference runs locally through Ollama, phi3.5 by default, orchestrated by FastAPI and watched live on a React dashboard over WebSockets. When an experiment ends, one click exports LaTeX tables, CSV and JSON, and a zipped reproducibility package holding the raw data, the protocol and the environment it ran in.
A result that cannot be rerun by a stranger is an anecdote. This one ships with everything the stranger needs. The project room is here.