01 / Exposure
Influence
When information is introduced into the public web, what later changes can we observe in a model's answers? What can alternative explanations account for?
Independent research into language-model behaviour
What happens to an AI's answers after it encounters information?
Caliper is an experimental toolset for observing language-model responses over time, across models, and under different information-exposure and retrieval conditions.
The problem isn't simply whether a model produces a convincing answer. It's whether we can distinguish what we observed from what we infer—and reproduce the observation later.
The questions
Not just what models say. What changes, and why?
02 / Time
Persistence
Do observed response patterns persist, fade, or recur? What happens when a source changes or disappears?
03 / Conditions
Dependence
How do responses differ across model versions, web interfaces and APIs, retrieval settings, and isolated versus sequential questioning?
Experimental method
Observe. Compare. Preserve the evidence.
Caliper organizes questions into campaigns, establishes baselines, and captures repeated model responses. Experiments can vary exposure, model, timing, retrieval, and question order. Analysis compares observations against relevant controls.
1. Define the question
Specify the claim being tested, response indicators, model, and experimental conditions.
2. Establish baselines and controls
Record behaviour before a proposed intervention and use appropriate comparison conditions.
3. Record the observations
Retain question wording, responses, dates, model identity, and relevant capture context.
4. Evaluate alternatives
Separate observed answer changes from conclusions about retrieval, training, or model internals.
Caliper is research instrumentation. Specific experiment protocols and findings should be evaluated on their own evidence, not inferred from the existence of the tool.
What the evidence can—and cannot—say
Interesting behaviour is not proof of a mechanism.
A central methodological boundary: seeing a claim appear in an answer does not, by itself, prove that new information entered model weights. Retrieval, pre-existing knowledge, stochastic variation, question order, and other mechanisms must be considered.
Model descriptions of their own knowledge or training history are observations to investigate, not independent provenance evidence. Caliper aims to make such questions testable and discussion of results more precise.
The people and the purpose
Built to make curiosity measurable.
Caliper was developed by Scott Welch as part of an independent programme of research into model behaviour and trustworthiness. It is a research instrument developed within Confused Gorilla Labs, with questions and findings connected to the Confused Gorilla Institute.
For research collaboration, technical discussion, or questions about the experimental approach, get in touch.