Auto
Introduction
Modern AI assistants (chatbots like ChatGPT or Bing Chat) often try to answer user questions and “show their work” by citing sources. However, studies show many answers have bad or missing citations. For example, Stanford researchers found that about half of AI chat answers contained unsupported statements or wrong citations (www.axios.com). In medical tests, new AI tools often gave answers not supported by the sources they cited (doaj.org). These problems mean we need better ways to test AI assistants’ citation behavior.
To understand how an AI picks what to cite, we propose a large-scale testing plan. We will create many synthetic queries (made-up questions) covering different subject areas and types of questions. We will run these through AI assistants automatically, gather their answers and citations, and label each cited source by its freshness (how recent), authority (how trusted), and structure (type or format). Then we use simple statistics to see which factors make it more likely an AI will cite a source. We will share all our data and tools openly. In this way, we can crowd-source improvements and keep monitoring AI citation behavior over time.
Designing a Synthetic Query Benchmark
To test citation rules in AI, we will use a synthetic query set. This means we generate a large number of example questions (prompts) using computers, rather than just collecting real user queries. Using synthetic prompts is common in research. For example, Google researchers have used AI systems to generate questions when data is hard to collect (research.google). Another study showed that AI-generated question-answer data can be a good stand-in for real test data in some settings (papers.cool).
Our queries will span many topics (verticals) and user goals. We pick a wide range of subjects such as science, history, health, finance, and everyday tasks. Within each topic, we cover different intents – the purpose of the question. For instance, some queries will be factual (like “What is the largest planet in our solar system?”), some will ask for how-to instructions (“How do I change a car tire?”), some for open-ended advice (“What should I consider when applying for college?”), and so on. By mixing informational, factual, argumentative, and social types of questions (papers.cool), we ensure our test covers many real-world uses.
We might use large language models themselves to help generate this query set. For each topic and intent, an AI can produce many questions. We can then review and filter them. The final set might contain thousands of questions to give us good statistical power.
Running Tests and Capturing Citations
Once we have our query set, we automate the testing. A script or program will send each synthetic question to the AI assistant (via an API or interface) and save the response. For each answer, we parse and extract any cited sources. AI assistants often provide references as links, footnotes, or embedded citations. We record each source (for example, a web link or article title) along with the question and answer.
This automation lets us collect data at scale. Instead of manually asking each question, the script can run through all hundreds or thousands of queries in a batch. We might do this for one or multiple AI systems. The output is a dataset of (question, answer, cited sources). We also note how often sources are cited and in what format.
Labeling Source Attributes
After collecting citations, we label each source by its key attributes. These attributes help us test what makes a source likely to be cited. The main attributes are:
- Freshness: How recent or up-to-date is the source? For example, a news article from last week is very fresh, while a textbook from 20 years ago is not. Freshness is important because newer information might be more relevant. Prior benchmarks have measured freshness as a quality signal for citations (papers.cool).
- Authority: How trustworthy or authoritative is the source? This could be based on who published it. For example, a university study or a government website usually has high authority, while an unknown blog may have low authority. SourceBench, a recent study, listed “authority” as a key page-level signal for quality (papers.cool).
- Structure: What type or format is the source? For instance, a source could be a research paper, a news article, an online forum post, or an official report. The structure might affect how an AI uses it. (Even if our prompt does not provide specific examples of structure, we define it as the document type or clarity of writing. SourceBench also mentioned “clarity” as a signal (papers.cool), which is related.)
We can annotate these attributes by automated checks or by looking up the source. For freshness, we might record the publication date. For authority, we could use known lists of reputable domains or the site’s description (for example, official domains like .gov or .edu tend to be more authoritative). This labeling can be manual or automated, but it gives us values like “freshness = new/old” and “authority = high/low” for each source.
Analyzing and Modeling Citation Patterns
With a dataset of many (question, answer, source attributes) examples, we analyze the data to find patterns. We ask: which features best predict whether an AI will cite a given source?
We might do this by simple statistical analysis or machine learning. For example, we can see if sources with high freshness or high authority are cited more often. We could use a logistic regression or a decision tree to model “cite vs not cite” as the outcome. This tells us which features (like freshness or topic or question type) have the strongest effect.
This step is similar to how researchers analyze survey or experimental data. It may reveal, for instance, that the AI prefers citing sources that are newer or more authoritative. We would look for the strongest predictors – those attributes that most increase the chance of a citation. By quantifying this, we essentially “reverse-engineer” the citation rules: we find out what the assistant seems to be looking for.
This approach is inspired by SourceBench and other studies that score sources on metrics like freshness and authority (papers.cool). By applying statistical modeling, we move beyond anecdotes to measured trends. This helps us understand if the AI has a bias toward certain sources (for example, always citing Wikipedia or only citing top sites).
Open-Source Benchmark and Continuous Monitoring
After building this benchmark, we will open-source it – publishing the queries, code, and analysis online for everyone. Open-source benchmarks are common in AI research because they let others use and improve the tests. For example, many large-scale datasets, like those for language translation or question answering, are shared publicly. We plan to put our query set and results on a platform like GitHub. This allows other researchers and developers to check their own AI assistants against our benchmark.
We also recommend ongoing monitoring. AI assistants change often (they get updated, new versions, etc.). We suggest a regular schedule for re-running the tests. For example, every time the AI gets a big update or on a fixed cycle (like monthly). This is similar to the idea of “lifelong benchmarks,” which expand and update test sets over time to avoid stagnation (huggingface.co). By continually adding new questions and re-running, we can catch any shifts in citation behavior. If the AI suddenly starts citing more out-of-date sites, or if it learns to cite better sources, we will see it.
Automating this monitoring means we can track trends. Teams can get alerts if something changes dramatically. This is like model drift monitoring in machine learning, but focused on citations. Combining our benchmark with regular runs ensures AI assistants stay on track in how they give sources.
Conclusion
In summary, we propose a comprehensive plan to probe AI assistants’ citation rules. By designing a large and varied set of test questions, running them automatically through AI systems, and carefully labeling the cited sources by attributes like freshness and authority, we can use statistics to uncover the factors that influence citation. Our method is backed by recent research that stresses the importance of verifiability (www.axios.com) (doaj.org) and innovative benchmarks that study source quality (papers.cool) and synthetic data (papers.cool) (research.google). By open-sourcing the benchmark and regularly monitoring results, we help create a transparent way to ensure AI assistants cite reliable information. This work can improve trust in AI by shedding light on the hidden rules of citations and guiding developers to make better, more accountable systems.
