Loop 070
The literature-search verification loop
A bounded literature-search workflow that deduplicates papers across live sources, verifies DOI metadata, measures topical relevance, and stops honestly when evidence is insufficient.
Ready-to-use prompt
Copy the loop
Search the current PubMed and Semantic Scholar APIs for papers about [topic] and produce a DOI-verified CSV. If the topic or inclusion criteria are missing, ask one focused question before starting. Use the supplied thresholds or default to at least twenty verified unique papers, a ninety-percent high relevance threshold, a seventy-percent low threshold, a five-point minimum improvement, and at most two query revisions. Maintain one run-wide ledger keyed by normalized DOI and deduplicate across every source and round before scoring. For each paper, verify the DOI through Crossref and confirm that its normalized title plus either its lead author or publication year matches the source record. Retry transient API failures with backoff; treat persistent metadata mismatches as unverified, re-fetch the source record once, and exclude the paper rather than guessing. Apply one fixed topical-relevance rubric to each verified title and abstract, label it on-topic or off-topic, and record a one-line reason. Never change the rubric during the run. Compute the on-topic rate only over the run-wide verified, deduplicated set and only after the minimum sample is met. Succeed when the set reaches the high threshold. Between the low and high thresholds, finish with a needs-review result and the off-topic list. Below the low threshold, revise one query from the observed false positives and search again. Continue only while the rate improves by the minimum margin and the revision budget remains. Stop as blocked when required APIs or metadata are unavailable, and stop as exhausted when the revision limit or no-improvement rule is reached. Never invent, infer, or autocomplete paper metadata. Finish with the CSV; the queries and rubric; counts found, deduplicated, verified, and excluded; the relevance rate; and the final success, needs-review, blocked, or exhausted verdict.
Verify / stop
The minimum-size literature set clears its relevance gate with matched DOI metadata.
Every retained row is unique across the full run, its DOI metadata matches the source record, its relevance decision follows the fixed rubric, and the final verdict follows the recorded sample size, rate, and retry budget.
Context and guidanceWhen to use it, steps, safety notes, and related loops
Use this when
Use this when a literature search must produce a high-precision, auditable paper set rather than an unverified list of citations.
How to run it
- Define the topic, inclusion rubric, minimum sample, relevance thresholds, improvement margin, and query-revision budget.
- Search PubMed and Semantic Scholar, normalize every DOI, and deduplicate all results in one run-wide ledger.
- Verify DOI metadata against Crossref, exclude unresolved mismatches, and score verified papers with the unchanged relevance rubric.
- Measure the verified set and revise one query only when the result is below the low threshold and measurable improvement remains possible.
- Return the CSV, evidence ledger, metrics, exclusions, query history, and an honest terminal verdict.
Why it works
Literature searches can look authoritative while containing duplicate, mistyped, invented, or irrelevant citations. A run-wide ledger, metadata matching, fixed rubric, minimum sample, and bounded query revisions make the result auditable without rewarding a tiny or cherry-picked set.
Implementation note
Use current public bibliographic metadata and respect each API's access and rate limits. Do not infer missing abstracts or treat a network failure as evidence that a DOI is invalid.