← series index

# Demo Script — Session 3: Reading, Notes & Knowledge Management

*AI for Researchers — presenter script · Landscape as of August 2026*

**Runtime:** ~25 minutes (Build ~5 min · Three questions ~9 min · Extraction table ~7 min · Verification ~4 min)
**Slides it follows:** slide 28 ("What We Do Next")
**Artefacts it exercises:** The Summary-Trust Triage (slide 14), The Extraction Column Template (slide 22), and the demo-corpus table (slide 23)

---

## 0. The demo corpus — five open-access papers, one question

Everything in this demo runs on **one** corpus and **one** question:

> **Does open access publishing increase citations?**

**The five papers.** All are open access with resolvable DOIs; all five DOIs were re-verified against Crossref on 2026-09-02, and open-access status was verified against Unpaywall on 2026-07-27. Attendees can rebuild this corpus themselves in about four minutes.

| # | Paper | DOI | Design | Why it is in the corpus |
|---|---|---|---|---|
| 1 | Davis, P. M., Lewenstein, B. V., Simon, D. H., Booth, J. G., & Connolly, M. J. L. (2008). *Open access publishing, article downloads, and citations: randomised controlled trial*. BMJ 337, a568. | `10.1136/bmj.a568` | **Randomised controlled trial** | The only randomised design in the corpus, and the only one that finds no citation advantage. |
| 2 | Ottaviani, J. (2016). *The Post-Embargo Open Access Citation Advantage: It Exists (Probably), It's Modest (Usually), and the Rich Get Richer (of Course)*. PLOS ONE 11(8), e0159614. | `10.1371/journal.pone.0159614` | Observational, matched control (3,850 opened vs 89,895 closed articles) | A middle-sized effect, and it carries a **published Correction** the notebook will not know about. |
| 3 | Piwowar, H., Priem, J., Larivière, V., Alperin, J. P., Matthias, L., Norlander, B., Farley, A., West, J., & Haustein, S. (2018). *The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles*. PeerJ 6, e4375. | `10.7717/peerj.4375` | Large-scale observational | The source of the "18%" everybody quotes. |
| 4 | Langham-Putrow, A., Bakker, C., & Riegelman, A. (2021). *Is the open access citation advantage real? A systematic review of the citation of open access and subscription-based articles*. PLOS ONE 16(6), e0253129. | `10.1371/journal.pone.0253129` | Systematic review (134 studies) | Contains the disagreement in structured form — and a risk-of-bias appraisal that reverses the headline. |
| 5 | Huang, C.-K., Neylon, C., Montgomery, L., Hosking, R., Diprose, J. P., Handcock, R. N., & Wilson, K. (2024). *Open access research outputs receive more diverse citations*. Scientometrics 129(2), 825–845. | `10.1007/s11192-023-04894-0` | Large-scale observational | Changes the **outcome variable**, not the answer. The trap for a naive synthesis. |

**Why this corpus, and not one from your own field:**

- It is **meta-research**, so no one in the room is a bigger expert than anyone else — the audience judges the *method*, not the subject.
- It **genuinely disagrees with itself**, and the disagreement is explained entirely by study design. That is the lesson of the session, sitting in the data.
- **You hold the ground truth.** Every number the notebook can produce is in sources `[28]`–`[32]` of this session's `sources.md`. When it says something, you can check it in three seconds.
- One paper **is not about the same outcome at all** — which is the single most common way an AI synthesis goes quietly wrong.
- Every paper is open access, so the corpus is legally shareable and the demo is reproducible by every attendee.

> **Say this out loud before you start:** *"I built this notebook from five papers I have read. That is the only honest way to demo a reading tool. If I hadn't read them, I would have no way of knowing whether anything it told me was true."*

---

## 1. Preparation (do this the day before — not the morning of)

| # | Item | Why |
|---|---|---|
| 1 | Download all five PDFs and put them in one folder named `oa-citation-demo`. | You will drag the folder in. Downloading live from five publisher sites is four minutes of dead air. |
| 2 | Build the notebook once, end to end, and **keep it**. | This is your primary fallback (§6). |
| 3 | Run all three questions and the extraction, and screenshot every output. | Second fallback. See the pre-capture checklist in §6. |
| 4 | Note which question the tool refused and *how* it refused. | Refusal behaviour is not stable across releases — and the current model generation is documented by its own vendors as abstaining *less* — so check it again the day before, every time. |
| 5 | Open the five PDFs in five browser tabs, in corpus order, ready for verification. | §5 needs you to jump into a PDF in two seconds. |
| 6 | Have `sources.md` open on your second screen. | It is the answer key. |
| 7 | Check the daily query quota on the account you are using. | The documented limit is 50 chat queries per day `[18]`. Rehearsal plus demo can approach that. |
| 8 | Set browser zoom so citation chips are visible from the back of the room. | Citations are rendered small by default, and they are the entire point of the demo. |

**Tool neutrality note.** This script names Gemini Notebook (formerly NotebookLM) because a script has to name something. Say so on the day: *"I am using this because I have an account and because its refusal behaviour is documented. Every tool on slide 7 does a version of this job, and you should pick on the basis of your corpus, your institution's data rules, and your budget — not my demo."* Elicit, SciSpace or SCiNiTO substitute directly into §4 (extraction). Any PDF-chat tool substitutes into §3.

---

## 2. Build the notebook (5 minutes)

### Step B1 — Create the notebook and name it

Name it for the **question**, not the topic:

```
Does open access increase citations? (5 papers, built 2026-08-XX)
```

**Say:** *"The name carries the question and the date. In six months you will not remember either, and a notebook whose scope you cannot reconstruct is not evidence — it is a pile."*

### Step B2 — Add the five PDFs

Drag all five in at once.

**Expected outcome:** five sources appear in the left-hand panel, each with a generated summary. Ingestion takes seconds, not minutes, for papers of this size.

**What to point at on screen:**

1. **The source list is the method section of this demo.** Anything not in that list cannot appear in an answer `[18]`.
2. **The checkboxes.** You can include or exclude sources per question `[16]` — that is how you interrogate one paper against the other four.
3. **The documented limits**, said once and moved past: 50 sources per notebook and 500,000 words per source; copy-protected PDFs will not import at all `[17]` `[18]`.

### Step B3 — The one-line sanity check

Before any real question, type exactly this:

```
List every source in this notebook by first author, year, and study design. Cite each answer.
```

**Expected outcome:** five rows, each with a citation chip. Click one chip; the source panel jumps to the quoted passage `[16]`.

**Why this step exists — say it:** *"I have just confirmed three things in fifteen seconds: it can see all five papers, it can tell them apart, and its citations actually go somewhere. If any of those fails, nothing after this point is worth watching."*

**If it gets a design wrong** (e.g. calls Ottaviani a randomised trial): stop and celebrate. That is a free, live example of a grounded tool being confidently wrong about a fact that *is* in the source. Point at the Summary-Trust Triage on slide 14. Current-generation models rarely miss this on a five-paper corpus — treat it as a windfall if it happens, and if it does not, say so: *"It passed. That tells you about this corpus today, not about the next one."*

---

## 3. Three planned questions (9 minutes)

### Question 1 — the naive synthesis question (2 minutes)

Type exactly this:

```
Does open access publishing increase citations?
```

**Expected outcome:** a short, confident, well-written paragraph. This is a dual-path check, and either path teaches. In 2025-era rehearsals it typically led with the positive finding — Piwowar's 18% `[30]` — and hedged the randomised trial into a subordinate clause; a current-generation model may well surface the design split unprompted. Re-rehearse the day before and know which paragraph you are likely to get — the lesson does not depend on it misbehaving.

**What to do on screen, in this order:**

1. **Read it aloud, then ask the room: "would you cite that sentence?"**
2. **Count the qualifiers.** Does it say *which* studies, *which* designs, *which* populations? Slide 10's finding is that the commonest failure is converting a quantified finding into a generic one `[1]`.
3. **Look for the RCT.** Davis et al. found *no* citation advantage `[28]`. If it is absent or softened, say so: *"The strongest design in the corpus is the one that disappeared."*
4. **Look for Huang.** That paper does not answer this question at all — it measures citation *diversity* `[32]`. If the synthesis folds it in as supporting evidence, that is a synthesis failure of exactly the kind the expert error schema names `[5]`.

**Expected outcomes — treat the first two as equally live paths, and re-rank from your own rehearsal:**

| Outcome | How likely | What to say |
|---|---|---|
| A balanced answer that names the design disagreement up front | Common with current-generation models | *"Good. Now ask yourselves: would you have noticed if it hadn't? It did this unasked — which means it might not next time, and you would not know. That is why the next prompt forces it."* |
| A confident paragraph led by the 18% figure, RCT hedged or absent | Still common | *"Everything in that sentence is true and the overall impression is wrong. That is the failure mode of the whole session."* |
| Huang's diversity finding presented as citation-count evidence | Occasional | *"It just merged two different outcome variables. Hold that thought for the table."* |
| A number that appears in no paper | Rare — and invisible without the check | The most dangerous outcome, because it is invisible unless you open the PDF. Open the PDF. |

### Question 2 — the nuance question (3 minutes)

Now ask the question the first answer should have answered. Type exactly this:

```
Answer separately for each of the five sources, with a citation for every
answer. For each source give: (a) the study design, (b) whether articles were
randomly assigned to open access, (c) the population or sample, and (d) the
reported effect on citations, in the paper's own numbers. If a source does not
report one of these, write NOT REPORTED. Do not merge the sources into a single
conclusion.
```

**Why the prompt is shaped this way — say it out loud:**

- **"Answer separately for each source"** is the direct counter to the multi-document synthesis failures in the expert error schema `[5]`.
- **"In the paper's own numbers"** blocks the generic-ification measured in the generalization-bias study `[1]`.
- **"Write NOT REPORTED"** is the abstain rule from slide 22. Without it, a gap becomes an inference.
- **"Do not merge"** is doing the most work of any clause here. A synthesis is what you ask for last, not first.

**Expected outcome:** a five-row answer, each row citing its own source. Davis now clearly reads *randomised · no citation advantage at one year* `[28]`; Huang clearly reads *NOT REPORTED* or *different outcome* on the citation-count column `[32]`.

**Land it:** *"Same corpus, same model, ninety seconds apart. The first answer flattened the disagreement; the second one made it the structure of the answer. The difference is entirely in the prompt."*

### Question 3 — the question the corpus cannot answer (4 minutes)

**Set-up line, before you type:** *"I am now going to ask it something none of these five papers studies. I want you to watch whether it says so."*

Type exactly this:

```
According to these sources, does open access publishing shorten the time from
submission to publication? Cite the specific passages you used.
```

**Why this question:** none of the five studies measures editorial or peer-review timing. It is adjacent enough to the corpus to be tempting, and specific enough that a fabricated answer would be obvious.

**Expected outcome (the good one):** an explicit refusal — the sources do not address submission-to-publication time. Google documents this behaviour: "If the answer isn't in the source material, it won't provide a response" `[18]`, and out-of-scope requests can return "Gemini Notebook can't answer this question" `[16]`.

**What to say when it refuses:**

> *"That refusal is the single most valuable thing this tool did today. A general chatbot would have written you three fluent paragraphs about publication lag, and you would have had no way to tell they came from nowhere."*

**Then immediately add the honest caveat** — this is the part presenters skip:

> *"Do not over-trust it. In a 2026-published evaluation across more than thirty models of the 2024–25 generation, refusal accuracy on multi-document tasks dropped below 50%, and some models refused over 60% of questions they could have answered"* `[6]`. *"Refusal is a real capability. It is not a guarantee — and nobody has re-measured it on the current generation yet."*

**Expected outcomes, and what to do with each:**

| Outcome | How likely | What to say |
|---|---|---|
| Clean refusal, naming the gap | Most common in rehearsal — but refusal is tuned per release, so confirm the day before | As above. Then run the caveat `[6]`. |
| Refuses, but vaguely ("I don't have enough information") | Common | *"It detected the problem but could not categorise it. Those are two separate skills, and the benchmark separates them too"* `[6]`. |
| Answers anyway, using open-access background prose from the papers' introductions | Occasional | **The money shot.** Click every citation. Show that the cited passages are introduction throat-clearing, not findings. Point at slide 17. |
| Answers with a number | Uncommon | Stop everything and verify it live against the PDF. Whatever happens, the room will remember it. |

**Backup question, if it refuses too easily and you want a harder test** — this one is answerable only in part, so it probes over-caution rather than over-confidence:

```
According to these sources, how much of the scholarly literature was open
access, and in which year was that measured? Cite the passage.
```

Only Piwowar reports this — "at least 28% of the scholarly literature is OA", and 45% for 2015 `[30]`. A model in over-caution mode will refuse a question it can answer; that is the other half of the RefusalBench finding `[6]`.

---

## 4. The extraction table (7 minutes)

**Transition line:** *"Question 2 gave me the right answer in prose. Prose is where findings go to hide. Now I want a grid."*

### Step E1 — The extraction prompt (verbatim)

```
Build a comparison table with one row per source and these columns:

1. First author and year
2. Study design (exactly one of: randomised controlled trial, observational,
   systematic review)
3. Were articles randomly assigned to open access? (Yes / No)
4. Population or sample, with the number of articles or studies
5. Outcome measured (citations, downloads, citation diversity, or other)
6. Reported effect, in the source's own numbers and units
7. What the authors themselves say limits this result

Rules:
- One citation per cell.
- If a source does not report a field, write NOT REPORTED. Do not infer it.
- Do not combine sources in any cell.
```

**Say while it runs:** *"That is the Extraction Column Template from slide 22, four times over. Name, instruction, format, abstain rule. It is the same form you would build for a systematic review — and column 7 is the one people forget."*

**Expected outcome:** a five-row table. Column 2 splits 1 RCT / 3 observational / 1 systematic review. Column 3 reads Yes only for Davis. Column 5 is where Huang separates from the others.

### Step E2 — Read the table critically, on screen

Work through in this order:

1. **Column 3 (randomisation).** One "Yes" in five rows. *"Everything in the citation-advantage literature rests on four observational studies and one trial."*
2. **Column 6 against column 2.** The randomised row says no advantage `[28]`; the observational rows say 18–19% `[29]` `[30]`. *"The effect size is a function of the design."*
3. **Column 5 (outcome).** Huang measures citation diversity, not citation counts `[32]`. *"If this column were missing, that row would look like a fourth confirmation. Adding the outcome column is what stopped a synthesis error."*
4. **Column 7 (limitations).** In rehearsal this is usually the thinnest column in the table — the finding from slide 11: limitations are what summaries drop first `[2]`. If it comes back full, do not wave it through: verify one limitations cell against the PDF before praising it.
5. **The systematic-review row.** Ask specifically for its risk-of-bias result. This is the demo's best single moment — see E3.

### Step E3 — The follow-up that reverses the headline

Type exactly this:

```
In the systematic review, how many included studies were judged to be at overall
low risk of bias, and what did those studies find? Quote the passage.
```

**Expected outcome:** three studies at low risk of bias — one found an advantage, one found none, one found it in subsets `[31]`.

**Land it hard:**

> *"The headline of that review is '47.8% confirmed the citation advantage.' The appraisal inside it says the three most trustworthy studies split one-one-one. Most summaries I have generated from this corpus in rehearsal led with 47.8% and buried the one-one-one — and whether yours does or not, notice that you only saw the split because we asked this follow-up. That is what 'flattening nuance' means in practice — not a false statement, a lost qualifier."* `[1]`

### Step E4 — Export it

Export to CSV or copy into your reference manager's notes.

**Say:** *"The table is the artefact, not the chat. The chat is disposable; the table goes into the paper's methods appendix and into Zotero as a note attached to the five items"* `[27]`.

---

## 5. The verification segment (4 minutes)

**Set-up line:** *"We are going to check two cells. Only two — because I want you to time me."*

Pick deliberately:

- **Cell A:** a number from an observational row (Piwowar's 18%, or Ottaviani's 19%).
- **Cell B:** the thinnest cell in the limitations column.

For each, run the Summary-Trust Triage from slide 14 on screen:

| Step | What you do | What you say |
|---|---|---|
| 1. **Click the citation** | The source panel jumps to the quoted passage `[16]`. | *"This is what grounding buys you: two seconds instead of twenty minutes."* |
| 2. **Open the actual PDF** | Switch to the browser tab. Find the same sentence. | *"The citation chip proves the quote is in the source. It does not prove the quote means what the answer says it means."* |
| 3. **Compare the strength** | Read the source sentence, then the table cell, aloud. | *"Weaker, narrower, or opposite?"* `[1]` |
| 4. **Check the qualifier** | Did the cell keep the population, the design, the units? | *"A number without its population is a generic"* `[1]`. |
| 5. **Check the record** | For Ottaviani, search the journal page for a correction. | *"There is a published Correction to this paper. Your notebook has no idea. Nothing in the PDF told it."* |

**Expected outcomes:**

| Outcome | How likely | What to say |
|---|---|---|
| Cell is accurate and the quote supports it | Most common | *"Good — and note it took forty seconds, and you must do it anyway."* |
| Cell is accurate but the qualifier was dropped | Common | The core teaching moment. Point at slide 10 `[1]`. |
| Limitations cell is empty or generic | Common — verify in rehearsal | *"The paper has limitations. The table does not. That is the gap you fill by reading"* `[2]`. |
| Cell cites a real passage that does not support it | Occasional | *"Real, correctly quoted, wrong. This is the failure grounding does not remove"* `[11]`. |

### Closing the loop (1 minute)

Ask the room:

> **"Which of these three outputs — the paragraph, the per-source answer, or the table — would you put in a paper's methods appendix?"**

Land it on:

1. **The paragraph was fluent and lossy. The table was ugly and honest.** Ask for grids, not prose.
2. **The refusal was the most trustworthy thing it said all session** — and the best available measurement (2024–25 cohort) puts it at only ~50% reliable on multi-document questions `[6]`, so you still test it.
3. **You checked two cells in four minutes.** A 20-paper, 7-column table is 140 cells. *"Budget the verification, or do not build the table."*
4. **If this is going into an evidence synthesis:** you must be able to demonstrate the AI use did not compromise methodological rigour, and any AI use that "makes or suggests judgements" must be transparently reported `[13]`.

---

## 6. Fallback plan (assume something breaks)

| If this fails | Do this instead |
|---|---|
| Venue network is down or slow | Run entirely from the pre-built notebook and pre-captured screenshots. Narrate as if live; **say** you are using captures. |
| PDFs will not upload (copy-protected, size, format) | Use the pre-built notebook. Then make the failure the point: copy-protected PDFs are documented as non-importable `[17]`, and that is a real constraint on real corpora. |
| You hit the daily chat-query limit | Switch to the pre-run outputs. Say the number out loud — 50 queries a day, the documented limit `[18]` — because attendees planning a 60-paper corpus need to hear it. |
| The tool answers Question 3 instead of refusing | **Do not treat this as a broken demo — it is a better one.** Click every citation and show where the text came from. Then run the RefusalBench caveat `[6]`. |
| The tool refuses Question 1 or 2 (over-caution) | Use the backup question in §3, and cite the over-caution half of the same finding: some models refuse over 60% of answerable queries `[6]`. |
| Extraction returns prose instead of a table | Re-issue the prompt with "Return a markdown table only." If it still refuses, switch to a dedicated extraction tool where the table is the interface `[21]` `[23]`. |
| Every cell checks out clean | Say so honestly, then check the limitations column — it is almost never complete. If it is, show the rehearsal capture of a dropped qualifier and note that rates are per-query, not per-tool. |
| The interface has changed since rehearsal | Say "this changed since last week" and move on. It reinforces the freshness caveat on every slide — the product was renamed on 16 July 2026 `[14]`. |
| Someone asks about a tool not on slide 7 | *"It is not on the slide because I have not read its documentation, not because it is bad."* Add it to the follow-up list. |

### Pre-capture screenshot checklist (capture all of these the day before)

- [ ] The notebook with all five sources loaded, source panel visible
- [ ] The B3 sanity check — five sources, five designs, citation chips visible
- [ ] Question 1: the naive synthesis answer, **in full**
- [ ] Question 1: the same answer with the citation chips expanded
- [ ] Question 2: the per-source answer, showing at least one NOT REPORTED
- [ ] Question 3: **the refusal** — this is the one you must have
- [ ] Question 3: the backup question's answer (Piwowar's 28% / 45%)
- [ ] The extraction table, full width, all seven columns
- [ ] The extraction table's limitations column, showing how thin it is
- [ ] E3: the risk-of-bias follow-up answer (the one-one-one split)
- [ ] Verification: a citation chip clicked, with the source passage highlighted
- [ ] Verification: the same passage open in the actual PDF, side by side
- [ ] The PLOS ONE page for Ottaviani 2016 showing the published Correction
- [ ] A copy-protected-PDF import failure, if you can produce one

---

## 7. Presenter notes

**On the corpus.** Do not swap in papers from your own field. You will defend the notebook's answer instead of testing it, and the room will notice. If you must localise, keep the *shape*: one randomised or quasi-experimental study, two or three observational studies with a bigger effect, one systematic review, and one paper that measures a different outcome. That shape is what makes the table teach.

**On the Ottaviani correction.** This is the detail attendees remember. A grounded notebook is grounded in *the files you gave it* — not in the journal's current record. Erratum, retraction, corrigendum, expression of concern: none of it reaches the notebook. Your reference manager and the publisher's page are still the source of truth. Say it once, plainly.

**On the refusal.** Resist the urge to sell it. The honest framing is: refusal is a documented product behaviour `[18]` and a measured, imperfect capability `[6]`. It is a good reason to prefer a grounded notebook over a general chatbot for corpus work, and not a reason to skip verification.

**Two objections to expect.**

1. *"This is just RAG, and RAG is solved."* The honest 2026 answer has two parts. First, grounding solved the failure it was built for — fabricated sources — and not the one this demo is about: a real, correctly-quoted passage cited for a claim it does not support `[11]`, and citation accuracy that still varies substantially between systems that all sound equally fluent `[7]`. Second, RAG is no longer the only mechanism: million-token-context models can ingest a corpus this size whole, which removes retrieval misses but not misreading. The workflow you are teaching — per-source interrogation, quote-backed cells, a refusal test — is exactly what survives both mechanisms and every model update.
2. *"Extraction accuracy is already better than mine."* Partly fair — AI-assisted extraction produced 9.0% incorrect items against 11.0% for human-only in a study within six systematic reviews (Claude 2.1–3.5, a 2023–24 cohort) `[8]`. But that was AI-*assisted*: a human adjudicated every cell. Take the human out and adjusted performance drops to "moderate" `[9]` — and the current generation's own model cards report accuracy and hallucination rising together, so the adjudication step has not been retired `[34]`.

**What not to do.** Do not generate the summary first and the table second. The whole session is an argument for the opposite order, and the demo should model it: interrogate per source, build the grid, and only then say what the corpus means.

---

*AI for Researchers · Session 3: Reading, Notes & Knowledge Management · Landscape as of August 2026*