← series index
# Session 3: Reading, Notes & Knowledge Management
*AI for Researchers — attendee handout · Landscape as of August 2026*
---
## At a glance
| | |
|---|---|
| **Session** | 3 of 7 — Reading, Notes & Knowledge Management |
| **You'll learn** | How paper Q&A and grounded notebooks actually work, exactly where a summary drops the nuance that changes a conclusion, and how to turn a stack of papers into a checkable comparison table. |
| **You'll practise** | Building a five-paper grounded notebook, interrogating it per source, extracting a methods/effect-size table, and testing whether it refuses a question it cannot answer. |
| **Prerequisite** | None. Session 1's CRIT template and Capability/Failure Map, and Session 2's Literature-Discovery Tool Matrix, are recapped in two minutes. |
| **Tools referenced** | Gemini Notebook (formerly NotebookLM), Elicit, SciSpace, Paperpal, SCiNiTO, Zotero and its plugin ecosystem — surveyed, not endorsed. |
---
## The one-paragraph version
A summary is a lossy compression, and what it loses is not random. Across 4,900 summaries from ten models of the 2025 cohort, LLM summaries were **twice as likely** as the original abstracts to state a generalised conclusion, three models overgeneralised in **26–73%** of cases, prompting explicitly for accuracy made it **worse** (OR 1.90), and against expert-written summaries of the same research the odds of a broad generalisation were **4.85×** higher [1]. This has not aged out: the same team's 2026 follow-up found ChatGPT-5 and DeepSeek-V3.1 rate generic claims as *more* generalisable than laypeople do — while human domain experts rate them as *less* [33]. Ask the authors of the papers being summarised and they name what vanishes: methods, limitations, and the attribution of ideas to prior work — while "AI-summaries present author suggestions as established facts" [2]. And reading the summary instead of the paper measurably costs strong readers comprehension (d = 0.83) [3]. None of this makes the tools useless. It changes what you use them *for*: a summary tells you **what to read** and **what to compare**; a per-source extraction table is what you can actually defend. Grounded notebooks help — they answer only from sources you chose, with a clickable quote behind every claim [16][18] — but "grounded" removes fabricated sources, not misread ones [11], and the refusal you are relying on was right **less than half the time** on multi-document questions in the latest published evaluation [6].
---
## The reading & extraction tool table
Every cell comes from that tool's own current documentation. No cell is a comparative judgement; row order is not a ranking.
| Tool | Best for | Access / cost | Key limitation |
|---|---|---|---|
| **Gemini Notebook** (formerly NotebookLM, renamed 16 July 2026 [14]) | Source-bound Q&A across a corpus you assembled, with in-line citations you can hover and click into the source [15][16]; AI-Pro tiers add a "secure cloud computer" that writes and runs code for "data analysis grounded in your sources" [14] | Documented limits: 100 notebooks, 50 sources each, 50 chat queries/day, 3 audio generations/day; an upgrade is required to raise them [18] | 500,000 words / 200 MB per source; copy-protected PDFs will not import; footnotes and comments are not imported from Google files [17][18] |
| **Elicit** | Extraction tables — one column per data point, across many papers, with the supporting quote behind every cell [20][21] | Free tier plus paid plans; figure/table extraction is restricted to the higher tiers [21] | Without the PDF, "Elicit can only pull information from the abstract" [19] |
| **SciSpace** | PDF chat plus a data extractor that exports comparison tables to CSV, Excel or RIS [22][23] | Free tier for Chat with PDF [22] | Maximum 50 extraction columns per table [23] |
| **Paperpal** | Chat across uploaded sources inside a manuscript-preparation workflow; also does reference checking [24] | Free and paid tiers [24] | Scope is a writing workflow; corpus limits are not documented publicly [24] |
| **SCiNiTO** | Structured insights from a single uploaded article, with follow-up questions and an analysis history [25] | Free individual tier; institutional tier (see Session 2) [25] | PDF Analysis is documented per article; cross-corpus extraction is not documented [25] |
| **Zotero + plugins** | Keeping the *result* — annotations become notes carrying page links and citations into Word, LibreOffice or Google Docs [27] | Free and open source [26][27] | AI arrives as third-party code, and "plugins have full access to your Zotero and your computer"; there is still no official plugin directory [26] |
*Landscape as of August 2026; tool capabilities and pricing change quickly — verify before relying on this table.*
---
## Checklist A — Setting up a grounded notebook
- [ ] **1. One question per notebook.** Name the notebook with the question and the build date. A notebook whose scope you cannot reconstruct is not evidence.
- [ ] **2. Write down your inclusion criteria before you upload.** Your source list *is* a search strategy — Cochrane, Campbell, JBI and CEE require any AI use that "makes or suggests judgements" to be transparently reported [13].
- [ ] **3. Use full texts, not abstracts.** Abstracts state claims more strongly than the discussion sections behind them [4], and abstract-only tools inherit that overconfidence [19].
- [ ] **4. Check what failed to import.** Copy-protected PDFs are rejected outright; sources over 500,000 words or 200 MB are rejected; footnotes and comments do not come across from Google files [17][18].
- [ ] **5. Run a sanity query first.** *"List every source by first author, year and study design, with a citation."* Confirms it sees all sources, tells them apart, and that its citations resolve [16].
- [ ] **6. Test refusal before you trust answers.** Ask something the corpus demonstrably does not cover. If it answers, you have learned something important about that notebook today [6].
- [ ] **7. Remember what the notebook cannot see.** Corrections, retractions and expressions of concern live on the publisher's record, not in the PDF you uploaded. Check the record separately.
- [ ] **8. Note the query budget.** 50 chat queries a day, per the documented limits [18], does not go far across a large corpus.
---
## Prompt templates
All three are the Session 1 **CRIT** skeleton (Context / Role and register / Instructions and constraints / Task, then iterate) applied to reading.
**Template 1 — the per-source interrogation** *(use this instead of "summarise these papers")*
```
Answer separately for each source, with a citation for every answer.
For each source give:
(a) the study design
(b) the population or sample, with numbers
(c) the [outcome you care about], in the source's own numbers and units
(d) what the authors themselves say limits the result
If a source does not report one of these, write NOT REPORTED. Do not infer it.
Do not merge the sources into a single conclusion.
```
*When to use:* always, before you ask for any synthesis.
*Watch out for:* the "do not merge" clause is doing most of the work. Multi-document synthesis failures are a named category in the expert error schema for scholarly QA [5], and a merged answer hides which paper each claim came from.
**Template 2 — the extraction column** *(one column at a time; this is the artefact from slide 22)*
```
Column name: [one phrase — e.g. "Primary outcome measured in the study"]
Instruction: [where to look, what counts, and what to exclude]
Format: [e.g. "mean ± standard deviation"; "one of: RCT / observational /
systematic review"; "Yes / No / Maybe"]
Precision: Be as precise as possible — "50 hours" is better than "2 days".
Abstain rule: If the paper does not state this, write NOT REPORTED.
Do not infer it.
```
*When to use:* whenever you would otherwise build an extraction form by hand.
*Watch out for:* the abstain rule is the load-bearing line — without it a gap silently becomes an inference. Constrain the answer space wherever you can (Yes/No/Maybe, or a fixed list) [20], and pilot your columns on three or four papers before running them across the whole corpus [21].
**Template 3 — the "what am I missing?" probe** *(run this last, every time)*
```
Based only on these sources:
1. Which of my questions can this corpus NOT answer? List them explicitly.
2. Where do the sources disagree with each other, and what explains the
disagreement — design, population, outcome measure, or time period?
3. Which claims in your previous answer rest on a single source?
Cite the specific passages for 2 and 3.
```
*When to use:* before you write anything based on the notebook.
*Watch out for:* question 1 is a refusal test, and refusal is unreliable in both directions — frontier models' refusal accuracy dropped below 50% on multi-document tasks, while some models refused over 60% of questions they could have answered [6]. That evaluation covers the 2024–25 cohort; no equivalent test of the current generation exists yet, which is a reason to run the test, not skip it. Treat a confident "nothing is missing" as a prompt to check yourself.
---
## Checklist B — Trust but verify a summary
Run this on any AI summary or extraction cell you intend to rely on. Stop at the first failure — you have already learned what you needed.
- [ ] **1. Is there a number in it?** Effect sizes, sample sizes, confidence intervals — go to the source. Always. A number without its population is a generic [1].
- [ ] **2. Has a qualifier vanished?** No population, no design, no boundary condition? The commonest documented failure is converting a quantified finding into a generic one — 26–73% of cases for three of the ten 2025-cohort models tested [1], with the bias confirmed on GPT-5-class models in 2026 [33].
- [ ] **3. Are the methods there?** Methods are the first thing summaries drop, and they carry "the validity of the findings" [2].
- [ ] **4. Are the limitations there?** Second thing dropped [2]. If the limitations column of your table is the thinnest one, that is the expected result, not a clean bill of health.
- [ ] **5. Does every claim carry a citation?** In a grounded tool, unsupported prose is a red flag, not a stylistic choice [16].
- [ ] **6. Click the citation, then open the actual PDF.** The chip proves the quote exists in the source; it does not prove the quote means what the answer says. Real-but-misdescribed citations are the failure grounding does *not* remove [11].
- [ ] **7. Compare the strength.** Does the source say *that*, or something weaker, narrower, or opposite?
- [ ] **8. Check the publication record, not just the file.** Corrections, retractions and errata are invisible to a notebook built from PDFs.
- [ ] **9. Did the corpus have the full text?** If the tool only had abstracts, expect claims stated more strongly than the papers support [4][19].
- [ ] **10. If this is going into an evidence synthesis:** can you demonstrate the AI use "will not compromise the methodological rigour or integrity" of the review, and have you transparently reported any AI use that makes or suggests a judgement? [13]
---
## Three things worth remembering
**1. Ask for a column, not a summary.** A summary is one blob of prose you cannot audit; a column is a defined question asked of every paper the same way, with a quote behind each cell [21]. The best evidence in this session is for AI-*assisted* extraction: 9.0% of items incorrect versus 11.0% for human-only across six ongoing systematic reviews, saving a median 41 minutes per study — with a human adjudicating every cell [8]. Remove the human and adjusted performance for data extraction drops to "moderate" [9]. Both studies used 2023–24 models (Claude 2.1–3.5; GPT-4), but the reason the discipline survives is current: vendors' own 2026 model cards report accuracy and hallucination rising *together* — Opus 5 is 11% more accurate and hallucinates 6% more than its predecessor [34].
**2. The column that decides the answer is "design".** In the session's five-paper demo corpus on open access and citations, the observational studies report an 18–19% citation advantage [29][30], the one randomised trial finds none [28], the systematic review finds 47.8% of 134 studies confirming an advantage — but of the three studies at low risk of bias, one found an advantage, one found none, and one found it only in subsets [31] — and the fifth paper measures a different outcome entirely, citation *diversity* [32]. A comparison table makes that visible. A summary makes it disappear.
**3. Let the notes outlive the tool.** Notebooks are disposable; your reference manager is not. Zotero annotations become notes that carry a link back to the exact PDF page and a citation you can insert into a manuscript [27] — the manual, fully auditable version of what a grounded notebook automates. If you add AI to Zotero through a plugin, read Zotero's own warning first: "plugins have full access to your Zotero and your computer. You should only install plugins from developers you trust" [26].
---
## Further reading
- Peters, U. & Chin-Yee, B. (2025). *Generalization bias in large language model summarization of scientific research*. Royal Society Open Science — https://doi.org/10.1098/rsos.241776 — the single most useful paper in this session. Open access, readable in twenty minutes, and the source of every 2025-cohort number in the critical-literacy segment. **Start here.**
- Peters, U., Bertazzoli, A., DeJesus, J. M., van der Velden, G. J. & Chin-Yee, B. (2026). *Generics in science communication: Misaligned interpretations across laypeople, scientists, and large language models*. Public Understanding of Science — https://doi.org/10.1177/09636625261425891 — the same team's follow-up, published 20 April 2026: on GPT-5-class models the bias persists, and it is directional — experts read a generic claim narrowly, LLMs read it more broadly than even untrained laypeople.
- Gartlehner, G. et al. (2025). *Artificial Intelligence–Assisted Data Extraction With a Large Language Model: A Study Within Reviews*. Annals of Internal Medicine — https://doi.org/10.7326/ANNALS-25-00739 — the best evidence we have on what AI-assisted extraction actually costs and buys, from six real ongoing reviews.
- Muhamed, A. et al. (2026). *RefusalBench: Generative Evaluation of Selective Refusal in Grounded Language Models*. EACL 2026 — https://aclanthology.org/2026.eacl-long.321.pdf — read it for the number that should govern how much you trust a grounded notebook's silence — measured on the 2024–25 cohort; nothing newer has replaced it.
- Hammond, K. & Newell, S. (2025). *Reading between the GenAI lines*. ASCILITE Publications — https://doi.org/10.65106/apubs.2025.2719 — two pages, twenty authors, and the clearest available account of what an AI summary of *your own paper* leaves out (summaries by ChatGPT-4; the categories, not the model, are the point).
- Flemyng, E. et al. (2025). *Position statement on AI use in evidence synthesis across Cochrane, the Campbell Collaboration, JBI and CEE 2025* — https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.ED000178/full — the rules you will actually be held to if an extraction table reaches a formal synthesis.
- Google (2026). *Frequently asked questions*, Gemini Notebook Help — https://support.google.com/gemininotebook/answer/16269187 — the documented limits, in the vendor's own words — re-verified unchanged on 2026-08-21. Re-read it before you plan a corpus; these numbers move.
*Full annotated source list for this session — 34 entries, 18 of them peer-reviewed — see `sources.md`. The five-paper demo corpus, with DOIs, is in `demo-script.md`.*
---
*AI for Researchers · Session 3: Reading, Notes & Knowledge Management · Landscape as of August 2026*