AI for Researchers

Session 5: Data, Code & Your AI Workflow

What it can do with your data, where it fails invisibly, and the four habits that make either one defensible

{{Presenter Name}}

Landscape as of August 2026

2-Minute Recap

The Series So Far

Session 1 — The Capability/Failure Map for Research Tasks: every task sits in Safe, Unreliable or Dangerous. "Generating analysis code" is listed under Unreliable.
Session 1 — The 2026 Tool Landscape Taxonomy: Tier 1 — General chatbots · Tier 2 — Research-specific tools · Tier 3 — Deep-research agents.
CRIT — Context / Role and register / Instructions and constraints / Task, then iterate — is our prompting template.
Session 2 — The Literature-Discovery Tool Matrix: six citation-grounded tools, compared from their own documentation.
Session 3 — The Summary-Trust Triage: when an AI summary obliges you to reopen the paper.
Session 4 — The Publisher AI-Policy Comparison Table: what you must disclose, and where.

Today (Session 5): the capstone. You have found the papers, read them, and written the draft. Now the data — and then we put all five sessions on one page you take home.

Learning Objectives

By the End of This Session You Will Be Able To…

Explain the four reproducibility guardrails — keep the code, pin the versions, set the seed, verify one number by hand — needed when using AI for data analysis or code generation.
Run an AI-assisted statistical analysis or visualization task on a sample dataset.
Apply a no-code analysis path to a dataset without writing code directly.

Objectives 4–6 continue on the next slide.

Learning Objectives

…And You Will Also Be Able To

Evaluate the output of an AI-assisted analysis for correctness and reproducibility before trusting it.
Assemble a personal AI workflow spanning the discovery, reading, writing, and analysis stages of the research lifecycle.
Identify strategies for staying current as agentic coding/analysis tools continue to evolve.

Section 01

What AI Actually Does to Your Data

Three jobs, one mechanism — and the mechanism is identical whether or not you ever see a line of code.

Literacy Foundation

Three Jobs You Are Actually Asking It to Do

Write the code. Load, clean, reshape, model — the script you would otherwise have written or commissioned.
Advise on the statistics. Which test, which assumptions, how to read the output.
Make the figure. And with it, the claim the figure implies to a reader who will never see the data.
These fail in different ways, so they need different checks — and only one of the three fails loudly.
All three sit in Session 1's Unreliable zone: never the only pass, always spot-checked.

Session 1's Rule 1 is the entry condition for all of today: if you could not catch the error, the task is not in the Safe zone.

The Mechanism, In Their Own Words

"No-Code" Is Not No Code. It Is Code You Have Not Read.

"For some data-analysis tasks, ChatGPT writes and runs Python code in a stateful Jupyter notebook environment." OpenAI, Data analysis with ChatGPT ^[1]

Excel: "Copilot uses Python-based analysis…" — and you may "expand the code to see how the answer was generated". ^[5]
Colab's data-science agent shows every suggestion and waits for Accept or Accept and run. ^[3]
But not every path shows you anything: one major assistant's own upload documentation does not state that it writes or runs visible Python at all. ^[4]
So the question is not "do I code?" It is "can I get the code out?" — a property of the tool, chosen before you upload.

Before You Upload Anything

Where Does Your Dataset Go?

Uploaded files may be used for training unless you turn it off: "Improve the model for everyone", in Settings > Data Controls. ^[1]
Execution sandboxes are isolated and have no internet; one vendor's containers "expire 30 days after creation". ^[2]
Python in Excel "runs within the compliance boundary of your organization"; "Data isn't persisted in the Microsoft Cloud". ^[5]
Retention differs sharply by product and tier — one tool deletes free-tier files after 1 hour of inactivity, 7 days on paid plans. ^[6]
And read the vendors' own warnings: a file-handling assistant "can be tricked into sending information from its context… to malicious third parties". ^[2]

Your institution's data-governance policy outranks every line above. Public, consented, de-identified or already-published data is the only safe default for a tool you have not had reviewed.

The Evidence — Measured, Not Asserted

How Well Does It Actually Do an Analysis?

Landscape as of August 2026. Figures from each paper's abstract, leaderboard, or the named independent evaluation; model vintages stated. Trends pair a 2024 baseline with its 2026 companion; rows without one are dated baselines. Rows are comparable to your expectations, not to each other.
Evaluation	What it measures	Best result reported	Read it with
ScienceAgentBenchICLR 2025	102 data-driven science tasks from 44 papers, 9 expert validators	32.4% (2024) → 33.3% (o3, Apr 2025), then frozen ^[9] ^[49]	Superseded by AstaBench: 58.0% overall, ~3% perfect end-to-end discovery ^[48]
DSBenchICLR 2025	466 realistic data-analysis tasks, long contexts, multi-table	34.12% of analysis tasks (2024-era models) ^[13]	Competition-derived — better specified than your question; no re-run found
SciCodeNeurIPS 2024	338 research-coding subproblems, 16 science sub-fields	Subproblems: ~33% (2025) → 60.2%, Claude Fable 5 ^[14] ^[50]	Board stale (10.8% main, early 2025); 60.2% via Artificial Analysis
StatQANeurIPS 2024	11,623 statistical-analysis items: choose and apply the method	64.83% (GPT-4o) ^[10]	Mid-2024; dormant since — successor below. Error type: next slide
StatABenchpreprint, June 2026	404 closed-form statistics questions, 18 topics	68.6% (GPT-5.1) ^[17]	Preprint, two years newer — gap narrowed, not closed
CORE-Benchpreprint · HAL	Reproduce a published result, code and data supplied	21% (2024) → 77.8%, Opus 4.5 + Claude Code ^[15] ^[49]	95.5% manually scored; HAL paused — no current-frontier entries ^[49]

Critical Literacy — The Key Slide

The Error That Matters Is Not Arithmetic

"LLMs primarily make applicability errors, whereas humans mostly make statistical task confusion errors." Zhu et al., StatQA, 11,623 items, NeurIPS 2024 ^[10]

An applicability error is silent. The test ran, the arithmetic is right, the assumptions did not hold — and nothing in the output says so.
Two years newer, still open: GPT-5.1 reaches only 68.6% on the successor benchmark — and half its audited failures are still tool selection. ^[17]
Given the same research questions as expert annotators, agents reproduced under 13% of the experts' modelling decisions (2024 cohort). ^[11]
Against a reference statistics package, discrepancies clustered in post-hoc tests, confidence intervals, sign conventions and non-parametric tests. ^[18]

The fair counterweight: six models chose the correct test in 100% of 20 textbook scenarios — basic scenarios, free tiers, one day in August 2025. ^[19] Choosing the test is close to solved. Applying it to your data is not.

The Evidence

Ask It Twice. The Answer Is Not the Same.

21–81%workflow reproducibility across 15 model pairs, 1,032 tasks, three task sets ^[12]

480runs of the same analysis, same settings: "considerable variation" ^[43]

75.8%of coding tasks gave zero identical outputs across repeat requests ^[44]

The finding you act on: inconsistent answers are also measurably less accurate — 64.4% vs 53.5%, p < 0.001. Instability is a free quality signal. ^[12]
And you cannot switch it off with a setting: "setting the temperature to 0 does not guarantee determinism in code generation". ^[44]
The authors' own recommendation: an LLM analysis "should be run multiple times independently and the distribution of results considered". ^[43]

Stated precisely: "reproducibility" in the first figure means consistency of the generated workflow, judged by an inspector model; the span covers three task sets (21–55%, 30–75%, 48–81%), and human experts scored 66.5% on the first. ^[12]

The Evidence

And in the Code Itself, Three Documented Failures

It solved a different problem. The largest category in a manual annotation of 1,380 generated snippets was "task requirement conflicts" — 43.53% of the hallucinations found, not crashes. ^[20]
It invented the library. Across 576,000 samples from 16 mid-2024 models, 5.2% of packages recommended by commercial models and 21.7% by open-source ones did not exist. ^[21] Re-measured on five 2026 frontier models: 4.62–6.10% — the spread collapsed; the threat did not (preprint). ^[51]
It wrote something insecure. In real repositories, 29.5% of AI-generated Python snippets carried a security weakness across 43 CWE categories — and static analysis plus the assistant fixed up to 55.5% of them. ^[22]

Stated precisely: 43.53% is a share of the hallucinations found, not of all generated code. ^[20] The one to fear is the first: code that runs, returns a number, and answers a question you did not ask.

The Evidence — Two RCTs That Disagree

How Fast It Feels Is Not How Fast It Is

−24%time developers forecast AI would save them ^[24]

−20%time they believed it saved, afterwards ^[24]

+19%time it actually took them — measured ^[24]

A different RCT, on one self-contained greenfield task, found the assistant group 55.8% faster — vendor-authored, and silent on whether the code was correct. ^[25]
In a controlled study, people with an AI assistant wrote less secure code while being more likely to believe it was secure. ^[23]

Honest caveats: n = 16 developers on codebases they knew for five years ^[24]; n = 47 students on security tasks with a 2022-era model ^[23]. The durable finding is the calibration gap, not the sign of the effect.

Section 02

Reproducibility Guardrails

Four habits. Three of them were published in 2013. AI did not create the problem — it multiplied the volume of code flowing into it.

The Evidence — The Pre-AI Baseline

Research Code Already Did Not Run

74%of 9,000+ deposited R files failed to run without error; 56% after cleaning ^[26]

26%of 204 Science papers under a code policy could be reproduced ^[28]

879of 27,271 notebooks from 3,467 biomedical papers reproduced their own results ^[27]

The dominant cause was not bad science. It was undeclared dependencies and hard-coded file paths. ^[27]
Measured directly: of 296 R projects deposited on a repository, 98.8% carried no formal dependency description; after rebuilding environments automatically, 25.87% ran without error. ^[45]
Under a sharing policy, artifacts could be obtained for 44% of papers — and findings reproduced for 26%. Policy is not practice. ^[28]
Read the last section against this baseline. The question is not whether AI is worse than a human analyst. It is whether your result survives being re-run.

Series Artefact

The Four Reproducibility Guardrails

Guardrails 1–3 are Sandve et al.'s Rules 1, 3 and 6, published in 2013 ^[29]. Guardrail 4 is the one AI adds — because for the first time the analyst who made the choices cannot be asked why.
Guardrail	What you actually do	Why — the evidence	If you skip it
1 · Keep the codebefore you close the tab	Ask to see it, save it beside the data, and re-run it outside the chat at least once	Sandboxes are ephemeral; one vendor's containers "expire 30 days after creation" ^[2]	You have a transcript, not a result
2 · Pin the versionstool, libraries, date	Ask it to print every library version; save an environment file; record the model and date	Undeclared dependencies were the top failure cause across 27,271 notebooks ^[27]	It runs today and fails at revision
3 · Set the seedand say where	Put the seed in the prompt and in the code; make the tool state where randomness entered	Rule 6 of the 2013 ten rules: "note underlying random seeds" ^[29]	Two runs, two answers, no explanation
4 · Verify one numberby hand, every time	Recompute one reported figure outside the tool. Check n. Ask what was dropped	Applicability errors are invisible in the output ^[10]; the same question gives a different workflow 21–81% of the time ^[12]	Silent subsetting and wrong-assumption tests pass unnoticed

The Rules, In Their Words

None of This Is Optional — It Is Already in the Policies

"If authors write or edit code using AI tools as part of their research, they should declare this in detail in the Methods section… The same standards for the validity and reproducibility of code apply whether the code was produced with AI assistance or not." Elsevier, Generative AI policies for journals, updated June 2026 ^[42]

Springer Nature: code must go to "a repository that assigns a permanent identifier, such as Code Ocean or Zenodo" — "providing a GitHub link only is not sufficient". ^[30]
COPE: AI used "in the collection and analysis of data" must be disclosed "in the Materials and Methods… how the AI tool was used and which tool was used". ^[31]
ICMJE: writing assistance goes in the acknowledgments — but "data collection, analysis, or figure generation" goes in the methods. ^[32]

Session 4's rule still holds: the publisher page gives you the shape, the journal's guide for authors gives you the wording. Check it the week you submit.

Section 03

If You Do Not Code

Everything so far applies to you unchanged — and one column of the next table decides whether you can obey guardrail 1 at all.

Series Artefact

No-Code Analysis Paths, From Their Own Documentation

August 2026; every cell from that tool's own documentation, fetched 2026-07-28. Rows are *paths*, not endorsements. **Column 3 decides whether you can obey guardrail 1.** New since compilation: Gemini Notebook (AI Pro) adds cloud code execution — press-reported, not vendor-documented ^[52] ^[53]; check code visibility first.
Path (examples)	What it does for you	Can you get the code out?	A documented limit to know
Assistant + code executionChatGPT, Claude	Upload a file, ask in plain English; it writes and runs Python	Yes — ask for it, and save it ^[1] ^[2]	No internet in the sandbox; ~50 MB CSV ceiling ^[1] ^[2]
Spreadsheet copilotCopilot in Excel	Answers questions about the workbook you already have	Yes — "expand the code to see how the answer was generated" ^[5]	Results "aren't refreshable"; it does not modify your workbook ^[5]
Notebook agentColab Data Science Agent	Plain-English request becomes code, into your notebook	Yes, by design — Accept, or Accept and run ^[3]	CSV and BigQuery sources only; runs only in your runtime ^[3]
Dedicated analysis toolJulius	Upload, ask, get tables and charts; reads SPSS files	Yes — notebooks "re-run, version, and share" ^[6]	Free-tier files deleted after 1 h idle; size limit undocumented ^[6]
Point-and-click stats + AIJASP 0.98, July 2026	Menu-driven tests; optional AI to annotate and interpret	The file "holds analyses and interpretations" alongside the data ^[7]	AI is opt-in with your own API key; data handling not documented ^[7]
Point-and-click stats, no AIjamovi	Menu-driven tests, publication-formatted output	Yes — Syntax mode emits "equivalent R code for each analysis" ^[8]	Sharing needs colleagues to hold the same R packages ^[8]

Section 04

Agentic Tools, in the Present Tense

From "write me a script" to "do the analysis" — that shift already shipped: Claude Science, Gemini Notebook's code execution, Elicit's Research Agent, Codex as a general work agent. The approval prompt is the last place a human is guaranteed to be standing.

The Mechanism, In Their Own Words

What "Agentic" Means, From the Vendors' Own Docs

The tool will "determine which files to make changes to, offer code changes and terminal commands for the user's approval, and iterate to remediate issues until the original task is complete." GitHub, Copilot features — agent mode ^[33]

One agent runs with "strict read-only permissions by default" and fail-closed matching: unmatched commands "default to requiring manual approval". ^[34]
Another defaults to a read-only sandbox with "network access turned off". ^[35]
Inside a notebook: "Agents request permission before writing files, running commands, or using MCP tools." ^[36]

And their own warnings, quoted: "no system is completely immune to all attacks" ^[34]; "prompt injection can cause the agent to fetch and follow untrusted instructions" ^[35]. The approval prompt is not a formality. It is the control.

The Evidence — The Trajectory, Re-Measured

The Trajectory Is Real. So Is the Gap.

16.9% → 64.44%Kaggle-medal rate on 75 ML-engineering contests, Oct 2024 → Feb 2026 ^[16]

21% → 77.8%reproducing a published result with code and data supplied; 95.5% manually scored ^[15] ^[49]

~3%perfect completion on AstaBench's end-to-end discovery tasks ^[48]

Where the task is well specified and scored by a metric — ML engineering, computational reproduction — two years moved the number by 40–50 points. ^[16] ^[49]
Where the task requires framing the question, grounding it in a field and carrying a discovery end to end, the best 2026 agent completes ~3% perfectly — and statistics still sits at 68.6%. ^[48] ^[17]
That second list is your job description. The reassurance survives — on new evidence, not old numbers. Plan for the first column to keep improving and the second to remain yours.

Stated honestly: MLE-bench's top entries are self-reported and it is not taking new submissions while fairness checks improve ^[16]; HAL is paused, so 77.8% is an Opus 4.5-class agent, not the current frontier ^[49]; the AstaBench figures are Ai2's own leaderboard ^[48].

Practice

Staying Current Without Reading Marketing

Preprints, for speed: arXiv cs.CL, cs.SE and stat.ML carry these benchmarks months before peer review — and filter nothing. ^[37]
Proceedings, for trust: the NeurIPS Datasets and Benchmarks Track, ICLR, ICML (PMLR) and the ACL Anthology — where this deck's evaluation papers landed. Its 2026 companion figures come from leaderboards — read their pause and provenance notes as data. ^[38]
News desks, for orientation: the Nature and Science news pages are staffed summaries — a lead to follow, never a citation. ^[39]
The one durable habit: when a number impresses you, go and find the task set — how many tasks, which model version, how many attempts, and who paid for it.
If there is no task set, it is not a result. It is an advertisement.

Section 05

The Capstone

Five sessions, five artefacts, one page. Two rows are deliberately empty — Sessions 6 and 7 fill them in.

Series Artefact · The Capstone

The Personal AI Workflow Canvas

One page, one project, one date. Rows are stages of your work, not products — so the canvas survives your tools being replaced. Fillable version: last page of the handout.
Lifecycle stage	What you hand to AI	Tier · Zone	Your non-negotiable check	Artefact that governs it
1 · FrameSession 1	The shape of the question; a weak prompt to sharpen	Tier 1 · Safe	Read every line you keep. Could you catch the error?	CRIT
2 · DiscoverSession 2	Search terms; candidate papers; a first synthesis	Tier 2 first, 3 for breadth · Unreliable	Every citation resolved in a database before it is used	Literature-Discovery Tool Matrix
3 · ReadSession 3	Q&A over your own PDFs; extraction columns	Tier 2, grounded · Unreliable	Reopen the paper whenever a number or a qualifier appears	Summary-Trust Triage
4 · AnalyseSession 5 — today	Analysis code; one test; one figure	Tier 1 + sandbox, or agentic · Unreliable	Keep the code · pin versions · set the seed · recompute one number by hand	The Four Reproducibility Guardrails
5 · WriteSession 4	Language and structure of text you wrote; the disclosure sentence	Tier 1 · Safe as editor, Dangerous as ghostwriter	Diff every touched paragraph; disclose in the journal's own format	Publisher AI-Policy Comparison Table
6 · Cross-field orientation — you fill this row in during Session 6
7 · Your discipline's specialised tools — you fill this row in during Session 7

Series Artefact

Filling It In: Three Rules

Rule 1 — Fill the check column first. If you cannot name the check, you have not decided to delegate the task; you have decided to hope. Session 1's Rule 1, applied to a whole project.
Rule 2 — One project, not a career. A canvas for "my research" is a wish. A canvas for "this paper, this dataset, this journal" is a workflow you can hand to a co-author.
Rule 3 — Date it, and re-check it in six months. Every tool named in these five sessions changed during the ten weeks we ran them. The stages did not.
And leave rows 6 and 7 empty. They are next session's homework, and the session after that.

The canvas is deliberately tool-neutral: it names tasks, tiers and checks. Nothing on it becomes wrong when a product is renamed.

Demo Preview

CRIT, Applied to an Analysis You Can Defend

C — Context: "I have uploaded a public CC-0 dataset of 344 penguins. I am a researcher, not a programmer."
R — Role and register: "Act as a careful research statistician doing a first pass on someone else's data. Plain language."
I — Instructions and constraints: "Show the code every time. Report missing values per column. Drop nothing without telling me. Use Welch's t-test. Set the seed to 42. Print every library version."
T — Task: "Describe this dataset and its data-quality problems." Then iterate — and then verify.

Demo: upload → explore → one test → one chart, in about thirteen minutes — and then seven minutes of checking: re-run the code outside the chat, recompute one group mean by hand, and check the conclusion against the peer-reviewed paper the data came from. ^[40] ^[41]

References

References (1–18)

OpenAI (2026). Data analysis with ChatGPT; File storage and Library in ChatGPT. help.openai.com/en/articles/8437071 — accessed 2026-07-28
Anthropic (2026). Create and edit files with Claude; Code execution tool. support.claude.com · platform.claude.com/docs — accessed 2026-07-28
Google (2026). Use the Data Science Agent. Colab Enterprise documentation. docs.cloud.google.com/colab/docs/use-data-science-agent — accessed 2026-07-28
Google (2026). Upload & analyze files in Gemini Apps. support.google.com/gemini/answer/14903178 — accessed 2026-07-28
Microsoft (2026). Get direct answers to your data analysis questions; Data security and Python in Excel. support.microsoft.com — accessed 2026-07-28
Julius AI (2026). What is Julius AI?; Files; Containers. julius.ai/docs/get-started — accessed 2026-07-28
JASP (2026, 2 July). Introducing JASP 0.98: Fully Integrated AI Support; Getting started. jasp-stats.org — accessed 2026-07-28
jamovi (2026). Combining jamovi and R. User manual §6. docs.jamovi.org/usermanual/um_6_jamovi_and_R.html — accessed 2026-07-28
Chen, Z., et al. (2025). ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery. ICLR 2025. arxiv.org/abs/2410.05080 — accessed 2026-07-28
Zhu, Y., Du, S., Li, B., Luo, Y., & Tang, N. (2024). Are Large Language Models Good Statisticians? (StatQA). NeurIPS 2024, Datasets & Benchmarks. proceedings.neurips.cc — accessed 2026-07-28
Gu, K., et al. (2024). BLADE: Benchmarking Language Model Agents for Data-Driven Science. Findings of EMNLP 2024. arxiv.org/abs/2408.09667 — accessed 2026-07-28
Zeng, Q., Jin, C., Wang, X., Zheng, Y., & Li, Q. (2025). AIRepr: An Analyst-Inspector Framework for Evaluating Reproducibility of LLMs in Data Science. Findings of EMNLP 2025. arxiv.org/abs/2502.16395 — accessed 2026-07-28
Jing, L., et al. (2025). DSBench: How Far Are Data Science Agents from Becoming Data Science Experts? ICLR 2025. arxiv.org/abs/2409.07703 — accessed 2026-07-28
Tian, M., et al. (2024). SciCode: A Research Coding Benchmark Curated by Scientists. NeurIPS 2024, Datasets & Benchmarks; official leaderboard. arxiv.org/abs/2407.13168 · github.com/scicode-bench/SciCode — accessed 2026-07-28
Siegel, Z. S., et al. (2024, rev. 2026). CORE-Bench: A Computational Reproducibility Agent Benchmark. arxiv.org/abs/2409.11363 — accessed 2026-07-28
Chan, J. S., et al. (2025). MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. ICLR 2025; official leaderboard. arxiv.org/abs/2410.07095 · github.com/openai/mle-bench — accessed 2026-07-28; leaderboard re-checked 2026-09-02
Zhu, Y., et al. (2026). StatABench: Dataset and Framework for Evaluating Statistical Analysis Capabilities of LLMs. Preprint. arxiv.org/abs/2606.22977 — accessed 2026-07-28
Shahrul, A. I., & Syed Mohamed, A. M. F. (2024). A Comparative Evaluation of SPSS and ChatGPT-4 in Statistical Analyses. Cureus 16, e72581. doi.org/10.7759/cureus.72581 — accessed 2026-07-28

References

References (19–36)

Shukla, M., et al. (2025). Evaluating… Large Language Models in Statistical Test Selection. Cureus 17, e94949. doi.org/10.7759/cureus.94949 — accessed 2026-07-28
Zhang, Z., et al. (2025). LLM Hallucinations in Practical Code Generation. ISSTA 2025. arxiv.org/abs/2409.20550 — accessed 2026-07-28
Spracklen, J., et al. (2025). We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs. USENIX Security '25. arxiv.org/abs/2406.10279 — accessed 2026-07-28
Fu, Y., et al. (2025). Security Weaknesses of Copilot-Generated Code in GitHub Projects. ACM TOSEM. arxiv.org/abs/2310.02059 — accessed 2026-07-28
Perry, N., Srivastava, M., Kumar, D., & Boneh, D. (2023). Do Users Write More Insecure Code with AI Assistants? ACM CCS '23. arxiv.org/abs/2211.03622 — accessed 2026-07-28
Becker, J., et al. (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. METR. arxiv.org/abs/2507.09089 — accessed 2026-07-28
Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023). The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arxiv.org/abs/2302.06590 — accessed 2026-07-28
Trisovic, A., et al. (2022). A large-scale study on research code quality and execution. Scientific Data 9, 60. doi.org/10.1038/s41597-022-01143-6 — accessed 2026-07-28
Samuel, S., & Mietchen, D. (2024). Computational reproducibility of Jupyter notebooks from biomedical publications. GigaScience 13. doi.org/10.1093/gigascience/giad113 — accessed 2026-07-28
Stodden, V., Seiler, J., & Ma, Z. (2018). An empirical analysis of journal policy effectiveness for computational reproducibility. PNAS 115. doi.org/10.1073/pnas.1708290115 — accessed 2026-07-28
Sandve, G. K., et al. (2013). Ten Simple Rules for Reproducible Computational Research. PLoS Comput. Biol. 9(10), e1003285. doi.org/10.1371/journal.pcbi.1003285 — accessed 2026-07-28
Springer Nature (2026). Code policy. springernature.com/gp/open-science/code-policy — accessed 2026-07-28
COPE Council (2023). Position statement: Authorship and AI tools. publicationethics.org/guidance/cope-position/authorship-and-ai-tools — accessed 2026-07-28
ICMJE (2025). Recommendations — Defining the Role of Authors and Contributors (II.A.4). icmje.org/recommendations — accessed 2026-07-28
GitHub (2026). GitHub Copilot features. docs.github.com/en/copilot/get-started/features — accessed 2026-07-28
Anthropic (2026). Claude Code — Security. code.claude.com/docs/en/security — accessed 2026-07-28
OpenAI (2026). Codex — Agent approvals & security. learn.chatgpt.com/docs/agent-approvals-security — accessed 2026-07-28
Project Jupyter (2026). Jupyter AI v3 documentation. jupyter-ai.readthedocs.io/en/v3/ — accessed 2026-07-28

References

References (37–53)

arXiv (2026). Category listings: cs.CL, cs.SE, stat.ML. arxiv.org/list/cs.CL/recent · /cs.SE/recent · /stat.ML/recent — accessed 2026-07-28
Official proceedings: ACL Anthology; NeurIPS Proceedings (incl. Datasets & Benchmarks Track); PMLR/ICML; ICLR. aclanthology.org · papers.nips.cc · proceedings.mlr.press · iclr.cc — accessed 2026-07-28
Nature news and Science news. Publisher news desks; orientation only, never a citation. nature.com/nature/articles?type=news · science.org/news/all-news — accessed 2026-07-28
Horst, A. M., et al. (2020). palmerpenguins: Palmer Archipelago (Antarctica) penguin data. R package v0.1.0. CC0; licence verified at CRAN and in LICENSE.md. doi.org/10.5281/zenodo.3960218 — accessed 2026-07-28
Gorman, K. B., et al. (2014). Ecological sexual dimorphism and environmental variability within a community of Antarctic penguins (genus Pygoscelis). PLOS ONE 9(3), e90081. doi.org/10.1371/journal.pone.0090081 — accessed 2026-07-28
Elsevier (2026, updated June 2026). Generative AI policies for journals — incl. the AI-assisted code FAQ. elsevier.com/about/policies-and-standards — accessed 2026-07-28
Cui, J., & Alexander, R. (2026). Same Prompt, Different Outcomes: Evaluating the Reproducibility of Data Analysis by LLMs. Preprint. arxiv.org/abs/2602.14349 — accessed 2026-07-28
Ouyang, S., et al. (2024). An Empirical Study of the Non-determinism of ChatGPT in Code Generation. ACM TOSEM. doi.org/10.1145/3697010 — accessed 2026-07-28
Saju, L., et al. (2025). Computational Reproducibility of R Code Supplements on OSF. Preprint. arxiv.org/abs/2505.21590 — accessed 2026-07-28
Baltes, S., et al. (2026). Guidelines for Empirical Studies in Software Engineering involving Large Language Models. Empirical Software Engineering (accepted). arxiv.org/abs/2508.15503 — accessed 2026-07-28
Zuckerman, A. L., et al. (2026). Ten simple rules for building a collaborative coding culture. PLOS Computational Biology 22(2), e1013970. doi.org/10.1371/journal.pcbi.1013970 — accessed 2026-07-28
Allen Institute for AI (2026, 30 April). AstaBench — spring 2026 update; ICLR 2026 paper. Ai2's own agentic-science leaderboard. allenai.org/blog — accessed 2026-08-21
HAL — Holistic Agent Leaderboard (2026). Princeton; standardized agent harness, currently paused for new models. hal.cs.princeton.edu · arxiv.org/abs/2510.11977 — accessed 2026-08-21
Artificial Analysis (2026). SciCode evaluation — 288 test-set subproblems, current models. Independent evaluator. artificialanalysis.ai/evaluations/scicode — accessed 2026-08-21
Churilov, A. (2026). The Range Shrinks, the Threat Remains: Re-evaluating LLM Package Hallucinations on the 2026 Frontier-Model Cohort. Preprint. arxiv.org/abs/2605.17062 — accessed 2026-09-02
Google Workspace Updates (2026, 16 July). NotebookLM is now Gemini Notebook. workspaceupdates.googleblog.com — accessed 2026-08-21
9to5Google (2026, 16 July). Gemini Notebook: cloud code execution for AI Pro. Press report; orientation only. 9to5google.com — accessed 2026-08-21