Augur Dispatch

Chain of evidence

Evidence for 2026-10-20

This frozen page shows Augur's claims and source links for one sent dispatch. Stored spot-checks appear only where the frozen edition supports them; absence is not presented as verification.

As of:

Bundle identity: evidence-bundle-v2-90c78171b4e4397527d45a0bfd1470c52c87c70e799012d648fd3e880c9a19c7

Format: evidence-bundle-v2 · 37 claims

Assertion 1

This spring GPT-5.4 for clinicians scored 59% on the HealthBench Professional subset, beating both GPT-5.5 and answers physicians wrote themselves Import AI Nate Jones AI Explained.

Assertion status: No spot-check verdict is published for this assertion.

In a diagnostic test, GPT-4 achieved a 90% accuracy rate while doctors achieved 74%, and doctors using AI achieved 76% accuracy.

Claim 10707 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Nate Jones

No stored spot-check names this claim in this edition.

In a study on medical Q&A, 45% of GPT-4's answers were rated as empathetic by medical professionals compared to 4% for doctors.

Claim 10708 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Nate Jones

No stored spot-check names this claim in this edition.

GPT-4 answered 80% of medical questions correctly in a study, outperforming the doctors in that specific metric.

Claim 10709 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Nate Jones

No stored spot-check names this claim in this edition.

GPT-5.5 achieved a 57% score on Humanity's Last Exam, outperforming Claude Opus 4.7 and 4.6 which scored 46%.

Claim 21752 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

AI Explained

No stored spot-check names this claim in this edition.

GPT-5.4 for clinicians achieved a 59% score on the HealthBench Professional subset, beating both GPT-5.5 and physician-written responses.

Claim 21760 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

AI Explained

No stored spot-check names this claim in this edition.

Anthropic researchers produce nearly ten times as much code now as they did during the company's first four years.

Claim 81669 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Import AI

No stored spot-check names this claim in this edition.

Claude Opus 5.5 achieved a score of 67.7% on Humanity's Last Exam as of the week of October 2026.

Claim 81670 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Import AI

No stored spot-check names this claim in this edition.

AI systems now achieve a 94% score on the FrontierMath test, up from 2% at its launch in November 2024.

Claim 81671 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Import AI

No stored spot-check names this claim in this edition.

Assertion 2

The same write-up recalls that in early 2024 GPT-4 erred on 35% of 983 patient-record questions written by 15 clinicians Stanford HAI.

Assertion status: No spot-check verdict is published for this assertion.

The source states that in early 2024, the GPT-4 model produced errors in 35% of its responses when evaluated against 983 patient-record questions written by 15 clinicians.

Claim 81587 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Stanford HAI

No stored spot-check names this claim in this edition.

The source reports that MedHELM benchmarks, developed with input from 29 clinicians, show that the best AI models have win rates between 0.53 and 0.89 across five categories, with the lowest performance in administration and workflow.

Claim 81588 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Stanford HAI

No stored spot-check names this claim in this edition.

The source claims that administrative spending in the U.S. healthcare system exceeds $1 trillion per year.

Claim 81589 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Stanford HAI

No stored spot-check names this claim in this edition.

Assertion 3

A Blue Cross Blue Shield Association analysis found that hospitals' AI tools for filing claims added $942 million to healthcare spending over two years TechCrunch AI.

Assertion status: No spot-check verdict is published for this assertion.

Hospitals' use of AI tools in submitting insurance claims resulted in an additional $942 million in healthcare spending over a two-year period, according to an analysis by the Blue Cross Blue Shield Association. (Fact)

Claim 68187 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

TechCrunch AI

No stored spot-check names this claim in this edition.

Assertion 4

FrontierMath needed a second version after an audit found errors in 42% of the original problems, so the climb from 2% to 94% may not compare like with like, since the evidence does not say which version produced the 94% Epoch AI Gradient Updates.

Assertion status: No spot-check verdict is published for this assertion.

Version 2 of the FrontierMath benchmark was released after an audit found errors in 42% of problems in the original version.

Claim 22003 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Epoch AI Gradient Updates

No stored spot-check names this claim in this edition.

Assertion 5

Reproducibility is improving on this front, and it deserves credit: UK AISI and EvalEval now publish verified results for HealthBench and FrontierMath, among others Hugging Face Blog.

Assertion status: No spot-check verdict is published for this assertion.

The release includes verified results for five benchmarks: HealthBench, FrontierMath, Humanity's Last Exam, SWE-Bench Pro, and Terminal-Bench 2.0.

Claim 65229 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Hugging Face Blog

No stored spot-check names this claim in this edition.

Assertion 6

On the public HELM leaderboard, the five models ranked best by average score and the five ranked best by their weakest scenario share no members at all across 54 models Hugging Face Daily Papers.

Assertion status: No spot-check verdict is published for this assertion.

On the public HELM leaderboard, the top-five models ranked by mean score and the top-five models ranked by weakest-link score are completely disjoint across 54 frontier models on ten scenarios.

Claim 40724 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Hugging Face Daily Papers

No stored spot-check names this claim in this edition.

Assertion 7

Factory Droid finished 11 minutes ahead, roughly 2.4 times the pace of Claude Code, and reached the finish with about half as many tool calls, the steps where the agent reaches for a tool like a file read or a search Cerebras.

Assertion status: No spot-check verdict is published for this assertion.

In a controlled experiment comparing Factory Droid and Claude Code using Claude Opus 4.6 to build a whiteboard app, Factory Droid completed the task 11 minutes faster (approximately 2.4x speedup) while using roughly half as many tool calls.

Claim 81482 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Cerebras

No stored spot-check names this claim in this edition.

Assertion 8

An earlier Faros test held the wrapper constant instead and swapped the model: Claude Code on GLM-5.2 slightly beat the same wrapper on Opus 4.8 at about half the cost per task Fireworks AI.

Assertion status: No spot-check verdict is published for this assertion.

Independent evaluation by Faros found that Claude Code running on GLM-5.2 slightly outperformed Claude Code on Opus 4.8 while costing roughly half as much per completed task.

Claim 37851 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Fireworks AI

No stored spot-check names this claim in this edition.

Assertion 9

Apple found that even compressed to about 493GB the model still needs more than one 512GB Mac Studio, and four of them is the only configuration with a published speed Latent Space Apple Machine Learning Research.

Assertion status: No spot-check verdict is published for this assertion.

Apple Research reports that running the Muse Spark Open model (816B parameters) on a cluster of four Mac Studios with M5 Ultra chips generates over 36 tokens per second.

Claim 80097 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Apple Machine Learning Research

No stored spot-check names this claim in this edition.

Apple Research reports that the Muse Spark Open model requires parallelization across multiple Mac Studios because its memory footprint exceeds the capacity of a single Mac Studio with 512GB of RAM.

Claim 80100 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Apple Machine Learning Research

No stored spot-check names this claim in this edition.

Apple Research reports that the Muse Spark Open model uses NVFP4 precision for evaluation, reducing its memory footprint to about 493GB.

Claim 80112 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Apple Machine Learning Research

No stored spot-check names this claim in this edition.

Meta released Muse Spark Open, an 816B-total, 42B-active Mixture of Experts model with open weights.

Claim 81517 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Artificial Analysis scores Muse Spark Open at 39 on its Intelligence Index, which is the highest score for a US open-weights model.

Claim 81518 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Meta states that Muse Spark Open's NVFP4 checkpoint fits on four Blackwell Ultra GPUs, eight H200s, or two 512GB M5 Ultra Mac Studios.

Claim 81519 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Nick Cammarata says a preliminary audit of Qwen3-32B finds about 90% of neuron signals have one main job.

Claim 81538 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Assertion 10

Artificial Analysis scores the model 39 on its Intelligence Index, top among US open-weight models, yet last of the nine strongest open models on its tests of multi-step agent work Latent Space.

Assertion status: No spot-check verdict is published for this assertion.

Meta released Muse Spark Open, an 816B-total/42B-active Mixture-of-Experts model with a 1M-token context window that accepts text, image, video, and audio input.

Claim 79908 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Artificial Analysis scored Muse Spark Open 39 on its Intelligence Index, ranking it first among US open-weight models.

Claim 79912 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Muse Spark Open ranked last among the nine highest-scoring open models on Artificial Analysis's agentic evaluations.

Claim 79913 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Nick Cammarata argues that most neurons in models up to trillions of parameters are legible, proposing that neural networks are "already clean."

Claim 79924 Label: opinion Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Cammarata states that his 2023 OpenAI automated-interpretability effort found only about 2% of neurons legible.

Claim 79925 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Cammarata asserts that many famous examples of polysemanticity were misread.

Claim 79926 Label: opinion Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Neel Nanda states that Cammarata's "already clean" argument does not change his cruxes, as he still expects representational superposition.

Claim 79927 Label: opinion Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Assertion 11

- Stanford HAI's MedHELM leaderboard is the place to check whether any model lifts the administration and workflow floor above the 0.53 it reports now. Stanford HAI

Assertion status: No spot-check verdict is published for this assertion.

The source states that in early 2024, the GPT-4 model produced errors in 35% of its responses when evaluated against 983 patient-record questions written by 15 clinicians.

Claim 81587 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Stanford HAI

No stored spot-check names this claim in this edition.

The source reports that MedHELM benchmarks, developed with input from 29 clinicians, show that the best AI models have win rates between 0.53 and 0.89 across five categories, with the lowest performance in administration and workflow.

Claim 81588 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Stanford HAI

No stored spot-check names this claim in this edition.

The source claims that administrative spending in the U.S. healthcare system exceeds $1 trillion per year.

Claim 81589 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Stanford HAI

No stored spot-check names this claim in this edition.

Assertion 12

- Artificial Analysis's rankings for multi-step agent work will show whether Muse Spark Open's last-place agent ranking moves in later updates. Latent Space

Assertion status: No spot-check verdict is published for this assertion.

Meta released Muse Spark Open, an 816B-total/42B-active Mixture-of-Experts model with a 1M-token context window that accepts text, image, video, and audio input.

Claim 79908 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Artificial Analysis scored Muse Spark Open 39 on its Intelligence Index, ranking it first among US open-weight models.

Claim 79912 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Muse Spark Open ranked last among the nine highest-scoring open models on Artificial Analysis's agentic evaluations.

Claim 79913 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Nick Cammarata argues that most neurons in models up to trillions of parameters are legible, proposing that neural networks are "already clean."

Claim 79924 Label: opinion Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Cammarata states that his 2023 OpenAI automated-interpretability effort found only about 2% of neurons legible.

Claim 79925 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Cammarata asserts that many famous examples of polysemanticity were misread.

Claim 79926 Label: opinion Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Neel Nanda states that Cammarata's "already clean" argument does not change his cruxes, as he still expects representational superposition.

Claim 79927 Label: opinion Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Latent Space

No stored spot-check names this claim in this edition.

Assertion 13

- OpenAI's new SQLite and software engineering evaluations point to a push into low-resource programming languages. Watch whether it publishes model scores on them, which would show how far behind those languages sit. OpenAI News

Assertion status: No spot-check verdict is published for this assertion.

OpenAI is working to improve ChatGPT's performance in low-resource programming languages.

Claim 81664 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

OpenAI News

No stored spot-check names this claim in this edition.

OpenAI has introduced new evaluations for software engineering.

Claim 81665 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

OpenAI News

No stored spot-check names this claim in this edition.

OpenAI has introduced new evaluations for SQLite.

Claim 81666 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

OpenAI News

No stored spot-check names this claim in this edition.

Assertion 14

- Transformers, Hugging Face's widely used model library, shipped 5.20.2 to stop inputs losing their type during generation for models that take images and text. Teams running such models on 5.20.x should pick it up and watch whether the 5.20 line needs another patch before it settles. Transformers Releases

Assertion status: No spot-check verdict is published for this assertion.

The Transformers v5.20.2 patch release, published on 2026-10-19, addresses an issue where inputs could lose their original type within the prepare_inputs_for_generation function of the generate method, which caused problems for multimodal models.

Claim 81786 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Transformers Releases

No stored spot-check names this claim in this edition.

Pull request #49731 by @Cyrilvallez fixes the casting of inputs dictionaries in the generate function.

Claim 81787 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Transformers Releases

No stored spot-check names this claim in this edition.

Pull request #49688 by @qgallouedec fixes the issue of zero expert gradients occurring in bfloat16 when using expert parallelism with ep_size less than fsdp_size.

Claim 81788 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Transformers Releases

No stored spot-check names this claim in this edition.

Assertion 15

The 35% error figure dates to early 2024 on a model since replaced, so treat it as history rather than a current rate Stanford HAI.

Assertion status: No spot-check verdict is published for this assertion.

The source states that in early 2024, the GPT-4 model produced errors in 35% of its responses when evaluated against 983 patient-record questions written by 15 clinicians.

Claim 81587 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Stanford HAI

No stored spot-check names this claim in this edition.

The source reports that MedHELM benchmarks, developed with input from 29 clinicians, show that the best AI models have win rates between 0.53 and 0.89 across five categories, with the lowest performance in administration and workflow.

Claim 81588 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Stanford HAI

No stored spot-check names this claim in this edition.

The source claims that administrative spending in the U.S. healthcare system exceeds $1 trillion per year.

Claim 81589 Label: fact Provenance: primary Eligible for corroboration · eligibility does not establish truth Recorded

Stanford HAI

No stored spot-check names this claim in this edition.

Keep exploring

Move on to Augur's other sections.