Auto Researching, not hyperparameter tuning: Convergence Analysis of 10,000 Experiments

Dimensions overall score 7.0

• A framework for LLM agents to autonomously design and optimize ML experiments through genuine architecture search. We answer this question by analyzing 10,469 experiments executed by two LLM agents (Claude Opus and Gemini 2.5 Pro) across a combinatorial configuration space of 108,000 discrete cells for dashcam collision detection over 27 days.problem 35%
  Evidencepartial

  A framework for LLM agents to autonomously design and optimize ML experiments through genuine architecture search. We answer this question by analyzing 10,469 experiments executed by two LLM agents (Claude Opus and Gemini 2.5 Pro) across a combinatorial configuration space of 108,000 discrete cells for dashcam collision detection over 27 days.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• When LLM agents autonomously design ML experiments, do they perform genuine architecture search -- or do they default to hyperparameter tuning within a narrow region of the design space? We answer this question by analyzing 10,469 experiments executed by two LLM agents (Claude Opus and Gemini 2.5 Pro) across a combinatorial configuration space of 108,000 discrete cells for dashcam collision detection over 27 days.method 35%
  Evidencepartial

  When LLM agents autonomously design ML experiments, do they perform genuine architecture search -- or do they default to hyperparameter tuning within a narrow region of the design space? We answer this question by analyzing 10,469 experiments executed by two LLM agents (Claude Opus and Gemini 2.5 Pro) across a combinatorial configuration space of 108,000 discrete cells for dashcam collision detection over 27 days.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• ScienceToStartup currently rates this 7.0/10 on the public viability pass. The agents' key contribution is discovering that V-JEPA\,2 video features with Zipformer temporal encoders achieve 0.9245 AP -- a configuration no human proposed -- and concentrating search on productive architectural regions: at $N = 50$, LLM-guided search reaches AP $= 0.985$ versus $0.965$ for from-scratch random search. A public repository is linked, so build verification can inspect implementation evidence instead of treating the paper as PDF-only.result 35%
  Evidencepartial

  ScienceToStartup currently rates this 7.0/10 on the public viability pass. The agents' key contribution is discovering that V-JEPA\,2 video features with Zipformer temporal encoders achieve 0.9245 AP -- a configuration no human proposed -- and concentrating search on productive architectural regions: at $N = 50$, LLM-guided search reaches AP $= 0.985$ versus $0.965$ for from-scratch random search. A public repository is linked, so build verification can inspect implementation evidence instead of treating the paper as PDF-only.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• AutoML moved forward this cycle; last verified April 2026. Public score 7.0/10. Implementation evidence is present through a linked repository.why now 35%
  Evidencepartial

  AutoML moved forward this cycle; last verified April 2026. Public score 7.0/10. Implementation evidence is present through a linked repository.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

Competitive landscape

A framework for LLM agents to autonomously design and optimize ML experiments through genuine architecture search.