daVinci-Env: Open SWE Environment Synthesis at Scale

Dimensions overall score 8.0

• OpenSWE is the largest fully transparent framework for SWE agent training in Python, comprising 45,320 executable Docker environments spanning over 12.8k repositories.technical 95%
  Evidencepartial

  We present OpenSWE, the largest fully transparent framework for SWE agent training in Python, comprising 45,320 executable Docker environments spanning over 12.8k repositories

  Implicationpartial

  Explicitly stated in the abstract with specific numeric values.

  Verificationpartial

  partial

• The project represents a total investment of approximately $1.47 million, with $891K spent on environment construction and $576K on trajectory sampling and curation.technical 95%
  Evidencepartial

  With $891K spent on environment construction and an additional $576K on trajectory sampling and difficulty-aware curation, the entire project represents a total investment of approximately $1.47 million

  Implicationpartial

  Directly stated in the abstract with specific cost breakdown.

  Verificationpartial

  partial

• OpenSWE-32B and OpenSWE-72B achieve 62.4% and 66.0% on SWE-bench Verified, establishing state-of-the-art among Qwen2.5 series.result 90%
  Evidencepartial

  OpenSWE-32B and OpenSWE-72B achieve 62.4% and 66.0% on SWE-bench Verified, establishing SOTA among Qwen2.5 series

  Implicationpartial

  Explicitly stated in the abstract with specific performance metrics.

  Verificationpartial

  partial

• SWE-focused training yields substantial out-of-domain improvements, including up to 12 points on mathematical reasoning and 5 points on science benchmarks, without degrading factual recall.result 90%
  Evidencepartial

  SWE-focused training yields substantial out-of-domain improvements, including up to 12 points on mathematical reasoning and 5 points on science benchmarks, without degrading factual recall

  Implicationpartial

  Directly stated in the abstract with specific performance improvements.

  Verificationpartial

  partial

• The system uses a quality-centric filtering pipeline that characterizes environment difficulty, filtering out unsolvable or insufficiently challenging instances.method 85%
  Evidencepartial

  Beyond scale, we propose a quality-centric filtering pipeline that characterizes the inherent difficulty of each environment, filtering out instances that are either unsolvable or insufficiently challenging

  Implicationpartial

  Strongly supported in both abstract and analysis, though specific filtering criteria are not detailed.

  Verificationpartial

  partial

• OpenSWE is built through a multi-agent synthesis pipeline deployed across a 64-node distributed cluster.method 85%
  Evidencepartial

  OpenSWE is built through a multi-agent synthesis pipeline deployed across a 64-node distributed cluster

  Implicationpartial

  Explicitly stated in the abstract with specific technical details.

  Verificationpartial

  partial

• The cost and complexity associated with maintaining such a large-scale environment are significant, posing a barrier for potential users.limitation 80%
  Evidencepartial

  The cost and complexity associated with maintaining such a large-scale environment are significant

  Implicationpartial

  Directly stated in the analysis section as a caveat, though not quantified.

  Verificationpartial

  partial

• Existing open-source datasets are limited in scale and repository diversity, while industrial solutions are opaque with unreleased infrastructure.market 85%
  Evidencepartial

  existing open-source datasets remain limited in scale and repository diversity, while industrial solutions are opaque with unreleased infrastructure

  Implicationpartial

  Directly stated in the abstract as motivation for the work.

  Verificationpartial

  partial

Competitive landscape

Building the largest open-source SWE environment for training scalable and verifiable software engineering agents.