Skip to main content
Back to Discussions

Who Grades the AI Models? | Ben Horowitz & Rayan Krishnan

Technology
Global
Started September 10, 2026

a16z’s Erik Torenberg, Ben Horowitz, and Jennifer Li sit down with Vals founder and CEO Rayan Krishnan to discuss one of AI’s increasingly difficult problems: how do you actually measure whether a model is getting better? As public benchmarks saturate and models get better at optimizing for the tests themselves, Rayan makes the case for independent, continuously evolving evaluations. They unpack why self-reported model scores can be misleading, how VALS evaluates models in the hours before a ...

Source Articles

Add Statement Analysis 0/5
Looking for a particular claim? Search all statements
🗳️ Join the conversation
10 statements to vote on • Your perspective shapes the analysis
📊 Progress to consensus analysis Needs 7+ participants and 20+ votes. Each statement joins once it has 3+ votes.
Participants 0/7
Statements (10+ recommended) 10/10
Total Votes 0/20
💡 Progress updates live here. Analysis can run once all three requirements are met.

Your votes count

No account needed. Your votes are saved and included in the consensus analysis. Create an account to keep a record of your votes.

CLAIM Posted by admin • Sep 10, 2026
Whoever grades AI models must maintain transparency about their evaluation methods and funding sources to avoid hidden conflicts of interest.
Vote options for this statement: agree, disagree, or unsure
💬 View Discussion
Be first to respond
Vote to see results
CLAIM Posted by admin • Sep 10, 2026
Requiring independent evaluations of AI models will slow down innovation and create unnecessary regulatory burden on developers.
Vote options for this statement: agree, disagree, or unsure
💬 View Discussion
Be first to respond
Vote to see results
CLAIM Posted by admin • Sep 10, 2026
AI model evaluations should prioritize real-world performance over controlled test conditions.
Vote options for this statement: agree, disagree, or unsure
💬 View Discussion
Be first to respond
Vote to see results
CLAIM Posted by admin • Sep 10, 2026
AI model evaluations should evolve continuously to keep pace with technological advancements.
Vote options for this statement: agree, disagree, or unsure
💬 View Discussion
Be first to respond
Vote to see results
CLAIM Posted by admin • Sep 10, 2026
Measuring AI model improvements should incorporate both quantitative and qualitative metrics for a holistic view of performance.
Vote options for this statement: agree, disagree, or unsure
💬 View Discussion
Be first to respond
Vote to see results
CLAIM Posted by admin • Sep 10, 2026
Evaluation systems must measure real-world AI performance on tasks that matter to users, not abstract benchmark scores.
Vote options for this statement: agree, disagree, or unsure
💬 View Discussion
Be first to respond
Vote to see results
CLAIM Posted by admin • Sep 10, 2026
The evaluation process for AI models must include diverse perspectives to avoid bias and promote fairness.
Vote options for this statement: agree, disagree, or unsure
💬 View Discussion
Be first to respond
Vote to see results
CLAIM Posted by admin • Sep 10, 2026
AI model evaluations should continuously evolve rather than rely on static standardized tests that models can overfit to.
Vote options for this statement: agree, disagree, or unsure
💬 View Discussion
Be first to respond
Vote to see results
CLAIM Posted by admin • Sep 10, 2026
AI developers should have a vested interest in the evaluation process, as their input is critical for accurate assessments.
Vote options for this statement: agree, disagree, or unsure
💬 View Discussion
Be first to respond
Vote to see results
CLAIM Posted by admin • Sep 10, 2026
A single standardized evaluation framework for all AI models is impossible because different models serve different purposes.
Vote options for this statement: agree, disagree, or unsure
💬 View Discussion
Be first to respond
Vote to see results

💡 How this works

  • • Add statements: Post claims or questions (10 to 500 characters).
  • • Vote: Agree, disagree or unsure on each statement.
  • • Respond: Add detailed arguments for or against, with evidence.
  • • Consensus: Once enough people take part, analysis shows opinion groups and where they agree.

Society Speaks is open and independent. Your support keeps civic discussion free from advertising and commercial influence.

Support us