Bad benchmarks and evals: Senior SWE-Bench, napkin math, and winter tires

2 comments

• stephantul 1 hour ago
  The (relevant) segue into tires elevates this post so much. I don’t understand why, but it does
  [-]
  • vintagedave 59 minutes ago
    Maybe that it's an example of the same pattern in a real-world, non-digital area? It makes it feel more grounded.
• alexpran 2 days ago
  [flagged]