GLM-5.3 Artificial Analysis Benchmarks

(artificialanalysis.ai)

77 points | by apitman 3 hours ago

8 comments

  • scotttrinh 2 hours ago
    I like to compare models with a similar score on cost per task and output tokens per task since those measure two things I'm interested in: cost efficiency and token efficiency. Here's how GLM-5.3 compares to other models in a similar score and against GLM-5.2 to save a few clicks for others who care about these metrics:

        Model                        Score    Cost / Task    Output Tokens / Task
        -------------------------------------------------------------------------
        GLM-5.3 (max)                 59.5          $0.68                  41,107
        GLM-5.2 (max)                 53.0          $0.56                  32,200
        Claude Opus 5 (high)          61.5          $1.52                  21,353
        GPT-5.6 Sol (max)             60.9          $1.23                  16,879
        Grok 4.6 (high)               60.9          $0.84                  21,735
        Kimi K3 (max)                 59.7          $0.84                  25,474
        GPT-5.6 Sol (xhigh)           59.0          $0.87                  11,098
        Claude Opus 5 (medium)        58.6          $0.98                  12,459
        Qwen3.8 Max                   58.1          $1.13                  38,287
        Qwen3.8 2.4T A95B             57.7          $0.95                  32,472
        Claude Opus 4.8 (max)         57.3          $1.65                  33,557
        GPT-5.6 Sol (high)            57.3          $0.52                   7,545
        Muse Spark 1.2 (xhigh)        56.8          $0.40                  30,430
        GPT-5.6 Terra (max)           56.6          $0.51                  20,838
        GPT-5.5 (xhigh)               56.3          $0.69                  16,893
        Gemini 3.7 Flash (high)       56.0          $0.40                  36,847
    
    Edited for accuracy and more models.
    • sourcecodeplz 2 hours ago
      Muse Spark has a nice balance. not to mentions the Contribs version is old deepseek flash prices.
      • glub 1 hour ago
        Tested muse spark 1.2 because it was rated so high on design arena, and I've missed a model that can do nice UI in the hands of an operator with no UI skills.

        It produced worse UI mockups than GPT and GPT models are already the bottom of the barrel here. The only model that performed well was Kimi K3 - insanely good, but expensive.

        It's hard to trust benchmarks these days.

      • sscaryterry 2 hours ago
        I found the sweetspot here: GPT-5.6 Sol (high) 57.3 $0.52 7,545

        (Edit: TLDR; It gets on with it, makes the same mistakes you would, without overthinking and overengineering, most of the time)

    • dudeinhawaii 1 hour ago
      It would make reading and comparing a bit easier if the data was sorted by a dimension.
      • ac29 17 minutes ago
        Cost per task:

          Model                        Score    Cost / Task    Output Tokens / Task
          -------------------------------------------------------------------------
          Muse Spark 1.2 (xhigh)        56.8          $0.40                  30,430
          Gemini 3.7 Flash (high)       56.0          $0.40                  36,847
          GPT-5.6 Terra (max)           56.6          $0.51                  20,838
          GPT-5.6 Sol (high)            57.3          $0.52                   7,545
          GLM-5.2 (max)                 53.0          $0.56                  32,200
          GLM-5.3 (max)                 59.5          $0.68                  41,107
          GPT-5.5 (xhigh)               56.3          $0.69                  16,893
          Grok 4.6 (high)               60.9          $0.84                  21,735
          Kimi K3 (max)                 59.7          $0.84                  25,474
          GPT-5.6 Sol (xhigh)           59.0          $0.87                  11,098
          Qwen3.8 2.4T A95B             57.7          $0.95                  32,472
          Claude Opus 5 (medium)        58.6          $0.98                  12,459
          Qwen3.8 Max                   58.1          $1.13                  38,287
          GPT-5.6 Sol (max)             60.9          $1.23                  16,879
          Claude Opus 5 (high)          61.5          $1.52                  21,353
          Claude Opus 4.8 (max)         57.3          $1.65                  33,557
        
        Benchmark score:

          Model                        Score    Cost / Task    Output Tokens / Task
          -------------------------------------------------------------------------
          Claude Opus 5 (high)          61.5          $1.52                  21,353
          GPT-5.6 Sol (max)             60.9          $1.23                  16,879
          Grok 4.6 (high)               60.9          $0.84                  21,735
          Kimi K3 (max)                 59.7          $0.84                  25,474
          GLM-5.3 (max)                 59.5          $0.68                  41,107
          GPT-5.6 Sol (xhigh)           59.0          $0.87                  11,098
          Claude Opus 5 (medium)        58.6          $0.98                  12,459
          Qwen3.8 Max                   58.1          $1.13                  38,287
          Qwen3.8 2.4T A95B             57.7          $0.95                  32,472
          Claude Opus 4.8 (max)         57.3          $1.65                  33,557
          GPT-5.6 Sol (high)            57.3          $0.52                   7,545
          Muse Spark 1.2 (xhigh)        56.8          $0.40                  30,430
          GPT-5.6 Terra (max)           56.6          $0.51                  20,838
          GPT-5.5 (xhigh)               56.3          $0.69                  16,893
          Gemini 3.7 Flash (high)       56.0          $0.40                  36,847
          GLM-5.2 (max)                 53.0          $0.56                  32,200
    • tw1984 35 minutes ago
      this is not very useful.

      for over 1 billion real world users living in China, they don't have the option of paying $1.52 per task to use Opus 5, they are banned doing that due to US politics.

      • Barbing 7 minutes ago
        What’s the reseller situation?
    • teravor 2 hours ago
      these $/task figures aren't very useful in my experience. it doesn't tell you how well it did the task.

      generally I choose models by their intelligence and then personal preference from direct experience.

      • kolinko 1 hour ago
        they make a ton of sense for tasks that are achievable with no prob by all models - e.g. writing scripts that do a specific thing etc.
        • teravor 1 hour ago

              > writing scripts
          
          you can use a Gemini model completely for free doing that. no agent. aistudio.google.com
  • glub 1 hour ago
    I've tested GLM 5.3 on the release day and Artificial Analysis is spot on. It's a really good model.

    But my main takeaway was something else. I've used closed weight models for long enough that I've forgotten how good it feels to see reasoning tokens.

    With GPT/Claude, you kind of hope that intent was captured well, that agent had all the information, all the tools it needed, because you won't see "hmmm it seems like nix flake isn't available here and I shouldn't install something globally" until it slopped out millions of tokens and wasted hundreds of dollars for 8 hours. With GLM and the likes, you just stop the disease right where it begins.

    • Havoc 1 hour ago
      Yes, not necessary often but being able to stop something that is going off the rails is super useful. Especially if the root cause is prompt ambiguity - inject a clarification & it recovers
      • glub 1 hour ago
        It's also starting to go beyond reasoning and it's becoming much more problematic. Reasoning is one thing, but codex, for example now encrypts agent-to-agent messages as well, and compaction. I've no idea what subagents are instructed to do, or what they reported back in native codex.

        The only thing that's keeping me is the value $200 subscription provides. If that value disappears, I see no reason why not to switch to something that isn't a black box.

    • tw1984 33 minutes ago
      With GPT/Claude, hiding those from users to waste their tokens is a feature, not a limitation.
  • BinRoo 2 hours ago
    Beware of the benchmarks listed. SciCode and EnterpriseOps for instance: https://shukla.io/blog/2026-08/gym.html
    • Onavo 1 hour ago
      The Chinese models also like to cut corners on stuff like science. Their scores on stuff like biotech and scientific knowledge is far from ChatGPT unfortunately. (Claude is pretty good but it just refuses all prompts).
  • markasoftware 2 hours ago
    Very impressive score for the size, though token use is higher than k3 and far higher than proprietary models, and its price to performance isn't all that far ahead of k3 as a result
    • Havoc 2 hours ago
      >token use is higher than k3 and far higher than proprietary models

      GLM sets effort to max by default historically.

  • Zaheer 2 hours ago
    Is it worth using these models if I have a claude code subscription already? The appeal of lower cost is nice but I haven't gotten over the switching cost yet.
    • robertn702 51 minutes ago
      FYI, you can use your Claude subscription pricing with OpenCode via Meridian[0], which also makes it easier to try out other models when they come out. You can also use your other subscriptions in OpenCode with CLIProxyAPI[1]. The switching cost was relatively high, mostly from claude code plugins but completely worth it. I'm now mostly using GLM-5.3 and Codex models via OpenCode and barely using Claude which seemed unfathomable less than two months ago.

      [0] https://github.com/rynfar/meridian

      [1] https://github.com/router-for-me/CLIProxyAPI

      edit: reworded for clarity

    • oceanplexian 1 hour ago
      I use the $200 plan w/ Anthropic and run out of tokens half way through the week and supposedly they are progressively reducing the limits on all their subs even further.

      At some point I will switch, $200 buys a lot of tokens on OpenRouter.

      • Sabinus 43 minutes ago
        Is the conventional wisdom that the subscription price/token is better than the API price/token not valid any more? Or is access to model diversity worth the increased per token costs?
      • kilroy123 1 hour ago
        Same here. I'm switching to the Codex plan. It just doesn't go very far now. Especially if you use fable at all.
    • culi 2 hours ago
      Use a unified proxy that lets you switch between models seamlessly. We are far from an equilibrium in this market and you will continue to have FOMO no matter who you pick if you go all in on one company
    • glub 1 hour ago
      If anything, it's going to be more expensive. Price/performance ratio isn't there yet for frontier open weight models.

      But regardless, you definitely should use a harness where switching models on the fly is easy. There's a reason why Anthropic uses their own proprietary formats/conventions anywhere they can - to lock you in when inference eventually commoditizes.

    • colingauvin 2 hours ago
      At least by API usage, they aren't yet lower cost than subscriptions. Not sure about GLM's subscription plans though.
      • glub 1 hour ago
        GLM subscription is better than API, but significantly worse than Codex, even when used outside peak hours.
    • notatoad 2 hours ago
      no, at subscription prices claude is a better value than GLM.

      They're only a better value if you're paying API rates

  • colingauvin 2 hours ago
    Tied for #1 by agentic index (with Opus 5).
  • colingauvin 2 hours ago
    ...do I take out a double mortgage to buy a 4 Spark cluster?
    • nvme0n1p1 2 hours ago
      No, you use openrouter and spend 10% as much as using a proprietary model.
    • lisplist 2 hours ago
      $20k is personal loan territory, not a second mortgage lol
  • fenestella 1 hour ago
    [flagged]