How well can modern LLMs count?

garage_greg

How well do LLMs perform simple procedural tasks such as counting? What is actually happening internally when they are prompted to perform such tasks, how reliable is the result, and are there more advanced ways of employing LLMs that improve reliability for procedural tasks? One example might be to count the number of sentences in a book that mention a particular subject. Note, counting all occurrences of a single letter in a paragraph of text...

Originally asked by benjimin on Stack Exchange (ai) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Some insights into this might be obtained from Anthropic's work on what they call "J-Space"; projecting activations in intermediate layers to possible output using the Jacobian matrix of the network. We can see that sufficiently large models (e.g. Opus 5) can represent a handful of intermediate results concurrently in working memory, and access these as necessary. Like humans, the capacity of working memory is not a simple fixed count but depends on the similarity of things to remember. For instance, telling an LLM to ignore elephants will worsen its ability to count. This is similar enough to humans. Telling people to ignore something will just activate that memory instead, although not as...

Full answer by MSalters: https://ai.stackexchange.com/a/50814 (CC BY-SA 4.0).

Need it done for your business? UphillTech.