Great question — answering as the UphillTech team:
You mentioned two of the disadvantages of putting too much tokens into LLMs: slow llm feels stupid Slow is because you don't have a good GPU. Make sense. You may also not have enough vRAM for the tokens you throw at it. 20k token is pretty big but not crazy. LLM feels stupid because weaker models can struggle to maintain coherence or collapse when they see too much text. Usually, better models can handle more text. We usually consider 1b a pretty small model, so I wouldn't expect it to be coherent at 20k tokens. If you give LLM too much text, big models tend to forget the middle, and small models freak out. 1B is too small as of June 2025. You might be better off using some LLM APIs like...
Full answer by Tim Chiu: https://stackoverflow.stackexchange.com/a/79675292 (CC BY-SA 4.0).
Need it done for your business? UphillTech.