Why would LLM generative AI produce certain words with higher frequency than...

bake_betty

In Delving into LLM-assisted writing in biomedical publications through excess vocabulary , the researchers found that frequency of some uncommon words (e.g. "delves") increased significantly in article abstracts since LLMs like ChatGPT became commonly available. Since generative AI is based on producing output based on statistical analysis of the input data, why would some word frequencies be exaggerated like this?

Originally asked by Barmar on Stack Exchange (ai) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Great question! Let's start in the beginning. LLMs are trained to predict the next most probable token given some prior context. Let's quickly equate this to humans. English is my first language, but I was lazy in my English classes and it probly shows in my writing ;) Others may have a much larger vocabulary than me. In fact, in writing this I could have said: Others may have a greater vocabulary than me Others may have a bigger vocabulary than me Others may have a more sophisticated vocabulary than me Each person may fill the missing Others may have a ____ vocabulary than me differently. Some might even put smaller . Why does this matter? Well, LLMs are trained to learn a statistical...

Full answer by Chrispresso: https://ai.stackexchange.com/a/48693 (CC BY-SA 4.0).

Need it done for your business? UphillTech.