Generating very long sequences with ollama

reno_queen

I'm trying to generate very long texts using ollama and python in a single run for research purposes but the generation stops with a stop status, despite everything I'm trying to increase the context length and set the stop sequence to an unlikely string: modelname = 'llama3.2:1' max_tokens = 30000 stop=["OloLol0"] ... completion = ollama.generate( model=modelname, prompt=prompt, stream=stream, options={ "temperature": temperature, "num_ctx":...

Originally asked by Nick Mikhailovsky on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

uphilltech

Good thread on this — worth a read before you pay anyone to build it.

Discussion: https://stackoverflow.com/questions/79333692/generating-very-long-sequences-with-ollama.

Want it set up for you? UphillTech.