Problem working with long text
#4
by
Kosh69
- opened
When trying to retell a chapter from a book (~120k tokens), he produces some kind of gibberish, despite using a bunch of words and phrases from the entire text, writing clearly and consistently... but essentially missing the point, and not just missing the point, but completely unrelated to the content of the text.
When I try to ask a question about the text, it presents its own fanciful version of the events it asked about.
Used unsloth q4_k_xl and I don't think it's a quantization issue...
Regarding the bandwidth and memory consumption - thank you! 5 stars!