Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This script, and any attempt to break a long string into multiple inputs, DOES NOT and CANNOT work. ChatGPT will forget 100% of all snippets, except for the last. #13

Open
exo-pla-net opened this issue Jul 9, 2023 · 1 comment

Comments

@exo-pla-net
Copy link

Though you can indeed split up a long text into chunks, and you then feed them all into a ChatGPT chat, this is irrelevant.

The maximum input length in a ChatGPT prompt is equivalent to the maximum context size for ChatGPT. Thus, ChatGPT will "forget" all the previous snippets, and it will only know about the final snippet.

Try it: use this script to break apart a long text. Then quiz it on a detail only present in the first snippet. ChatGPT will have no idea about it, since it's outside of its current context, which is ONLY THE FINAL SNIPPET. (And maybe a bit of the second-to-last snippet, if the final snippet is less than the maximum input length.)

This problem is impossible to solve in a fully satisfactory way. I'd love to have a longer context, too, but this is unfortunately not solvable by breaking apart a long text. The best you can do is have ChatGPT iteratively summarize previous snippets, then feed the summary into the next snippet. You can programmatically do this, and it might work for your purposes.

As for this project, it should have a disclaimer at the top that this script is useless / just for fun, because it's currently just wasting people's time. If you want to salvage this project into something useful, then here's what it could look like:

  • Allow user to input the full text of their long document.
  • Output the first part of the document, along with a request to summarize.
  • Allow the user to input the summary of the first part.
  • Output the next part of the document, along with the summary, and request ChatGPT to summarize both.
  • Repeat until the entire long document is summarized.

The above is a script that would indeed be useful. But the current one is a waste of time and should ethically have a disclaimer at the top.

@mo9a7i
Copy link

mo9a7i commented Aug 23, 2023

I can confirm that. Using GPT-4 and fed it 22 sections of a document I have, it just keeps answering within the context of the last two.
When I tried to be smart and confront it 😂 this is what it said:

Unfortunately, I don't have access to the earlier sections of the document, 
so I'm unable to verify if there are any missing procedures. 
If you have any specific questions or need further details about 
the sections I've summarized, please let me know! 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants