File not found error if token amount is greater then 4k

[Shyvadi]

After some testing, I've noticed that if the webpage crawl is big enough that the chatgpt file may fail to create itself because of a token limit exceeded (Since there is no response back from chatgpt)

[Hacking-Notes]

I see, it think the solution is then to make multiples requests