Not OP, but I've had the same experience, and that's with tools that purport to handle the context for you.
And frankly, if you can't automate context, then you don't have an AI tool that can realistically replace a programmer. If I have to manually select which of my 10000 files are relevant to a given query, then I still need to be in the loop and will also likely end up doing almost as much work as I would have to just write the code.
I see that you deleted your previous response which was unnecessarily snarky while my question was genuine and simple I suppose.
> And frankly, if you can't automate context,
How about ingesting the whole codebase into the model? I have seen that this is possible with at least one such tool (Devon) and which I believe is using gpt model underneath meaning that other providers could automate this step too. I am curious if that would help in generating more legit large scale changes.
> I see that you deleted your previous response which was unnecessarily snarky while my question was genuine and simple I suppose.
You edited your comment to clarify that you were asking from a place of ignorance as to the tools. Your original comment read as snarky and I responded accordingly, deleting it when I realized that you had changed yours. :)
> How about ingesting the whole codebase into the model? I have seen that this is possible with at least one such tool (Devon) and which I believe is using gpt model underneath meaning that other providers could automate this step too. I am curious if that would help in generating more legit large scale changes.
It doesn't work. Even the models that claim to have really large context windows get very distracted if you don't selectively pick relevant context. That's why I always talk about useful context window instead of just plain context window—the useful context window is much lower and how much you have depends on the type of text you're feeding it.
I don't think my comment read as snarky but I was surprised to see the immediate downvote which presumably came from you so I only added the last sentence. This is a stupid way of disagreeing and attempting to shut down the discussion without merits.
> It doesn't work. Even the models that claim to have really large context windows get very distracted if you don't selectively pick relevant context.
I thought Devon is able to pre-process the whole codebase and which it could take up to a one single day for larger codebases so it must be doing something, e.g. indexing the code? If so, this isn't a context-window specific thing, it's something else and it makes me wonder how that works.
> I don't think my comment read as snarky but I was surprised to see the immediate downvote which presumably came from you so I only added the last sentence.
I can't downvote you because you are downthread of me. HN shadow-disables downvotes on all child and grandchild comments.
I'm the one who upvoted you to counteract the downvote. :)
> How about ingesting the whole codebase into the model?
You keep referring to this vague idea of "ingesting the whole codebase". What does this even mean? Are you talking about building a code base specific rag, fine tuning against a model, injecting the entire code base into the system context, etc.?
It is vague because the implementation details you are asking me for are closed source, for obvious reasons. I can only guess what it does but that's besides the point. The point is rather that Devon or 1M window context qwen model might be better or more resilient towards the "lack of context" than what the others were suggesting.
And frankly, if you can't automate context, then you don't have an AI tool that can realistically replace a programmer. If I have to manually select which of my 10000 files are relevant to a given query, then I still need to be in the loop and will also likely end up doing almost as much work as I would have to just write the code.