I’ve found that whenever it gets something really wrong, it’s either corrected by a follow up prompt (telling it that its wrong helps), or rephrasing the question.
Nope, not my experience - unless it's a trivial problem. In my experience it almost always gets stuck in a loop where it's rotating between two or more incorrect implementations. It isn't doing iteration - it's just looping. Even if I point out the problem directly, it will just say "sorry, you're right - here's the correction!" and the correction has the same response from an earlier item that I pointed out had a problem!
Have not had that experience with GPT-4. But that is almost unusable for me anyway because it's crazy slow (during US working hours anyway, it's faster at night) and the rate caps get hit before we get anywhere.