If you believe in the AI doom scenario then yes, you do need to do that. Because it's very important that your "less ethical" and "less good" competitors do not get to the machine god first.
If you don't believe that, or you don't believe that the frontier labs believe that, then sure, it makes no sense. But they probably do. The people at these companies literally dedicated their lives to building this specific thing that, up until people had to make tradeoffs between "that looks risky" and "that looks useful", virtually everyone agreed would be a dangerous technology.
What apparently many people on HN failed to appreciate is that the thing that makes it dangerous is the fact that it grows in utility.
In reality there is basically no mechanism by which a large company (i.e. with a board) can do this so long as there is 1) significant profit motive and 2) no government intervention.
Companies don't have to do that. If they're getting into actually dangerous territory, they can stop as soon as they want to.