The problem is that the metrics can be gamed, and there's incentive to do so.

Nobody is arguing against testing. They're arguing about doing tests where you know the results will be deeply flawed.