Though the gov't might want research to be distributed, the underlying problem here has more to do with the scientists themselves than anything else. We've had ArXiV around for a long time and yet I ask who's adopted it other than mathematicians and (many, but not all) physicists? Why hasn't it been adopted by NIH-funded life scientists?
The fact that years ago, life scientists easily could have adopted an ArXiV-like model for publishing, and chose not to, is quite telling. It suggests a far deeper problem with incentives in (general) academic culture to publish and that article availability is not going to affect that at all. As someone who spent >6 years Ph.D./PostDoc (bioinformatics, stats and CS), I can say that the vast majority of researchers have no genuine incentive to take action. Protesting against Elsevier online in the comfort of your office is one thing, but having to publish X>10 papers/yr to get tenure/brownie points within your department is another.
You're absolutely right that academics are torn between two things that they want:
1. open access for their research (both for altruistic reasons, because they want to spread knowledge as widely as possible, and for selfish reasons, because greater access to their research means more citations of their research, which helps them with career advancement)
2. publishing in the most prestigious journals they can, because that helps with career advancement.
The NIH, which funds all research into biology and medicine in the US, just cares about 1. above. They just care about maximizing distribution for taxpayer-funded research, as long as the distribution is done so in an economically sustainable way.
The NIH drove the open access mandate in 2008. My sense is that in the short term, driven mainly by the demands of funding agencies, all journals will move to 'open access models', where publishers pay to publish their research in the journals, and for readers, it's free to access. This satisfies both 1. and 2. above: academics get both open access and the validation of being published in prestigious journals.
Longer term, I think alternative credit metrics will replace the credit metric that's governed research for hundreds of years, i.e. the brand value of the journal you were published in.
I think when that happens, the power of journals will disappear altogether. Right now, the journals' role in driving the discovery of research has been disrupted a lot. In the pre-web days, people used to walk down to their libraries, and check out what had been published in the journals recently. In those days, the journals used to drive a lot of research discovery.
Nowadays, pretty much all research discovery happens online, and it's driven by things like search engines (Google, Google Scholar, Pubmed), and social platforms (Twitter, arXiv, Academia.edu; general communication technologies like email and IM are also used a lot).
The journals still have a strangle-hold on the validation of research. Which journal you were published in is still incredibly important, as far as the validation of your paper is concerned.
Other credit metrics are emerging, however. Citation counts are one metric, and that has a big role in driving research discovery on search engines like Google Scholar. Hiring committees are starting to look at the kinds of credit metrics that apply to general web content, e.g. page views.
As alternative credit metrics emerge, there will come a point when they do a good enough job at the validation of research that academics feel they no longer need to submit their papers to journals.
If pre-publication peer review goes out of fashion, then another possibility if that the brand of major journals becomes more important. We still need a quick gauge of the quality of an article, other than its Google rank or number of page views. Nature can retract popular articles that are later proven flawed; I don't think Google would attempt to wield that kind of authority.
Relevant example: You published these two posts in TechCrunch to get a wide audience. (And I'm glad you did!) I read them partly because they appeared in TechCrunch.
The fact that years ago, life scientists easily could have adopted an ArXiV-like model for publishing, and chose not to, is quite telling. It suggests a far deeper problem with incentives in (general) academic culture to publish and that article availability is not going to affect that at all. As someone who spent >6 years Ph.D./PostDoc (bioinformatics, stats and CS), I can say that the vast majority of researchers have no genuine incentive to take action. Protesting against Elsevier online in the comfort of your office is one thing, but having to publish X>10 papers/yr to get tenure/brownie points within your department is another.