Don't Stop, Don't Look, Don't Listen

I know I've blogged about this before, but the extent of the robots.txt file at http://www.whitehouse.gov is truly breathtaking.

For the uninitiated, this particular file is created by some webmasters to inform search engines like Google or services like the Internet Archive as to which pages and directories are effectively off-limits -- online information contained therein is not supposed to be searched or cached.

A quick look at the Internet Archive shows how the Clinton Administration didn't even have any restrictions like this in place. Under Bush's watch, this off-limits list has grown and grown, making it increasingly difficult to keep track of how his administration revises history under our very noses.

Here's the bit on Talking Points Memo today that brought this issue back to mind.