Entire Library of Congress

One often hears statements like 'transmitting the Library of Congress in 15 minutes' or 'equivalent of the Library of Congress on a key fob.'  I suppose I've contributed to that by putting all of WorldCat on an iPod (not hard to do, but the iPod isn't up to interacting with it yet), but how realistic is LC on a key fob?

The Library of Congress is larger than I thought.  The site claims 29 million books, 2.7 million recordings, 12 million photographs, 4.8 million maps and 58 million manuscripts!  Sometimes people equate a volume to a megabyte (the typical novel is around that), but more realistically, you'll need a scanned image of those pages, around 100Kbytes/page.  At 500 pages/volume that gives us about 50 megabytes per volume.  At 500 megabytes/recording, 2 megabytes/photo, 5 megabytes/map and 50 megabytes/manuscript I get: 30m x 50mb + 3m x 500mb + 12m x 2mb + 5m x 5mb +60m x 50m = 6 petabytes.  This doesn't include video.  At 5 gigabytes/video, it only takes 300,000 videos to match the scanned size of all the books, so lets call the collection an even 10 petabytes.  This is quite a bit larger than the size people often use, but more realistic.

My son recently put a terabyte together for around $1,000.  I just bought a one-gigabyte Memory Stick for $100 (about 100x the cost of disk).  10 petabytes is 10,000 terabytes and 10,000,000 gigabytes, so we've got something between 4 and 7 orders of magnitude in cost reduction to wait for.  Assuming the cost of storage continues to decline at 50% per year, it takes about three years to get a 10x reduction in price.  So it will be 12 years before you could stuff the contents of the Library of Congress on your personal storage server, 21 years before you could get it on your key fob.

That's not too bad.  15-20 years is a long way to extrapolate, but the cost decline has held up for the last 40.  I got asked the same question in 1970 though, and didn't have any easy answer then as to what it would mean.  We were talking microfilm, but it was still the same question.  Of course I suppose the contents of the Library of Congress fall into the "limited canon of book-centered knowledge", so the interest in this has probably already declined.  Still, it does give you an idea of the capabilities we might have and how hard it's going to be for publishers to keep everyone from having a copy of everything they come in contact with.

--Th

Comments

[Andy Havens]

I read an article awhile ago -- I think it was in "Wired" -- saying that data storage technology has actually been outstripping Moore's Law pretty significantly. I tried to find the exact article on Wired's site, but was unable to do so.

In searching around, though, I found this neat site: http://www.littletechshoppe.com/ns1625/winchest.html on the history of the cost of data storage.

Now expense does not, of course, translate directly into size. So it may take that 15-20 years to get the LOC on a key fob. What may happen more quickly, though, with the advent of high-speed, wireless Internet access being upon us, is the ability to access the entire LOC from a device the size of a key fob, even if the data isn't stored right there.

Which is, of course, kinda better. Since as soon as you put all that data on any kind of media, it's immediately out-of-date. And you can't connect with it to other libraries. Or other stuff.

Here are the related questions about connectivity, then: When is my wireless phone going to be so small that it gets worn as an ear-ring or implanted in my jawbone? When is the screen/monitor/HUD for the device going to be embedded in my glasses or something that can project an image right onto my retina? When is the speaker going to be the size of a hearing aid? When will it all be so small and cheap that the devices themselves are disposable?

Don't laugh... any of us over the age of 30 should think about how bizarre the words "disposable camera" would have sounded in 1975.

[Thom]

That's a great site. (The URL didn't display right in one of my browsers. Here it is again as a link.) Size and price seem so highly correlated that they are almost the same thing for storage. I too have seen articles showing the exponential drop in the price of storage has actually accelerated (to everyone's surprise), but I'd hate to base projections on that continuing. In truth, it will be a bit surprising to see the current exponential drop in price continue for 25 more years, but as I said, it's happened before.

--Th

[Lisle]

OK, sign me up. I want the LC keyfob that seamlessly connects and updates and interacts with the mothership in DC and anywhere else I want and the monitor in my glasses (rimless, please, no thank you laser surgery) and the input device wherever and I am willing to wait 20 years, no problem. It sounds so scifi and Gibson and Stephenson seemed so out there fifteen years ago. But hey, you are giving me hope! The intellectual property battles are what we can't predict.

[Raphael]

I put 'Library of congress size' into google and got this article.

3 years later, and $100 will get you a 4Gb stick. (The biggest flash drive is supposedly around the 1.5Tb mark - but $30k is a bit much)

$600-700 will get you a 2Tb HDD (Biggest I could find) With $1000 maybe you could manage 3Tb?

It's interesting to run into the article 3 years on and see changes in size and cost...

Right on target.

Good article