In an earlier post I talked a little about our new VIAF infrastructure and claimed that it is going to let us update VIAF faster. Up until this month other issues, such a finishing our port and spending the time to make it easier to run (plus the holidays at the end of 2012) have kept us from delivering on that promise, but this month we managed to build a VIAF database on the morning of January 15th. Since we process whatever data is ready for matching on the 15th of each month, this is as good as it gets unless/until we start real-time updates, and at least a week earlier than our previous best efforts.
Since we made as many changes and added as much new data to VIAF this month as normal, I attribute this to our new system and some fairly extensive work to streamline the process.
Lorcan was reminded of a post he did in 2005 (WorldCat in your pocket) reporting our experience in putting the WorldCat data on an iPod. The WorldCat we were experimenting with then had 56 million bibliographic records (current count is 289 million), something that would now fit on a micro-SD card, which is visible, but weighs less than half a gram and is small enough to lose in your pocket. The current WorldCat data would just fit on two 64-gigabyte cards. Literally fit hundreds of those little cards would fit in that rather thick (15 mm) 2005 iPod case.
The VIAF clusters themselves compress down to about 7 gigabytes (http://viaf.org/viaf/data), but the 135 million bibliographic and authority records those clusters are built from would be quite a bit larger, although they might just fit on one of those half-gram 64-gigabyte cards. Since the databases behind the VIAF interface total about 150 gigabytes, it would be a squeeze to have not just the VIAF data in your pocket, but actually run the whole system on a pocketable smart phone, but we aren't far from it!
Not that we are planning to do that.
--Th