I saw this article about a new scientific advance that extrapolates to 50 petabyte (a million gigabytes) storage device and I wondered what this might mean. Now various practical limitations would probably constrain an actual production device to less than 50 petabytes but this really doesn’t matter. The makers of large storage devices, like EMC, use arrays of relatively cheap commodity drives readily available along with some ingenuous other hardware, racking, cooling, software, etc., to easily build (at a substantial price, not exactly home computer ready) 5 petabyte and larger devices today. So assuming this new technology or just the steady advance of conventional magnetic oxide technology, 50 petabytes in a single and sorta affordable device is a likely event in the next couple of decades. In fact most estimates of storage on the net already uses the next measure of exabytes already, so petabytes is not really so big, but that’s distributed over a vast array of servers.
So I thought it would be fun to translate this into tweets, i.e. what is some reasonable set of assumptions to determine how many tweets we all have to do to fill up one of these monsters.
I start first with tweet size, nominally 140 bytes, but not really. I have no data but based on scrolling through lots of tweets I might even think people use half that much, but I’ll round that to 100 as the average length. Now given the actual contents of tweets substantial compression is possible (I doubt Twitter does this, why waste compute time when storage is so cheap, but I’ll do it for my estimate). I’d bet 3X is a reasonable guess but with tinyurl’s and lots of proper names and diverse set of languages, I’ll opt for 60 bytes on average.
Now the second guess, how many people, realistically, would ever be the entire twitterverse. I’ve seen numbers around 150million today (vs 850million for Facebook) but since we’re talking about a new technology, probably decades from deployment, I’ll just go with approximately 1/3 of humanity, or 2 billion people.
So, 50,000,000,000,000,000 / 60 = 833,333,333,333,333 / 2,000,000,000 = 416,667
Now I’ve seen 78.5 years as life expectancy (in U.S., who knows average for my 2,000,000,000 global users?) and let’s assume no tweeting in first five years, so that means the average person lives for
(78.5-5) * 365.25 = 26,846 days. Now let’s assume we’re all awake for 16 hours a day, which means 429,534 hours of non-sleeping life.
Or essentially a tweet an hour for your (all 2 billion of “you”) entire life to fill up one of these storage devices.
Have you met your quota?
But other than such a silly number is there any more to this than hypothetical arithmetic?
And I think there is. While tweets don’t exactly do it (Facebook is probably better) we are nominally reaching the point where we’re recording at least some of the daily life of most of humanity. And all that information can be processed in a wide variety of ways, that used to be unimaginable (the combination of commercial use of “big data” and spooks spying on us all is leading to lots of new data analysis techniques), to extract more condensed findings about any event or issue.
So if you count all the social media (and whatever might be left of “real” news), blogs (someone somewhere writes about anything), social media (Facebook and all the other true confessions sites), tweets (even pull in texts as we know the spooks are doing when they get the raw data from “tower dumps”) we probably have enough information we can create a reasonably complete history of any event, esp. as we can trace not just the instant information in the moment but the re-analysis, disclosures, rewriting, opinion, etc., that will refine the event over time.
I’m a fairly big fan of Ken Burns documentaries but especially his use of the dramatic technique of reading actual letters written by real and more-or-less ordinary people, plus showing some press accounts, to get history to real-life sense of what really happened and how people dealt with the events. As we go forward and continue burning up the planet some future historians of the small fraction of living humanity in their underground caves may be able to actually figure out what we did to destroy the future.
So tweet away and put all your personotrivia in Facebook and record your brilliant essays in blogs as history awaits what we have to say. And maybe in the future you can refer back to all this recorded and processed information and answer the question many of us over-55 types can, “what were you doing when you heard about JFK?”. Now that can be expanded in 50 years in the future to “what were you doing when you heard about Katie and Tom?” – isn’t progress wonderful!