"Collectively, this presents potential risks that scientific data may become less available to end users", I wonder what would the Soviet space program do. Shoot the horse, it's got pustules coming out of the tentacles coming out of its neck! they have you by the short and curlies. You can check in anytime you like, but you can never leave (unless you pay us a lot of money). You can also change your choices at any time, by hitting the If we assume an average size of 100KB, thats around 90 petabytes; if the average size is 1MB, thats 900 petabytes almost an exabyte! The folks at the Tech Expectations blog have bravely tried to answer the How much data does x store? question, but even they couldnt locate a decent number for Amazon S3, Microsoft OneDrive, or Google Drive. If you scour the web, though, some rough ballpark figures emerge: To put these figures into perspective, an average computer probably has a 500GB or 1TB hard drive, and a petabyte is 1024TB. Just to funny that no one thought about the access charges. Like the porn story, well first start with some theoretical numbers, and then move onto some real-world figures (and hardware) from Backblaze, a cloud backup provider. This site may earn affiliate commissions from the links on this page. And then theres cloud computing. Now they have to pay for each and every data access!! We know that, as of last year, one million files were being saved every five minutes so today, with four times as many users, thats 800,000 files per minute. Lets discuss some real-world numbers and real-world hardware! I'm just a casual home user, and I recently migrated my offsite backups (a couple of TB) from a hard drive in my bank's safe deposit box to S3. Still, NASA should've done their research. If people say no to these cookies, we do not know how many people have visited and we cannot monitor performance. Is that where they shunt your data out of a side door and you have to pay to go back in through the front entrance again? Just because NASA isn't going anywhere doesn't mean the project isn't going to get moth-balled at some point. That's with 16TB spinning disks. The power of the cloud is vested in the fact that it can be coerced and shoehorned into tasks as disparate as a cloud-based supercomputer, to webmail, to simple document storage. While storage might not be as sexy as terabytes of RAM and thousands of CPU cores, it is the most reliable way of measuring the size of the cloud, especially when we factor in bandwidth usage. ", So I pointed to the five year old computer on my desk and said "this machine holds about ten times as much data as you're talking about. Perhaps it is time for those bean counters everywhere to think again. That will save you millions. At any one time, streaming adult videos probably utilize around 30% of the internets total bandwidth, which equates to around 6 terabytes of porn being consumed every second. No one ever pays list cost for AWS services. If that's all you're using it for, you may be better off with one of two other options: Backblaze B2 has an API, and is $5/TB/month. I dare you, go look at their published costs for all the services you can use and try to figure out what it will cost to do something. wouldn't consider that maybe storing data in a cloud had costs at some point? Because data is intellectual property, not actual property. What kind of a bean counter are you?? This might sound like a lot, but when you consider that Google, Amazon, Facebook, and Microsoft regularly roll out data centers with floor plans of over 30,000 square meters (300,000+ square feet), its really not that much. In other words, the current model is that end users have unfettered access to data (because, it's their data and there's no cost associated with retrieving that data from on-prem storage). End-stage crapitalism. Put another way 50 racks of ESS/DSS is not that much dif. We automatically transfer to Standard IA after 30 days. Japan's Asteroid-Smashing Probe Reveals a Surprisingly Young Space Rock, Big Rigs Begin To Trade Diesel For Electric Motors, For every complex problem, there is a solution that is simple, neat, and wrong. But wait, as Backblaze B2 Cloud Storage comes online, that will certainly add a petabyte or two or a hundred to our 2016 numbers. Well, sorry, it's the law. Over the last 5 years the company I work for has leased all of its buildings. It's not different from serving a website on AWS, they charge egress for that too. He probably would if it got him a DoD contact or two. opex? They realize Amazon is in this business for profit, right? At the very least NASA could hire them to design and build the storage and to train staff. What the Soviets were *really* good at was heavy lifting and they still are. You get nickled and dimed to death. For the most part, real numbers from the big companies, such as Google, Facebook, Amazon, and Microsoft, are few and far between. Amazon S3, which is significantly larger than Dropbox, handles 650,000 requests per second.. This doesnt seem like a situation where the cloud is going to save you money over running your own infrastructure. With that said, what we do know is that Backblaze has a little over 200 petabytes of customer data. The agency expects to transfer data off-premises for years to come. Capitalism - how to get to pay blood in order to eat shit, with an additional post-dated charge if you ever fail to confirm how beautiful the experience of eating shit is. For example, *NASA* shouldn't be at risk of evaporating tomorrow. On the other hand, if I need an entire server, I might as well buy one and only pay for colocation. The space agency therefore projects that by 2025 it will have 247 petabytes to handle, rather more than the 32PB it currently wrangles. You can keep using GitHub but automatically, NASA needs 215 more petabytes of storage by the year 2025, and expects Amazon Web Services to provide the bulk of that capacity. From an administration perspective it's a basically a doddle, *if* you are an experienced in GPFS. This is taxpayer $ they beg for; if they can't manage it as well as AWS then flush out the upper layers of the hierarchy. In the grand scale of things, a lot more space is dedicated to servers (i.e. At the very least, then, Microsoft and Facebook data centers play host to more than 100,000 hard drives. And thats a live question because a March audit report [PDF] from NASA's Inspector General noticed EOSDIS hadnt properly modeled what data download charges will do to its cloudy plan. Asking the in-house sysadmins what they think about moving to the cloud is a bit like asking in-house developers what they think about outsourcing or what the support staff thinks about a chat bot AI, whatever legitimate concerns they have is going to get mingled with very self-serving arguments. Then they wonder why our expenditures are so high. > The company we are leasing from bought the site specifically for the purpose of leasing to us, so their purchase price is in the public record. The data in question will come from NASAs Earth Science Data and Information System (ESDIS) program, which collects information from the many missions that observe our planet. Yeah, IMO if you have enough data to fill a datacenter, it would be cheaper to have your own than pay someone else to rent space. When it comes to the geographically distributed i. I would think that NASA would be able to implement this. ..how many satellites forgot to pay their ISP bill. Including that dandified grandson of a whore-monger currently in the White House and planning his 3rd term. It's good to have a "cloud practitioner [amazon.com]" on staff to run your cloud ideas by. Roughly $16,000 in tapes + The storage library robots. The bigger the cache, the lower the egress charges. And it will if NASA can afford to operate it. I've contracted out for those. On a single cloud cluster, Google can host and serve petabytes of YouTube videos and store all of your email and documents. We measure how many people read us, Digital file lockers, such as Rapidshare and Megaupload, account for around 10% of traffic worldwide. It's when you get to a couple of racks full that the costs might turn into your favor. The Soviets just didn't fly computers or do stuff in software if it didn't fit within the weight limits. Your revenue stream is pretty solid unless thousands of your customers go broke "unless thousands of your customers go broke". Particularly egress charges. I am not sure 16TB disks are an option yet, but I am just scaling up from the system at work. Sure, but people are really bad at estimating how much traffic their site will generate. If I need a datacenter, well, it would be better to have my own. Re: What happens to your data when you can't pay? Dropbox, a year ago, stored 10+ petabytes of data. However, the space agency. Also, while it is trendy to discuss IT equipment, same applies for real estate. Oops. Specialization is one of the better ones. So my point was that 'but if you bring it inhouse, you need to have the skillset' doesn't work in this context, because you would need to have the skills in house either way. PCMag Digital GroupExtremeTech is among the federally registered trademarks of Ziff Davis, LLC and may not be used by third parties without explicit permission. From the total amount of storage we can also work out the cost of cloud storage and from there, we can finally work out why the likes of Google, Microsoft, and Dropbox are falling over themselves to provide cloud storage services. These cookies collect information in aggregate form to help us understand how our websites are being used. That means EDSIS wearing cloud egress costs. He currently writes and rants about drive stats, Storage Pods, cloud storage, and more. In some SaaS contexts the hosting service does reduce the client-side skill needed (though I feel like this is mostly a failing of software development in giving up on making their software easy to deploy and has gotten worse as of late). Once NISAR and SWOT are operational and providing sufficient data, complete an independent analysis to determine the long-term financial sustainability of supporting the cloud migration and operation while also maintaining the current DAAC footprint. Yes, 200 petabytes is a big numbera two followed by 17 zeros with five commas thrown in. They perform functions like preventing the same ad from continuously reappearing, ensuring that ads are properly displayed for advertisers, and in some cases selecting advertisements that are based on your interests. Customise Settings. But NASA is tired of managing all that infrastructure, so in 2019, it picked AWS to host it all, and started migrating its records to the Amazon cloud as part of a project dubbed Earthdata Cloud. This was done in the context of trying to determine what would happen if the data in Googles data centers was in fact stored on punch cardsspoiler alert, Boston gets crushed. However, if your agency heads pushing this contract are doing so for their personal enrichment or their colleagues personal enrichment, then, this doesn't really matter. Which is why, to this day, they are the heavy lift ki. They already had the all of. Nowadays, it seems like no-one gets into trouble for defective recurrent costs. Glad it's not just me. That's literally considered a selling point. Without building custom hardware, you can squeeze 48 drives into a 4U enclosure. It's just that for some reason this year, management forgot that rental tends to cost more than owning over the long term. people downloading files from their Dropbox), but its probably in the region of 10 to 20Gbps. Consideration must be given to how to efficiently replicate across sites, protecting against data loss and allowing optimum servicing of large access requests. Oh, you want it back? However, even I know that to actually read out and download that data, it would cost me about as much as buying a whole new hard drive of that capaci. Buying two servers and renting colocation in two datacenters would most likely be cheaper over time than renting the same amount of power in the cloud, since renting means I buy the server for the provider and also pay for profit.