Screen Shot 2011-12-13 at 6.54.11 PM

Crashplan Online Backup LOST MY ENTIRE BACKUP ARCHIVE

After almost an entire week of not being able to backup to Crashplan because of “archive maintenance” on their part, I was informed today that instead of merely maintaining my backup archive, Crashplan LOST THE ENTIRE THING. That’s right, Crashplan lost all of my data, unrecoverably. It’s gone – my entire online backup archive of all of my data – my life’s work – 2.4 tb of everything – completely gone. And Crashplan doesn’t seem to have any remorse.

I’ve been using online backup for a number of years. Before I switched to Crashplan, I was using Backblaze for online backup. Backblaze was great – a lightweight, easy to use front end, fast backup speed, low system resources, good support response times, and most importantly, a reliable service. However, the one thing Backblaze didn’t do at the time was offer unlimited, or very long retention. Backblaze would delete data from my archive if it didn’t see that data connected to my computer for a month – so if I went on vacation for 2 months and stored away my external hard drives, my data would be automatically expunged at the end of the first month. Because of this little quirk of their service, I decided to switch backup providers to Crashplan. Note that nowadays, I believe that Backblaze retains data for 3-6 months, which is much much better.

Anyway, I decided that having longer than 1 month retention times for my data was important, and switched to Crashplan. My initial impression of Crashplan was good – a flexible service with lots of configuration options. However, over the last few months of using it, and leading up to this giant Crashplan data loss disaster, have seen a darker side. Backup upload speed was nothing like I saw using Backblaze. I would usually only see 100kbps or so upload speed, even when I knew I had much more bandwidth. The backup service would load slowly, and would frequently re-scan my computer. The UI was finicky – if the wrong settings were changed, my entire backup archive would be expunged by the system – with no time to go back and undo anything. Sure they give big warnings of this, but a small grace period would be nice. And throughout all of these issues, Crashplan support sucked. I’d wait days to get a response, had multiple support agents dealing with my case on multiple threads – the whole thing was a mess.

So when I received the below email this afternoon, I was extremely disappointed and immediately concerned for the safety of my now un-backed-up data, but unfortunately I wasn’t very surprised – this is just the kind of disaster I’ve come to expect from Crashplan.

Below you’ll find a transcript of my current support history, starting from the time when I reported my “unable to backup” issue, and ending with my response to their meager attempt to remedy the situation.

I had a brief call with tech manager Brad W. this evening about the issue – he was a nice, knowledgable guy, and offered me a 2tb seed drive to re-seed my data. He also confirmed that I’m the only account that this data loss situation happened with, and that they’re interested in finding a suitable solution. I haven’t accepted a solution yet.

For now, the situation is unresolved, and my data is still gone. And since I’m still traveling at the moment, I don’t even know when I’ll be able to get home to re-seed all of my data to a drive, even if they do overnight one to me.

What should I do? Should I accept their 2tb seed drive resolution and move on? Should I jump ship and backup with somebody else?

Crashplan Support Thread

“What happened to my data?! “Unable to restore due to a backup archive I/O error”

Jad
Dec-09 01:58 pm

I’m very concerned.. I was looking into my crashplan settings to see
why my backup was taking so long, and noticed that although it says
I’m connected, it’s saying the backup destination is unavailable. Also
online I get a notice when trying to view my flies “Unable to restore
due to a backup archive I/O error”. What’s going on here? Is all my
data gone? Why am I not backing up? Do you need any diagnostic info
from my computer or anything? Help help – there have been way way too
many hiccups with my backup so far since I switched from backblaze to
crashplan. Once again thinking of switching back. I just don’t want to
be always worrying about my backup crashing all the time – that is not
good.

screenshots attached. thanks!

-Jeffrey
Screen Shot 2011-12-09 at 2.54.43 PM.png (quick view)
Screen Shot 2011-12-09 at 2.53.16 PM.png (quick view)

==

Dec-10 2011 02:05 pm
David W.
CrashPlan
Hello,

First, your data is still on our servers and safe. Your archive is in queue for maintenance. Unfortunately, there are a few large archives ahead of yours in the queue. Archive maintenance is necessary to ensure reliable data for restoration. This is also what caused the error that you saw when you went to the Web Restore portal.

As soon as archive maintenance has run on your archive, your backups will resume normally. Unfortunately, you’ll have to wait for the other archives in front of yours in the queue first.

Please let us know if you have further questions.

Sincerely,
~David

==

Dec-10 2011 03:16 pm
Jad
Hi David, thanks for your response!

It seems like my archive has been in the queue for quite some time, and as a result I’ve been unable to back up for a few days – how can this happen? One of the reasons I chose Crashplan was the promise of “Real-time continuous backup” – which I took to mean that my files would always be backed up, as my upload bandwidth permitted – so that when I added or changed files, they would be backed up ASAP. I’m a little disappointed that maintenance on your side would result in me being unable to backup for so long. Can anything be done restore my backup functionality I paid for? Also, I understand that planned system maintenance is probably expected – and bet that I actually agreed to it in the license agreement I accepted – although I can’t find any type of availability reference in this doc – http://support.crashplan.com/doku.php/eula – but I’m not a lawyer and am almost definitely not reading/understanding it correctly.

Anyway, let me know if anything can be done to get my backup restarted, or if it would be at all possible for me to get a drive to backup to while I wait for my online backup to be restored.. Could I do another seed drive and have it added to my existing backup archive (not replace it)?

Thanks

-Jeffrey

==

Dec-10 2011 03:48 pm
David W.
CrashPlan
Hello,

Unfortunately, another Seed Drive would replace your archive rather than be added to the current archive. Also, the only thing we can do while your computer is in the maintenance queue is to wait for it to complete. This is part of the reason that we highly recommend that you use CrashPlan to backup to more than one destination (i.e. another computer or an external harddrive).

Sincerely,
~David

==

Dec-10 2011 04:38 pm
Jad
This is rediculous. It’s been 4 days since I was able to back up, and now I’m receiving warning emails from your system – for a problem that I have no control over! I actually DO backup to other locations with Time Machine – however that has nothing to do with the fact that Crashplan+, which I pay for, has not allowed me to backup for 4 days! What is going on here? Why in the world would I pay for such shitty service? Since I switched from backblaze to crashplan, I’ve had nothing but problems.

Please fix this, immediately.

-Jeffrey

==

Dec-12 2011 02:35 pm
Chad V.
CrashPlan
Hi Jeffrey,

Thanks for calling–give me a call back when that scan finishes!

~Chad

1-877-680-7526

==

Dec-12 2011 02:46 pm
Chad V.
CrashPlan
Hello Jeff,

Could you please send me your CrashPlan log files so I can take a closer look at the behavior you’re seeing? Please do the following:

– Open CrashPlan
– Double click the CrashPlan logo in the upper right
– Type: getlogs [REDACTED]
– Press enter

CrashPlan will automatically zip up your log files and attach them to this ticket.

~Chad

==

Dec-12 2011 02:49 pm
Chad V.
CrashPlan
Hi Jeffrey,

Okay–got a response back *right* away! Apparently after the archive move we ran a maintenance job on it (which is normal) and yours says it’s still pending because there were a few folks ahead of you in line. I’ve ‘bumped’ you to the top of the line so your archive should run maintenance and finish tonight, hopefully. It should be ready to access later on tonight/tomorrow. I hope this helps.

~Chad

==

Dec-12 2011 02:58 pm
Jad
Ok, thanks very much for bumping me up to the front of the line – hopefully I’ll be able to backup soon.

Is suspending a paying customers backup for 4 days normal or acceptable? This seems like a bit long to go without backing up.. what do you think?

-Jeffrey

==

Dec-12 2011 03:18 pm
Eugene O.
CrashPlan Support
Hello Jeffrey,

Thank you for your patience as we looked into this issue. Unfortunately, I have some bad news regarding your maintenance job. Your backup archive had corrupted data, and the archive could not be properly repaired by archive maintenance in this situation. This is a very rare situation. We are still looking into the root cause, but unfortunately much of the archive was affected and very little was able to be rebuilt. I sincerely apologize for this situation and the inconvenience of this event.

We definitely want to get you backed up and secured again, and I can expedite shipment of a seed drive to you so that we can upload your data again. We will cover the cost of the seed drive and all shipment, and we can get this out to you today if that would work for you. Could you contact us back with the shipping address, or simply confirm that the shipping address we previous used is the one you wish to use again?

From our records, I have the shipping address as:
[redacted]

Please let us know if we should use this one, or if you’d prefer a different address. Again, please accept our apology for this issue. Please contact me if you have any questions.

Regards,
~Eugene

==

Dec-13 2011 03:17 pm
Jad
Eugene-

Thanks for the heads up on the issue. After not being able to backup for almost a week, and now being informed that my ENTIRE BACKUP IS LOST, I’m extremely unhappy. How in the world can this situation possibly occur? Crashplan advertises itself as a reliable, real time backup solution. I was sold on the promise that my data with Crashplan would always be protected, maintained, and available to me at all times through the app, website, and mobile. app. To tell me that my data not only became corrupted, but was also unrepairable and then completely lost is absolutely unacceptable, for any level of service. Crashplan was my primary backup, and I’m extremely scared now that my data is unprotected and GONE.

I understand how competitive the online backup industry has become recently, and that one of the main hallmarks of marketing and competition is the reliability and ease of use of your service. You advertise both “Files secured in data centers worldwide” and “Real-time continuous backup”, yet I have experience none of this. My backup has been stalled for almost a week, and now your data center has completely lost my data.

Up until I started having issues with Crashplan last week, I had actually been recommending you to various friends and family. In fact, I set up both my Mother and Sister with Crashplan. How do I explain to them that the service I had once recommended has completely lost my data?

Regarding your “solution” – it’s weak, and highlights the fact that you really don’t care about me or my data. You’re offering to send me ONE measly 1tb seed drive? Did you not look at my account and see that I have 2.5 tb of data selected? At very very very least, you could offer to seed all of my current data to your online data center. A 1TB seed drive service costs $124.99 – so are you saying that all of my data – my life’s work – is only worth $124.99 to you? That’s insane.

I understand that periodically data issues do occur, and I appreciate you owning up to the issue. I’d like to have my confidence in your service restored and continue out the rest of my 4+ year prepaid account. I’d like a swift and appropriate resolution to this issue. Please let me know what you can do for me. Also, look forward to my blog post about this, and associated information sent over to Consumerist, ETC.

Your un-backed-up almost former customer,
Jeffrey

==

Dec-13 2011 05:00 pm
Brad W.
CrashPlan
Hello Jeffrey,

Here’s a recap of our phone call.

Here is what we can do to get you backed up again as quickly as possible.

We can send you a 2TB hard drive so you can seed your data to that. The remainder of your backup archive will need to be uploaded via the internet. We can comp you for the time it takes for the last part of your backup archive to upload to our servers.

We have your email address so we can send out a drive overnight once you make your final decision as to what you want to do.

~Brad

Dec-13 2011 05:26 pm

**UPDATE 2011-12-14 13:03 EST**

Looks like this is actually not an isolated issue, despite what Crashplan told me point blank on the phone last night. Facebook user Diane Dusek just posted on the Crashplan Facebook wall that she is having a very very similar issue. Screenshot of the Facebook conversation:

** UPDATE 2011-12-14 17:30 EST**

It’s 5:30 – the end of the day. I’ve called Crashplan twice today to talk to somebody about resolving my issue and moving on. On the first call I was put on hold for a brief minute, and when the guy came back on the line I was told that somebody could call me back shortly. 30 minutes later and no call, I called back and was once again put on hold, then told that somebody would definitely call me back within 30 minutes. That was 45 minutes ago. I’m simply being brushed off by Crashplan. Do they thin that this issue will simply go away, like all of my data did? I couldn’t have imagined how they could make the situation of losing a customer’s data worse, but somehow they managed to.

** UPDATE 2011-12-14 19:09 EST **

I just received a personal call from Matthew Dornquast of Code 42 software, makers of Crashplan. He was extremely apologetic, and did a great job of being extremely nice and understanding, and explaining the entire situation to me in as much detail as he could. I took a few notes from the call, and with Matthew’s permission, am publishing them. In the end, Crashplan is sending me out a 3tb seed drive so I can get my entire archive back online, and they’re also issuing me an account credit for the previous seed drive service I had purchased. Also of note, Matthew mentioned that he had not in fact read my blog post on the subject, but did read through the support thread, and heard from the Crashplan social media person that there was an issue that needed his attention.

Notes from call with Code 42’s Matthew Dornquast

  • The problem with data being lost was due initially to a hardware failure in one of the data centers Crashplan uses
  • Crashplan manage 62 petabytes of data across its data centers,  which is a huge amount.
  • In this incident, a single server had a hardware issue, which normally wouldn’t take down the whole thing and cause any data loss.
  • However, there was also an element of human error in this case, which ultimately caused a loss of data for about 20 customers.
  • All 20 customers did not lose all of their data – only some did.
  • Only 20 users were affected, out of the millions of Crashplan customers
  • Human error was a blind spot for Crashplan’s system, and there was no process in place to prevent it.
  • As of now, there’s a new process to prevent this type of human error from happening again.
  • This year Crashplan is expecting 380% growth. As a growing company, they are obviously still having growing pains.
  • For the 20 people affected by this snafu, Crashplan is offering free seeding to get data back into cloud.
  • Crashplan is a backup company, not an archive company. Don’t put all eggs in one basket – multiple backups is the way to go so that if one backup fails, there is a second backup. [ed. Which I do – I use Apple’s Time Machine as my local backup]
  • In this issue, software in the Crashplan system software detected data issue, and started healing immediately.
  • System was able to repair a lot of the data, and of the 20 people affected, very few actually lost all of their data.
  • Crashplan normally only offers 1tb seeding drives. But in this special case I can be sent a 3tb drive for seeding my 2.4 tb of data. [Although Brad told me last night that it was technically impossible for them to import more than 2tb of data from a seed drive into their system, Matthew assured me that his team would figure out a way to make it happen on his request]
  • Looking forward, Crashplan is coming out with new mobile stuff – iPad, android, windows phone 7 apps
  • New features for travelers – don’t use mobile access points, don’t run on battery power.
  • Summary – screwup by software people, 20 people affected, not the end of the world, working hard to get affected customers backed up again, putting systems in place to prevent this from happening again.
  • Responsible message to remind people of: Crashplan isn’t an archive service, it’s a backup service. Important to remember that multiple backups are the way to go for better security.

**UPDATE 2011-12-15 13:20 EST**
Crashplan released an official statement on the issue that caused the loss of my data:

On Dec 13th a storage node in one of our Minneapolis data centers experienced a hardware failure. While typically a non-event, additional human error escalated this failure into backup archive corruption for 20 of our customers, affecting their backup in varying degrees.Since CrashPlan automatically detects and heals around archive corruption, the affected customers don’t need to do anything – their backup archives will automatically heal over the internet. (However, to accelerate this process, we have offered overnight free seeding services to those customers.)While CrashPlan manages over 60 petabytes of consumer data globally and this event was limited to 20 people, we nevertheless take this failure very seriously and sincerely apologize for this lapse.Further, our operations team has modified our processes to avoid this human error in the future.



**UPDATE 2012-01-07 22:43 MST**

I’ve redacted my actual ticket number from the command line noted above. It seems that a user was able to successfully add their own crashplan’s log files to my support thread by using this line.. not good at all.

**UPDATE 2012-01-07 10:33 EDT**

Reader and beauty photographer Ashley Karyl just commented below that she’s been having big issues with crashplan too. She’s also noticed some new behavior. According to network monitoring tool Little Snitch running on Ashley’s computer, Crashplan made a connection to Amazon Web Services S3 cloud storage service this morning. The behavior deviates from the standard Crashplan behavior of connecting directly to Crashplan’s servers for backup. Crashplan has obviously had its share of issues lately. There’s even a static message posted on its support site indicating that they’re getting more support issues than normal”

“We’re currently seeing unprecedented demand for technical support, due to our recent rapid growth. As a result, our support response times are significantly longer than we want them to be.
Realtime live phone support and chat is available for customers experiencing urgent issues (help restoring files, lost or stolen hardware).
We are working diligently to resolve the delayed response times and apologize for the inconvenience.”

So could this Amazon AWS connection attempt indicate that Crashplan is making an emergency switch over to S3 to fix their own server farm’s I/O issues? Maybe, maybe not. Also to be considered is the fact that Crashplan just updated its software, which existing customers computers will automatically download and update themselves with. Very frequently developers use S3 to distribute software downloads – so it’s very likely that Ashley’s S3 connection was simply to download the latest version of the Crashplan client. In fact, I think this is very likely the case.

However, it’s still worth noting that things have been a bit bumpy for Crashplan lately, as evidenced by the static support message, and increased number of comments posted here.

Below is Ashley’s email explanation, and screenshot of her Little Snitch connection log.

Hi Jeffrey,
I didn’t grab a screenshot of the message as it popped up in Little Snitch, however I did just find it in the rules and I’ve attached a screenshot of that instead. Immediately afterwards the upload started to work normally and has continued for the last few hours. It could be that Crashplan are simply using S3 for the download of updated software, which many developers do, however it would seem strange in a case like this given the network they have.At the moment it is uploading nicely but this is after a week of problems and I’m still looking at a month to upload the rest assuming there are no further delays. I’m still on the fence about signing up but some options are simply too expensive and it’s a pain for me to start again with this slow connection while I’m waiting for fibre to be enabled in this area.ThanksAshley
PS I’m a photographer as well and want my archives backed up offsite to ensure I am covered if there was a fire or theft at the house.

 

**UPDATE 2012-04-05 17:13 EDT**

In light of the torrent of recent comments, and in the interest of getting some sort of official response from Crashplan on this ongoing issue, I’ve just tweeted directly to Code42 Software’s CEO Matthew Dornquast. During my initial issues with Crashplan, which prompted me to write this blog post, I had a phone call with Matthew – he was generally very nice, knowledgeable, and pleasant to talk to, and helped speed me through the process of getting my backup re-seeded. Hopefully he’ll take heed of this post and let everybody know what’s going on here, and why users are continuing to experience issues.

My Original Tweet
Matthew Dornquast’s Public Twitter Stream


**Update 20120-09-26**

After having crashplan crash multiple times on my Macbook Air running 10.8.2, I wrote in to support to ask for advice. Their response? “you’re running out of memory and need to manually increase memory”. Really? I have to manually fix your app that keeps crashing? Terrible.

Here’s my initial problem:

Whats going on here? My computer has been on and connected to the internet just fine lately… Error message I received: “Computer “My Computer Name” has been unable to reach any backup destinations for 3 DAYS. Back up to multiple destinations to reduce your risk of losing data.”

I forwarded Kelly my Crashplan log files. Then her response after reviewing my log files:

Good day Jeffrey,I believe that you may not be backing up due to out of memory errors. Looking through your logs it appears that the CrashPlan backup engine is running out of memory. Please follow the instructions below to allocate more memory to the CrashPlan backup engine.Edit the CrashPlan engine’s com.crashplan.engine.plist (“the plist file”) file to allow it to use more java memory. You will need to use Terminal for this, and edit the file using the ‘sudo’ command. When prompted, type in your computer’s Admin password. Note that you will not see the text-cursor move when you do – this is normal.1. Stop the backup engine by typing this into the Terminal application:sudo launchctl unload /Library/LaunchDaemons/com.crashplan.engine.plist2. Run this command to edit the backup engine:sudo nano /Library/LaunchDaemons/com.crashplan.engine.plist3. In /Library/LaunchDaemons/com.crashplan.engine.plist, find this line: -Xmx512m4. Edit that line to something larger such as 640, 768, 896, or 1024. E.g.: -Xmx1024mThis sets the maximum amount of memory that CrashPlan can use. CrashPlan will not use that much until it needs it. I would recommend starting out setting it to 768, and go higher only if you continue experiencing problems. You can increase it above 1024 if you have a really large file-selection.5. Hold the Control key and tap the x key to exit. Choose “y” to confirm it.6. You’ll see the prompt “File Name to Write.” Hit enter to save to the existing location.7. Start the backup engine by typing:sudo launchctl load /Library/LaunchDaemons/com.crashplan.engine.plistIf you have any problems with this process, you can call us directly at 1-855-411-4242. We are available during the week 7-7 Central Time. Please let me know if you have any questions.Regards,
~Kelly

So, for an out of memory problem, they’re trying to instruct me on how to go in and manually fix it. For a consumer facing product, this seems much much much too complex to expect me to do on my own. Crashplan, fix your app.

And it seems like they are working on fixing it with a Native Mac Crashplan App!

I got a nice tidbit of information from support rep Kelly – Crashplan is working on a Mac Native Crashplan App. That’s right, native code, no java wrapper crappyness. Here’s Kelly’s words:

The reason that it is not native is because CrashPlan is a cross platform program. Currently we are working on a native Mac client, but this does not have a time frame on when it will be released.

– Kelly S., Crashplan Support

Hoping for that new native crashplan client to be released soon…

**Update 2014-09-06**

Still no native MacOS App… let me know if you hear any updates on this..

201 comments

  1. What surprises me (actually, after so many occurrences “Disappoints” is a better word) is how many people do not TEST their backups (by doing a restore to an alternate site and validating all content/functionality)….Additionally having only one backup target is a high risk activity.

  2. Gdan Coleman says:

    I had a computer crash a few days ago due to a hard drive failure. I had no worries though because I knew that Crashplan had my back. I replaced the failed hard drive and reinstalled windows. This whole process has taken about a week and finally, today (Fri 9/12/2014), I was ready to restore my files from Crashplan. I began the restoration about 8 AM. Just before 1 PM, the restoration stopped at about 30% complete and a message appeared that said the following: “Destination Unavailable – backup location is being moved.” It is now 9:00 PM, I just checked and it is still saying the same thing. I sure hope they are available over the weekend. I need my data!!

  3. I think this story tells what Crash Plan already told: You cannot trust any backup 100%. Mabye 99.99% but not 100%.
    Even if CrashPlan did everything right with 10 copies of your data around the world, they can still go bankrupt and the curator can get all hard drives smashed so he can sell the servers to pay the bills.

    You must have the amount of backup you are satisfied with. You can just have CrashPlan, and count on being able to establish a new one without dataloss when (and for any company it is just a question of time) they are no longer there.

  4. Richard Scott says:

    Crashplan is great at backing up date, but that’s not the important part of their service.

    Has anyone who “loves Crashplan” actually tried to restore files you’ve lost? I had lost files, and tried to restore my Linux server and my account at Crashplan was corrupt and it would not restore ANYTHING!!!!. 756GB of data lost!

    I stupidly thought that Crashplan was a backup solution but it is not.

    Compare the cost of crashplan to the personal cost/value of your files (to you). I bet your files are worth far more than the monthly Crashplan subscription.

    Just imagine what you would do if you lost all your photographs you’d ever taken, all your documents you’d ever written and all your music you’d ever purchased.

    ***** STAY AWAY FROM CRASHPLAN IF YOU VALUE YOUR FILES ****

    • Rik Hemsley says:

      If you only had one place you were backing up to (CrashPlan) and didn’t test restoring your files until you actually needed them, you can’t blame CrashPlan. That’s exactly saying ‘I backed up to tape for years and never tested restore, then found they were corrupt. Never back up to tape!’

      Backup is your responsibility and unfortunately there is no ‘pay for service, forget about it’ solution.

      • Jimmy says:

        It is certainly unwise to not have a secondary backup. However, it’s in no way an excuse for selling a backup service that doesn’t work, assuming the story is true. It’s not the first time I’ve seen this same story, and if the situation still haven’t improved I will probably not renew my Crashplan subscription.

      • Rik Hemsley says:

        I’m not giving any excuse for CrashPlan here. They already have continuous monitory of backup archives to test for corruption so obviously there’s something missing at a higher level to inform this user that there is a problem (assuming the fault lies with CrashPlan here and not the user).

        CrashPlan appear to have many happy users, so I’d say the service does work, but of course there can be flaws in any system. One as complex as backup will of course have problems – which is why the responsibility for being sure your backups work will always lie with you, not with some company you pay a fee to for helping you with part of your backup strategy.

        Use multiple backup systems (not storage points, systems!) and test them regularly, or you put yourself at great risk of being unable to restore data when you need to.

      • adam says:

        Huh? How can he not blame crashplan for the corrupt archive? It wasn’t the flying spaghetti monster that screwed up his files – crashplan were incompetent not keeping his backup, when their only function is to backup files.

  5. Vincent says:

    I’m with SOS Online Backup. I searched the internet for online backup companies/services to find the 20 best and most recommended for end-users.
    I checked the internet for reviews, user rating, and other information.
    Then I opened trial accounts, during this time i contacted support to get information, including technical behind the scenes to find out for myself if their systems seem reliable to me.

    Every company where support took longer then 24 hours I got rid of.
    Every company that offered software solutions which crashed, unstable, too slow to handle my 8TB of data to handle I got rid of.
    Every company which didn’t have their own backup solution I got rid of.
    Every company that forbade me as end user to use a server 2012 system and told me to use business plans as private user simply because I use a server OS, i got rid of.

    The only one that remained was SOS Online backup.
    Support replied within 5 minutes.
    Technical guy on the phone in the Ukraine I talked to told me they have their own Raid implementation to handle the loss of several HDD and still ensure data integrity and on top of that, he told me they mirror the data to a 2nd raid server always. So even if one rig completely breaks down, fire, lightning, etc, their customers data is still available. But he was very fair pointing out: in case of a major incident spanning more then 1 country: combined power grids of countries, solar flare, war, etc. or the simple fact that sudden death of a 2nd system within a time-span of several days (they need that to rebuild the first server and move back all the data) that would mean data loss.
    So, although such events are highly unlikely, there is no 100% guarantee. But if I want 99,9999999% guarantee he told me to have all data stored local, a backup local, a 2nd backup at a friends place, AND use their online backup.

    So that’s what I did. I setup a oVPN between me and my fathers house. I have my own server2012R2 with 32TB, at my fathers I put a copy of that system with Server2012R2, also 32TB. And then I backup everything to SOS Online Backup as well.

    Also, this is not 100%, I’m very well aware of this. Even without solar flares targeting continental power grids or events of war, there’s still the minimal possibility 50 HDD drives fail all within a time-span of less then a week. Very unlikely, but not impossible!!!

    So, since 100% security doesn’t exist, even if Crashplan promises that, it can’t promise it, it must be a lie!

    Everyone has to figure out how important their data is versus the costs of having proper backup plans.
    My plan thus, the main server can handle 2 HDD breaking down at once.
    In case a 3rd breaks, I still have the system at my dads.

    According to the plan, it’s highly unlikely I ever need SOS Online Backup for restoring my data…
    but if it ever gets to that, it’s highly unlikely even more that at the same time their 2 systems with my data also breaks down within the time I need to restore my data.

    The costs are high, 4000EUR for my own backup systems. Then the monlthy fee to SOS Online Backup.

    So, it’s up to everyone to think about: How important is my data to me versus costs and efforts to get 99% security, 99,9% security, 99,999% security…
    or 99,9999999% security..

    But 100,000000%? You won’t get it.

    • Robin says:

      Thanks, I was skeptical as this sounded almost like someone promoting SOS but then I remembered that I gush about things that I like, evernote for example and I’ve been accused of ‘shilling’ or being an evernote PR or employee. Anyhoo… I decided to check out SOS backup which I had somehow never heard about and found lots of great reviews and mentions including WSJ. I’m not done my homework and no guarantee I’ll go with them but still, thank you for your story pointing to them as an option.

      To everyone else, Caveat emptor. Do your own research.

  6. Adam says:

    Crashplan’s latest is that they prune old versions (even where you have told the software not to) and compress your archive every couple of months.

    There’s no notice, no warning and in my case I have zero access to backup or archives for almost 2 weeks each time.

    When their support finally responds to tickets they are useless they basically said stiff, you just have to deal with losing access whenever we feel like archiving, and there is no notification – one day the backup service is just gone.

    So you won’t have access to the service for over 10% of the time you pay for, and will literally spend weeks without access – don’t trust these people with your data, you simply cannot rely on the service.

  7. Ben Fletcher says:

    Hey man sorry to hear about your data… It bites, but at least you were smart enough to have more than 1 copy…
    I use CrashPlan – initially I wanted it for all my data, but not a chance. So for me it is part of a fairly costly, but fairly bullet proof system.
    I have a Datatale thunderbolt RAID set up in RAID10. 4x6TB Mirrored. This is my main drive – keeps my Mac fast and performance up the wazoo (Until I can afford SSD’s in that bad boy ;) ) It is backed up instantly to a Drobo 5N with dual disk redundancy. Time machine backs up the OS system to a 4TB external, which is then backed up to the Drobo also. Then, I have 3 4TB externals that make a copy of my data every weekend, and then get taken to my partner’s office to be swapped with their another set. Lastly, all my mission critical data (clients photos that haven’t been delivered, folio, family photos has a further copy stored in the CrashPlan cloud.
    It’s a pretty extreme setup for a home business, and cost me a fair few grand, but I can sleep at night!

    I must say the thing I find the most annoying about CrashPlan (apart from them losing folks’ data!) is that every time you disconnect an external drive that is a backup destination, it performs that irritating “block-synchronisation” – which with 4TB data over gigabit ethernet takes roughly 24 hours. So that means that were I to use it to back up my entire archive, I can NEVER turn my servers or iMac off!! Or if I want to access my backups when my computer is on I need to give it 24 hours notice, so to speak. It’s ridiculous!

    I would not recommend CrashPlan to anyone but the casual computer user who stores only a few photos and movies on their laptop. Mac Backup Guru is the way forward. Simple, easy, creates bootable copies and versioned backups using hard links. It’s effectively a bootable Time Machine for every disk you’ve ever owned.

    Good post – wish I’d come across it BEFORE I dropped coin on CrashPlan….

    Ben

Leave a Reply