There’s been a LOT of noise about this incident in the past day or two, the very definition of a cloud nightmare. Git/SVN & Project Management SaaS Code Spaces has been hacked and completely deleted by a hacker.
It started off with a large scale DDoS attack (the likes of which Feedly and Evernote have also seen this month) – and in all cases the malicious parties have been asking for ransom money to stop the attacks.
This time however, with Code Spaces – somehow the malicious parties got access to their Amazon console..which gave them access to delete everything, including snapshots and backups. This has rendered Code Spaces totally inoperable, and unfortunately it looks like they won’t be able to recover from this.
Source code hosting provider Code Spaces has suffered the ultimate cloud nightmare, having been effectively forced out of business by the actions of an attacker who managed to gain access to its Amazon EC2 control panel.
The devastating incident began on June 17 when Code Spaces – a company that claimed to offer “Rock Solid, Secure and Affordable Svn Hosting, Git Hosting and Project Management” – became the target of a DDoS attack from an unknown party who demanded “a large fee” to make it stop.
This isn’t the first such incident in recent weeks. Evernote and Feedly were each knocked down on June 10, reportedly by criminals trying to extort money. Both managed to restore their services, albeit only after extended outages.
The difference this time is that in addition to having access to a formidable botnet, Code Spaces’ assailant had also gained access to the company’s Amazon EC2 control panel, giving him control over the data it had stored using Amazon’s Elastic Block Store (EBS) and S3 cloud services.
“We finally managed to get our panel access back but not before he had removed all EBS snapshots, S3 buckets, all AMI’s, some EBS instances and several machine instances,” the company wrote in a message posted to its homepage. “In summary, most of our data, backups, machine configurations and offsite backups were either partially or completely deleted.”
The ironic part is they make claims about having bulletproof ‘offsite’ backups, when all they seem to mean is they put some backups in a different Amazon region – not even on a different cloud platform – so anyone having access to their Amazon console..could nuke everything. You can see their claims on Archive.org here.
We have invested a great deal of time and effort in developing a real-time backup solution that allows us to keep off-site, fully functional backups of your data – at high performance. We literally backup everything you do, as soon as you do it.
Also why weren’t they using 2 Factor Authentication for such an important account? I’m really very interested in how the extortion DDoS gang got hold of their Amazon credentials.
The net effect was that, once the smoke cleared, Code Spaces no longer had any service to offer its customers.
“Code Spaces will not be able to operate beyond this point,” the company’s statement reads, “The cost of resolving this issue to date and the expected cost of refunding customers who have been left without the service they paid for will put Code Spaces in an irreversible position both financially and in terms of ongoing credibility.”
The source-sharing site said it is now working to export whatever data remains so that customers can regain access to their files. All Git repositories and some svn repositories are reportedly available for export, although their backups and snapshots have been erased, and some svn repositories have been destroyed altogether. All of Code Spaces’ EBS-hosted database files have also been wiped clean, along with their backups and snapshots.
The company said it is experiencing “massive demand” for support and has asked customers for patience as it struggles to get to everyone.
“On behalf of everyone at Code Spaces, please accept our sincere apologies for the inconvenience this has caused to you, and ask for your understanding during this time!” the statement reads. “We hope that one day we will be able to and reinstate the service and credibility that Code Spaces once had!”
In a way I’m glad this happened though, nothing against Code Spaces (never even heard of the service) – but it’s a lesson to be learnt – Cloud DOES not make you any more secure, in many ways it makes you less secure.
All your eggs in one basket? Then yeah, you are asking to get burnt. Set up pull only backups on an entirely different platform (the desktop in your office is fine, or on Linode/Rackspace/Softlayer etc).
It costs $5 to run up a Digital Ocean VPS and pull backups from your production servers – just do it.
There’s further discussion on Reddit here too – Hacker Puts Code Spaces Out of Business.
Source: The Register