Poor mans offsite backup

Ghworg — Sat, 02 May 2009 07:00:34 +0000

Before I got some proper offsite backup space* I used to make tarballs of files and dump them in my gmail. Not the most elegant solution but it worked and is free (I know about gmailfs, but I could never get it to work reliably).

This is the script I used to send the files:

#!/usr/bin/env python
import getpass
import optparse
import smtplib
import sys

from email import Encoders
from email.MIMEBase import MIMEBase
from email.MIMEMultipart import MIMEMultipart

opts = optparse.OptionParser()
opts.add_option(‘-u’, ‘–username’)
opts.add_option(‘-p’, ‘–password’)
(options, args) = opts.parse_args()
if len(args) < 1:
print "Usage: send_file_to_gmail.py [-u user] [-p pass] filename"
sys.exit(0)

if options.username == None:
options.username = raw_input("Username: ")
if options.password == None:
options.password = getpass.getpass()
if options.username[len(options.username)-10:] != "@gmail.com":
options.username += "@gmail.com"

msg = MIMEMultipart()
msg[‘Subject’] = "Backup of %s" % args[0]
msg[‘From’] = options.username
msg[‘To’] = options.username

msg.preamble = "Backup of %s" % args[0]
msg.epilogue = " "

fp = open(args[0], "rb")
atfile = MIMEBase("application", "octet-stream")
atfile.set_payload(fp.read())
fp.close()
Encoders.encode_base64(atfile)
atfile.add_header("Content-Disposition", "attachment", filename=args[0])
msg.attach(atfile)

s = smtplib.SMTP()
#s.connect("localhost", "8025")
s.connect("smtp.gmail.com")
s.ehlo()
s.starttls()
s.ehlo()
s.login(options.username, options.password)
print "Sending %s" % args[0]
s.sendmail(options.username, options.username, msg.as_string())
s.close()

The above code is made available under the MIT license.

* While I am writing this the Dreamhost backup space I use has been down for about two weeks (it came back briefly a couple of days ago but is back down again now). I can’t complain too much as it is a free extra they throw in with the hosting package but I may have to restart my gmail sending if this keeps up.

Offsite backup Catch-22

Ghworg — Thu, 26 Feb 2009 17:10:02 +0000

Over the past couple of days I’ve been setting up a secure offsite backup for my home machines. Dreamhost started offering 50GB of space for backups for free to their customers, this is vastly superior to the “send an archive to gmail every now and then” method I was using before.

I decided to use duplicity as it will use scp as the transfer (among manu others), encrypts all the data and automatically does incremental backups transferring only the changed files. Newer versions even support gmail and amazon s3 as backends, which is really cool but not something I need.

I had one problem setting it up, which was the obscure and unhelpful error reporting.

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 463, in 
    with_tempdir(main)
  File "/usr/bin/duplicity", line 458, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 451, in main
    incremental_backup(sig_chain)
  File "/usr/bin/duplicity", line 186, in incremental_backup
    bytes_written = write_multivol("inc", tarblock_iter, globals.backend)
  File "/usr/bin/duplicity", line 87, in write_multivol
    globals.gpg_profile,globals.volsize)
  File "/usr/lib/python2.5/site-packages/duplicity/gpg.py", line 217, in GPGWriteFile
    file.write(data)
  File "/usr/lib/python2.5/site-packages/duplicity/gpg.py", line 125, in write
    return self.gpg_input.write(buf)
IOError: [Errno 32] Broken pipe
close failed: [Errno 32] Broken pipe

That is not a useful error message (and I’ve programmed in python), the actual error turned out to be a typo in the GPG key id I had entered. Once I’d worked that out everything went fine.

The real problem is the catch-22 between security and recoverability. In order for the data to be secure it needs to be encrypted, for that you need either a key or a password (or both for an encrypted key). Therefore in order to restore the data you need a copy of that key/password, but the situations in which you are going to be restoring from the offsite backup are ones in which all your data and local backups have gone poof. In which case the key will have gone poof too.

Okay, if you used just a password and memorized it then you’d be fine. But a password you can memorize is probably too weak to be secure, certainly for me it is with my terrible memory.

So, in order to make the data recoverable the key used for encryption needs to be backed up offsite too, but then your security goes out the window. So the key itself needs to be encrypted before sending it offsite, and there we have the catch-22.

Data needs to be encrypted to be secure, but we need to save the key to make the data recoverable, which makes the data insecure.

The Troll-Range » backup

Poor mans offsite backup

Offsite backup Catch-22