RFC: Plesk backup to Amazon S3 – it works!

First thanks to Uli for the first step towards some idea’s and Armijn giving good guidelines how to use some variables together with commands.
cause I am a newbie on bin/bash scripting.. but with lots of testing tonight I was able to backup several ‘clients’ content towards buckets and objects on Amazon S3. Probably this is a first good step towards the usage of a simple script for the backup of your server’s content to Amazon S3

Needed: Python, S3cmd, data and the correct settings

note: this script is not finished yet as I am manually executing it and the removal of data from Amazon S3 is not well tested (it works, but I need to check it during the time the stuff is running.


### ###

4 thoughts on “RFC: Plesk backup to Amazon S3 – it works!

  1. @sorin,
    the idea’s is nice, but as I have multiple ‘clients’ and domains I want them to have them as clients in my buckets. I succeeded now with my scripts and crontab settings ;-)

  2. Its a shame that there isn’t yet an S3 option within the backup manager on Plesk.

    I currently use s3cmd and pleskbackup to achieve a backup per domain

    Each domains backup is in a single tar
    No scheduled backups are required in plesk control panel
    To restore: an ftp user is set up on localhost with a home directory of /home/backups – the backups then show up in the ftp repository on the control panel.
    All backups have the same date stamp – set when the cronjob starts – the %H%M format could be removed…
    Only backs up domains with physical hosting (ie domain has a directory in /var/www/vhosts )

    I currently have this line in crontab… on ubuntu hardy 8.04, Plesk 9.3 and s3cmd

    /bin/ls -ls /var/www/vhosts | /usr/bin/awk -v mydate=$(date +%Y%m%d%H%M) ‘/\w+\.\w+/ {system (“/opt/psa/bin/pleskbackup domains-name ” $10 ” –skip-logs –output /home/backups/backup_” $10 “_” mydate “.tar”)}’; /usr/bin/s3cmd –delete-removed sync /home/backups/ s3://mybucketname; /usr/bin/s3cmd du | /usr/bin/awk ‘NR==1 {print $1/(1024*1024*1024)” Gb used, ” 5- $1/(1024*1024*1024) ” Gb remaining”}’

    to do…
    1. Use a default S3 configuration unless a specfic one exists for a domain (or client) – allowing domains to back up to specific S3 accounts – making the
    best use of the free tier

    2. Stop using s3cmd sync – reducing disk space required on server. Delete all but the latest backup from the server.

    3. Change backup_ prefix to daily_ weekly_ or monthly_ dependant on whether day matches last day of month or a sunday or not.
    prefix=”daily_”; [ `date +%w` -eq ‘0’ ] && prefix=”weekly_”; [ `date -d tomorrow +%d` -eq ’01’ ] && prefix=”monthly_”;echo $prefix
    # I’m sure there’s a better way to do this – but this works…

    4. set up deletion of backups – so that I have a backup for the last n days, n weeks & n months – i.e I’d like to keep the last 5 days backups and keep the backups from Sunday for the last 5 weeks and keep the backups from the last of each month for the last 5 months… resulting in a rolling set of 15 backups for each domain. Maybe have n variable dependant on (backup quota for s3 account)/((average size of daily backups for the s3 account)*3)

    5. check integrity of each backup file after upload to S3

    6. report an exception if size of backup is more than 5% different to previous backup.

    7. report an exception if S3 account’s quota is exceeded.

    8. and all the other things that I wish were already in Plesk

  3. thanks for this info Jason!
    indeed shame that it is not available from plesk

    note: my script needs modification when used under 10, cause 10 has different setup between users/resellers/clients

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.