MySQL Backup on Yandex Disk with Cron

This article has been written before more than 24months, information might old.

I wrote a very small python script that works in conjunction with another MySQL backup script to sync your backup on Yandex disk service.

First, in order to do this, you should have Yandex client installed. You can get the packages from Yandex repo at http://repo.yandex.ru/yandex-disk/ .

More specific in Debian/ubuntu case just do a:

wget http://repo.yandex.ru/yandex-disk/yandex-disk_latest_amd64.deb

After that install the Yandex Console client as usual doing:

dpkg -i yandex-disk_latest_amd64.deb

After installing the package, basic yandex commands should be available(commands like yandex-disk start, yandex-disk sync and others).
The script uses the yandex-disk commands.

Next, you should set up your yandex client with  the command yandex setup.
After the yandex setup command, you should place both yandex script and the MySQL script you can download Mysql Backup script from GitHub with:

wget https://github.com/kubes/backupscripts/blob/master/mysqlbackup.py

(Anyway I will provide a download link for both scripts later just in case the previous link will not work anymore.)

The Yandex Backup script is as following:( I should note that in case that the yandex token will expire the script will make a new token using your credentials. Also is important that you change all variables from the script according to your case.)

#!/usr/bin/env python3

import sys
import string
import shutil
import getopt
import os
import os.path
import syslog
import errno
import logging
import tempfile
import subprocess
import time

from datetime import datetime, date
from operator import itemgetter

"""
-------------- -------- ------------------ --------------- ------------- ---------
A cloud Yandex script to sync backups made by a MySQL backup script made by Dennis E. Kubes.


Program: Yandex Sync Script
Author: Andrei O.
Date: April 28, 2016
Revision: 0.1

Revision      | Author            | Comment
-------------- -------- ------------------ --------------- ------------- ---------
20161028-0.1    Andrei O.     Initial creation of the script.
-------------- -------- ------------------ --------------- ------------- ---------
"""
# Edit with appropriate values

MysqlScriptDatabases = "database1,database2" # separated by commas
MysqlDaysToKeepBackup = "21"
YandexBKSStorePath = "/home/andrei/yandex_backup/01.140/DB" # Where you want your store your backups beware this should be a folder under Yandex Sync
MysqlUser = "root"
MysqlPassword = "******"
MysqlHost = "localhost"
MysqlScriptPath = os.path.dirname(os.path.realpath(__file__)) + "/" + "mysqlbackup.py"
YandexStopDaemonAfterSync = True
YBKSLogPath = os.path.dirname(os.path.realpath(__file__)) + "/" + "YBKSLog.log"
YBKSEnableLog = True
YandexReadOnly = False # This option controls if old backups are deleted form Yandex
YandexUserName = "andrei0x309" 
YandexUserPass = "******"
YandexAuthTimeOutSec = 600

class YandexMBKS:

    def __init__(self):
        
        print(YBKSLogPath)
        
        if YBKSEnableLog: sys.stdout=open(YBKSLogPath,"a")
        MYsqlScriptArgs = "-k "+ MysqlDaysToKeepBackup +" -d "+ MysqlScriptDatabases +" -t "+YandexBKSStorePath + " -u " + MysqlUser+ " -p " +MysqlPassword + " -s " + MysqlHost
        currTime = datetime.now().strftime("%Y-%m-%d %H:%M:%S") 
        
        try:
            retcode = subprocess.call(MysqlScriptPath+ " " + MYsqlScriptArgs, shell=True)

            try:
                p = subprocess.Popen(['yandex-disk', 'sync'], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
                
                while True:
                    line = p.stdout.readline()
                    if line.find('Syncing') != -1 :
                        print("{0}: Not Syncing probably auth error, request new token".format(currTime), e, file=sys.stdout)  
                        pt = subprocess.Popen(['yandex-disk', 'token ' +YandexUserName], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
                        while True:
                            try:
                                if line.find('Password') != -1:
                                    pt.communicate(YandexUserPass, YandexAuthTimeOutSec)
                            except TimeoutError as e:
                                pt.kill()
                            time.sleep(0.7)
                        if not p.poll() is None:
                            retcode = subprocess.call("yandex-disk sync", shell=True)
                            break
                    time.sleep(0.7)
                    if not p.poll() is None:
                        break                
                
                if YandexStopDaemonAfterSync:
                    try:
                        retcode = subprocess.call("yandex-disk stop", shell=True)
                    except OSError as e:
                        print("{0}: Stoping Yandex Daeon failed".format(currTime), e, file=sys.stdout)    
            except OSError as e:
                print("{0}: Execution Of Yandex Sync failed:".format(currTime), e, file=sys.stdout)
        except OSError as e:
            print("{0}: Execution Of Mysql Script failed:".format(currTime) , e, file=sys.stdout)
        
        sys.stdout.close()
        

YandexMBKS()      
        

After that you should place a CRON entry, I recommend one that executes weekly.
You should be aware that the Yandex script uses python3 syntax so your system should have python3 installed in order to properly work. But most new distros even the ones on LTS have python3 support.

I use the fallowing crontab entry:

@weekly andrei /home/andrei/yandex_backup/bkscripts/YandexMBKS.py

Don’t forget to grant executable rights to both script files using chmod +x.
Practically the whole idea of the script is to not keep yandex-disk daemon on, just use the sync once a week then stop the daemon. I am pretty sure that a lot of improvements can be made to the script but i just want to write one that does the job.
Almost forgot here is a link to an archive with both scripts:

Share the joy

Leave a Reply