For any data that’s even somewhat important, we all know backups are a must – but what about when it comes time to backup the backups?
So there I was, a client of mine runs a dozen or so VM’s on a linux-based hypervisor. That implementation had an inbuilt feature for backups, but a single backup server in the same room as the backup source does not a resilient backup solution make… So what could I do? I had plenty of hardware for a secondary backup server, and sure, it’s easy enough to just schedule an
rsync command in cron, but what about error handling and email alerts?
So off I went, searching the webs for a reasonably small app or script to automate the mundane work for me, however I wasn’t able to find anything that really worked how I wanted it to… After an hour or so of that, I determined that it would be best to cook up my own solution and the result was Niftsync.
I won’t lie, I’m a bit proud of this little setup. In my usage scenario, I had two servers I wanted to replicate data between. Server A received backups directly from the virtualization platform, and server B was set up for the sole purpose of having the backups on Server A replicated to it. In my case, I setup an NFS share on Server B and then wrote an entry in
fstab on Server A so that said NFS share on Server B would be automatically mounted at boot. I won’t go into depth on the details of NFS or how it compares to SAMBA in any respect, but for my setup transfer speed was important and from my tests NFS seemed to perform the best.
*Note: Niftsync works with both SAMBA and NFS network shares, so you have options.*
How does it work?
So all things said, the utility is pretty basic. When it is run, it works based off of directories that have been specified in the configuration area of the
sync.sh script. First it checks the directory where the remote network share is supposed to be mounted to and attempts to mount that resource (using
mount -a) if it is not already mounted (this setup assumes you have set the network share to be mounted automatically via
fstab). If it’s able to mount said resource or if the resource is already mounted, then it proceeds spool up a
rsync -raq --delete operation to mirror the two directories.
From the beginning to the end of the script itself, every action is logged to the
logs directory (by default, logs are retained for 30 days). The script can be configured to always send an HTML-formatted Email report (this is the default M.O) when it’s run, or to only send an alert if it encounters an error. In addition, Email reports contain the hostname of the machine the script is run on, so multiple instances of the utility can be run on a single network and reports/alerts can be easily differentiated by hostnames.
If that wasn’t enough, custom alert messages give insight into he problem(s) encountered by the script as a way to aid in troubleshooting. The reports also contain a timestamp of how long the replication process took from start to finish (if the script is set to always notify) or how long the replication process ran before it encountered an error – so I ask, what more could you possibly ask for?
Any extra pointers about Niftsync?
First off, the default configuration is setup to use a Gmail account for sending alerts. I chose to go with this setup because it should be a solution that most people can use and it required minimal setup. It goes without saying but any account used for this utility (whether Gmail or otherwise) should be dedicated for that use only. As the utility makes use of the
package for sending emails, it must be configured with the email username and password stored in clear-text in the
file – meaning any user on the system with access to this configuration file will have those email credentials. A
ssmtp.conf nother consideration to be made is that by default, ssmtp is configured with the FromLineOverride= option enabled – this allows alert emails to have a custom “From” parameter specified (set to “Niftsync Service” by default) but if a different user on that system decides to use the ssmtp utility to send an email, this setting would allow them to customize the “From” parameter also…
*Edit: Niftsync was originally written to use the
ssmtp utility for sending emails, more recently however that utility has been depreciated and so the code has been updated to use the
msmtp utility instead. These changes have been reflected on the GitHub repository.* – 10/24/2019
How to Get Started
As mentioned in the README.md file on GitHub, the utility relies on a few dependencies. These are
rsync. After installing/updating those packages and then performing a quick
cd to the directory where you want to download the utility, feel free to move on to the next step.
The easiest way to obtain the utility is to pull it from GitHub, where (I should add), more in-depth instructions on configuration exist. With the
git utility, this can be accomplished using the command below:
git clone https://github.com/bobbygecko/niftsync.git
After the clone operation completes,
cd into the “niftsync” folder and edit/move the configuration files as needed using the aforementioned README.md file on GitHub for reference. After configuring the settings to reflect your setup, you can make the script executable and then run it for the first time like so:
chmod +x sync.sh
./sync.sh & (the ampersand forks the process so that the command line can still be accessed while it is running)
Assuming everything was configured correctly, you should get an email shortly with a report on the first sync. I should also add that the email reports are responsive, so they view well on most devices. To demonstrate that:
That more or less wraps things up for this article. If you like what you’ve seen here or it’s been helpful, consider helping your favorite lizard out.
This site is a Brave Verified Publisher so if you are running their web browser then making a BAT contribution is as simple as clicking the “Brave Rewards” icon in Brave. If you aren’t but would like to lend some aid anyhow, you can download Brave using my affiliate link: You get a fast browser with built-in Ad-Blocking, secure private browsing on the Tor network, and some other great features while I get a few bucks to put toward some coconut wine… Seems like a rather decent arrangement does it not?