Protecting your data files with a good data backup plan is one of the most important tasks you can perform to keep your data safe from hard drive failure, malicious software, or just accidental file changes or deletions.
For me, I have been talking about various data backup plans since I started this blog, and with over 600GB of personal photos and videos stored on an external hard drive, I also take data backup very seriously.
I recently bought and setup a Plex media server that I use to stream my media throughout my home. After setting up Plex I also decided that the server would also make a good data backup server as well.
After some extensive research, I managed to create a server that provides redundancy for both my Plex media files and my personal data files. The server will also backup data files automatically online while keeping them safe from malicous software that may be installed on another computer on my LAN.
The data backup plan requirements
When I decided to use my Plex media server as a data backup server I wanted to ensure that three data backup requirements were met:
- All data files need to be protected from accidental changes and deletions.
- Allow digital photos from non-connected devices (digital cameras) to be added.
- Photos and videos from smartphones and tablets need to be copied automatically to the server.
- A second local copy of the data files needs to be automatically created.
- All data files need to be automatically uploaded to cloud storage.
- All hard drives containing the data files need to be monitored.
The requirements list that I created was a result of years of backing up my data, and then looking at how I could do it better. Unfortunately, at the time I didn’t have the equipment to satisfy all the requirements, but with my Plex media server, I realized that I could probably accomplish what I needed.
Let’s look at each of the requirements so I can explain how I managed to satisfy each one.
1. All data files need to be protected from accidental changes and deletions
The most important requirement when storing my data files is to ensure they are protected from accidental changes and deletions. This requirement needs to address both user and software changes and deletions.
When I just had my desktop and an external hard drive, I would simply turn off my hard drive when not in use. The issue was when it was on and connected to my desktop and the entire hard drive could be written to without any issue.
This meant that I could accidentally change or delete files, or any malicious software that was installed could do the same thing.
To avoid the same problems with my Plex media server, I connect the external hard drive to the Plex media server, create a share to the drive containing my data files that I can then access from my desktop using a local Plex media server account.
Note:
My Plex media server runs without any logged in user, and I never access the Internet from the server. The only time I log into the server is when I need to perform some maintenance.
I changed the permissions on the share to allow read-only access for the local Plex media server account that I use to access the share from my desktop.
Note:
By default, Windows creates shares with the group “Everyone” having access to the share. This is insecure, so take the time to remove “Everyone” from the share permissions and manually add the users and groups that need access.
This prevents me, or any software running on my desktop, from changing any data on my data share. Of course, this also means I can’t add any new files to the share, or make any changes to any of the files, if needed.
2. Allow digital photos from non-connected devices to be added
The first requirement and this one seems to be at odds with how I can access my data files. While my data files are protected when the files are accessed from my desktop, or another machine on my network, I still need to be able to add new files or make occasional changes to existing files.
How to protect my files, while at the same time allowing me to add files or make changes?
Simple – create a staging area.
The staging area is a write-able share on the server that points to a folder on the server. This folder only contains the data files that will be moved to the actual data files external hard drive location.
Basically, when I want to add new files, or replace files that already exist on the server’s external hard drive, I simply connect to the staging share, create the folder structure I want to create on the external hard drive, and then copy the files.
I created a small application that is running as a service on the server that monitors the staging folder. If any files are detected in the staging folder or any subfolders in the staging folder, the service will move the files from the staging folder to the external hard drive automatically. The entire process is fairly quick and requires no interaction from me other than copying the files into the staging folder.
The best part is that the share that contains my actual data files can remain read-only from the desktop as all the updating takes place directly on the server, and not from my desktop. The only share that is write-able is the staging share.
Since the files are moved from the staging share to the actual data files folder very quickly, any malicious software, such as ransomware, that could be running on a machine that is connected to the staging share, may not have enough time to modify all of the files, or may not be able to modify any files.
While the staging share takes care of manually backing up photos, I also needed to be able to backup photos and videos from mobile devices, as well.
3. Photos and videos from smartphones and tablets need to be copied automatically to the server
Many of the new photos I backup are taken with a smartphone. Sometimes my kids also take photos from a tablet, or two. At the moment I have two iPhones, an iPad, an Android smartphone and an Android tablet that can have new photos or videos stored that need to be backed up.
My requirement was to create an automatic process for backing up all photos and videos from mobile devices to the server. Finding the easiest and automatic method to perform such a task did require some research, but I managed to find an app that runs on both iOS and Android.
PhotoSync, while not free, makes it easy to backup all photos and videos from my mobile devices to my Plex media server very easily. There is an app for both iOS and Android, and the look and feel is very similar on each platform.
Using this app to backup to my server did require me to install a free Windows application called PhotoSync Companion. The Windows application that is available from the links on the PhotoSync website didn’t allow my to run as a service on Windows, meaning I had to have a user logged in on the server to have the PhotoSync Companion application running all the time.
I contacted technical support for PhotoSync, and very quickly I was informed that there is a Windows service version of the PhotoSync Companion. The only issue is that I can’t send files to my mobile devices with this version as I could with the standard Windows application. This didn’t matter to me as I only need to send photos and videos to the server, and not the other way around.
Once the PhotoSync Companion service was installed on my server, configured it to point to the staging folder as the destination folder to copy the files from the mobile devices. From there, the same service application that monitors the staging folder will then move the files to the correct folders.
The PhotoSync app keeps track of which files have already been copied so it won’t recopy any files again, which is good because it would keep recopying the files to the server since the files get moved by the staging folder monitor service on the server.
I could, however, manually re-select the files for recopy if I wish.
Now that I have managed to copy files to the correct location on the server, I wanted to then create a local duplicate copy of the files so I would lose any files if a hard drive failed.
4. A second local copy of the data files needs to be automatically created
Let me be blunt: all hard drives will fail. It is a fact of computing, and is something that you need to prepare for if you value the data stored on a hard drive. This is is one of the main reasons for backing up data.
This is also the reason I decided to invest in a second hard drive for my data files.
For years I relied on a single external hard drive, but last year that hard drive began to show signs of failing. The S.M.A.R.T. data on the hard drive began showing issues related to hard drive failure. To avoid losing data, I bought a new Western Digital My Book 3TB and copied my data files to it. All my data was copied without issue.
This year I bought a second Western Digital My Book 3TB to act as a mirror to the first – meaning the two drives were an exact copy of each other. But I wanted to have the mirroring done automatically so I wouldn’t have to manually, or have my staging application, copy the file to each hard drive.
I had been using a great piece of software called Stablebit DrivePool to mirror my Plex data drives, so I decided to do the same with my data files drives.
I setup a drive pool and included both of my data file external hard drives in the pool. A drive pool is simply a virtual hard drive that combines the available storage of any drives you add to the pool. In my case, the drive pool equaled just under 6TB – the combined storage space of the two My Book external hard drives.
Once the pool was created, I then enabled 2x duplication on the pool. This tells DrivePool to duplicate each file in the pool. Because I have two drives in the pool, each file is copied to both drives.
When my staging folder monitor service moves a files to the data files directory in the pool, DrivePool will then copy the file to the second hard drive automatically.
If one drive in the pool starts to fail, I can still access my data files from the other hard drive, while I replace the damaged drive with a new one, and then DrivePool will then mirror the files to the new hard drive automatically.
While having duplicate files locally protects me from one hard drive failure, it won’t protect me if something were to happen to my home. This is where cloud storage comes in.
5. All data files need to be automatically uploaded to cloud storage
I have talked about why I use Backblaze as my cloud backup provider and implementing their backup service into my automatic backup workflow was the easy part. Backblaze was already backing up my data automatically anyway.
I moved my Backblaze account from my desktop to my server, and setup the folders that I wanted to backup within the Backblaze application. Since I now store my data files in a drive pool, I simply added the folders from the pool to Backblaze for monitoring.
When I add a file to the drive pool, Backblaze will automatically backup the file to the cloud storage without any manual effort on my part.
This was the easiest requirement that I completed because of the fact that Backblaze handles the automatic backup, all I had to do was tell it what folders I wanted to backup.
Now that all the automatic file backup was in place, I wanted to add one more requirement to help monitor the integrity of the hard drives. If a hard drive is about to fail, I would like ample opportunity to copy off my data files before it really does fail.
6. All hard drives containing the data files need to be monitored
Along with DrivePool, Stablebit also offers a utility that will automatically scan and monitor the health of hard drives called Stablebit Scanner. I bought both applications together, since they work together and there is a substantial discount when buying both together.
Note:
Each machine that uses DrivePool and Scanner will need a separate license. After the first machine, there is a discount in license fee for each additional machine. The license is a lifetime license, so you get all future upgrades.
Stablebit Scanner essentially monitors the S.M.A.R.T. data, temperature, and the availability of all hard drives connected to a machine. The application will also run scans of the drive to ensure there are no issues, as well.
If there are any issues with a hard drive, such as a high temperature, a drive disconnection, or S.M.A.R.T. data warnings, Scanner will send me an email that describes the issue. This helps me stay ahead of any serious problems.
Scanner and DrivePool are also closely integrated. If Scanner detects that a hard drive looks to be failing, DrivePool will stop copying files to that hard drive, and duplicate files to another free hard drive in the pool.
If Stablebit Scanner detects that the temperature of a hard drive is getting high, DrivePool will throttle any activity on the drive until the temperature returns to a more normal operating temperature.
The monitoring and email alerts have worked well since I started using Scanner. I received a few temperature warning emails when I first setup my data file drive pool. All the file copies causes the temperature of one of the drives to reach the warning temperature.
I received another email when I accidentally turned off the power strip connected to the external hard drives for a fraction of a second. There was no impact to the drive pools because the hard drives briefly turned off.
One of the best features of both DrivePool and Scanner is that I can install a second copy of each application on my main desktop, and then connect to the server to see the statuses of my drive pools and the hard drive monitoring data.
No additional license is required for using the applications in this manner from my desktop.
Summary of how I backup data files automatically
I have been backing up my data files since 2002 (the year I bought my first digital camera and began taking photos), and over the years I have been developing and modifying my backup process.
Since buying and setting up my Plex media server in December 2015, I realized I could use it to incorporate a more automatic approach to backing up and protecting all of my files.
Today, I am able to keep my data files protected, access them in a read-only manner from any device on my network, while being able to add additional files at any time. All of the mobile devices can now backup both photos and videos to the server
All data files on the server will automatically be duplicated to a second local hard drive, and then automatically uploaded to Backblaze.
With Stablebit Scanner running on the server, I will be able to replace hard drives before they fail. All of this has been working flawlessly to keep my irreplaceable data files safe and secure.