Technically Easy

Category: Data Backup

All posts in this category provide information on data backup, including services and helpful advice on protecting your files.

Tips on Reducing Ransomware Attack Risks

As a business owner, there are many things you need to keep an eye on. This includes watching the business finances, dealing with staffing, and making sure you have the right services and equipment in place for efficient operation. Another thing you need to do is protect your business, and one of the things you need to protect against is cybercrime.

With businesses now more reliant than ever on digital technology, the risks in relation to cybercrime activities have rocketed. Technology has become increasingly sophisticated over recent years, but so have the methods used by cybercriminals. This means that it is up to you as a business owner to ensure you have measures in place to protect your data, files, and systems from problems such as ransomware attacks, which can cause a huge amount of damage to businesses in a variety of ways. In this article, we will look at some tips to help you to reduce the risk of ransomware attacks.

What you can do

There are a number of steps that you can take in order to protect your business against cybercrime such as ransomware attacks. Some of the ways in which you can do this are:

Educate your employees

One of the things you need to do is to prevent these attacks from occurring in the first place, and this can be done by educating your employees. There are various ways in which ransomware can enter your systems, and often this is due to staff members being unaware that they are doing something that could potentially put the business at risk from this type of attack. This includes opening suspicious attachments or clicking on links in phishing emails. So, make sure your staff are trained to identify the possible risks.

Have a reliable backup

Another thing you need to do is to ensure you have a reliable backup of your files and data in case your systems are infected. If you are hit with a ransomware attack, you can wipe the infected files and replace them from a reliable backup source. This can save you a lot of headaches and stress as well as money. Using a reliable cloud backup solution is something that a lot of businesses do, as it offers affordability, ease, efficiency, and convenience. It also means that you can benefit from the advanced security that is put into place by the operators of the third-party servers.

Protect your backup

Of course, you must never underestimate the expertise of cybercriminals when it comes to conducting their questionable activities. These days, they can also affect your backed up data, and this means you also need to protect this for additional peace of mind. Using solutions such as Veeam Object Lock can add that extra layer of protection so that cybercriminals cannot access and change your files and data. You can also protect your data with a robust cloud-to-cloud backup solution that backs up data stored in one cloud backup service to another, making it virtually impossible for critical data to be lost from cyber attacks.

These are some of the key steps that you can take to protect your business from this type of activity.

July 23, 2021
Backblaze: How Much Does It Cost?

There are 2 main products under the Backblaze name: the sibling Personal Backblaze and Backblaze Business which are both used for cloud backups and the Backblaze B2, an IaaS platform.

Both solutions allow you to store and backup your data on the cloud, however, their intended user bases are different. Thus, it only makes sense that the pricing structure between these services is also different.

Regular Backblaze vs Backblaze B2

The regular Backblaze pricing is a set price per year or per month. Meanwhile, the Backblaze B2 is charged on a per GB of data downloaded and stored basis.

BackBlaze Personal and Backblaze Business

The regular Backblaze solution has a pricing structure per computer since you can only backup one device with every license. It is possible for you to buy more than one license at a time, however, there is no discount for doing so.

If you sign up for every month, you are billed $6 per month. This totals $72 for a year.

If you want a cheaper option, you can simply go with the yearly plan. It costs you $60, meaning you only pay $5 a month. Lastly, for the cheapest pricing per device, you can sign up for 2 years upfront for $110 per device. This translates to $55 a year or $4.58 a month.

Backblaze B2 pricing

B2 offers a storage price of $0.005 per GB per month. Take note, that the first 10GB of storage is free. Download costs, on the other hand, are $0.01 per GB per month. It also comes with 1GB of free download per day.

In addition, you are also charged for API calls. The first 2,500 calls each day are free and will cost $0.004 per 10,000 transactions after.

For the B2 customers, Backblaze also offers mailed physical devices instead of restoring via download. This will cost you $99 for a USB flash drive with 256GB capacity and $189 for an 8TB external hard drive.

The Backblaze B2 costs significantly lower than its major competitors the likes of Google Cloud Storage, Microsoft Azure, and Amazon S3 (previously known as Amazon Cloud Storage).

The cloud storage pricing for these giants comes in at 320 – 260 percent more costly than that of Backblaze. In addition, the download costs for these three solutions are 400 – 700 percent higher per GB of data than Backblaze.

Why is it so cheap?

With the ability to backup unlimited data at a lower price point, some people might be doubtful of the quality of service. However, the reason for this is that most Backblaze users only backup a relatively small amount of data, despite it being unlimited.

Therefore, Backblaze can offer unlimited data to power users at a cheaper cost since the overall average cost per computer stays low.

So, if you are looking for a cheap way to back up your data and don’t mind buying a new license per device, then Backblaze is an excellent option.

May 17, 2021
How to Backup Files on Windows 10
I have talked about how I automatically backup my files from my server, but one thing I have discussed is how I backup my files from my Windows 10 desktop.

Most of my important files are stored on my server, but I do have some data files, like game related files, that I would like to backup in case the hard drives in my desktop crash or begin causes data corruption in my files.

In keeping with my automatic backup policy, I wanted to have my Windows 10 desktop automatically backup my files to the server, which will then backup to Backblaze. With Windows 10, it is actually easy to do.

The steps below explain how to backup files on Windows 10 automatically.
1. Determine where you want to store the files. If you plan to backup the files to an external hard drive, then connect the hard drive into the Windows 10 computer. For network shares, you will need to setup the share on the machine that contains the share and set the permissions.
2. On the Windows 10 computer, click the Start button and select Settings->Update & Security->Backup.
3. Click the Add a drive option.
4. Select either the external drive or a network path for your backups.
  
  Note:
  
  If you chose to use a network path to store your backups then you may be prompted to enter the user ID and password to use to connect to the network share. Check the remember checkbox to ensure the credentials are stored so they will always be used to connect to the share and your backups will work without any interruption.
5. By default, the backups will run every hour and will keep the backups forever. If you you would like to change this, click the More options option.
6. From the More options window you can change how often the backup is run, how long to keep the backups, additional folders to backup and which folders to exclude. Pay attention to which folders are being backed up and ensure that all folders you want to backup are included.
7. From this point you can click the Back up now button or wait for the next scheduled backup.
At this point your Windows 10 computer will now automatically backup your data to the location you specified, at the scheduled time you specified.

As I mentioned earlier, I backup my files to my server, which will automatically make of a second copy of the files on a second external hard drive, and then backup to the cloud with Backblaze.

Do you automatically backup your computer? If so, how do you do it?
December 12, 2016
Getting Smart With Backups: Testing My Backblaze Restore
I have been using Backblaze for many years and haven’t had any issues with backing up my files. The backup process from Backblaze has worked like a charm for me. I have integrated Backblaze into my process of automatically backing up my important files without any issues.

The one thing that I have always wondered about, and for some reason I haven’t tested out the feature, is how the process of restoring my files would work. Restoring my files successfully whill help me feel confident that the files I restore from Backblaze will be the same as the files I backed up.

After all, a backup process is only as good as its restore process.

I have decided to begin testing out restoring my data files from Backblaze to ensure that I will be ready in case I need to restore any of my files. Waiting until something does happen to my files is not the best idea.

Restore Methods

Backblaze provides three options when it comes to restoring files. I decided to restore my files using two of the options. The two options I will test will be:

1. Web restore with zip download.

This method basically allows me to select files using their Web interface and then Backblaze will zip the files and let me download the zip file from my account in a Web browser or from their download utility. This post will examine the results of this method. This method is suited for smaller restores and not for my entire library of files.

2. USB hard drive.

To restore all my files, I will use this method. For $189 Backblaze will copy my files to a hard drive and then mail the hard drive to me. From there I can restore my files from the hard drive. I can get my money back if I return the hard drive to them. This method allows a maximum of 4 TB of files to be restored. I will explore this method in a future post.

The third option is using a USB flash drive which has a file restore maximum of 128 GB. Since the first two options are enough for me to restore either a subset or all of my data, I won’t be looking at the USB flash drive option.

My current backup status

All of my important files – meaning files I never want to lose – are backed up with Backblaze. I chose Backblaze mainly because of the promise of backing up unlimited data for $5/month. The amount of data I backup increases every year, so if it wasn’t for the flat $5/month rate, online cloud-based backup would be too expensive for me.

To see what I mean, here is my current file backup status.

The current status of my file backup with Backblaze.

As you can see from the screenshot above, I currently have the following backed up:

Files Total Size

164,418 741,973 MB or 741.97 GB

Since this post will look at restoring files through the Web and download application from Backblaze, I won’t be able to restore all of my files. Instead I will focus on a small subset for this test.

Preparing the file restore from the Backblaze Web interface

Let’s begin with the restore of some files.

Using the Web interface is simple enough. I basically log into my Backblaze account and select the “View/Restore Files” option from the menu on the left. By default, the Web option is selected as the restore method, so I then use the folder tree at the bottom of the Web page to select the folders and files I want to restore.

Once I have selected my files I click the “Continue with Restore” button and that’s it. Backblaze will now prepare the zip files with the files I selected and then notify me, by email, when it is ready for download.

Note:

Backblaze will inform you if your download will be too large. The Web option is used mainly for restoring a small number of files. For any restore that is over 20 GB in size, they recommend the USB flash drive or USB hard drive options instead.

When I click the “My Restores” option from the menu on the left I can see my restore is currently being prepared.

My Backblaze restore is being prepared.

After about ten minutes of waiting I receive an email from Backblaze indicating that my zip file is ready to be downloaded. I returned to my account and to the same “My Restores” page and noticed that my zip file was now available for restore.

At this point I also can see that the size of the zip file is 3.39 GB so I have a rough idea of how long it may take to download and how much drive space I will need.

My Backblaze restore is available for download.

Now that my restore was ready it was time to download and see if the files are what they should be – meaning they are an exact duplicate of the original files.

Restoring the files

When downloading from the Web interface, Backblaze provides two ways of downloading the zip file: using the download button on the Web page, or using their Backblaze Downloader utility.

The Downloader utility is a portable application that you extract anywhere on your desktop. From there you just run the executable, enter your Backblaze login information and it will go and download the last zip file that was prepared for your account.

I wasn’t sure which option would be faster. I have a 100 mb/s (12.5 MB/s) Internet connection, although I do manage to get 120 mb/s (15 MB/s) when downloading games from Steam, so I wasn’t sure what to expect when downloading through Backblaze.

In any case, I tested both options to see if there was any download performance difference.

Download from the Web

Using the “Download” button on the Web form from my Web browser (Google Chrome), I began to download the zip file.

The download speed jumped between 2.5 MB/s to 8.5 MB/s with the average around 4 MB/s. This is a lot slower than I was expecting, but I was more concerned with the fact that my files were an exact duplicate of the originals, so speed wasn’t a major concern for me.

It took just over 15 minutes to download the 3.39 GB zip file from Backblaze using Google Chrome. This worked out to an average of 31.38 mb/s (3.9 MB/s) on my 100 mb/s connection. It wasn’t bad, but not great either.

I wondered if the Downloader utility would be faster, so I downloaded the zip file again using that utility.

Download using Backblaze’s Downloader utility

I downloaded the zip file containing the Downloader utility, unzipped the file and then ran the executable. I entered my Backblaze credentials and left all the other options at their default values. I clicked the “Sign in to Start” button and let the utility do its job.

The utility displays a progress bar during the downloading activity as well as the amount transferred and the transfer speed.

The Backblaze Downloader downloading my zip file restore.

The Backblaze Downloader took 19 minutes to download the same zip file. This worked out to an average of 25.72 mb/s (3.2 MB/s) download speed. This was slower than using the Web browser to download the zip file.

Bear in mind, however, that in order to get a completely accurate picture of the download speeds, I would have needed to run both download tests multiple times. Since I am less concerned with download speeds as I am with actually getting my files, I didn’t perform any additional tests.

Verifying my restored files

Once I unzipped the zip file containing the files I selected to be restored, I could actually see the number that I had selected. I basically selected an older directory (10 years old) to be restored but I was unsure of how many files were in that directory.

The files in the directory were JPEG files, so the zip file size was very close to the uncompressed size because JPEG files are already compressed to begin with. There were also some RAW image files from my old Canon G2 camera I was using back in 2006.

In total 1,417 files were in the folder, and the files were 3,636,151,770 bytes (3.6 GB) in size. This represents a very small sample size compared to how many files I actually have backed up. The table below shows the number of files and sizes that I restored and the percentage of the total backed up.

Files Size (bytes)
1,417 (0.86% of 164,418) 3,636,151,770 (0.49% of 741,973,000,000)

While it is a small sample size, it does allow me to test out restoring a smaller number of files, and also files that have been backed up to Backblaze for many years – since I first started using their backup service.

The most important aspect of restoring files is to determine if the files restored have no issues and that they match the originals byte-for-byte.

To verify the files I performed two tests:
1. I randomly opened image files to ensure that they could be opened without any issue. Opening all 1,417 files in many directories would take a while, so I would use the second test to verify all files.
2. Create a SHA256 hash of each restored and original file and compare the hashes to see if they are identical. If one byte in a restored file was changed, then that file would have a completely different SHA256 hash than the matching file.
I randomly selected files to open – both JPEG and RAW – and had no issues opening any of the files. I even displayed the thumbnails of directories in Windows Explorer and all thumbnails were displayed without any problems. So far so good.

I then ran a small application I created that would generate the SHA256 hash for both the restored and original files. After all the hashes were calculated and compared, the application would display a message box that indicated if any files were identical, different, or missing. Since I selected a single directory to be restored, I can easily ensure that all files were restored and none were missing.

After several minutes of creating the hashes and performing the comparison I was relieved to see that my application indicated all 1,417 files were matched successfully and that no file was missing from the original files.

This indicated that the restoration of 10 year old files from Backblaze was 100% successful.

Conclusion

I have been using Backblaze for many years to automatically backup my files in the cloud. During that time I haven’t tested their restore option, and I didn’t want to wait until I lost my files in order to use their restore option.

I managed to successfully restore 1,417 of my files that were a total of 3.6 GB in size without any problems. The download speeds, however, were a little disappointing, but as I mentioned earlier, my main concern was ensuring the files I restored were exact duplicates of the originals without any missing files.

This goal was achieved.

Next, I will restore all of my files using the USB hard drive method and then compare 100% of my files to the original to see if a complete backup of my files can also be successful.

Being able to restore all of my files successfully is now my goal.
October 31, 2016
How I Protect My Files From Malware

I have a lot of important files stored on two external hard drives beside my computer. What makes these files important is that they contain photos and videos of my life since 2002 that can’t be replaced. Since that time my life has changed substantially – marriage and raising a family. Losing the files that captured all moments over the past 14+ years is not an option.

With the rise of malware, more specifically ransomware that encrypts files until a ransom is paid, it has become increasingly more important for me to protect my data files. I already have implement methods of protecting my data – logging in with a standard user account, using OpenDNS and blocking many harmful sites, using browser plugins that block JavaScript until I allow it, as well as having devices outside my home to connect to a segregated guest network.

Even with the precautions I have in place on both my network and computers, there is still a chance that some form of malware can get access to my files. With my kids using the computers more and more these days, the chances of malware infection increases.

I decided to revamp how I backup and protect my data files. In turn, making it more difficult to modify those files, even while on the network. Here are the steps I have taken to protecting my data files.

Step 1: Backing up my files

I have always had a backup process in place but it did require some manual intervention. I wanted to make a more automated backup process. A process that not only included a second local copy, but also an offsite copy of the files.

Several months ago I setup and Plex Media Server using a desktop that I bought off eBay. The desktop is more than powerful enough to stream media from Plex to any device in my home. Since it won’t be streaming movies constantly, I decided to also use the server as my file storage.

Local drive mirroring

My Plex media files are stored on a Western Digital Duo hard drive enclosure. The two hard drives in the enclosure are mirrored using Stablebit DrivePool. I decided to use DrivePool to also mirror my data files from one external hard drive onto another. The drive mirroring provides redundancy in case one drive fails. With DrivePool running, I automatically have two copies of my data locally by simply copying the files to my drive.

On top of that, I also use Stablebit Scanner to scan my data files external hard drives once a month. Stablebit Scanner also monitors the S.M.A.R.T. data on the drives and reports any issues to me through email. This setup allows me to be proactive in resolving any problems, to avoid any chance of losing data.

Offsite backup

The next step is to have my files automatically backup offsite to the cloud. My cloud storage service of choice is Backblaze. I have been using Backblaze for many years without any issues.

Once I copy my files to my DrivePool drive, the Backblaze service automatically finds the changes and then backs up those same files to the cloud. At the moment I have almost 700 GB (241,740 files) backed up with Backblaze. I can access my files from anywhere even from my mobile devices.

So I now have two local copies of my data and an offsite copy of the same files. I also have an external hard drive that I store offsite that I update once a month with any changes.

Of course, all this will be moot if I also didn’t protect my files from malware on my local network and computers.

Step 2: Restricting access to the server and files

One of the best ways you can protect your data files is to prevent users from accessing the server. You can also provide read-only access to the files. This is what I have done with my server and files.

Standard user account but not logged in

The one issue I have is that I am storing my files on a media server that can be access from devices on my network. This can be an issue if a security issue is discovered in the Plex application. Such an issue could allow a miscreant, or malware, to gain access to my files.

I reduce the chances of a malware from using Plex to gain access to my files by simply running Plex under a standard user account. This account can’t access my data files. If something tries to exploit a security hole in Plex, and attempt to access the server using the same account that Plex uses, they would be limited to what they can do.

To add to the user access, I also don’t log into the server, unless I need to manage things on the server. Since everything on the server runs as a service (Backblaze, Stablebit DrivePool, and Stablebit Scanner), I don’t need to log into the server to ensure my data is backed up – it happens as long as Windows is up and running.

Read-only access to my files

When I need to access my data files, I map a drive letter to a share on the server from my desktop. The user I use to map to the server has read-only access and can’t change any data file. This means that if malware exists on my desktop, it can’t modify my data files because they are read-only.

I treat the folders that contain my data files as a master copy. I rarely change the master copy of a file. In fact, when I edit my photos, I use Lightroom, which doesn’t make any changes to my photos. Lightroom simply stores the changes in a database file and then applies the changes internally in Lightroom.

If I do need to make changes to a file, I will copy the file locally and then make the changes. I never make changes to the master file directly.

Of course, this begs the question as to how do I modify or add files to the server if I connect using an account that is read-only? Simple – I use a staging area.

Step 3: Using a staging area

Since I connect to the data files with an account that is read-only, making changes to my files (such as my Lightroom database) is not possible.

The way I get around this is I use a separate folder as my staging area.

What is a staging area?

The staging area is simply a write-able share on the server that is monitored by a service on the server for changes. When files are added to the staging area, the service will move the files to the proper folder within the data folder.

How my staging area works

The files are moved, and not copied because once all the files have been processed by the service, the staging area will be empty. Moving the files helps to prevent malware from making changes to any file after it has been moved. This prevents the staging service from updating malware-changed files from being copied over any master file.

Is this system perfect protection from malware? Nope, but it does reduce any chances of infecting my data files. The entire file moving process from the staging area to the master files area takes seconds. The short time a file is in the staging area may prevent malware from changing the file before being moved. Once the file has been moved, it is now in the read-only share and can’t be changed.

While this may seem like a tedious way of copying my files to the server, it really isn’t. I just need to map the staging area share on my desktop and copy the files. Everything else – the move to the master files folder, the mirroring, and the cloud backup – are done automatically for me.

Bringing it all together

While I have taken steps to protect my data at a network and computer level, I wanted to do more to protect my data files. Using a desktop computer I can easily have my data mirrored locally and backed up offsite automatically.

Restricting access to my data files, I can make it more difficult for any malware within my network to change my files. By using a staging area, I can add and update my data files without removing any of the restrictions.

I will continue to make necessary security changes as the world of security (or insecurity) changes.

Do you have data files that you need to keep safe? If so, how do you keep the files safe?

July 25, 2016
How I Backup Data Files Automatically So I Won’t Worry About Losing Files
Protecting your data files with a good data backup plan is one of the most important tasks you can perform to keep your data safe from hard drive failure, malicious software, or just accidental file changes or deletions.

For me, I have been talking about various data backup plans since I started this blog, and with over 600GB of personal photos and videos stored on an external hard drive, I also take data backup very seriously.

I recently bought and setup a Plex media server that I use to stream my media throughout my home. After setting up Plex I also decided that the server would also make a good data backup server as well.

After some extensive research, I managed to create a server that provides redundancy for both my Plex media files and my personal data files. The server will also backup data files automatically online while keeping them safe from malicous software that may be installed on another computer on my LAN.

The data backup plan requirements

When I decided to use my Plex media server as a data backup server I wanted to ensure that three data backup requirements were met:
1. All data files need to be protected from accidental changes and deletions.
2. Allow digital photos from non-connected devices (digital cameras) to be added.
3. Photos and videos from smartphones and tablets need to be copied automatically to the server.
4. A second local copy of the data files needs to be automatically created.
5. All data files need to be automatically uploaded to cloud storage.
6. All hard drives containing the data files need to be monitored.
The requirements list that I created was a result of years of backing up my data, and then looking at how I could do it better. Unfortunately, at the time I didn’t have the equipment to satisfy all the requirements, but with my Plex media server, I realized that I could probably accomplish what I needed.

Let’s look at each of the requirements so I can explain how I managed to satisfy each one.

1. All data files need to be protected from accidental changes and deletions

The most important requirement when storing my data files is to ensure they are protected from accidental changes and deletions. This requirement needs to address both user and software changes and deletions.

When I just had my desktop and an external hard drive, I would simply turn off my hard drive when not in use. The issue was when it was on and connected to my desktop and the entire hard drive could be written to without any issue.

This meant that I could accidentally change or delete files, or any malicious software that was installed could do the same thing.

To avoid the same problems with my Plex media server, I connect the external hard drive to the Plex media server, create a share to the drive containing my data files that I can then access from my desktop using a local Plex media server account.

Note:

My Plex media server runs without any logged in user, and I never access the Internet from the server. The only time I log into the server is when I need to perform some maintenance.

I changed the permissions on the share to allow read-only access for the local Plex media server account that I use to access the share from my desktop.

Note:

By default, Windows creates shares with the group “Everyone” having access to the share. This is insecure, so take the time to remove “Everyone” from the share permissions and manually add the users and groups that need access.

This prevents me, or any software running on my desktop, from changing any data on my data share. Of course, this also means I can’t add any new files to the share, or make any changes to any of the files, if needed.

2. Allow digital photos from non-connected devices to be added

The first requirement and this one seems to be at odds with how I can access my data files. While my data files are protected when the files are accessed from my desktop, or another machine on my network, I still need to be able to add new files or make occasional changes to existing files.

How to protect my files, while at the same time allowing me to add files or make changes?

Simple – create a staging area.

The staging area is a write-able share on the server that points to a folder on the server. This folder only contains the data files that will be moved to the actual data files external hard drive location.

Basically, when I want to add new files, or replace files that already exist on the server’s external hard drive, I simply connect to the staging share, create the folder structure I want to create on the external hard drive, and then copy the files.

I created a small application that is running as a service on the server that monitors the staging folder. If any files are detected in the staging folder or any subfolders in the staging folder, the service will move the files from the staging folder to the external hard drive automatically. The entire process is fairly quick and requires no interaction from me other than copying the files into the staging folder.

The best part is that the share that contains my actual data files can remain read-only from the desktop as all the updating takes place directly on the server, and not from my desktop. The only share that is write-able is the staging share.

Since the files are moved from the staging share to the actual data files folder very quickly, any malicious software, such as ransomware, that could be running on a machine that is connected to the staging share, may not have enough time to modify all of the files, or may not be able to modify any files.

While the staging share takes care of manually backing up photos, I also needed to be able to backup photos and videos from mobile devices, as well.

3. Photos and videos from smartphones and tablets need to be copied automatically to the server

Many of the new photos I backup are taken with a smartphone. Sometimes my kids also take photos from a tablet, or two. At the moment I have two iPhones, an iPad, an Android smartphone and an Android tablet that can have new photos or videos stored that need to be backed up.

My requirement was to create an automatic process for backing up all photos and videos from mobile devices to the server. Finding the easiest and automatic method to perform such a task did require some research, but I managed to find an app that runs on both iOS and Android.

PhotoSync, while not free, makes it easy to backup all photos and videos from my mobile devices to my Plex media server very easily. There is an app for both iOS and Android, and the look and feel is very similar on each platform.

Using this app to backup to my server did require me to install a free Windows application called PhotoSync Companion. The Windows application that is available from the links on the PhotoSync website didn’t allow my to run as a service on Windows, meaning I had to have a user logged in on the server to have the PhotoSync Companion application running all the time.

I contacted technical support for PhotoSync, and very quickly I was informed that there is a Windows service version of the PhotoSync Companion. The only issue is that I can’t send files to my mobile devices with this version as I could with the standard Windows application. This didn’t matter to me as I only need to send photos and videos to the server, and not the other way around.

Once the PhotoSync Companion service was installed on my server, configured it to point to the staging folder as the destination folder to copy the files from the mobile devices. From there, the same service application that monitors the staging folder will then move the files to the correct folders.

The PhotoSync app keeps track of which files have already been copied so it won’t recopy any files again, which is good because it would keep recopying the files to the server since the files get moved by the staging folder monitor service on the server.

I could, however, manually re-select the files for recopy if I wish.

Now that I have managed to copy files to the correct location on the server, I wanted to then create a local duplicate copy of the files so I would lose any files if a hard drive failed.

4. A second local copy of the data files needs to be automatically created

Let me be blunt: all hard drives will fail. It is a fact of computing, and is something that you need to prepare for if you value the data stored on a hard drive. This is is one of the main reasons for backing up data.

This is also the reason I decided to invest in a second hard drive for my data files.

For years I relied on a single external hard drive, but last year that hard drive began to show signs of failing. The S.M.A.R.T. data on the hard drive began showing issues related to hard drive failure. To avoid losing data, I bought a new Western Digital My Book 3TB and copied my data files to it. All my data was copied without issue.

This year I bought a second Western Digital My Book 3TB to act as a mirror to the first – meaning the two drives were an exact copy of each other. But I wanted to have the mirroring done automatically so I wouldn’t have to manually, or have my staging application, copy the file to each hard drive.

I had been using a great piece of software called Stablebit DrivePool to mirror my Plex data drives, so I decided to do the same with my data files drives.

I setup a drive pool and included both of my data file external hard drives in the pool. A drive pool is simply a virtual hard drive that combines the available storage of any drives you add to the pool. In my case, the drive pool equaled just under 6TB – the combined storage space of the two My Book external hard drives.

Once the pool was created, I then enabled 2x duplication on the pool. This tells DrivePool to duplicate each file in the pool. Because I have two drives in the pool, each file is copied to both drives.

When my staging folder monitor service moves a files to the data files directory in the pool, DrivePool will then copy the file to the second hard drive automatically.

If one drive in the pool starts to fail, I can still access my data files from the other hard drive, while I replace the damaged drive with a new one, and then DrivePool will then mirror the files to the new hard drive automatically.

While having duplicate files locally protects me from one hard drive failure, it won’t protect me if something were to happen to my home. This is where cloud storage comes in.

5. All data files need to be automatically uploaded to cloud storage

I have talked about why I use Backblaze as my cloud backup provider and implementing their backup service into my automatic backup workflow was the easy part. Backblaze was already backing up my data automatically anyway.

I moved my Backblaze account from my desktop to my server, and setup the folders that I wanted to backup within the Backblaze application. Since I now store my data files in a drive pool, I simply added the folders from the pool to Backblaze for monitoring.

When I add a file to the drive pool, Backblaze will automatically backup the file to the cloud storage without any manual effort on my part.

This was the easiest requirement that I completed because of the fact that Backblaze handles the automatic backup, all I had to do was tell it what folders I wanted to backup.

Now that all the automatic file backup was in place, I wanted to add one more requirement to help monitor the integrity of the hard drives. If a hard drive is about to fail, I would like ample opportunity to copy off my data files before it really does fail.

6. All hard drives containing the data files need to be monitored

Along with DrivePool, Stablebit also offers a utility that will automatically scan and monitor the health of hard drives called Stablebit Scanner. I bought both applications together, since they work together and there is a substantial discount when buying both together.

Note:

Each machine that uses DrivePool and Scanner will need a separate license. After the first machine, there is a discount in license fee for each additional machine. The license is a lifetime license, so you get all future upgrades.

Stablebit Scanner essentially monitors the S.M.A.R.T. data, temperature, and the availability of all hard drives connected to a machine. The application will also run scans of the drive to ensure there are no issues, as well.

If there are any issues with a hard drive, such as a high temperature, a drive disconnection, or S.M.A.R.T. data warnings, Scanner will send me an email that describes the issue. This helps me stay ahead of any serious problems.

Scanner and DrivePool are also closely integrated. If Scanner detects that a hard drive looks to be failing, DrivePool will stop copying files to that hard drive, and duplicate files to another free hard drive in the pool.

If Stablebit Scanner detects that the temperature of a hard drive is getting high, DrivePool will throttle any activity on the drive until the temperature returns to a more normal operating temperature.

The monitoring and email alerts have worked well since I started using Scanner. I received a few temperature warning emails when I first setup my data file drive pool. All the file copies causes the temperature of one of the drives to reach the warning temperature.

I received another email when I accidentally turned off the power strip connected to the external hard drives for a fraction of a second. There was no impact to the drive pools because the hard drives briefly turned off.

One of the best features of both DrivePool and Scanner is that I can install a second copy of each application on my main desktop, and then connect to the server to see the statuses of my drive pools and the hard drive monitoring data.

No additional license is required for using the applications in this manner from my desktop.

Summary of how I backup data files automatically

I have been backing up my data files since 2002 (the year I bought my first digital camera and began taking photos), and over the years I have been developing and modifying my backup process.

Since buying and setting up my Plex media server in December 2015, I realized I could use it to incorporate a more automatic approach to backing up and protecting all of my files.

Today, I am able to keep my data files protected, access them in a read-only manner from any device on my network, while being able to add additional files at any time. All of the mobile devices can now backup both photos and videos to the server

All data files on the server will automatically be duplicated to a second local hard drive, and then automatically uploaded to Backblaze.

With Stablebit Scanner running on the server, I will be able to replace hard drives before they fail. All of this has been working flawlessly to keep my irreplaceable data files safe and secure.
February 15, 2016
How I do More with My Plex Media Server by Keeping it Simple
When I was looking at setting up my Plex Media Server, the one thing that I really wanted to make sure was that I kept the entire setup simple. I wanted to set up and manage a server that could stream media files to my devices, and not much beyond that idea. This meant I didn’t need a high-end and powerful computer to accomplish the goal, but I did have to make sure the system could handle at least three media transcoding operations at a time.

Once I had settled on the server, installed the operating system (Windows 10 Pro) the next thing I did was to decide how I wanted to set up my Plex media files. I also had some other services running on my desktop that I figured I could have my server manage, such as my regular data file management and backup. At this point I decided to look at how best to use the server to manage as much of my files as possible, while at the same time automating as many processes as possible.

I began to think about some ideas.

My Plex Media Server Ideas

I was running the Plex media server from my main desktop for a year before investing in the dedicated server. All my Plex media files were stored on a Western Digital Duo 8TB external drives connected to my desktop through USB 3.0, which I wanted to keep as is.

The Western Digital Duo external connection does contain to Western Digital Red drives, and one drive was a mirror of the other. For the mirroring, I wanted this to be more automatic, but without the use of RAID 1. On top of the automatic mirroring, I also wanted to see if I could also monitor the health of my Plex media drives so I can take action if one of the drives was having issues.

Since my Plex usage wasn’t great, the amount of idle time on the server would be quite substantial. This go me thinking about also use the server to manage my data files, which were stored on a Western Digital My Book external hard drive, connected via USB 2.0 (my desktop only supports USB 2.0). I figured with the always on option of the server, and the USB 3.0 interface, I could move my data files external hard drive to the server as well.

The added benefit of moving my data files external hard drive to the server is I could also monitor that drive as well to make sure that there are no issues with the drive. Between the data files external hard drive and the Plex media hard drives, the data files external hard drive is more important as it has files I can’t recreate, such as family photos and personal videos.

To better protect my data files, I could add a second external hard drive, and mirror the current data files external to the new drive. I could also transfer my Backblaze account to the server from my desktop so my data files can always be backed up since the server is always on.

I will continue to edit my photos on my desktop, but I will set up shares on the server that will allow me to map to the photos from the desktop for editing. This will also provide more protection for my data files from such threats as ransomware or other malware that can cause me to lose my database. I will talk more about how I handle this in another post.

With all my ideas for my new server, it was time for me to put it into action.

The Plex Media Server Setup

To help understand what I would like with my Plex Media Server, here is a quick summary:
- My Plex media files on my Western Digital Duo external drives to be mirrored automatically.
- My personal data files on my Western Digital My Book external drive to be mirrored to a second external hard drive.
- My personal data files to be automatically backed up to Backblaze.
- All hard drives in my server to me monitored to make sure there are no issues.
The good news is that all the above points can be managed with very little hardware, software, and with minimal setup. To accomplish my goal, I would need the following:

Western Digital Duo 8TB

This as an external enclosure that has two 4TB Western Digital Red drives. I bought this about a year ago to store my Plex media files, and I just needed to connect it to my server.

Western Digital My Book 3TB

I bought this drive when the 2TB Western Digital external drive that contain my data files showed signs of failing – bad sectors. That drive was over 5 years old, so it was time to replace the drive with a newer model.

Western Digital My Book 3TB #2

This external drive I had just bought a few days ago so I could create a mirror of the first 3TB external hard drive – the drive that has my data files.

Stablebit DrivePool

This is the software that is going to duplicate both my Plex media files and my data files to the correct drives automatically. Stablebit DrivePool easily combines the disk space from both external hard drives into a single drive.

Stablebit Scanner

Software that will easily allow me to monitor all my hard drives, and integrates effortlessly with DrivePool. Stablebit Scanner will also notify me, via email, if there are any issues with a hard drive.

Backblaze

My unlimited online cloud backup solution. This software runs as a service and will continuously monitor my data files for changes and then automatically upload the changes to the Backblaze cloud service.

Before getting into the details of how it all will come together, let me show you how it will be setup with a diagram:

The Plex Media Setup

The main reason I setup the server was to host and stream my Plex media files, so I will start with the Plex media storage idea of the server. The diagram below shows what is involved with setting up my Plex media on my server.

All my Plex media files are stored on my Western Digital Duo external device. This device is considered a DAS (direct-attached storage), because it connects directly to my server through a USB 3.0 port instead of over the network. The device includes two Western Digital Red drives (4TB each), which I setup as JBOD (Just a Bunch of Disks). Setting up the drives in this way allows me to manage the storage setup of the disks using DrivePool

I create a single drive pool for my Plex media files from within DrivePool and added both hard drives from the DAS to the pool. This produced a pool of 8TB in storage size. After the pool was created, I enabled duplication in DrivePool so that each file stored on one of the drives would be automatically copied to the other drive. This essentially halves the amount of storage space in the pool because each file is stored twice.

If one of the two hard drives in my DAS was to fail, I wouldn’t lose all of my media since all files are duplicated on the other drive. It is, of course, possible for both drives to fail at the same time since they share the same DAS unit, and in such a situation I would rebuild my media libraries. I don’t consider my Blu-ray, DVD, and CD libraries to be critical enough to have a full backup solution as I do my personal data files.

The Data Files Setup

The most important data I have has always been stored on an external hard drive, but it was connected to my desktop . I would turn off the external hard drive when I didn’t need to get access to my data, but it would sometime annoy my wife because she couldn’t easily access digital photos. To make life easier for me, and her, I decided to connect the external hard drive that has my personal data files to the server so it can be accessed at any time.

The diagram below shows how my data files are connected to the Plex server.

After seeing how easy it was to set up and manage a drive pool for my Plex media files, I figured it would be a great option for my personal data files. I have always made multiple copies of those files, but having multiple local copies was something that I didn’t have. I purchased a second Western Digital external hard drive to contain the copies of my data files.

My data files setup is very similar to my Plex media files, with the exception of using two external hard drives instead of a single DAS device. I create a second drive pool (the first being my Plex media), and then added the two external hard drives to the pool. Because the second external drive was new, and didn’t contain any of my data files, DrivePool took some time to copy over the files from the first drive onto the second drive after I setup file duplication.

In addition to the two local copies, I also have an offsite, cloud copy of my files stored with Backblaze. I have had this setup on my desktop for many years, and I wanted to transfer it to my server so my data files can continue to be backed up online.

I followed the instructions on the Backblaze website about moving my account information to the server, changed the drive in the software to point to the new DrivePool drive and then Backblaze went to work verifying that all files were backed up. After Backblaze quickly went through the list of files in the drive pool, it indicated that all files were backed up and everything was good again.

The result of the above setup is that when I add or change a data file in the drive pool, the file will automatically be duplicated to the second hard drive by DrivePool and then automatically be backed up to the cloud by Backblaze. This means two copies of the file are automatically created without any work outside of adding or modifying the original file.

Monitoring the Hard Drives

The one aspect of managing my data that I had neglected in the past is to actively monitor the hard drives where my data was stored. When I setup my Plex Media server, I wanted to make sure that I also enabled monitoring so I can be more proactive of replacing hard drives that were about to fail. I was lucky with my previous external hard drive as it had just begun to fail and I was able to copy off all of my data. I didn’t want to take the chance again.

For monitoring I installed another Stablebit application called Drive Scanner.

Once I installed Drive Scanner on my server it immediately detected all the hard drives – both internal and external – that were connected to my server. The application began reading the S.M.A.R.T data for each drive and alerting me if there were any issues.

Note:

The S.M.A.R.T. data from my Western Digital external hard drives (not my Western Digital Duo) could not be read and reported by the Drive Scanner application. I had to change the partition of the external hard drives from MBR to GPT for Drive Scanner to read and report the S.M.A.R.T. data from the two external hard drives.

The most common issue I have at the moment is the drive temperatures getting high because of all the copying, especially my data files drive when DrivePool was duplicating the data from one drive to another. When any issue happens, I receive an email from Drive Scanner that describes the issue.

While duplicating files with DrivePool can cause the Drive Scanner to issue heat warnings, the best part of the two applications is that DrivePool will start to throttle the duplication process to help reduce the heat. The two applications work together to make sure that your drives are running without any issue. The integration between the applications is seamless and only requires both to be installed on the same system.

In addition to the S.M.A.R.T. monitoring, Drive Scanner will also check the disks for any issues. The disk scan is set on a monthly schedule and you can see the results directly in Drive Scanner. If needed, I can also start the scan manually. If a drive once again reaches a heat warning, the scanning process will throttle in an attempt to reduce the heat. I usually only scan my drives overnight when there is very little IO on the drives.

My Plex Media and Backup Server

By utilizing my Western Digital Duo, two external hard drives, Stablebit DrivePool, Stablebit Drive Scanner, and Backblaze, I now have a server that can stream all my media to any device I own, as well as automatically duplicate my personal data files locally and offsite in the cloud.

The best part is that this server is running headless (no monitor), and without any user logged in so even if Windows does reboot, such as after an update, I don’t have to remember to go and log in as everything will automatically run without any problems.
January 29, 2016
Backblaze or Carbonite: Which Do You Need?

A few years ago my cloud backup provider changed their pricing plans. The change made backing up my data with them very expensive. I also knew that I would be having much more data that needed to be backed up each year, so the price would continue to rise. Fortunately, then I had about one and a half years left in my current plan with that provider, so I had time to review other cloud backup providers before I had to change.

During that time I looked at many different providers and many had pricing packages that had limits to the amount of data I could store. Of course, the larger the storage space and more money it would cost. I looked at two services – Backblaze and Carbonite. Each provided unlimited storage for a flat, affordable, rate per month. Just what I was looking for to back up my data. In the end I went with Backblaze, and my reasons are outlined below.

(more…)

February 21, 2014