What Backup/Archive Storage Path to Take

• 

  theguwithnoname on 29/04/2023 - 12:32

  +2

  Just a heads up on your previous storage solution, duplicating files on 2 drives isn't ideal, especially if you leave them unplugged and expect to be able to recover your work years down the line (they might not work). You will want to plug these drives in and let them spin up at least every year. For maximum reliability, you'll want to also encrypt your data and put it onto AWS glacier or some similar cloud storage provider, look into rclone for this.

  Raid1 is good, but it doesn't constitute a proper backup, you're SOL if your house catches on fire, for example.

  • 

    tempestswitch on 29/04/2023 - 13:59

    Thanks for the info. I thought that was the case with SSD's to plug em in every once in a while. But I'll do that with my drives too.
    Once the drive is full I offload it to my parent's place. Not the best solution but at least it's an extra backup in the event of a fire.

    • 

      theguwithnoname on 29/04/2023 - 14:02

      Look into tape or Blu rays for long term storage. Hard drives are bad for this because if you leave them lying around for years the bearing lubricant drains to one side and when you plug it, the platter might not spin up properly.

      • 

        rumblytangara on 29/04/2023 - 15:31

        "Hard drives are bad for this because if you leave them lying around for years the bearing lubricant drains to one side and when you plug it, the platter might not spin up properly."

        Do you have a source for this?

        I have been personally using random hard drives as long term backups for years, never come across a failure mode attributed to "draining lubricant". This is just personal anecdote, and those used by my other tech friends.

        Less anecdotally, I have been around datacentres for decades and sometimes we have servers sit powered down for literally years before we bring them online again (spares that we swap into production). We've never heard a "boot it up occasionally to get the lube running" idea.

        But this aside, I am full agreed that a couple of local hard drives is not a great solution:
        - drives are kept in one location, so there is risk of fire/flood/theft
        - if drives are online then there's additional risk of corruption/electrical surges

        @OP:

        Something like Backblaze will give an 'all you can eat' amount of storage for a single computer, and 30 days archive (unless you pay for more archival retention). The only real downside is the speed of initial backup can take weeks, and if you need to restore everything, it will also take weeks.

        In a real "complete failure" situation, some services let you pay extra for them to courier the entire restore to you on a hard drive, rather than having to restore over crappy NBN speeds.

        Edit: I just read the other comments, funny that Backblaze is mentioned so much. I'm using them, and have been happy. Mainly because it's a flat rate for unlimited backup space and because they are the only company to release their drive reliability data. They are a good company.

        • 

          theguwithnoname on 29/04/2023 - 15:52

          @rumblytangara: Here is a stackexchange answer that says the same thing.

          I think I was wrong in confidently stating that it's always due to lubrication issues: there doesn't seem to be a general consensus on a single cause that makes HDDs unreliable to long term storage, some other issues seem to include the magnetic domains on the platters demagnetising after several years, etc.

          However, there does seem to be a consensus that HDDs shouldn't be used for long term storage if you want to reliably access the stored data in 5, 10 or 20 years time.

          • 

            rumblytangara on 29/04/2023 - 16:12

            @theguwithnoname: I can't think of anything that's good for 10-20 years. Physical media degrades, changing hardware standards make old storage obsolete. I've personally tested a box of ten year old LTO tapes, something like >25% of them failed. People used to think that CDs would be good for archival, then realised that writable CD/DVDs used dyes that lasted nowhere near as long as a factory-printed disk… and then people just stopped using DVD drives altogether.

            Best solution I can think of now really is online- the underlying hardware will be continually refreshed by the cloud provider. And if the cloud provider goes down, you'll likely have some warning and you also have your local storage and can make new backups elsewhere.

• 

  deme on 29/04/2023 - 13:14

  +1

  How much data do you have?

  RAID1

  https://www.raidisnotabackup.com/

  Does this provide you a solution? https://www.backblaze.com/backup-pricing.html

  • 

    tempestswitch on 29/04/2023 - 14:00

    I did look into BackBlaze and it seems like a good idea. I probably go through 6TB a year of photos/video/audio for work.
    I like the idea of cloud… Just don't like the idea of trying to access that data with crappy NBN.

    • 

      deme on 29/04/2023 - 15:05

      I think if anything terrible happened you could just upgrade your internet or use a friend's/work for a once time thing

• 

  saitaris on 29/04/2023 - 13:19

  Use a NAS with a cloud backup.

• 

  scrimshaw on 29/04/2023 - 13:24

  +1

  If you are doing old school backup: i.e. you are not using cloud, not using NAS and not using automated backup software like cron+rsync you will need to be quite mindful of how often you are backing up stuff, how many copies you are making of each file and whether the drives are "in sync". The other issue is that lack of logging — unless you're writing it down you probably won't know when you last performed a backup.

  If the data is super important then you might also want to store a copy of it on the cloud, that way you actually have a proper 3-2-1 backup solution for at least some of your files. If there is a house fire or flood for example all of your local data would be destroyed, and the only surviving copy would be whatever is in the cloud.

  RAID1 is not a replacement for backup solution. It may guard against one hard drive failure but if something happens to the NAS or computer that holds that RAID array you will still suffer data loss.

  • 

    tempestswitch on 29/04/2023 - 16:22

    My process is very old school but with some tweaks. I'll try to explain:
    My camera has two memory card slots.
    Card 1 > External Hard Drive 1 - Lives at home
    Card 2 > External Hard Drive 2 - Lives at home
    Card 1 > External SSD - Lives in car

    Once I edit photos/video and deliver them to clients, I delete them off the external SSD but leave them on the 2x External Hard Drives.
    Once the External Hard Drives are full, I leave one at my place and one at my parents.

    I guess the best solution for true redundancy is going into the cloud next.

    • 

      localhost on 29/04/2023 - 17:04

      First, you don't need to keep raw file from camera, just edited client files. Also consider local computer is not a backup - it's a working station and once you finish with files move them to backup archive.
      Possible solution:
      1. one storage with raid 1 drives(Qnap or something else) with 2x8GB drives. This is backup archive #1.
      2. create an automated backup from there to an external drive (2 disks x8gb). This is backup #2 and 3. Use one disk offsite and ship it to your parents house. Once disk 2 is delivered, get disk 3 from there and use it for next backup.

• 

  Hybroid on 29/04/2023 - 14:49

  +2

  3-2-1 backup strategy bare minimum

  https://www.backblaze.com/blog/the-3-2-1-backup-strategy/

• 

  Marty-69 on 29/04/2023 - 17:38

  Off-side backup.
  I use Backblaze but there are plenty of other options. The main reason for me to use Backblaze is that they can send a disk with all your data via courier and it's all encrypted with my own key.