Advice on storing digital photos

New Topic
This topic has been archived, and won't accept reply postings.
 gammarus 08 Feb 2021

After a lockdown slide-scanning epic I've filled most of my laptop's SSD storage with photographs, with copies on OneDrive. What storage systems do other folk use? Would a couple of external hard drives be best? All suggestions welcomed.

Removed User 08 Feb 2021
In reply to gammarus:

External SSD/Flash must be preferred over a harddrive in term of robustness and data safety?

 ChrisJD 08 Feb 2021
In reply to gammarus:

BackBlaze for added online back up.

... will take a long time (could be weeks) to do the first back up if you have multiple TB

I've adopted:

- Primary HD storage

with Back-ups via:

- Secondary Ext HD

- 4 Bay NAS

- Backblaze

Post edited at 10:38
 Dan Arkle 08 Feb 2021
In reply to Removed User:

More robust, but way more expensive. 

I've got a backup external HDD at home, and one at a mate's house in case my house burns down.

I did try an automated online version like Chris suggests. It worked well for a while, then the company stopped operating. 

Removed User 08 Feb 2021
In reply to Dan Arkle:

I'm using IDrive at the minute which seems good, used it in anger just once so far.

 ChrisJD 08 Feb 2021
In reply to Dan Arkle:

It's good to have redundancy in the back up system. Multiple lines. 

Which company was it that went bust?

I'm in my 5th year now with Backblaze.

 ChrisJD 08 Feb 2021
In reply to Removed User:

I had an (early) SSD main drive fail last year.

 Dan Arkle 08 Feb 2021
In reply to ChrisJD:

CrashPlan stopped doing personal storage, and I didn't find out about it until months down the line. I was left with no online back-ups, and my auto home backups were unusable proprietary encrypted files.

I was a bit peeved especially as it took months to upload all my data to them.

Anyway, I'm not arguing against online backup, just that it has its weaknesses too.

 Robert Durran 08 Feb 2021
In reply to Dan Arkle:

> I've got a backup external HDD at home, and one at a mate's house in case my house burns down.

This is what I do, except I keep the second backup at work.

 dread-i 08 Feb 2021
In reply to gammarus:

If you have amazon prime, they do free unlimited storage. I've not used them and I'd read the t&c's just in case you give up copyright.

Similar story with google photos.

If you have an external hard drive, and your house burns down, then its toast. If you like your data, keep it in multiple, unrelated places. So NAS at home, and an online service or two.

Also have a think about the automation side of things. If you delete a pic, you probably don't want it deleted in all places at the same time. It might be that you run sync jobs at different intervals. Possibly daily, weekly and monthly, but try and automate it, so you don't have to remember what you backed up and when.

But if you really like your pics, then get some printed and stick them in an album. Your great grand kids, probably wont go digging through various online accounts. A photo book can store them, without additional power or ongoing cost, for centuries. 

 wilkesley 08 Feb 2021
In reply to gammarus:

I back mine up to a computer that is used to store media and runs the Emby multimedia server.  This has mirrored hard drives, so if one fails you can still recover the data. In addition I do incremental backups during the night to my pCloud account. If you haven't heard about pCloud, you make a one off payment for a lifetime storage amount. My paranoid addition is to use MultiCloud, which allows you to sync between various cloud accounts such as Amazon, Google, pCloud and Microsoft. 

Post edited at 12:03
 The Lemming 08 Feb 2021
In reply to gammarus:

I use the 3-2-1 principle.

Three copies of the same thing.

Two of those copies can be in the same location and stored safely. The most important bit is that the copies ARE NOT physically connected to your computer or home network in any shape, way or fashion. This will help stop your backups getting encrypted by Ransomware.

One copy somewhere else, preferably in a different physical location such as a mate's house.

Some people use an external hard drive to back up images from their laptop or desktop, mainly because their laptop or desktop computer's storage is filling up. They then delete the images from the computer to get that space back. People who do this, don't realise that they no longer have a back-up. They only have one copy and if that external storage dies then their images are gone, probably for ever.

I use another option of using storage connected to my home router. Its a Network Access Storage device that keeps all my photos and because its connected to my home network, every device at home can see the photos. And if set up in a specific way, then the images can be seen from anywhere in the world. I've basically got my own "Home Cloud". I don't have to worry about the Cloud Storage company going bust, changing practices or simply having a brain fart and deleting all those images.

 Graeme G 08 Feb 2021
In reply to gammarus:

I notice nobody has mentioned iCloud? I don’t use it, but have started thinking I should have a Cloud backup. And as I use a Mac, iCloud seems logical?

 SouthernSteve 08 Feb 2021
In reply to Graeme G:

iCloud for me is really slow. I have multiple cheap disks.  Steve

 Graeme G 08 Feb 2021
In reply to SouthernSteve:

> iCloud for me is really slow. I have multiple cheap disks.  Steve

Thanks. Good to know.

OP gammarus 08 Feb 2021
In reply to gammarus:

Thanks everyone. Before this morning, I had never heard of NAS drives. What is the advantage of having physical NAS storage over something cloud-based? say, OneDrive? I have 1TB of OneDrive storage with my MS Office subscription.

 Chris Craggs Global Crag Moderator 08 Feb 2021
In reply to gammarus:

I have two 2T hard drives, with a copy of all the important stuff on both. If one should die - which happens - I just copy the info on the  surviving one over onto a new disc,

Chris

OP gammarus 08 Feb 2021
In reply to Chris Craggs:

Chris, do you use cloud storage as well?

And is your 2x2TB system one of these NAS jobs?

 Dan Arkle 08 Feb 2021
In reply to gammarus:

NAS doesn't offer you any more security - just being able to access your files from afar. 

To repeat what Lemming said, 

You need more than one copy at home, which could be an external HDD, a NAS drive or a RAID array (in order of price). 

Then you need something off site. Either cloud, or a HDD somewhere else. 

ChrisJD's system is the best described here.

Mine is adequate but cheaper. 

 HeMa 08 Feb 2021
In reply to gammarus:

Difference between NAS and cloud. Simple. NAS does not require onlinE connection and is faster (unless you have a really crap router). Also NAS tends to be a lot bigger. 
 

e.g. My NAS is 10TB and about half full. So about 5x your OneDrive. But I mostly work with video nowadays, so need a lot more space than just pics. 

 Chris Craggs Global Crag Moderator 08 Feb 2021
In reply to gammarus

I don't use Cloud Storage, we are often in places with poor Wifi and I generate a lot of data when photoing for the books. The discs are just standard 'fag-packets' as I call them.  It is a system that works for me when we are (or used to be) away from the UK for most of the year,

Chris

 Durbs 08 Feb 2021
In reply to gammarus:

Just a reminder (if needed) that something like OneDrive/Dropbox/GoogleDrive is NOT a true backup solution - if it's mirroring your hard-drive, it will also update if you accidentally delete a file, or if a file gets over-written or corrupted.

BackBlaze (like other online options) will retain a rolling record of your files, so if you delete something, you can download a backup from last week.

 ChrisJD 08 Feb 2021
In reply to gammarus:

Hi Steve

An NAS is a great solution to home and small business storage, especially if you are hard wired (which we are).

I have a 4 disk NAS, each disk is 3TB.

You then 'RAID' the system (RAID 5 in my case): it does some crazy magic and spreads the data over the multiple disks; this reduces the 'useable' storage capacity, in my case down to 8TB.

The beauty is, if a disk fails, you just take it out and put another in. No data lost, you can keep using the system before you replace the failed disk; the system then rebuilds 'the RAID' when new disk put in.

One of the disks in the NAS did fail last year, system worked a treat. Replaced the disk next day via Amazon.  ... it would have been bad luck if another went in that sub 24 hour period! But all backed up elsewhere. I should get a spare disk for immediate hot-swapping.

You can also divide the NAS into 'Volume's. I use two, so the NAS appears as two mapped external Drives; e.g. P: and N: 

I'd recommend QNAP for NAS. 

 Marek 08 Feb 2021
In reply to ChrisJD:

> The beauty is, if a disk fails, you just take it out and put another in. No data lost, you can keep using the system before you replace the failed disk; the system then rebuilds 'the RAID' when new disk put in.

Just a word of caution: RAID 5 is actually not that good from a data resilience point of view and generally not used in professional environments. The main issue is that when a disc dies and you replace it, the system gets quite thrashed in doing the rebuild and the incidence rate of early life failures (and UREs) in SATA HDD is significantly high. A full rebuild put a lot of stress on the remaining drives and low-cost SATA drives were never designed for that sort of usage pattern. End result is a significant risk of a uncorrectable error or failure during the rebuild and loosing at least some your data at that point. From what I can remember (a couple of years back) the probability of rebuilding a typical 8TB SATA array successfully was less than 50%. Worryingly low if you've been thinking "RAID 5 means I can't loose data".

 The Lemming 08 Feb 2021
In reply to ChrisJD:

> Hi Steve

> An NAS is a great solution to home and small business storage, especially if you are hard wired (which we are).

> I have a 4 disk NAS, each disk is 3TB.

> You then 'RAID' the system (RAID 5 in my case): it does some crazy magic and spreads the data over the multiple disks; this reduces the 'useable' storage capacity, in my case down to 8TB.

Up until the start of this month I had a 2 bay NAS box with two 8TB drives in it mirroring each other. This gave me 8TB of storage and allowing for one HHD to bite the dust and be replaced.

However I now have a 4 bay NAS box with four 8TB drives in it. You'd think that this would give me 16TB of storage allowing for a HHD to bite the dust. I was pleasantly surprised to learn that Synology have their own form of RAID which allows me to have the storage capacity of three 8TB drives with a safety margin of one HHD biting the dust. This gives me a massive 22TB of storage, when I was only expecting 16TB.

I'm one happy little rodent.

 ChrisJD 08 Feb 2021
In reply to Marek:

Cheers Marek, points noted.

I'm happy so far with the QNAP 453A. On every day, for ~12 hours a day.

The 4 drive are 3TB each, so not crazy big ones, so the RAID 5 2nd disk failure risk during rebuild is lower.

The critical data is backed on an external drive to the NAS, then on another External HD and again via Backblaze. 

But I'll make sure the next NAS goes RAID 6

Counter view here wrt RAID 5: https://louwrentius.com/raid-5-is-perfectly-fine-for-home-usage.html#:~:tex....

Post edited at 22:05
 ChrisJD 08 Feb 2021
In reply to Marek:

That being said, the QNAP is 3.5 years old now, so probably should be thinking of replacing it within the next year; so will go RAID 6 then.

 ChrisJD 08 Feb 2021
In reply to Marek:

Someone else debunking the 50% failure rate for 8TB RAID 5:

http://www.raidtips.com/raid5-ure.aspx

They calculate it as 0.1% for 4 bay @2TB each under RAID 5:

https://www.raid-failure.com/raid5-failure.aspx

The RAID-0 lifetime is worth playing around as well, out of interest.

 SouthernSteve 09 Feb 2021
In reply to ChrisJD:

We have successfully rebuild RAID 5 drives (Pegasus and G-Technology) several times over the years. We do give our arrays a little bit of space (c 75% full) and intermittently run directory checking / optimisation software. 

In reply to stevesmith:

As a means of protecting your photos and also helping with accessibility for use (if you don't have an offline cataloguing programme), consider running a batch copy to medium sized JPEGS so that these can be rapidly examined / identified whilst not completely filling your hard drive. Alternatively use Lightroom or similar and make decent quality previews and again your photos done need to be attached when you are looking at them casually or sorting them out for a presentation etc.

The most useful thing I did in the past was culling the many bad photos, where at one point I would keep everything which is completely pointless. 

Also I left my slide scans very large - way beyond what I would ever need 2000 slides at 140 MB was happily reduced to 20ish MB per slide. 

edited for extra comment

Post edited at 07:21
 Marek 09 Feb 2021
In reply to ChrisJD:

> The critical data is backed on an external drive to the NAS, then on another External HD and again via Backblaze. 

That sopund like a pretty rubust approach.

Old internet adage: For every internet opinion stating X you can find an internet opinion stating notX.

 Marek 09 Feb 2021
In reply to SouthernSteve:

> We have successfully rebuild RAID 5 drives (Pegasus and G-Technology) several times over the years...

And I've never had HDD failure of any sort at home...

The trouble is that small scale experiences (like yours -perhaps- or mine at home -certainly-) give very little indication of actual risk. Unless you are using 100s-1000s of drives you cannot get reliable statistics on error/failure rates, particular since on HDDs they tend to be quite heavily clustered (for various reasons).

My original comment about RAID 5 was really aimed more at readers who might equate 'RAID' with 'backup' and or think that RAID systems don't lose data. That's probably not you.

> The most useful thing I did in the past was culling the many bad photos, where at one point I would keep everything which is completely pointless. 

Absolutely. I think these days, because image storage/retrieval is so easy compared to film we get suckered into thinking we have to keep everything. Until a few years down the line you realise you have to massive and fragile data integrity problem (you don't know what you have and you don't know if you have what you think you have) and no easy way to resolve it. I try to have a 'culling' session about once a year and also have a clear distinction between what I really don't want to lose (proper multiple onsite/offsite backups) and what I keep because I can (multiple local drives). 

 wilkesley 09 Feb 2021
In reply to Marek:

There has been a lot of comment recently about drive manufacturers using "sharding" on their disks. This can cause significant problems if you are rebuilding a RAID5, which can fail. I always use RAID1 where you have two disks which are mirrors of each other. If you have a drive failure it's much easier to recover than with RAID5. You can treat the unfailed disk almost the same as a non RAID drive if you want to move your data to something else while you rebuild the RAID.

 Tringa 09 Feb 2021
In reply to gammarus:

Two external drives(one at home, one elsewhere) is an easy and cheap option, but it needs good discipline for it to work well. Hard drives will fail eventually so the drives will need replacing now and again, but if you can cope with the logistics then it will work.

Storage online is also fairly cheap. The drawbacks are you are at the behest of the company who might change the T&Cs/cost. If online storage is your only method then you have to rely on their security and backup systems and if you do not have a fast internet connection, uploading could take ages.

Dave

 The Lemming 09 Feb 2021
In reply to gammarus:

Just backed up my Synology NAS to an external hard drive, plugged into a USB 3 port, using the Synology software "Hyper Vault" to back the entire NAS.

This took a glacial 82 hours to complete. And I haven't even verified the data yet.

82 hours?

And this is marketed at small businesses?

I think I shall stick with using FreeFileSync to backup the NAS to an external drive. Its quicker by at least three days. Days!

 jethro kiernan 09 Feb 2021
In reply to gammarus:

Use some form of star or colour coding, you will over time accumulate a lot of crap photos and whilst it is worth backing them up for that obscure photo of your mate 5 years ago, it’s probably not worth backing up your whole catalog multiple times.

I back up any stared pictures in my extra extra back up in raw format, with large JPEG’s in the cloud as well. (This includes nice family snaps as well as landscape etc)

I do need to get a NAS sometime soon as I need to consolidate all my back ups 😏 so following with interest

 HeMa 09 Feb 2021
In reply to jethro kiernan:

> I do need to get a NAS sometime soon as I need to consolidate all my back ups


Out of curiosity, do you do editing (and also viewing) from multiple computers?

If not (so predominantly only editing from one device... at a time). You really don't need a NAS (Network Attached Storage). A much more simpler DAS (Direct Attached Storage) would do just fine, and offer greater speeds. Non big and spendy NAS generally have 1Gbs Ethernet as the speed... and that is provided all your cabels and routers are also 1Gb, with no other load on it... and certainly not using WiFi to connect. DAS then to be at least USB 3.x, so starting from 5Gbs... if your devices support TB3, USB4 or TB4, then the speeds will be 10, 20Gbs.

So a DAS will most often be faster, but the extra hassle of connecting to it might be there. NAS will offer more flexibility (can even do stuff via WiFi or some solutions allow even access from outside your own network...) and also allow multiple users (e.g. have your movies stored on the NAS and watch them from yer smart telly, show photo-albums using your tablet etc.

 Robert Durran 09 Feb 2021
In reply to Tringa:

> Two external drives(one at home, one elsewhere) is an easy and cheap option, but it needs good discipline for it to work well. Hard drives will fail eventually so the drives will need replacing now and again.

Is it worth using solid state external hard drives? Are they less likely to fail? When I bought my current lap top, I had the main hard drive swapped for a solid state one. I am considering doing the same with my two backups.

 HeMa 09 Feb 2021
In reply to Robert Durran:

well yes, SSD is by all accounts less likely to break than a spinning disk. SSD has limited amounts of write cycles (not of any real concern for backups imho). But they can also break...

And also they are way more spendy.

If you don't have a huge space need, then certainly SSD is worth it (e.g. Samsung T5 1Tb can be had for like 130 or so Eur). But if you require more space than 2Tb, then you will be looking at a considerable costs (which might be OK for those professionals, but certainly not for a hobbyist).

For the record, I aim to get my stuff sorted out. So that I have two identical 4Tb drives to which I keep at different locales and backup on monthly. Then I have an 8Tb DAS (so desktop external) that has the stuff I'm not currently working on. And the 12Tb NAS (RAID 5, so real space is 8Tb), to which I also store my stuff (mostly used to house movies etc).

And a bunch of those Samsung T5 drives to be used for project stuff (when on the road or simpy, 'cause video files take a lot of space... so my laptop is full)... ie. scratch drives.

 Marek 09 Feb 2021
In reply to Tringa:

> ... but it needs good discipline for it to work well...

Actually, that's the biggy. It all too easy to 'leave it up to the technology', but like with security, data management is mainly about people and process, not technology. Discipline and a good process with adequate technology will always trump (sorry!) fancy technology and poor process/discipline.

 Marek 09 Feb 2021
In reply to jethro kiernan:

> I do need to get a NAS sometime soon as I need to consolidate all my back ups ...

Alarm bells! 

(a) NAS is not (should not be seen as) a backup. NAS is just a convenient way to make you files available to multiple consumers (PCs, tablets, phones...)

(b) 'Consolidate'='all eggs in one fragile basket' unless you are very careful.

 Marek 09 Feb 2021
In reply to Robert Durran:

> Is it worth using solid state external hard drives? ...

Probably not. Almost certainly not. The only reasons to use SSDs are (a) you need the read performance, (b) resilience against physical shock, and (c) power consumption. All these make sense for a system drive in a laptop, but not so much in a backup drive. Unless by backup drive you mean an external USB connected drive that you carry around a lot with your laptop.

 The Lemming 09 Feb 2021
In reply to jethro kiernan:

> I do need to get a NAS sometime soon as I need to consolidate all my back ups 😏 so following with interest

I had a Synology DS 214 Play and it served me very well. I just got greedy storing everything like a hoarder and ran out of space. I could have bought two 12TB HDD's however I chose to invest in a 4-bay NAS and buy two smaller HDD's. Worked out similar, with a lot of wiggle room for my hoarding of data.

You could go second-hand with a NAS and it would be more than capable of what you want.

This one was roughly the same as mine, spec wise. I had a DS214Play.

https://uk.webuy.com/product-detail?id=snetsynds214sedl&categoryName=ha...

Shove in a couple of HDD's and you get the latest Synology operating system and all the software that Synology has to offer.

Post edited at 13:56
 Marek 09 Feb 2021
In reply to wilkesley:

> There has been a lot of comment recently about drive manufacturers using "sharding" on their disks. ...

Hmm, any references? 'Sharding' is (traditionally) a database technology intended to promote load sharing across multiple servers/networks. Not sure how it relates to  discs - except as another (marketing) term for 'striping'.

 Marek 09 Feb 2021
In reply to Marek:

> Hmm, any references? 'Sharding' is (traditionally) a database technology intended to promote load sharing across multiple servers/networks. Not sure how it relates to  discs - except as another (marketing) term for 'striping'.

Also sharding is something which require intimate knowledge of (a) your applications and (b) your users' query patterns - neither of which seems relevant to (or even available to) 'drive manufacturers'. Odd.

 sbc23 09 Feb 2021
In reply to Marek:

> Hmm, any references? 'Sharding' is (traditionally) a database technology intended to promote load sharing across multiple servers/networks. Not sure how it relates to  discs - except as another (marketing) term for 'striping'.

It's not sharding, it's 'Shingled' magnetic recording' (SMR) that's allegedly the problem vs. Conventional magnetic recording. (CMR). Less reliable due to the track overlap. 

Western Digital (WD) in particular have been selling 3-8Tb WD Red 'NAS optimized drives', that are actually SMR drives re-labelled and not declared as such. Synology have now removed some of these drives from their approved hardware list. It's confusing because you can buy one model of a WD Red 6tb that's CMR and another model that's SMR. Putting a CMR and an SMR drive in the same raid array is apparently a bad idea.

https://blocksandfiles.com/2020/04/14/wd-red-nas-drives-shingled-magnetic-r...

 Marek 09 Feb 2021
In reply to sbc23:

> It's not sharding, it's 'Shingled' magnetic recording' (SMR) that's allegedly the problem vs. Conventional magnetic recording. (CMR). Less reliable due to the track overlap. 

That makes sense. I'm aware of shingling, but I've been 'out-of-the-loop' - aka retired - for a few years and then SMR was a bit touch-and-go as far as reliable recording technology goes. Perhaps it's improved now. I would certainly be very wary of using any SMR drive where data integrity (as opposed to areal density) was more than trivially important.

Post edited at 15:25
 HeMa 09 Feb 2021
In reply to sbc23:

> Putting a CMR and an SMR drive in the same raid array is apparently a bad idea.

Actually the common rule is to replace the drives from the same manufacturing batch for RAID.


So if one drive breaks, you don't just buy out one replacement drive... you buy as many. Then you replace them all.

Not that regular people do that, but that is how you're supposed to do it. I recall all RAID manufacturers actually stating this (even Buffalo), that you should actually have the following serial number on all drives.

 Marek 09 Feb 2021
In reply to HeMa:

> Actually the common rule is to replace the drives from the same manufacturing batch for RAID.

> So if one drive breaks, you don't just buy out one replacement drive... you buy as many. Then you replace them all.

> Not that regular people do that, but that is how you're supposed to do it. I recall all RAID manufacturers actually stating this (even Buffalo), that you should actually have the following serial number on all drives.

Which is part of the reason why simply having hot/cold spares fitted and ready to 'soft-swap' makes sense. And of course the whole hot-vs-cold debate will then rage...

Or indeed "if you have hot spares why don't you just use them"...

Post edited at 15:34
OP gammarus 09 Feb 2021
In reply to gammarus:

There's a huge amount of technical detail to digest here - many thanks for all the contributions. I think a useful zero-cost first step is to remove all the crap pics I shouldn't have stored in the first place - a good lockdown task! I think 2-3 HDD 2TB external drives (and OneDrive) look like my most cost-effective solution.

 Mike_d78 09 Feb 2021
In reply to gammarus:

For anyone interested in drive failure info beyond personal experience... here's the data on the drives used by Backblaze with a generally low failure rate but variability between model and manufacturer...


 The Lemming 09 Feb 2021
In reply to gammarus:

>  I think 2-3 HDD 2TB external drives (and OneDrive) look like my most cost-effective solution.

Sounds like a good plan. I suggest that you get hard drives with storage capacity three times larger than what you have at the moment. Gives you room to grow.

You can then use FreeFileSync to manage swapping files from your computer to the hard drives and for getting the hard drives to be exact copies of each other.

https://freefilesync.org/

And as an added bit of "Tin Foil Hat" reassurance get yourself a fire-proof safe to store some of your hard drives.

https://www.amazon.co.uk/Master-Lock-LCHW20101-LCHW20101-Electronics/dp/B01...

 Graeme G 10 Feb 2021
In reply to The Lemming:

> And as an added bit of "Tin Foil Hat" reassurance get yourself a fire-proof safe to store some of your hard drives.

Mmm....i like the idea, but only rated for 30 minutes at 843°C.  Surely you’re house would burn for lot longer than that? Although maybe not as hot...

 The Lemming 10 Feb 2021
In reply to Graeme G:

I'd hope the fire bobbies would have put the fire out in 30 minutes.

 Mr. Lee 10 Feb 2021
In reply to Durbs:

> Just a reminder (if needed) that something like OneDrive/Dropbox/GoogleDrive is NOT a true backup solution - if it's mirroring your hard-drive, it will also update if you accidentally delete a file, or if a file gets over-written or corrupted.

> BackBlaze (like other online options) will retain a rolling record of your files, so if you delete something, you can download a backup from last week.

Dropbox can be restored to any point in the last 30 days. I'd personally rather Dropbox automatically syncs everything than rely on myself to do so. I'm not that disciplined or organised. Backblaze does sound good value though, although I think I value the automatic syncing to much to switch. The 2TB Dropbox package is fine with me as well for the time being.

 ChrisJD 10 Feb 2021
In reply to Mr. Lee:

> Backblaze does sound good value though, although I think I value the automatic syncing to much to switch. The 2TB Dropbox package is fine with me as well for the time being.

It is amazing value.

And it is automatic syncing (not immediate), it just sits there doing its thing.  But it is not a file storage/sharing app like Dropbox.

If you had a massive data loss, they can send you portable hard drives with all your data on, rather than you having to download it all.

 artif 10 Feb 2021
In reply to gammarus:

At work, we have to archive media for 100 years, the problem being accessing them, traditional paper records are fairly straight forward but digital is an increasing problem. Digital media files are supposed to be copied every 5 years to update any format issues that might be encountered. 

Personally I have thousands of images and numerous movies stored all over the house on various storage media from slides to hard drives (VHS C anyone?), some are already inaccessible due to them being on out of date formats. Without question they should nearly all be binned, as they are of no interest to any one except me, and I could care less about most of them.

 Iamgregp 10 Feb 2021
In reply to artif:

You're not alone in this - I work in media management and archiving and you wouldn't believe the amount of footage that has been lost due to tape formats becoming obsolete, or not cost effective to transfer.  Honestly, I've heard of organisations junking tens of thousands of hours as they just can't deal with it. 

Now that we've moved away from videotape-based archiving to file based the problem hasn't gone away completely but I see this as being one of the main benefits of cloud based archiving.  It's now the cloud services provider's responsibility to migrate your files to the latest forms of storage media, and to ensure that there is a robust backup and DR protocol.

Post edited at 16:16
 Marek 10 Feb 2021
In reply to Iamgregp:

The issue with consumer-focused* (as opposed to business/professional) cloud based data services is what come-back you have if  one day they say "Oops, sorry, seem to have lost it" or "We have decided to discontinue this service, thank you for your custom". The answer pragmatically seems to be 'none'. So its great if you need something like NAS, but don't want to maintain it yourself, it's OK if you use it as a 'live' backup (as long as it's not the main backup), but probably not very good for archiving.

* T&Cs probably say something like "... no liability for lost data...".

 Blue Straggler 10 Feb 2021
In reply to artif:

> Without question they should nearly all be binned, as they are of no interest to any one except me, and I could care less about most of them.

I've silently been pondering this, throughout the thread. 

I recently threw away about 120 litres (measured approximately based upon capacity of a large bin) of photo prints of mine, and with them, quite a lot of negatives. 

In parallel, I transferred a load of scanned photographs from old CD-r and DVD-r formats, to a portable hard drive. Mostly concert and festival photographs from the 00s, with a few climbing pics amongst them. Many of them had been forgotten even by me. It was nice to view a few along the way, and I'll get around to sorting them properly and singling out a few, some day soon. 

Looking at the many terabytes required by a lot of people on this thread, has been mind-boggling! I cast no aspersions on any of the contributors here, or their photographs and videos and how much they mean to them. For me personally, if I lost all my digitally stored photographs tomorrow and could only have what I happen to have physical versions of, I'd be alright. Actually this nearly happened once, around 10 years ago, when my then-workplace was broken into and a load of laptops stolen. For some reason at that time, my only hard-drive "backup" was an external hard drive that I kept in a drawer by my work desk and I'd assumed for around 4 days, that that had been stolen (I'd thought it was ON TOP of my desk and had gone, and didn't check the drawers for a while). I was mildly upset, then I got over it, then I found the hard drive, and when I checked it and found my backlog of photos, I was a bit disappointed to find that they weren't worth crying over  

 Graeme G 10 Feb 2021
In reply to ChrisJD:

> > Backblaze does sound good value though, although I think I value the automatic syncing to much to switch. The 2TB Dropbox package is fine with me as well for the time being.

> It is amazing value.

> And it is automatic syncing (not immediate), it just sits there doing its thing.  But it is not a file storage/sharing app like Dropbox.

> If you had a massive data loss, they can send you portable hard drives with all your data on, rather than you having to download it all.

Pretty poor reviews on Trustpilot?

https://uk.trustpilot.com/review/www.backblaze.com

Post edited at 17:19
OP gammarus 10 Feb 2021
In reply to Chris Craggs:

Chris, which brand of fag-packet do you use? (I'm serious!)

 HeMa 10 Feb 2021
In reply to Blue Straggler:

> Looking at the many terabytes required by a lot of people on this thread, has been mind-boggling!

It is...

For photos, it isn't that bad (I mean really, most RAWs are about 50 Mb per shot or less).

But for 4K video, things are a bit different. E.g. I edit using Prores 422 (since I haven't shelled out the dough for a studio license of Resolve to edit directly 10bit H265 from my GH5). Meaning that I transcode all my media into Prores... Prores seems to go for 220 GB/h. That adds up quite fast... Even if you bin all the bad parts (and still, you do need some b-roll stuff).

 Blue Straggler 10 Feb 2021
In reply to HeMa:

Sure. Video was on my mind as I was typing; I don't do any video so don't know how big it can get but I would guess "quite big quite quickly" so I do get that.
FWIW I do generate very large data files at work. 3D x-ray CT data. 8Gb is average for a "sensible" one. For large objects it can be 120Gb for a single scan (with well over 120Gb of corresponding 2D data to build the 3D) at which point my workstations can't handle them as they are only 192Gb RAM. Our best ones are 1.5Tb RAM. 

 Iamgregp 10 Feb 2021
In reply to Marek:

I think cloud services are ideal for archiving.

I think the likelihood of AWS/Azure losing your data, or discontinuing any service and deleting your data is pretty remote, certainly less likely than me f*cking up trying to archive data myself at home.

In any case how would you store the data?  HDD, SSD, LTO all have issues so there's a decision to be made there.  What happens if your house burns down?  Do you have a second copy offsite somewhere?  How do you monitor and manage the health of the drives and migration to new media...

Let's face it, given the history we all have with trying to manage our own archiving at home (and from people's stories upthread), chances are it's going to end up with us all ditching a load of our files at some point.  

I'd rather pay a reputable organisation to deal with it for me. 

Post edited at 18:05
 Iamgregp 10 Feb 2021
In reply to gammarus:

Surely it's LaCie ruggeds?  Decent drives those...

 The Lemming 10 Feb 2021
In reply to Mr. Lee:

>  I'd personally rather Dropbox automatically syncs everything than rely on myself to do so. I'm not that disciplined or organised. 

If you are a Windows 10 user then you can set up File History to backup all your photos to an external HDD or Network Access Storage box plugged into your router.

You only have to set it up the once and job done, your photos are automatically backed up to something else other than your computer's internal storage.

Post edited at 18:57
1
 Chris Craggs Global Crag Moderator 10 Feb 2021
In reply to gammarus:

For general storage, I currently have a Western Digital Passport and a Samsung with everything duplicated. Plus I have a Toshiba for the Mac's Time Machine back-up, all 2T. I prefer to mix and match, less chance of two dying in rapid succession,

Chris

 Mr. Lee 10 Feb 2021
In reply to The Lemming:

Useful to know, but the remote backup was the main reason I got Dropbox rather than the automation. I've been burgled twice in my life and lost photos in the process on one of those occasions, so a remote backup is critical for me. I think I'm also better protected against randomware etc if important files are held on completely separate network.

Edit: I would also expect the likes of Dropbox to have multiple server backups in case they were hit with something major. 

I've also got a hard drive hidden in the bedroom which I also use as a physical backup, but the nature of it being hidden means I only do a proper full backup regularly enough. Maybe just a few times a year. I think if this was more accessible for backing up then it would probably be more accessible to thieves (again). 

Post edited at 19:55
 Marek 10 Feb 2021
In reply to Iamgregp:

As evidenced by this thread the simple answer to "What's the best way to backup mt photo?" is "Depends." All the methods discussed are basically a set of less-than-perfect compromises based on individual requirements, means and foibles.

Personally, I start by being pretty aggressive about what is really important and what is nice to have. For former amounts to a couple of percent of the total. The important stuff get backed up multiple ways (multiple local copies on PC and USB drive & NAS) and off-site in the cloud. The vast majority ('nice'), just get a couple of copies locally (although I do happen to have a fireproof safe). Apart from the 'live' cloud backup, the rest is done manually - but then I'm pretty process-driven and disciplined with looking after my data, so it's no great stress. I also don't worry too much about what happens if I lose my 'nice' data. And compared to some people here, I have very little!

Like I said, the process above work for me, but I wouldn't necessarily say it's the *right* process for anyone else.

 The Lemming 10 Feb 2021
In reply to Mr. Lee:

> Edit: I would also expect the likes of Dropbox to have multiple server backups in case they were hit with something major. 

And what if your computer is encrypted and that encryption is then replicated to Dropbox?

Anything attached to your network, either at home or in the cloud, is fair game to ransomeware.

 Marek 10 Feb 2021
In reply to Mr. Lee:

> ... I think if this was more accessible for backing up then it would probably be more accessible to thieves (again). 

It shouldn't be too difficult to hide/bury/entomb a USB drive somewhere within cable reach of your PC/Mac? Or go wireless to a hidden NAS?

 Marek 10 Feb 2021
In reply to Mr. Lee:

> ... I think if this was more accessible for backing up then it would probably be more accessible to thieves (again). 

It shouldn't be too difficult to hide/bury/entomb a USB drive somewhere within cable reach of your PC/Mac? Or go wireless to a hidden NAS?

 The Lemming 10 Feb 2021
In reply to Marek:

Is there an ecco in here?

 Marek 10 Feb 2021
In reply to The Lemming:

Hmm, weird. I certainly didn't type it twice. Must be a backup!

Post edited at 22:07
 FactorXXX 10 Feb 2021
In reply to Marek:

>   I also don't worry too much about what happens if I lose my 'nice' data. And compared to some people here, I have very little!

How about your 'niche' data?
I've got loads of that and I certainly wouldn't want to lose it and I definitely wouldn't want someone else to get hold of it... 🙄

 Iamgregp 10 Feb 2021
In reply to Marek:

Sounds you’ve got it all covered, backups of backups in different locations and by different means. 

The point you’ve made about selecting the best of the best and adding provision for that stuff is very valid. I’ve had to do similar with video collections at work as often companies just don’t have the budget to be able to keep everything forever.

 Mr. Lee 11 Feb 2021
In reply to The Lemming:

> And what if your computer is encrypted and that encryption is then replicated to Dropbox?

> Anything attached to your network, either at home or in the cloud, is fair game to ransomeware.

I'm not too worried about this, since Dropbox has a 30 day backup. I can wipe my pc and reinstall everything from scratch, and then set my Dropbox back to before the attack. I think 30 days is going to be fine in most cases, unless I am missing something.

Edit: I appreciate that it might take much longer than 30 days to realise that files had been corrupted/encrypted by malware, which is food for thought. 

Post edited at 19:00
 Mr. Lee 11 Feb 2021
In reply to Marek:

> It shouldn't be too difficult to hide/bury/entomb a USB drive somewhere within cable reach of your PC/Mac? Or go wireless to a hidden NAS?

Yes good point, but I partly go for the disconnected drive in the back of the wardrobe approach so that it is disconnected from the network in the event of a malware attack. I think I'd still want a cloud backup in the event of my home network getting totally compromised somehow. 


New Topic
This topic has been archived, and won't accept reply postings.
Loading Notifications...