Got digital noise in your photos?

New Topic
This topic has been archived, and won't accept reply postings.
 The Lemming 08 Feb 2022

I have just started using Topaz software for my Micro Four Thirds camera.

I have to say that it is a truly stunning piece of software that performs magic on your RAW files.

Give the free trial a go and you will be amazed.

I'm using this YouTube Tutorial to sexy up some wedding photos taken in poor mixed lighting.

youtube.com/watch?v=80aasB7Oaw8&

3
 galpinos 08 Feb 2022
In reply to The Lemming:

Is that the Denoise AI? The Sharpen AI looks pretty neat as well, there's a few photos I slightly missed focus I'm gutted about!

Not available for iOS though unfortunately......

Post edited at 10:21
OP The Lemming 08 Feb 2022
In reply to galpinos:

Sure is. Its a phenomenal bit of kit, especially for my camera with a small sensor compared to a Full Frame camera. I'm now going to revisit all my older astro photos and see what magic this can perform with obviously noisy images.

 galpinos 08 Feb 2022
In reply to The Lemming:

> Sure is. Its a phenomenal bit of kit, especially for my camera with a small sensor compared to a Full Frame camera. I'm now going to revisit all my older astro photos and see what magic this can perform with obviously noisy images.

I'm on APS-C so would probably help. All my editing is done on the iPad though so I'll have to wait a bit....

 Paulhesketh 08 Feb 2022
In reply to galpinos:

It helps just as much on FF, you just end up taking shots with higher ISO which is useful for fast moving subjects in low light. I've recently started using Topaz Denoise, as you say it is very good, and much better than Lightroom alone. The trick is to avoid running everything through Denoise and Sharpen and doubling your processing time......

I here DoO DeepPRIME is also very good?

 Marek 08 Feb 2022
In reply to The Lemming:

I'm not sure if there's any connection, but there was an MIT paper published in Nature last year describing the use of an AI-based denoising method for low SNR microscopy called ... "Topaz-Denoise':

https://www.nature.com/articles/s41467-020-18952-1

As is often the case, the software is freely available (e.g., via Anaconda, pip...)

http://cb.csail.mit.edu/cb/topaz/

As with most AI stuff, the training is critical, which can be a good thing (if you do it right) or a bad thing (if you don't). I don't think the commercial software allows you to train the algo yourself - which is a pity since different sensors tend to have different noise characteristics. It even more important in Sharpen AI since the blurring you're trying to remove is likely to vary image to image (miss-focus, motion, optical aberration), across the image (e.g., astigmatism) and between different lenses.

Looks like it'll be worth a play with (the free software that is) when I've got some time.

 Solaris 08 Feb 2022
In reply to The Lemming:

Interesting topic, and follow up comments.

I had a look at it when it was reduced a couple of months back and was prepared to make the outlay, but having downloaded the free trial (Denoise, I think, but it may have been Sharpen - sorry) and experimented with it on a variety of images, I decided against.

Why? It seemed to over-sharpen (even after some fiddling with settings) and introduce artefacts.

I was using it on Fuji (X-T2) RAW images, and it seemed that other Fuji users had similar experiences, so maybe it's something to do with the Fuji sensor.

But I'm glad others are pleased with it.

 galpinos 09 Feb 2022
In reply to Solaris:

I'm using at X-T1 so that's worth bearing in mind....

OP The Lemming 09 Feb 2022
In reply to Solaris:

> Why? It seemed to over-sharpen (even after some fiddling with settings) and introduce artefacts.

I too have noticed some artefact on some images when using Sharpening. For those images Lightroom works well. For images with no artefact Topaz is outstanding.

Post edited at 11:57
 Solaris 10 Feb 2022
In reply to galpinos:

I put my Fuji RAWs through Capture One 20, which seems to me to do a pretty good job with its basic import settings, especially with noise (but maybe I'm too tolerant).

I can see that if I was a fanatical pixel peeper I might have a use for Topaz from time to time, but I have found C1 adequate for my general purposes.

If you want to follow up and get more opinions – some pro and some con – there was a thread on the Fuji section of DPReview a couple of months back.

Post edited at 22:06
OP The Lemming 11 Feb 2022
In reply to Solaris:

My ten year old Lightroom 5.7 can't open my RAW images. I have to convert them to DNG files first.

I'm on the back foot from the outset. I refuse to pay monthly for software, however happy to pay a licence fee.

1
 65 11 Feb 2022
In reply to The Lemming:

It looks like you need to look elsewhere. Check out Affinity. It’s very cheap, upgrades are free and it’s a pretty powerful tool. The online support community is pretty good too. I use it in preference to photoshop. Not that I use it much, Capture1 does almost everything I want but it’s not cheap. 
 

I’ve read good things about Topaz especially for RAF files but I’ve not needed it and I’m on Mac.

 JDal 11 Feb 2022
In reply to The Lemming:

There are performance issues, the software needs a powerful video card with, according to Topaz, at least 4 meg of VRAM. I'd say it needs 8 meg.

I use both SharpenAI and DenoiseAI. I had a load of hassle at the outset because my hardware wasn't up to the job, in particular the video card with it's measly 1Meg VRAM. I had an old Dell desktop which was no longer expandable, so I had to upgrade my whole machine to get a suitable video card. I now have an NVidia 3060 card with 8GB of VRAM and the AI software performs acceptably well. 

Functionally, it's great. I mostly use DenoiseAI which has a bit of sharpening built in. It'll process RAW files directly but it tends to screw up the colours somewhat.

OP The Lemming 11 Feb 2022
In reply to JDal:

> Functionally, it's great. I mostly use DenoiseAI which has a bit of sharpening built in. It'll process RAW files directly but it tends to screw up the colours somewhat.

I'm beginning to notice colour problems myself. I thought it was just me but it is noticeable with my calibrated monitor going from Lightroom to Topaz and back again.

 JDal 11 Feb 2022
In reply to The Lemming:

It does say somewhere they are aware and its an area of development. They need profiles for each camera I think, like LR.

> I'm beginning to notice colour problems myself. I thought it was just me but it is noticeable with my calibrated monitor going from Lightroom to Topaz and back again.

 Marek 11 Feb 2022
In reply to JDal:

Speculation on my part since I haven't tried Topaz, but I tend to be wary of software which claims to "work on RAW files" for the simple reason that there are a lot of less obvious pitfall therein. Specifically, RAW data has to be colour balanced BEFORE debayering (with the exception of a few application specific debayering algos like drizzle). You can't always rely on the data in the EXIF field to provide accurate colour balancing info. Personally I'd rather use other tools to colour balance and debayer rather than relying on some 'best effort' from some software which is focussed on other value-add like denoising. They may of course claim that their denoising algo works better on RAW data - perfectly plausible - but then they should denoise the RAW data and output a DNG file (for instance).

Or perhaps I'm just being too picky - the above is very relevant for astrophotography, but perhaps less so for landscapes.

OP The Lemming 11 Feb 2022
In reply to Marek:

May I ask what work flow you suggest for my Panasonic RAW files to be converted to DNG files?

At the moment I use "Adobe DNG Converter".

 Marek 11 Feb 2022
In reply to The Lemming:

> May I ask what work flow you suggest for my Panasonic RAW files to be converted to DNG files?

> At the moment I use "Adobe DNG Converter".

Not sure - I don't think I've ever converted Panasonic RAW to DNG. I would have suggested the one you've used. I tend to use either dcraw.exe (either undebayerd output or convert to 16b linear TIFF with an explicit white balance) or RawTherapee, but I also very rarely resort to RAW development on my G9 - it's mainly for wildlife. For fussy stuff (e.g., astro) I use Canon.

OP The Lemming 11 Feb 2022
In reply to Marek:

As you mentioned astro. If you are photographing a landscape, not DSO, how do you choose WB?

I like doing little time-lapses or stacking images. This is my most recent attempt to solve the WB conundrum.

youtube.com/watch?v=1EXgaUIks94&

 Marek 11 Feb 2022
In reply to The Lemming:

> As you mentioned astro. If you are photographing a landscape, not DSO, how do you choose WB?

OK, first you have to decide: Do you want the colours to be accurate or do you want it to look nice/interesting? They are not always the same*. In reality most amateur astrophotographs are more art than science. If you want accurate, then assuming you are not using any filters, I tend to think the best way is to use star colours to calibrate your lens/sensor. Target a suitable star field, look up the expected colours in say Stellarium and then adjust your red/blue multipliers until they are as close as you want. It'll be close to 'daylight'. If you are using any filters like UHC or pollution filters, or a modded camera, then all bets are off - choose whatever white balance you like, they won't be 'true' colours in any normal sense of the word.

* The most obvious example is if you take colour balanced image of a proper dark sky, most people expect it to be dark blue (in so far as it has any colour at all). In reality it's greenish. Show a green sky to most people and they'll say "Nah, that's wrong!" But that's the true colour of airglow. Of course for most DSO objects you'll subtract out the airglow in the workflow, so it won't be obvious. That doesn't work so well if you have foreground though - the foreground isn't affected as much by airglow.

 Solaris 11 Feb 2022
In reply to The Lemming:

Yeah, I was on Lr 5.7 but was (I thought) forced to new software when I bought an X-T2 (with which I am very pleased). Like you, I wasn't prepared to subscribe to Adobe and although I'd found a way round RAF worms, I am pleased I changed to C1.

Hope you find the technically well-informed posts from others helpful. They're making me think I'd need to get a new computer and try Topaz again!

OP The Lemming 11 Feb 2022
In reply to Solaris:

Would you like a clean copy of LR 5.7?

It's old.

 Solaris 12 Feb 2022
In reply to The Lemming:

I think I'm OK with what I've got, but thanks all the same.

 JDal 12 Feb 2022
In reply to Marek:

> ... They may of course claim that their denoising algo works better on RAW data - perfectly plausible - but then they should denoise the RAW data and output a DNG file (for instance) ....

It can output DNG for any file type you open. It's standard processing has 4 models to choose from -  Standard, Severe Noise, Clear, and Low Light".  When you open a RAW file it adds a "RAW" model to this list. If you use that, and save as DNG, it has no need to debayer the data except to show the monitor image as you are processing. I've been playing with the latest release with a few Olympus RAW files and the colours are pretty close when the images are then processed in Lightroom.

 Marek 12 Feb 2022
In reply to JDal:

> ...  When you open a RAW file it adds a "RAW" model to this list. If you use that, and save as DNG, it has no need to debayer the data except to show the monitor image as you are processing...

Sounds like they've done it "the right way" (IMHO).

 Marek 16 Feb 2022
In reply to The Lemming:

My interest was peaked by this tread, so being too stingy to invest in Topaz, I had a dig into the similar open source codes. Count me impressed.

I believe Topaz denoise is based on an ML algorithm called Noise2Noise. The code is available, but a bit awkward to use. I found a derivative algo called Noise2Void which is simpler to train and also is available as a plugin to ImageJ. So I generated some noisy ISO25600 images on both my full-frame Canon 6d and my m4/3 Lumix G9, battled with installing all the right versions of ML libraries (another story) and did some testing. I've put the results on... 

https://drive.google.com/drive/folders/1-fY5CLsi1xe5g5_BNEWzKgCCVm5K9w7U?us...

compare-h.jpg (100%), compare-h2.jpg (400%)

I initially ran the training for a minimal 30 epochs (~30 mins) on the 6d and the G9 images and then ran the images through their 'native' and 'foreign' network models. It looks like the result are noticeably better with the native model.

compare-h3.jpg

The compares the native model result with a low-ISO (ISO1600) version of the same image (a sort of ground truth).

compare-h300.jpg

Went out climbing this morning, so left the computer training a 300 epoch model (Canon). Applied it to both the 6d and G9 images. For the 6d - which is relatively low noise anyway - there was a small improvement over the 30 epoch quicky version, but I was gobsmacked how well it worked on the very noisy G9 image. NOTE: training the model takes a long time, but once you have it, denoising an image takes only 2-3 secs.

compare-h300-lowISO,jpg

As above, but comparing the 300 epoch results with low-ISO version.

The relevant files are all there on the drive if you want to look deeper. One thing I did find was that the denoise worked much better on pure pixel noise - i.e., RAW. Once interpolated, the debayer algo (AHD) leaves more structured noise which is much more difficult to eradicate. Hence all the above testing was done on debayered-but-uninterpolated 16b TIFF files created by 'dcraw -h'. I wouldn't bother with JPGs!

Anyway, I thought others might be interested, so there you have it. I can provide more info if required (e.g., ML library installation - easy once you figure out which instructions are wrong!).

Post edited at 15:32
OP The Lemming 16 Feb 2022
In reply to Marek:

You could at least put a little more effort into this.

😀

I just wish I understood a small part of this.

OP The Lemming 16 Feb 2022
In reply to Marek:

With your experiments, did the denoise alter the colour balance at all?

Even though Topaz does a great job denoising the DGN files, it buggers up the colours and no tinkering with temperature and tint will get close to the original.

 Marek 16 Feb 2022
In reply to The Lemming:

> With your experiments, did the denoise alter the colour balance at all?

Yes, when I used a 'foreign' model (e.g., one created from Canon images to denoise a G9 image) and if there's a difference in the colour balance between the two, then the colour balance was messed up in the result. In the samples shown above, I carefully matched the balance in the 6d and G9 images (hence the "-wb" in the filenames) and then there was no problem.

OP The Lemming 17 Feb 2022
In reply to The Lemming:

If anybody is interested in my woes with colour changes while using DGN files and Topaz Denoise.

I contacted Topaz with a sample DNG file for their advice, and the CEO personally replied saying that my camera was set in a Colour Profile not recognised by Topaz. My camera was set up for Display P3 colour profile. And to get the most out of Topaz it would be convert my images to either ProPhoto, Adobe RGB, or sRGB to get the best out of the software.

I don't even remember putting my camera into P3. Time to do a bit of research about setting my camera profile for photo work.

 ChrisJD 17 Feb 2022
In reply to The Lemming:

Looks like that profile was created for the film industry (made by Apple, it looks like)

But isn't the Colour Profile irrelevant to the actual data in the RAW file - its just a 'tag' the camera adds to the file to show your Colour Space preference when you process?

Anyway, I set it to Adobe RGB, then output jpgs in sRGB, else you can screw up some picture viewers/browsers that can't deal with rendering Adobe RGB correctly.

Post edited at 15:32
OP The Lemming 17 Feb 2022
In reply to ChrisJD:

My GH5 is primarily for little home videos and over the last couple of years I've fallen down a rabbit hole of colour spaces and colour profiles for simple colour grading of my little videos. Nothing fancy and the occasional dalliance with the classic "Orange and Teal" look for personal amusement.

For this reason I set my camera up with the widest colour space available to capture as many possible colours that I could. That way I could destroy my video work as creatively as possible messing with the colours in post production. 🤣.

I just need to do a bit of reading about being able to separate my colour spaces for film and photo work.

Post edited at 15:32
 ChrisJD 17 Feb 2022
In reply to The Lemming:

see my post edit

OP The Lemming 17 Feb 2022
In reply to ChrisJD:

I thought a large colour space would not affect converting to something like sRGB but the CEO at Topaz thinks otherwise.

I'll do a few test shots to see.

 ChrisJD 17 Feb 2022
In reply to The Lemming:

https://photo.stackexchange.com/questions/89977/how-does-the-in-camera-colo....

Colour space choice does not affect the RAW 'image' data stored in a RAW file.

Maybe Topaz gets confused with the colour space 'tag' in a RAW file, 

Post edited at 16:17
 jezb1 17 Feb 2022
In reply to The Lemming:

I have a short cut set up on my Sony to turn picture profile on / off, definitely don’t want a video’s pp on when shooting stills.

OP The Lemming 17 Feb 2022
In reply to ChrisJD:

Maybe you're correct, which I believe that you are.

But what can I do other than try to jump through their hoops?

OP The Lemming 17 Feb 2022
In reply to ChrisJD:

>  Maybe Topaz gets confused with the colour space 'tag' in a RAW file, 

I have just gone through every single setting of my camera and its set to sRGB. There is no way to adapt or amend the P3 colour space. And I agree Colour Space should not affect the RAW image data.

At least I know I've got my camera set as requested. 😥

OP The Lemming 17 Feb 2022
In reply to ChrisJD:

>  Looks like that profile was created for the film industry (made by Apple, it looks like)

I did a little experiment with my only other available camera, Google Pixel 3a phone. Its all that I had.

I took a DGN image of my xRite Colour-checker Passport and put it through the Topaz denoise washer. I could only save/export to TIFF because some how the Topaz DGN copies would not import to Lightroom.

The only difference I could see while in Lightroom and looking at the histogram was that Topaz moved everything to the left.

Once I corrected the White Balance using the 18% gray chip on the original DGN and the new TIFF files and then did a bit of tinkering with the exposure to get both images in the same lighting, I could not spot any colour shift.

Maybe there is some setting in my GH5 that is buggering things up. Just wish I knew what it was and how to fix it.


New Topic
This topic has been archived, and won't accept reply postings.
Loading Notifications...