Improving photography: getting rid of excessive noise, film grain, compression artifacts &c.

over on the passive income thread, I discovered that my photos need improvement.  

When I submitted my photos to Shutterstock, they rejected them because: Image contains excessive noise, film grain, compression artefacts, and/or posterization.

I don't really know what this means.  But I don't mind not knowing so long as I can find out how to not do it.  

I want to find out if using the manual or semi-manual settings on my current camera are sufficient.  While I'm at it, using the manual settings on my camera will get me ready for the DSLR camera I'm planning to get in 2020 or 2021.  It's no use me investing in such expensive equipment if I don't know how to use it.


(hint - click on the photos to see the full size image)

The photos I submitted to Shutterstock were outdoor shots.  It was suggested that I may be using too low an ISO.  So I made an experiment.

I took the same photo three times.  One on auto, one with low ISO and one with high ISO.  This is taken at dusk in low light.  I rested the camera on the fence post to keep the shot as steady as possible.  These are unedited.  (and I know, they aren't great photos - I just choose something that looked good in real life to see if I could capture it)

Now the question: is there any difference in this noise/grain/whatever?

huge difference in noise between the top two and the bottom one. (what iso were the top two on? it shows in the data for the file) look bottom left just above the power lines in the bottom photo, make it 100% and you can really see the noise in the photo, it looks a bit like static on an old TV set the top photo contains some in the same place but nowhere near as much.

Can you take a photo of something flat with a pattern on it? like a piece of your cloth. the trees are not very sharp in any of the photos but I'm not sure if that's just the the file over the internet, (Photo viewer cannot show as high resolution as a "proper" program, do they look fuzzy to you?) the camera or if there is some condensation or dirt inside it.  I was trying to take a similar photo earlier for comparison, but we don't have any pine trees here so it wasn't really comparable.

The first one - auto - ISO 400

The second one - ISO 80

Last one - ISO 3200

For me (and they may not agree) the noise in either of the top two is acceptable, with the top photo having better exposure/colour. back lit photos are always difficult to get, many of the ones you see online are actually two photos merged.

Here's something worrying.  When I look at several of my photos at full size, there's a strange horizontal or verticle line.  

I would say first things first - the time of the day (think golden hour), relative position to the Sun (what the golden hour light hits and what remains in shadows) and feeling of contrast (which objects stay light and which are dark) is way more important than camera settings.
Here is one of my imperfect attempts to get it right, the time window to capture majority of details being not too dark and not to light was very short.

r ranson wrote:over on the passive income thread, I discovered that my photos need improvement.  

When I submitted my photos to Shutterstock, they rejected them because: Image contains excessive noise, film grain, compression artefacts, and/or posterization.

I don't really know what this means.  But I don't mind not knowing so long as I can find out how to not do it.  

I want to find out if using the manual or semi-manual settings on my current camera are sufficient.  While I'm at it, using the manual settings on my camera will get me ready for the DSLR camera I'm planning to get in 2020 or 2021.  It's no use me investing in such expensive equipment if I don't know how to use it.


(hint - click on the photos to see the full size image)

OK, so what are these things they're talking about?

These are essentially pretty similar defects that show up for different reasons.

Noise: For instance, you take a photo of a perfect blue sky or a perfectly smooth yellow wall, and when you blow up that part of the picture, it is mottled. Like it was full of little random dots or a stucco-like texture. It's not totally smooth like the original subject was. Here is an example from this web page:

[Edit: Damn, link didn't work, attached below]

Film grain: A throwback to the days when people used film. Photographic film is not perfectly smooth, it has a grain to it, let's just say those are imperfections in the film itself. Consider the size of a frame of old super-8 reel-to-reel film. Tiny! But it had to be projected big, on one of those old white-sandpaper screens. Old super-8 films look kind of blurry and mottled. That's because this tiny image on a piece of film had to be blown up to maybe 1 x 1.5m. Same with photographs. "Professional" images are ones that are expected to be able to be blown up to a very, very large size and still look great: sharp, good highlight and shadow detail, and the smooth areas must look smooth. The images on billboards, posters and glossy magazines come from somewhere, you know, and if you blow up an image to 1600% of its original size, all the defects of the substrate it was taken on are plain to see. Film grain looks pretty much like the noise example above.

Compression artefacts: Most digital images are hugely compressed, usually in JPEG format. This is because uncompressed digital images are huge space hogs. The huge files are difficult to work with (obviously depends on the pixel dimensions of the photo). To take professional photos, you MUST have a camera that saves your images in "RAW" format, an application to accept and manipulate your raw files, and you need to get good at compressing them into .JPG format yourself. JPEG compression artifacts show up dependably around "borders" in the image, light/dark especially, and also highly contrasting colours. You'll find that the tone is exaggerated in a line on both sides of the border. I have to attach this image, so I'll continue in another post. The image is from this website.

I continue for a moment and I must go water my plants.

Posterization is the simplification of what should be (or what originally were) smooth, gradual color transitions in an image. These days, this is usually the result of unmindful image processing, since nearly all modern devices are capable of displaying 256-bit colour, or another term you might occasionally hear, 16+ million colours (= 256 x 256 x 256). That means that there are enough "shades" of each of the basic colours to fool the human eye into thinking that there is a smooth transition. So if all the devices involved are slightly more sensitive than the human eye, you're good to go. There is also a "posterize" filter in image processing apps like Photoshop that do this purposefully for artistic effect.

The main example below came from wildlifesouth.com but seems to have been removed from their site.

Then there's artisitic posterization, like this:

Good advice and feedback.  Thank you.

I often wish I was a painter.  What I see and what I can capture on film are very different - I think what I see is often edited by what interests me and I hope one day to learn how to capture that with a lense.

I added a fourth picture to my sunset trees above.  I use basic editing to try to reproduce what I saw.  The more I edit a photo, the more this noise stuff seems to show up.


I always thought my photos were less than perfect due to pure human error.  I think it must be a big contributing factor so I'm going to work on this first.  If I can take better photos, then there will be less time and money in postproduction.  


The library has 10,070 odd books about photography, so I put a few dozen on hold.  I've also looked into some local classes on DSLR - the problem with them is that we need a DSLR to take the class and I don't want to invest that kind of money into a machine until I know how to use it.  Catch 22.  I wonder if I can rent one to take the class with?


Going to read and reread what you wrote.  Thank you so much for taking the time to help me improve my photogrpahy.

Damn sorrell was mobbing my italian pepper transplants and shading them. Comfrey was doing the same to the tomatoes. All's well now. I continue.

Book:

A good book to read for all these issues is the latest you can find in the Photoshop Artistry series by Barry Haynes. Co-authors in the last book in the series I know of, Photoshop Artistry: For Photographers Using Photoshop CS2 and Beyond were Wendy Crumpler and Sean Duggan. Barry was a serious film-based photographer and worked at Apple when Photoshop first came out, so he got right on the digital photography bandwagon at the very beginning, and with the eye of an artist. He even influenced the early development of Photoshop, I believe.

Maybe some of the Photoshop techniques in the book are outdated since Photoshop CS2 is officially "ancient" now, but the rest of the book is thoroughly worth a read if you can find it. Actually, the Photoshop techniques are probably almost all still usable even if the interface has changed. They're always inventing more efficient ways of doing things, but the "old ways" usually still work. And if you're using another serious image editing program, you can usually find equivalents of all the techniques he shows you. You just will need to poke around the interface for them. But if you read the book, you'll know what you're looking for.

You can have some serious fun with the techniques in this book and will be amazed with the images you can create. And he goes super in-depth on all these digital quality issues. He's a complete stickler for the highest quality. Just telling you about this one book because it's the only one I ever needed.

Back to the image discussion:

Even if you have good equipment, you might find that a photo you've taken has a bit of noise or posterization. In the book or elsewhere you can learn techniques for processing these things out. Adding a little bit of noise (yes sometimes you need to!), blurring, sharpening, masking, overlaying identical photos taken at different exposures, on and on. Really professional photos (like you'd want to take for that RM category of Alamy's) can have a lot of post-processing time attached to them. It's not just shoot and upload.

Last is equipment. A seriously good digital camera back is necessary for very high quality pictures. It needs to be silly high resolution, so the images you take can be blown up to huge size if necessary, and also very sensitive. It needs to be able to pick up details in the shadows in really low lighting conditions, and also pick up plenty of detail in the brightest areas of the image. This is where consumer grade cameras fall down. Also SLR of course so you can actually see your depth of field and such.

I know more about image processing than about the actual photography and equipment part, so I better quit now before I get in over my head and allow a real photographer to say more about the front end of the process.

My camera was set to "fine" instead of "superfine" compression.  In the manual, it says that superfine gives a better shot, but fewer of them (2000 pictures on the card instead of 5000).  I think this happened last summer when I was getting someone who said they knew a lot about photography and if I let them see my camera for a minuite he would adjust the settings to take better pictures.  

I thought I had put everything back how it was supposed to be, but I missed this setting.

Am I right?  Will superfine compression give me better results?  It's nearly doubled the file size compared to how it was before.  

It doesn't look like you have any dirt in there there's no randomly fuzzy bits, just the normal focus bleeding off towards the edges. It's in the slightly out of focus edges it's easier to see the noise, more apparent on the bottom picture. the picture below I just took in my living room which is pretty dark, it's a 1/30s speed at F3.5 and iso 2500 The lighter and better one is taken outside in the sun and is ISO 100 (1/125s F 6.3)  I think the only difference with a "better" camera is probably that one can be lazier and it compensates better! There's no point using higher compression not with todays storage capacities so always use superfine in your case the photos below are around 14MB in "Fine" jpg on my camera and 25MG in RAW format.

These two photos also show the importance of light source direction.. but hey I didn't take them to be "good" photo's!

From a technical perspective:
ISO (sensitivity): This is the amplification factor by which the camera electronics have to "boost" the signal from the sensor to get the colours in the image. Just as with any amplifier, more amplification also means more noise. If you can get a stronger signal (more light) into the amplifier, you get a better signal (image) out. The sensor itself has a fixed (actually temperature dependent) noise level, which cannot be removed. A high amplification, this becomes too strong to ignore. In theory there is also "shot-noise", but that is not relevant with today's camera sensor (unless you have a laboratory-grade camera).

Lens aperture (f) The light passing through the lens is inversely proportional to the square of the F-value. flux ∝ 1/f²
So a smaller F-value gives you more light and thus less noise. However, it also means a smaller depth-of field, less contrast and sharpness.

Compression in the camera: Basically the bigger the files, the more information they contain.

Editing: If the camera produces RAW files, the images can be edited on a computer (I use darktable). This is possible because the file contains the information that was read out of the sensor. (No actual colours, just a lot of brightness values) The raw data is then combined with information light white-balance, known lens-aberration (if the distortions of the lens are known, most of them can be corrected), and colour mapping (monitors can display far less than the camera captures, so it needs to be squished to fit it somehow) and various other algorithms, to create the final image.

When the camera produces JPEG files, all above steps are done already and only some information is left. In my opinion JPEG images cannot be edited with good results.
Part of the problem is, that every transformation of the image has some losses and Jso you end up with less information. RAW files have about 10x more information that the best output picture, so there is plenty to "use up".
Another part is that JPEG already stripped out the "not so important parts according to human vision", but most transformations will turn them into "important parts", which are then missing or wrong.

I've started a new project to help me focus on learning about photography.  https://permies.com/t/118634

There is so much to learn that I'm overwhelmed.  

So I'm starting simple - composition.  Then I'll work my way up/along/wherever from there.  That's basically what that thread is about.



But I'm also curious about the picture quality of the ones I've uploaded so far.  I don't think it's very good as there seems to be an extra colour around the outline of the trees and everything is hazy.  I cleaned the lens before going out, but there's dust inside the lense that I don't know how to clean.  It shows up when I have a light background.  Composition aside, I don't think these would be good enough quality to sell or print.  I'm wondering if my camera is at fault or if I need to try different settings?

so here's a new error:

Composition: Distracting elements are entering the frame obscuring the main subject or the horizon line is unintentionally crooked/skewed.

Do you have any thoughts on why this (attached) photo would get this error?  this photo of simular flowers made it okay.

(by the way, thank you, everyone, for your earlier help.  It's helped me improve my photos tremendously)  

I think it's because the background in this one is kind of distracting from the subject. It's blurred just enough to cause my eyes to try to focus there, to figure out what it is, instead of focusing on the beauty of the flowers.