> MozJPEG use a lower value in the first position of the quantization just like this article suggests
It has lower value in the first position of the base table, i.e. the table which is used for q=50. With lower qualities this value scales up. This delays color banding from q=50 to roughly say q=40, after that the same effect is appears.
The simple question: if all the existing formats already solve the image compression problem, why a new image formats (WebP, HEIC, AVIF, JPEG XL etc) appears?
It was not badly picked for regular use cases. Just, before the retina displays appeared no one was interested in extreme low bitrate and no one knew that different artifacts had different impact with high density.
> the employed solution only modifies first element from [16,17] to [10,16].
Correction: 16 and 17 are values from the base tables, which means this table is used with q=50. With q=25 it will be [32, 22, 24, 28, 24, 20, 32, 28…] (in zigzag order). The employed solution is to always limit the first value by 10 regardless of q: [10, 22, 24, 28, 24, 20, 32, 28…]
mozjpeg chosen the different approach: it still scales all values based on q, but has significantly changed the default base table. It helps, but doesn't eliminate color banding completely (you can still see it on the example from issues/76).
Each resampling algorithm will internally produce some high-precision result before cutting it to 8 bits. For Pillow-SIMD it is 32-bit integers. Currently, I haven't considered dithering, but it is a very interesting idea. Do you have any links for further reading about downsampling banding and dithering?
About IPP's features: the comparison is pretty fair: the same input, the same algorithm and filters, pretty much the same output. If IPP uses more resources internally with the same output, so, maybe it shouldn't.
Shame on me, I still haven't added the link to IPP's test file I used. Here is it: https://gist.github.com/homm/9b35398e7e105a3c886ab1d60bf598d...
It is modified ipp_resize_mt program from IPP's examples. If you have installed IPP, you'll easily find and build it.
> Do you have any links for further reading about downsampling banding and dithering?
Sadly, no, I wish I did. I just made some expensive mistakes printing giclee images from downsampled digital files, and whipped up my own dither for converting 16bit to 8bit. It wasn't until it bit me that I noticed Photoshop does it better than most apps because dithering is on by default. That's when I went looking and found an option for it in Photoshop's settings.
The main banding problem when downsampling is with slow changing gradients. Sky and interior walls, for example. I bump into it a lot with digital art too, since the source images don't have any noise. But even when there's noise in the source image, downsampling 2x or more with a good filter can eliminate the noise and cause gradients to stabilize and show their edges in 8bit color. In my experience, the problem is more common with print than on-screen resized images, but it's still pretty easy to spot on a screen, especially in the darks, and especially when jpeg compressing the results.
Implementation wise, the 16-to-8 bit dither is nowhere near as sensitive as the dithers we normally see converting 8bits to black & white or when posterizing. Almost anything you come up with will do. You don't need any fancy error diffusion or anything like that. Here's what I do: imagine the filtered 16 bit result as an 8.8 fixed point number in the [0-256) range, so the least significant bits are in the [0-1) range. I add a random number between -0.5 and +0.5 before rounding to the nearest integer. Viola, drop the low 8bits and the result is a dithered 8 bit value.
What I just described will be way slow in your world if you call a random number function every pixel, so don't do that. :P For Pillow-SIMD you'd want a random number lookup table or something slightly smarter than a random() function. And I dither on the color channels separately, but there might be some way to make it blaze by dithering the brightness and rounding all three channels up or down at the same time. I've just never tried to optimize it the way you're doing, but if you find a way and release anything that dithers, I would LOVE to hear about it.
Well, 3680x2456 / 0.06 seconds ≈ 150 Mp/s or 75 Mp/s/per core. Pillow-SIMD's current implementation runs on ≈ 700 Mp/s/pre core for bicubic (which is closer to this implementation).
Obviously, your implementation could be further optimized, but quality drawbacks will remain the same.
I've since made a small change that doubled performance and I'm operating on >5x more data per pixel (32x4 vs 8x3). So a 10x factor makes this already faster without any explicit SIMD yet. Proper benchmarking on the same machine would need to be done obviously.
On the quality drawbacks I'd have to do some more checking. This algorithm is closest to what the image would be if you had a camera with that native sensor size. The standard filtering approach may very well have plenty of cases where it produces better output but it can also cause issues so I wonder if this isn't a conservative solution.
Color space conversion is a hard topic in terms of performance. First of all, not all images are stored in sRGB. Most of them have another color profiles (such as P3 or ProPhoto). So, sRGB conversion is not enough, you need the full color management.
Second, you'll see a real profit of color management only on a few images. Most time you'll see the difference only when you see both images at the same time on the same screen.
For now, I came up to the resizing in original non-linear color space and saving the original color profile with the resulting image.
By far most images in the wild are sRGB, and those that aren’t should typically be tagged with their color space.
Resizing in gamma-adjusted space (sometimes) causes nasty artifacts when resizing. If you can afford the CPU use, always convert to an approximately linear space first, then downsize, then convert back. If you get the gamma curve slightly wrong (e.g. gamma = 2.0 vs. 2.2) it’s not too big a deal, the resulting artifacts won’t really be noticeable, so feel free to use a square root routine or something if it has better performance.
But note if you go to a linear space, you need to bump up the precision of your numbers. 8 bits per channel doesn’t cut it anymore, you’ll want 12 or 16 bits per channel (or even better, a floating point number). This might have a big effect on performance.
You don't really need the full color management for resizing, only fixing the gamma curve. I believe Apple's implementation of P3 uses the common 2.2 gamma, which is also approximately what sRGB uses. Unfortunately ProPhoto uses 1.8.
It's a bit of a misnomer to talk about a distinction between convolution and FFT. By the convolution theorem, the two are mathematically equivalent. In addition, on paper, FFT-based convolution scales much better than traditional convolution because it reduces the complexity from O(n^2) to O(n log n).
I haven't done the benchmarks for a fully optimised implementation, but comparing naive implementations you can easily tell that FFT-based is much faster (even with all of the tricks that MATLAB does to optimise sparse matrix operations).
It originally was written two years ago, so some things have changed. But in general, it is still correct: for most browsers, you need to combine several ugly technics to get suitable results. Though, the quality will have nothing common with quality when you have direct access to hardware.
It has lower value in the first position of the base table, i.e. the table which is used for q=50. With lower qualities this value scales up. This delays color banding from q=50 to roughly say q=40, after that the same effect is appears.