FR: Colour saturation on Kobo Libra Colour

[tymmej]

Does your feature request involve difficulty completing a task? Please describe.
Kobo released new devices (#11737) with Kaleido 3 screen, same as in PocketBook InkPad Color 3.

Someone made comparison at https://kurotimedesign.blogspot.com/2024/05/kobo-libra-colour-pocketbook-inkpad-3.html

Colors on Libra looks washed out comparing to PocketBook, but they are using same screen.

I wonder if it’s possible to add some saturation/vibrance to Kobo.

I assume currently colors on stock app in Kobo looks the same as on Koreader on Kobo and Koreader on PocketBook pre-#10446 (#10446 (comment))

Looking at #10446 by “default” PocketBook is also washed out, but it offers some API to improve colors. If I understand correctly it’s only available on PocketBooks as it calls some stock API.

Describe the solution you'd like
Any method to increase saturation of colours.

Describe alternatives you've considered
CSS filters, as in #10336 but it looks like they are not supported. Even if, it would only affects books, not whole UI.

Additional context
None

[Frenzie]

MuPDF (as well as DjVu) also has a contrast setting btw.

[pazos]

Someone made comparison at https://kurotimedesign.blogspot.com/2024/05/kobo-libra-colour-pocketbook-inkpad-3.html
Colors on Libra looks washed out comparing to PocketBook, but they are using same screen.

Yup. That's pretty bad.

I wonder if it’s possible to add some saturation/vibrance to Kobo.

No idea. There're no kernel sources yet.

If Nickel offers a feature to change contrast or color saturation we can mimic that in KO. In that case please strace a nickel session and attach the output here.

Maybe the answer is a few FW upgrades ahead :)

[tymmej]

I made some quick and dirty PoC (slow, each refresh of screen adds more saturation):

local Math = require("optmath")

local function _fix_colors(bb, x, y, w, h)

    local change = 1.5

    local Pr = 0.299
    local Pg = 0.587
    local Pb = 0.114

    for vy=y, h-1 do
        for vx=x, w-1 do
            pix = bb:getPixelP(vx, vy)[0]:getColorRGB32()
            org = pix
            org.r = pix.r
            org.g = pix.g
            org.b = pix.b
            local P = math.sqrt(pix.r * pix.r * Pr + pix.g * pix.g * Pg + pix.b * pix.b * Pb)

            pix.r = Math.clamp(P+((pix.r)-P)*change, 0, 255)
            pix.g = Math.clamp(P+((pix.g)-P)*change, 0, 255)
            pix.b = Math.clamp(P+((pix.b)-P)*change, 0, 255)

            bb:setPixel(vx, vy, pix)
        end
    end
end

add call _fix_colors(bb, x, y, w, h) in mxc_update function in ffi/framebuffer_mxcfb.lua in line 2.

Algorithm is from first result in google: https://alienryderflex.com/saturation.html

Device with cover is unmodified, without modified. Both 50% brightness.

#8872 (comment) give some pointers.

I think there's some improvement, especially in first cover (https://www.publio.pl/czysty-wymysl-jak-japonska-popkultura-podbila-swiat-matt-alt,p1299512.html).

I left ghosting visible because each refresh increase saturation.

[NiLuJe]

If Nickel offers a feature to change contrast or color saturation we can mimic that in KO. In that case please strace a nickel session and attach the output here.

AFAICT, they don't. (It doesn't even dither anymore on color panels, despite the driver being able to do that perfectly well via the media accelerator o_O).

There's a "reduce rainbow effect" toggle that supposedly does something (they have started bundling OpenCV), but I clearly haven't seen it do anything yet. I don't even know in which context it's supposed to do something. It certainly doesn't seem to do anything on either highlights or in-KePub images.

[NiLuJe]

@tymmej: Haven't had time to play with it much, but when testing in FBInk when using the proper waveform mode (GCC16) it's very subtle, and boosting the sat too much very easily ruins contrast (while still not really netting you much more "pop" ;p).

I'll try some more stuff during the week (including the aforementioned FBInk experiment, which was just a quick'n dirty PoC that hasn't been merged).

[NiLuJe]

Okay, pushed the same HSP algo in FBInk to ease experimentation.

I'm.... torn. On most of the covers I tried, the effect is fairly subtle at +50%, and starts actively murdering content by eating contrasts over that.

On the other hand, I've seen a very very few examples where +75% did noticeably improve things. I've also seen a few examples where +50% was already murdering details (stuff like the TV tie in cover of The Handmaid's Tale being a fairly good example of that).

Soooo, yeaaah. Dunno.

FWIW, test conditions (which match how images will be rendered in KOReader on those devices; which differs from what was tested by the OP as the PR was incomplete at the time):

./fbink -W GCC16 -f -i image.png -D FS
vs.
./fbink -W GCC16 -f -i image.png -D FS -J 50 (or -J 75 or -J 100 for shits & giggles).

(It also happens to be a ~100ms pass because floating point, which doesn't help endear it to me :/).

[NiLuJe]

(stuff like the TV tie in cover of The Handmaid's Tale being a fairly good example of that)

Possibly related to the fact that reds don't appear to be Kaleido's strong suit.

I mean, it does red better than it does yellow (which is a fairly low bar, because, let's face it: it doesn't really do yellow at all :D).

Greens appear to be somewhat salvageable when boosting saturation.

Blue is... meh. More so than red.

(I'm no color science guy, but this strangely correlates with how well humans perceive colors: i.e., we're ace at distinguishing greens).

[tymmej]

I know that Kaleido screens are not the best at colors and boosting by more than 50% usually kills image (I tested it by modifying framebuffer_linux first).

From my limited tests it looks like low values has no downsides besides refresh time.

I have no hope that image will look as good as OLED screen ;) but boosting slightly seems to be a good idea.

Maybe we can try a more sophisticated algorithm? Looking at linked comment (“change saturation and value of bitmap pixels as in HVS color representation”, “adjustAreaWithLabColorsSaturation”, “adjustAreaWithVibrance”) I feel that PocketBooks modifies displayed data, not change screen settings.

it would be nice if someone with PocketBook could check how long ‘adjustAreaDefault’ executes.

[NiLuJe]

FWIW, the original tests I did were manual with fancier (or... not? Apparently HSV isn't the silver bullet it's supposed to be for that use-case?). (Specifically, ImageMagick's -modulate and a bunch of GIMP color thingies).

Given the extremely limited color gamut of these screens, I doubt we'd actually see much difference in the end result, so my priority for choosing an algo would be ease of implementation & performance ;).

[tymmej]

By the way, are you sure you implemented clamping correctly in FBInk?

https://github.com/NiLuJe/FBInk/blob/9c61a1a5e85dd14a4dc2f692ae83b0f5765ab50d/fbink.c#L10753

R, G, B can be negative (at least they were in lua) and it’s not obvious to me that casting to uint8_t will handle it correctly.

[NiLuJe]

can be negative

Can it?

I was assuming it couldn't because a bunch of multiplications; but I'm severely maths-impaired, so ^^.

[tymmej]

In lua they were, if two components were close to 255, and third close to 0.

here’s subtraction :) https://github.com/NiLuJe/FBInk/blob/9c61a1a5e85dd14a4dc2f692ae83b0f5765ab50d/fbink.c#L10749

[NiLuJe]

Huh.

Brute-forcing it seems to confirm that, yeah.

#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <math.h>

int main(void) {
#	define Pr 0.299f
#	define Pg 0.587f
#	define Pb 0.114f
#	define change 1.5f

	for (uint16_t r = 0; r <= UINT8_MAX; r++) {
		for (uint16_t g = 0; g <= UINT8_MAX; g++) {
			for (uint16_t b = 0; b <= UINT8_MAX; b++) {
				float R = r;
				float G = g;
				float B = b;

				float P = sqrtf(R * R * Pr + G * G * Pg + B * B * Pb);

				R = P + (R - P) * change;
				G = P + (G - P) * change;
				B = P + (B - P) * change;

				if (R > UINT8_MAX) {
					printf("R overflow @ %02X%02X%02X (%g)\n", r, g, b, R);
				} else if (R < 0) {
					printf("R underflow @ %02X%02X%02X (%g)\n", r, g, b, R);
				}

				if (G > UINT8_MAX) {
					printf("G overflow @ %02X%02X%02X (%g)\n", r, g, b, G);
				} else if (G < 0) {
					printf("G underflow @ %02X%02X%02X (%g)\n", r, g, b, G);
				}

				if (B > UINT8_MAX) {
					printf("B overflow @ %02X%02X%02X (%g)\n", r, g, b, B);
				} else if (B < 0) {
					printf("B underflow @ %02X%02X%02X (%g)\n", r, g, b, B);
				}
			}
		}
	}

	return EXIT_SUCCESS;
}

[tymmej]

I have two ideas for optimization:
create full LUT. Kaleido has 4096 colors, assuming 4 bits per channel size should be reasonable
LUT[R>>4][G>>4][B>>4]=…

[NiLuJe]

Unfortunately we don't actually ever see the quantized palette there (we leave dithering to the HW), so we operate on the full RGB scale, which makes this a tad more complex, AFAICT?

On a somewhat related note, as a quick fun fact: the 8x8 ordered dither we usually use when we do handle dithering ourselves is extremely effective at making the CFA grid pattern very apparent, so it wouldn't be desirable here.

Other than that, yeah, the theory was sound: quantize/dither each component down to 16 shades like we usually do for grayscale, and you get 16 ^ 3 = 4096 for the highly theoretical color gamut of Kaleido ;). I was actually happily surprised the media accelerator's dithering pass handled it this well (even in what I think is supposed to be an ordered dither pass. We instead opt for a Floyd-Steinberg error diffusion dither, though. [For context, the HW exposes three different dithering options: not-a-dither-at-all (i.e., plain quantize), Ordered & Floyd-Steinberg]).

[tymmej]

The other idea I have is to make “fast path” for RGB(0,0,0) and RGB(255,255,255) which should help when image isn’t full screen.

But that’s just ideas for later.

[NiLuJe]

The other idea I have is to make “fast path” for RGB(0,0,0) and RGB(255,255,255) which should help when image isn’t full screen.

But that’s just ideas for later.

That's less of a problem ;).

Outside of PDF/CBZs, we'd only be processing the actual image content.

[tymmej]

https://www.reddit.com/r/kobo/comments/1csdy4d/if_you_think_your_libra_colour_looks_washed_out/l44l1d5/

Looks like there's some option in developer menu.

(I'm not talking about VCOM, but cfa mode)

[NiLuJe]

Welp, shame on me. I saw it, thought that wouldn't do anything so I never even tried it ;p.

Turns out, it does, and we can make use of it easily.

I'm going to expose this in FBInk to ease testing.

[NiLuJe]

Okay, pushed that to FBInk ;).

I've renamed the SW sat boost mentioned earlier to -j, --saturation, and this is controlled via -J, --cfa.

Looks like G2 roughly matches a 50% sat boost, so it does look like a fairly safe bet (I mean, I can find content where it gets a little bit too destructive, in which case the software HSP approach looks ever so slightly better [while still mangling parts of the content, mind you], but it generally does more good than harm ;)).