Björn Ottosson

How software gets color wrong

2020-12-03T01:00:00+01:00

How software gets color wrong

Most software around us today are decent at accurately displaying colors. Processing of colors is another story unfortunately, and is often done badly.

To understand what the problem is, let’s start with an example of three ways of blending green and magenta:

Perceptual blend – A smooth transition using a model designed to mimic human perception of color. The blending is done so that the perceived brightness and color varies smoothly and evenly.

Linear blend – A model for blending color based on how light behaves physically. This type of blending can occur in many ways naturally, for example when colors are blended together by focus blur in a camera or when viewing a pattern of two colors at a distance.

sRGB blend – This is how colors would normally be blended in computer software, using sRGB to represent the colors.

The example only reads correctly if you have a reasonably calibrated display and trichromatic color vision.

As you can see, the sRGB example looks quite different. As the colors blend they shift to be much darker and bluer. Why is the standard way for blending colors different from both human perception and the physical behavior of light?

As it turns out, the reason is that the color representation used in most software is based on how CRT displays in the 1990s worked. How color processing works is just a consequence of that representation. Because of this, working with color in software is harder than it needs to be and there are many unnecessary pitfalls working with image processing.

The rest of this post will cover the basics of how color perception works, how it is modelled in software and how almost all software, doing any sort of color manipulation, gets color wrong.

In a series of follow up posts I will look at better ways of doing processing with color.

Introduction to color perception and color management

Skip this section if you are familiar with things like cone responses and the CIE 1931 XYZ color space

Color is a peculiar thing. Everyday and all around us we perceive color. It wouldn’t be odd to expect color to be simple to explain and that relating it to the world around us would be easy. Instead color perception is a complicated process that takes a spectrum of light through complex and non-linear transformations. Thankfully, centuries of research has produced a decent understanding of this process and models for predicting perception exist.

One key result is that our eyes don’t measure the full spectrum of light. Photoreceptor cells in the eye react to certain ranges of wavelengths, roughly corresponding to what we perceive as colors red, green and blue. The cells reacting to color are called cones. The different types of cones are referred to based on what wavelengths they are sensitive to: long, medium and short or LMS – long corresponds to red, medium to green and short to blue.

Light sensitivity for L, M and S cones for different wavelengths

A consequence of this is that many different light spectrums can result in the same perceived color, as long as the LMS response is the same. This is the basis for many everyday things: e.g. color displays produce a wide range of colors by combining red, green and blue light. Print works similarly by using cyan, magenta and yellow ink and color paint is made by mixing various pigments to achieve the desired color.

This also makes it possible to describe colors mathematically, by describing it as a mix of three primary colors. A standard exists for this called the CIE 1931 XYZ color space, which represents colors as three values: X, Y and Z

Another important result about human perception is that the mapping between perceptual qualities such as hue, brightness, colorfulness, etc. is not a fundamental property of light. The only way to numerically model them is by fitting non-linear models empirically to match human perception or by simulating human biology.

Representing color in software

To describe colors in software, conventions for describing colors with numerical values is needed. A specific way of mapping colors to numbers is called color space. Normally color spaces use three numbers to describe colors. RGB color spaces are the most common, with the three values representing red, green and blue light.

To describe a typical RGB color spaces a few things are needed: a set of red, green and blue primary colors and a non linear transfer function (sometimes referred to as gamma correction function). To represent a color in a color space two steps are performed:

The light intensities for each of the three primary colors are calculated so that the perceived color matches the desired color. If the desired color is already represented as intensities of another set primary colors (such as the standard CIE 1931 XYZ color space), this transformation can be done with a 3x3 matrix multiplication:

$\begin{pmatrix} R_{linear} \\ G_{linear} \\ B_{linear} \end{pmatrix} = \mathbf{M} \times \begin{pmatrix} X \\ Y \\ Z \end{pmatrix}$

The calculated intensity for each primary is normalized and transformed using the non linear transfer function.

$\begin{pmatrix} R \\ G \\ B \end{pmatrix} = \begin{pmatrix} f(R_{linear}) \\ f(G_{linear}) \\ f(B_{linear}) \end{pmatrix}$

A common choice of the non linear transfer function is a power function: $f(x) = x^\gamma$ , with $\gamma = 1/2.2$ .

Examples of color spaces structured like this are: sRGB, Adobe RGB, P3, Rec2020 (although they don’t all use a power function).

Color gamut

The choice of which RGB primaries to use affects what portion of visible colors can be represented, referred to as color gamut. Choosing a wider color gamut means that more colors can be represented, but it also means that more data is needed to store colors with a high precision and that normal displays might not be able to show all the colors.

Non-linear transfer function

The reason for the non linear transfer function existing is a bit odd. Many of the commonly used color spaces today were originally designed with CRT displays in mind, or are derived from color spaces that were, and in CRT displays the relationship between voltage and brightness is non-linear. The transfer function is added to encode the colors so that they can be used directly to drive a CRT display, without additional processing.

A happy accident is that the same transfer function somewhat correlates with human perception of brightness. As a result colors encoded for CRT displays can be stored with fewer bits than ones stored linearly, which makes the encoding useful even though CRT screens are rarely used today.

Color management

To consistently reproduce color across devices, information about what color spaces are used needs to be encoded and color translated to the right color space for a device. This is referred to as color management.

Over the last couple of decades a lot of effort has gone into color management and this now works decently. Many displays today are calibrated fairly well out of the factory and a lot of software is built to correctly convert between color spaces as needed. Images typically have embedded color space information (using ICC profiles) which allows them to be accurately reproduced.

For applications not actively managing color sRGB has become the standard. This way, as long as the device and operating system correctly manage color, even applications without any explicit color management will work consistently. The web is a great example of this. Whenever you see a color defined as #DAF7A6, rgb(85%, 97%, 65%) or hsl(118, 52%, 65%), you are probably looking at a color in sRGB.

While there still are things to improve, color management overall is a success story.

Desirable operations with color

Before diving into how colors are processed in software today, let’s look at what types of operations with color make sense.

Common use cases for doing operations involving color in software are things like: realistically rendering images, stylizing photos, digital painting, manipulating photos, picking colors in a color picker, blending transparent elements and making color gradients etc.

To support that I can see three types of operations:

Matching the behavior of real light and materials. E.g. Add artificial lens blur, add a layer of fog over an image, simulate light bouncing of a material, etc.
Matching the human perception of color, to achieve intuitive and predictable control. E.g. Make a color more colorful, while keeping the hue and lightness constant.
Stylized operations, producing interesting results but not grounded in realism or human perception. E.g. Operations like cross processing photographic film.

Color processing in software today

Given what we know about color, it would seem reasonable that process colors is done using representations designed to either mimic the real world interaction of color or human perception. Is this commonly done? Unfortunately not. Instead color spaces designed for reproducing colors are used for doing calculation with them. This is where the problems start appearing.

The most common approach is to simply use sRGB. In color managed software (such as Photoshop), the same color spaces are typically used for doing operations as for storing colors. This means that how operations with colors behave can be different from image to image, depending on the color space. It also means that changing color space to something that behaves better is possible, but this is not something most users do and it is hard to understand.

The result is that color processing, as normally implemented, neither capture how colors behave in the real world nor how humans perceive them – it is just a byproduct of how the color space has been defined! This why image processing behavior today is determined by how CRT displays used to work in the 90s.

Not all bad

To be fair things are slightly better than this in some specialized image processing software. Over the years ad-hoc functionality has been added here and there that better match human perception or the behavior of light. I’m also sure the developers are aware of the underlying issues, but taking a large step is probably impossible for them, since they need to keep backwards compatibility.

There is also one class of software that does this fairly well: Software for rendering realistic 3D images. Here the accuracy matters much more since they need to mimic the interaction of light to achieve acceptable results.

Two main approaches exist: doing computation using the RGB primaries of a color space, but without the non-linear transfer function and fully simulating the spectrum of light. The spectral approach is of course the most accurate, but also more costly. The RGB approach is completely accurate for operations and has been proven to be a reasonable trade-off practically.

Some studies have been made to compare rendering using linear RGB color spaces and full spectrums:

Comparisons

Let’s look at some more examples of blending of colors, to see how these problems surface more practically. The examples use strong colors since then the differences are more pronounced. This is using the same three ways of blending colors as the first example.

White and green

White and yellow

White and red

White and blue

White and black

Blue and yellow

Cyan and red

Green and magenta

As the examples show, the differences are quite significant:

Neither sRGB nor linear are able to blend colors with white and keep a constant perception of hue.
sRGB darkens colors significantly when mixing saturated opposing colors
sRGB and perceptual color spaces blend white and black similarly, while linear blending is much brighter. This is the reason sRGB sometimes is referred to being a perceptual color space, despite its other shortcomings.

Overall sRGB does not match color perception nor the physical behavior of light, which makes it unsuitable for image processing.

What about HSV and HSL?

For picking colors and sometimes editing images another representation is often used: HSL (hue, saturation, lightness) and HSV (hue, saturation, value). They are not actually color spaces, they are ways of remapping RGB color spaces to parameters that are easier to work with. This means they share the properties of the RGB color spaces they are based on, which also means they share their weaknesses. HSL and HSV representations of sRGB in particular are everywhere – almost all color pickers are based on this.

As an example, here is one by Google, that you can find by googling “color picker”:

This blue color is again a clear example of a hue shift in sRGB. From the top right corner the hue shift to purple as the color gets lighter to the left, despite the HSL hue value staying constant.

If a perceptual model was used instead, the same color picker would look something like this:

What can we do?

The solution is simple at a high level: instead of relying on what color space a file or display uses, color operations should be done with the use case in mind, to either model human perception or the physical behavior of light.

Unfortunately doing this well in practice is quite complicated, which is probably part of the the reason it isn’t done often. Getting started is easier though.

One easy thing to do is to use a linear version of sRGB when doing any sort of blending, blurring, resizing or transparency blending. This can be done simply by applying the inverse of the sRGB nonlinear transform function, doing computations, then switching back. For colors in sRGB in range 0.0 to 1.0 this can be done by applying these functions component wise (provided in C-like pseudocode):

float f(float x)
{
    if (x >= 0.0031308)
        return (1.055) * x^(1.0/2.4) - 0.055
    else
        return 12.92 * x
}

float f_inv(float x)
{
    if (x >= 0.04045)
        return ((x + 0.055)/(1 + 0.055))^2.4
    else 
        return x / 12.92
}

The same thing can also be done with color management libraries where available, which also helps when dealing with more color spaces.

Perceptual representations for image processing is a less explored area. There are models aimed at understanding color appearance that can be used for processing as well, but doing so isn’t without issues. A common problem is that models with high accuracy, also tend to be complicated, and when using them for image processing the results can be hard to predict for a user. This post has used OSA-UCS, but it also has drawbacks. The largest issue is that the conversion to the representation doesn’t have a closed-form inverse. More work is needed to find a suitable perceptual representation for image processing.

Conclusion

Instead of making it as easy as possible to work with color, most software make it unnecessarily hard, by doing image processing with representations not designed for it. Approximating the physical behavior of light with linear RGB models is one easy thing to do, but more work is needed to create image representations tailored for image processing and human perception.

In a series of future posts I will look more practically at how colors can be represented for image processing and how a better color picker can be designed.

The images in this post have been generated using python, jupyter, numpy, matplotlib, colorio and colour.
The perceptual color blending has been done with OSA-UCS

A perceptual color space for image processing

2020-12-23T01:00:00+01:00

From personal project to industry standard

Introduction added in 2025

When introduced Oklab in 2020, I never expected it to reach as far as it has. In a few years Oklab has, among other things, found its way into:

Photoshop – Now the default interpolation method for gradients

Web browsers – Part of CSS Color Level 4 and 5, supported by major browsers

Game engines – Used in Unity’s gradients and Godot’s color picker

Open source libraries – Too many to list! For python, I recommend colour.

I developed Oklab in my free time and the results are freely available and open source. Want to support my work? Here are some options:

I’m available for contracting work in color science, computer graphics, game dev and more.

I’m an indie game developer making Island Architect, a cozy town building game. Consider sharing and wishlisting. It is of course using Oklab here and there!

Below you find the original blog post introducing Oklab, from 2020.

A perceptual color space for image processing

A perceptual color space is desirable when doing many kinds of image processing. It is useful for things like:

Turning an image grayscale, while keeping the perceived lightness the same
Increasing the saturation of colors, while maintaining perceived hue and lightness
Creating smooth and uniform looking transitions between colors

Unfortunately, as far as I am aware, while there are color spaces that aim to be perceptually uniform, none are without significant drawbacks when used for image processing.

For this reason I have designed a new perceptual color space, designed to be simple to use, while doing a good job at predicting perceived lightness, chroma and hue. It is called the Oklab color space, because it is an OK Lab color space.

Before diving into the details of why a new color space is needed and how it was derived, here is the everything needed to use the color space:

The Oklab color space

A color in Oklab is represented with three coordinates, similar to how CIELAB works, but with better perceptual properties. Oklab uses a D65 whitepoint, since this is what sRGB and other common color spaces use. The three coordinates are:

$L$ – perceived lightness
$a$ – how green/red the color is
$b$ – how blue/yellow the color is

For many operations, $Lab$ -coordinates can be used directly, but they can also be transformed into polar form, with the coordinates lightness, chroma and hue, $LCh$ :

$C={\sqrt {a^2+b^2}}, \qquad h^{\circ}=\text{atan2}(b,a)$

From $C$ and $h^{\circ}$ , $a$ and $b$ can be computed like this:

$a=C\cos(h^{\circ}),\qquad b=C\sin(h^{\circ})$

Lets look at a practical example to see how Oklab performs, before looking at how the $Lab$ coordinates are computed.

Comparing Oklab to HSV

Here’s an Oklab color gradient with varying hue and constant lightness and chroma.

Compare this to a similar plot of a HSV color gradient with varying hue and constant value and saturation (HSV using the sRGB color space).

The gradient is quite uneven and there are clear differences in lightness for different hues. Yellow, magenta and cyan appear much lighter than red and blue.

Here is lightness of the HSV plot, as predicted by Oklab:

Implementation

Converting from XYZ to Oklab

Given a color in $XYZ$ coordinates, with a D65 whitepoint and white as Y=1, Oklab coordinates can be computed like this:

First the $XYZ$ coordinates are converted to an approximate cone responses:

$\begin{pmatrix} l \\ m \\ s \end{pmatrix} = \mathbf{M_1} \times \begin{pmatrix} X \\ Y \\ Z \end{pmatrix}$

A non-linearity is applied:

$\begin{pmatrix} l' \\ m' \\ s' \end{pmatrix} = \begin{pmatrix} l^{\frac 1 3} \\ m^{\frac 1 3} \\ s^{\frac 1 3} \end{pmatrix}$

Finally, this is transformed into the $Lab$ -coordinates:

$\begin{pmatrix} L \\ a \\ b \end{pmatrix} = \mathbf{M_2} \times \begin{pmatrix} l' \\ m' \\ s' \end{pmatrix}$

with the following values for $\mathbf{M_1}$ and $\mathbf{M_2}$ :

$\mathbf{M_1} = \begin{pmatrix} +0.8189330101 & +0.3618667424 & -0.1288597137 \\ +0.0329845436 & +0.9293118715 & +0.0361456387 \\ +0.0482003018 & +0.2643662691 & +0.6338517070 \end{pmatrix}$

$\mathbf{M_2} = \begin{pmatrix} +0.2104542553 & +0.7936177850 & -0.0040720468 \\ +1.9779984951 & -2.4285922050 & +0.4505937099 \\ +0.0259040371 & +0.7827717662 & -0.8086757660 \end{pmatrix}$

The inverse operation, going from Oklab to XYZ is done with the following steps:

$\begin{pmatrix} l' \\ m' \\ s' \end{pmatrix} = \mathbf{M_2}^{-1} \times \begin{pmatrix} L \\ a \\ b \end{pmatrix},\qquad \begin{pmatrix} l \\ m \\ s \end{pmatrix} = \begin{pmatrix} {(l')}^{3} \\ {(m')}^{3} \\ {(s')}^{3} \end{pmatrix},\qquad \begin{pmatrix} X \\ Y \\ Z \end{pmatrix} = \mathbf{M_1}^{-1} \times \begin{pmatrix} l \\ m \\ s \end{pmatrix}$

Table of example XYZ and Oklab pairs

Provided to test Oklab implementations. Computed by transforming the XYZ coordinates to Oklab and rounding to three decimals.

X Y Z L a b

0.950 1.000 1.089 1.000 0.000 0.000

1.000 0.000 0.000 0.450 1.236 -0.019

0.000 1.000 0.000 0.922 -0.671 0.263

0.000 0.000 1.000 0.153 -1.415 -0.449

X	Y	Z	L	a	b
0.950	1.000	1.089	1.000	0.000	0.000
1.000	0.000	0.000	0.450	1.236	-0.019
0.000	1.000	0.000	0.922	-0.671	0.263
0.000	0.000	1.000	0.153	-1.415	-0.449

Converting from linear sRGB to Oklab

Since this will be a common use case, here is the code to convert linear sRGB values to Oklab and back. To compute linear sRGB values, see my previous post.

The code is in C++, but without any fancy features so should be easy to translate. The code is available in public domain, feel free to use it any way you please. It is also available under an MIT licensee if you for some reason can’t or don’t want to use public domain software. The license text is available here

struct Lab {float L; float a; float b;};
struct RGB {float r; float g; float b;};

Lab linear_srgb_to_oklab(RGB c) 
{
    float l = 0.4122214708f * c.r + 0.5363325363f * c.g + 0.0514459929f * c.b;
	float m = 0.2119034982f * c.r + 0.6806995451f * c.g + 0.1073969566f * c.b;
	float s = 0.0883024619f * c.r + 0.2817188376f * c.g + 0.6299787005f * c.b;

    float l_ = cbrtf(l);
    float m_ = cbrtf(m);
    float s_ = cbrtf(s);

    return {
        0.2104542553f*l_ + 0.7936177850f*m_ - 0.0040720468f*s_,
        1.9779984951f*l_ - 2.4285922050f*m_ + 0.4505937099f*s_,
        0.0259040371f*l_ + 0.7827717662f*m_ - 0.8086757660f*s_,
    };
}

RGB oklab_to_linear_srgb(Lab c) 
{
    float l_ = c.L + 0.3963377774f * c.a + 0.2158037573f * c.b;
    float m_ = c.L - 0.1055613458f * c.a - 0.0638541728f * c.b;
    float s_ = c.L - 0.0894841775f * c.a - 1.2914855480f * c.b;

    float l = l_*l_*l_;
    float m = m_*m_*m_;
    float s = s_*s_*s_;

    return {
		+4.0767416621f * l - 3.3077115913f * m + 0.2309699292f * s,
		-1.2684380046f * l + 2.6097574011f * m - 0.3413193965f * s,
		-0.0041960863f * l - 0.7034186147f * m + 1.7076147010f * s,
    };
}

The matrices were updated 2021-01-25. The new matrices have been derived using a higher precision sRGB matrix and with exactly matching D65 values. The old matrices are available here for reference. The values only differ after the first three decimals.

Depending on use case you might want to use the sRGB matrices your application uses directly instead of the ones provided here.

This is everything you need to use the Oklab color space! If you need a simple perceptual color space, try it out.

The rest of the post will go into why a new color space was needed, how it has been constructed and how it compares with existing color spaces.

Motivation and derivation of Oklab

What properties does a perceptual color space need to satisfy to be useful for image processing? The answer to this is always going to be a bit subjective, but based on my experience, these are a good set of requirements:

Should be an opponent color space, similar to for example CIELAB.

Should predict lightness, chroma and hue well. $L$ , $C$ and $h$ should be perceived as orthogonal, so one can be altered without affecting the other two. This is useful for things like turning an image black and white and increasing colorfulness without introducing hue shifts etc.

Blending two colors should result in even transitions. The transition colors should appear to be in between the blended colors (e.g. passing through a warmer color than either original color is not good).

Should assume a D65 whitepoint. This is what common color spaces like sRGB, rec2020 and Display P3 uses.

Should behave well numerically. The model should be easy to compute, numerically stable and differentiable.

Should assume normal well lit viewing conditions. The complexity of supporting different viewing conditions is not practical in most applications. Information about absolute luminance and background luminance adaptation does not normally exist and the viewing conditions can vary.

If the scale/exposure of colors are changed, the perceptual coordinates should just be scaled by a factor. To handle a large dynamic range without requiring knowledge of viewing conditions all colors should be modelled as if viewed under normal viewing conditions and as if the eye is adapted to roughly the luminance of the color. This avoids a dependence on scaling.

What about existing models?

Let’s look at existing models and how they stack up against these requirements. Further down there are graphs that illustrate some of these issues.

CIELAB and CIELUV – Largest issue is their inability to predict hue. In particular blue hues are predicted badly. Other smaller issues exist as well

CIECAM02-UCS and the newer CAM16-UCS – Does a good job at being perceptually uniform overall, but doesn’t meet other requirements: Bad numerical behavior, it is not scale invariant and blending does not behave well because of its compression of chroma. Hue uniformity is decent, but other models predict it more accurately.

OSA-UCS – Overall does a good job. The transformation to OSA-UCS lacks an analytical inverse unfortunately which makes it impractical.

IPT – Does a great job modelling hue uniformity. Doesn’t predict lightness and chroma well unfortunately, but meets all other requirements. Is simple computationally and does not depend on the scale/exposure.

JzAzBz – Overall does a fairly good job. Designed to have uniform scaling of lightness for HDR data. While useful in some cases this introduces a dependence on the scale/exposure that makes it hard to use in general cases.

HSV representation of sRGB – Only on this list because it is widely used. Does not meet any of the requirements except having a D65 whitepoint.

So, all in all, all these existing models have drawbacks.

Out of all of these, two models stand out: CAM16-UCS, for being the model with best properties of perceptual uniformity overall, and IPT for having a simple computational structure that meets all the requirements besides predicting lightness and chroma well.

For this reason it is reasonable to try to make a new color space, with the same computational structure as IPT, but that performs closer to CAM16-UCS in terms of predicting lightness and chroma. This exploration resulted in Oklab.

How Oklab was derived

To derive Oklab, three datasets were used:

A generated data set of pairs of colors with the same lightness but random hue and chroma, generated using CAM16 and normal viewing conditions. Colors were limited to be within Pointer’s Gamut – the set of possible surface colors.
A generated data set of pairs of colors with the same chroma but random hue and lightness, generated using CAM16 and normal viewing conditions. Colors were limited to be within Pointer’s Gamut
The uniform perceived hue data used to derive IPT. From this data, colors were combined into pairs of colors with equal perceived hue.

These datasets can be used to test prediction of lightness, chroma and hue respectively. If a color space accurately models $L$ , $C$ and $h$ , then all pairs in lightness dataset should have the same value for $L$ , all pairs in the chroma dataset the same value for $C$ and all pairs in the hue dataset the same values for $h$ .

To test a color space it is not possible to simply check the distance in predictions in the tested color space however, since that will depend on the scaling of the color space. It is also not desirable to exactly predict ground truth values for $L$ , $C$ and $h$ , since it is more important that our model has perceptually orthogonal coordinates, than that the model has the same spacing within each coordinate.

Instead the following approach was used to create an error metric independent of the color space:

For each dataset all the pairs are converted to the tested color space.
Coordinates that are supposed to be the same within a pair are swapped to generate a new set of altered pairs:
- For the lightness dataset, the L coordinates are swapped between the pairs, and so on.
- These altered pair would be equal to the original pair if the model predicts the datasets perfectly.
The perceived distance between the original colors and the altered colors are are computed using CIEDE2000.
The error for each pair is given as the minimum of the two color differences.
The error for the entire dataset is the root mean squared error of the color differences.

Oklab was derived by optimizing the parameters of a color space with the same structure as IPT, to get a low error on all the datasets. For completeness, here is the structure of the color space – the parameters to optimize are the 3x3 matrices $\mathbf{M_1}$ and $\mathbf{M_2}$ and the positive number $\gamma$ .

$\begin{pmatrix} l \\ m \\ s \end{pmatrix} = \mathbf{M_1} \times \begin{pmatrix} X \\ Y \\ Z \end{pmatrix}$

$\begin{pmatrix} L \\ a \\ b \end{pmatrix} = \mathbf{M_2} \times \begin{pmatrix} l^\gamma \\ m^\gamma \\ s^\gamma \end{pmatrix}$

A couple of extra constraints were also added, since this error doesn’t alone determine the scale and orientation of the color model.

Positive b is oriented to the same yellow color as CAM16
D65 (normalized with $Y=1$ ) should transform to $L=1, a=0, b=0$ and
The $a$ and $b$ plane is scaled so that around 50% gray the ratio of color differences along the lightness axis and the $a$ and $b$ plane is the same as the ratio for color differences predicted by CIEDE2000.

Using these constraints a fairly good model was found, but based on the results two more changes was made. The $\gamma$ value ended up very close to 1/3, 0.323, and when looking at the sRGB gamut, the blue colors folded in on themselves slightly, resulting in a non-convex sRGB gamut. By forcing the value of $\gamma$ to 1/3 and adding a constraint the blue colors to not fold inwards, the final Oklab model was derived. The error was not noticeably affected by these restrictions.

Comparison with other color spaces

Here are the errors for the three different datasets across color spaces, given both as root mean square error and as the 95th percentile. The best performing result is highlighted in bold in each row (ignoring CAM16 since it is the origin of the test data). Since the lightness and chroma data was generated using CAM16 rather than being data from experiments, this data can’t be used to say which model best matches human perception. What can be said is that Oklab does a good job at predicting hue and its predictions for chroma and lightness are close to those of CAM16-UCS.

	Oklab	CIELAB	CIELUV	OSA-UCS	IPT	JzAzBz	HSV	CAM16-UCS
L RMS	0.20	1.70	1.72	2.05	4.92	2.38	11.59	0.00
C RMS	0.81	1.84	2.32	1.28	2.18	1.79	3.38	0.00
H RMS	0.49	0.69	0.68	0.49	0.48	0.43	1.10	0.59
L 95	0.44	3.16	3.23	4.04	9.89	4.55	23.17	0.00
C 95	1.78	3.96	5.03	2.73	4.64	3.77	7.51	0.00
H 95	1.06	1.56	1.51	1.08	1.02	0.92	2.42	1.31

To be able to include HSV in these comparisons, a Lab-like color space has been defined based on it, by interpreting HSV as a cylindrical color space and converting to a regular grid.

JzAzBz has been used with white scaled so that Y=100. This matches the graphs in the original paper, but it is unclear if this is how it is intended to be used. (See this colorio Github issue for a discussion of the topic)

Munsell data

Here is Munsell color chart data (with V=5) plotted in the various color spaces. If a color space has a chroma prediction that matches that of the Munsell data, the rings formed by the data should appear as perfect circles. The quality of this data is a bit hard to assess, since it isn’t directly using experimental data, it is a color chart created from experimental data back in the 1940s.

Oklab and CAM16-UCS seem to predict the Munsell data well, while other spaces squash the circles in the dataset in various ways, which would indicate that Oklab does a better job than most of the color spaces of predicting chroma.

Oklab

CIELAB

CIELUV

OSA-UCS

IPT

JzAzBz

HSV

CAM16-UCS

Luo-Rigg dataset and full gamut

These plots show three things, ellipses, scaled based on perception of color differences from the Luo-Rigg dataset, the shape of the full visible gamut (the black line corresponds to pure single wavelength lights) and a slice of the sRGB gamut.

A few interesting observations are:

The shape of the full gamut is quite odd in CIELAB and OSA-UCS, which likely means that their predictions are quite bad for highly saturated colors
Except for CAM16-UCS the ellipses are stretched out as chroma increases. CAM16 explicitly compresses chroma to better match experimental data, which makes this data look good, but makes color blending worse

Oklab

CIELAB

CIELUV

OSA-UCS

IPT

JzAzBz

HSV
Data missing. Plot broken since HSV does not handle colors outside sRGB gamut

CAM16-UCS

Blending colors

Here are plots of white blended with a blue color, using the various color spaces tested. The blue color has been picked since it is the hue with the most variation between spaces. CIELAB, CIELUV and HSV all show hue shifts towards purple. CAM16 has a different issue, with the color becoming desaturated quickly, resulting in a transition that doesn’t look as even as some of the other ones.

Oklab

CIELAB

CIELUV

OSA-UCS

IPT

JzAzBz

HSV

CAM16-UCS

Conclusion

This post has introduced the Oklab color space, a perceptual color space for image processing. Oklab is able to predict perceived lightness, chroma and hue well, while being simple and well-behaved numerically and easy to adopt.

In future posts I want to look into using Oklab to build a better perceptual color picker and more.

Oklab and the images in this post have been made using python, jupyter, numpy, scipy matplotlib, colorio and colour.

sRGB gamut clipping

2021-01-25T01:00:00+01:00

sRGB gamut clipping

When doing image processing, rendering or converting between RGB color spaces, it is easy to end up with RGB values outside of the 0.0 to 1.0 range (or 0 to 255 for 8-bit colors). This normally means that it isn’t possible to encode the color and it needs to be adjusted somehow before being displayed or encoded (scRGB is an example of an exception, and has a different range). The range of supported colors in a color spaces is referred to as a its gamut.

Many of the colors outside the gamut are still valid colors, they just can’t be encoded in the given color space. The colors that can’t be encoded can be categorized as follows (for RGB color spaces without imaginary primaries):

One or more RGB values are larger than one, but none are negative
These colors are too bright for the color space, but could be encoded at a lower luminance.
One or more negative values
These color are more colorful than the color space is able to encode. Some of these can be real colors, but they can also be colors more colorful than physically possible. This is a consequence of linear encoding of colors, which is able to encode colors that can’t exist in real life.

The simplest technique for dealing with colors outside the valid range is to simply clamp $R$ , $G$ and $B$ individually. The simplicity of this makes the technique widespread, but unfortunately it has significant drawbacks and can lead to unpleasant color distortions.

Here is an example of taking a photo and increasing its exposure and colorfulness significantly, with results displayed as sRGB. Colorfulness has been increased by scaling Oklab chroma. RGB values here are simply clamped to the 0.0 to 1.0 range.

Unprocessed

Processed with out of range values clamped

This is of course problematic. Hues are heavily distorted and all the details in the flowers are gone. Better methods are needed. Thankfully, this problem isn’t new and has been researched a lot over the years.

Two main research topics exist:

Tone mapping, which is concerned with mapping high dynamic range images to a lower dynamic range. This typically involves altering both colors inside and outside the gamut and sometimes involves different kinds of spatial filtering to keep details while reducing overall dynamic range.

Gamut mapping, which is concerned with mapping colors from one color space to another and dealing with the mismatch in what colors can be encoded. This can either be done using gamut compression or gamut expansion, which adjusts all colors of an image, or using gamut clipping, which leaves colors that exist in both color spaces intact, while mapping colors outside the gamut back in range. For an overview see the paper “The Fundamentals of Gamut Mapping: A Survey”.

Tone mapping, gamut expansion and gamut compression are all useful techniques, but will not be the focus of this article. Instead we will limit the scope to creating a practical alternative to clipping sRGB values, by looking at gamut clipping in the sRGB color space and on relatively simple, robust and portable algorithms. The source code for the operations shown is also available, at the end of the article.

The techniques investigated are also, with some additional work, applicable to other color spaces, and some of the building blocks can be for other things, such as gamut compression. Combining gamut clipping with tone mapping can also be useful since some tone mapping techniques may still produce colors outside the target gamut.

Gamut Clipping

The goal of gamut clipping is to take colors that are outside the target gamut, and map them back to valid colors in a way that avoids perceived distortion and artifacts. In this article we also want to do this in a way that is reasonably efficient and simple to implement. Many techniques require complex iterative algorithms per pixel (or storing the results in 3D lookup tables) – those will not be considered here, but can be useful for many applications.

Instead we will take this commonly used approach:

Work in a perceptual color space.
Keep hue constant, only change lightness and chroma.
Only project color along straight lines in the perceptual color space.

For an overview of different approaches, see the paper The Fundamentals of Gamut Mapping: A Survey.

As a color space, we will use the Oklab perceptual color space, since it models hue, chroma and lightness well while being easy to work with mathematically. In particular, with Oklab, the shape of the gamut of an RGB color space is described by cubic polynomials, which are quite practical to work with compared to the equations you get with other spaces.

Since we have decided to keep hue constant, it is now possible to plot the problem in two dimensions, using lightness, $L$ , and chroma, $C$ . To better understand the shape of the sRGB gamut, let’s look at of slices of constant hue. These are sampled every 30 degrees:

Our goal is to find a way to map colors outside the gamut, shown in grey in the graphs, back into the gamut using straight lines. This problem can be split into two parts:

Deciding along which line to project the color onto the gamut
Finding the intersection between the line and the gamut

Lets start by tackling part 2.

Intersecting lines with the gamut

The math here is quite involved, so this sections just contains an overview of the approach. If that is still too much math, skip to the next section. For more details, see the source code.

Expressed mathematically, the problem we want to solve is: Given a color expressed as $(L_1, C_1)$ and $h^{\circ}$ , and a point $(L_0, C_0)$ that is known to be inside the gamut, we want to find a value $t$ so that $(L_t, C_t) = t (L_1, C_1) + (1-t) (L_0, C_0)$ is a point on the gamut surface. We will also set $C_0=0$ since the projections we want to make later always pass through the $L$ axis.

Using the definition of Oklab here, we get this conversion from $LCh^{\circ}$ to linear $sRGB$ :

$\begin{pmatrix} l' \\ m' \\ s' \end{pmatrix} = \mathbf{M_2}^{-1} \times \begin{pmatrix} L \\ C\cos(h^{\circ}) \\ C\sin(h^{\circ}) \end{pmatrix},\qquad \begin{pmatrix} l \\ m \\ s \end{pmatrix} = \begin{pmatrix} {(l')}^{3} \\ {(m')}^{3} \\ {(s')}^{3} \end{pmatrix},\qquad \begin{pmatrix} r \\ g \\ b \end{pmatrix} = \mathbf{M}_{XYZ\_to\_sRGB} \mathbf{M_1}^{-1} \times \begin{pmatrix} l \\ m \\ s \end{pmatrix}$

The gamut is then defined by the following inequalities:

$0 < r < 1,\qquad 0 < g < 1,\qquad 0 < b < 1$

This is enough information to solve the problem numerically, but involves solving six cubic polynomials. We are after a faster solution that uses more knowledge about the problem.

By studying the gamut graphs and the equations, we can draw a few conclusions:

For a given hue, the lower limit is always a straight line passing through $(0,0)$
The gamut shape is always close to a triangle, with two of its corners in $(1,0)$ and $(0,0)$
If we can find the third point, we know exactly the lower limit and we have good first approximation of the upper limit

The location of the third point, $(L_{cusp}$ , $C_{cusp})$ , only depends on $h^{\circ}$ . To find it we use a fitted numerical approximation followed by a single step of Halley’s method, this gives an error less than $10e6$ (except for one point where the derivative $\partial C_{cusp}/\partial h^{\circ}$ is close to infinite). Since this only depends on $h^{\circ}$ it is also possible to use a 1D lookup table for it.

With the location for the cusp, it is now possible to find the intersection with the triangle. If the intersection is with the lower edge, this is everything we need to do. If the intersection is with the upper half, the triangle is only an approximation and we need to refine the solution. Again, one step of Halley’s method is enough to do this with high accuracy.

Altogether, this is everything needed to find the intersection between a line and the sRGB gamut. To derive the numerical approximation of the location of the cusp, this Google Colab Notebook was used. With same method is should be possible derive approximations for other RGB color spaces. The complete source code is available at the end of this article.

Choosing which line to project along

Starting with a point $(L_1, C_1)$ , we want to decide a point $(L_0, C_0)$ within the gamut to project towards. The choice is a matter of preference and can depend on context.

Let’s start by testing a few simple existing methods and see what the projections look like for different hue slices and when applied the test image from the beginning.

Keep lightness constant, only compress chroma

Lightness is kept constant if it is between zero and one, otherwise it is clamped. This is done by projecting towards $(L_0, C_0) = (clamp(L_1, 0, 1), 0)$

$h^{\circ}=30^{\circ}$

$h^{\circ}=120^{\circ}$

Gamut clipping test

Projection towards a single point, hue independent

Colors are projected towards the middle of the valid greyscale colors: $(L_0, C_0) = (0.5, 0)$

$h^{\circ}=30^{\circ}$

$h^{\circ}=120^{\circ}$

Gamut clipping test

Projection towards a single point, hue dependent

Colors are projected towards the lightness of the cusp of the gamut for the current hue: $(L_0, C_0) = (L_{cusp}, 0)$

$h^{\circ}=30^{\circ}$

$h^{\circ}=120^{\circ}$

Gamut clipping test

Can we do better?

The choice of projection will always be a matter of preference. Overall I think preserving lightness does the best overall of the above methods (and others agree, it is a commonly used technique). One challenge with preserving lightness though is that all colors with $L>1$ or $L<0$ get projected to a single point, and that highly saturated colors can become almost completely desaturated if light or dark enough, even though sacrificing just a small amount of lightness would have resulted in a much closer match.

For this reason I’ve looked into creating new methods that mostly preserve lightness, but behaves closer to the single point projection methods for high and low lightness colors. The result is two new methods. They are novel as far as I am aware, but there are a lot of gamut mapping papers I haven’t read.

The first method is based on projection towards $L=0.5, C=0$ and the second based on $L=L_{cusp}, C=0$ . Both have a parameter $\alpha$ to control to what extent lightness is preserved. $\alpha$ is a real number larger than 0, where values close to zero will behave as pure chroma compression, and larger values will behave closer to the single point projection methods. Here is an overview of the two methods:

Adaptive $L_0$ , hue independent

This method blends between pure chroma compression and projection towards a single point, $L_0=0.5, C_0=0$ .

$L_0$ is computed with the following equations:

$L_d = L_1-0.5$

$e_1 = 1/2 + |L_d| + \alpha C_1$

$L_0 = \frac{1 + sgn(L_d)(e_1 - \sqrt{e_1^2 - 2 |L_d|})} 2 ,\qquad C_0 = 0$

Here are some results with different values for $\alpha$ :

$\large \alpha = 0.05$

$h^{\circ}=30^{\circ}$

$h^{\circ}=120^{\circ}$

Gamut clipping test

$\large \alpha = 0.5$

$h^{\circ}=30^{\circ}$

$h^{\circ}=120^{\circ}$

Gamut clipping test

$\large \alpha = 5$

$h^{\circ}=30^{\circ}$

$h^{\circ}=120^{\circ}$

Gamut clipping test

Adaptive $L_0$ , hue dependent

This method blends between pure chroma compression and projection towards a single point, $L_0=L_{cusp}, C_0=0$

$L_0$ is computed with the following equations:

$L_d = L_1-L_{cusp}$

$k = 2 \begin{cases} (1-L_{cusp}) &\text{if } L_d >= 0 \\ L_{cusp} &\text{if } L_d < 0 \end{cases}$

$e_1 = \frac k 2 + |L_d| + \frac {\alpha C_1} k$

$L_0 = L_{cusp} + \frac {sgn(L_d)(e_1 - \sqrt{e_1^2 - 2 k |L_d|})} 2 ,\qquad C_0 = 0$

Here are some results with different values for $\alpha$ :

$\large \alpha = 0.05$

$h^{\circ}=30^{\circ}$

$h^{\circ}=120^{\circ}$

Gamut clipping test

Overall, doing gamut clipping using Oklab seems like a useful tool, and is fairly easy to implement. The results here though are not enough to determine how it compares with other algorithm or how results are perceived if comparing wide gamut images with clipped versions. Doing more studies and comparisons would be interesting. In particular, should compare the results against methods more complex methods that minimize color distances, such as the ones evaluated in the paper: “Gamut Mapping for Digital Cinema”

Personally, of the methods evaluated here, I think a good default choice would be adaptive $L_0$ with $\alpha = 0.05$ , but in specific cases other methods and $\alpha$ values can definitely perform better. At $\alpha = 0.05$ the choice between $L_0 = 0.5$ and $L_0 = L_{cusp}$ seems to matter very little visually. $L_0 = 0.5$ is then preferable, since it is cheaper to compute, and can be implemented with a single 2D lookup table if desired (From hue and $L_0$ to $L$ and $C$ ).

There is also a lot of potential future work:

Would be nice with a solution that could work directly with RGB matrices, instead of optimizing some constants separately. This way a solutions could be created quickly for any RGB color space.
One issue, that is related to the sRGB gamut rather than the clipping algorithm, is the sharp ridges of high chroma that sometimes get created. A smoothed version of the sRGB gamut could be interesting to clip to instead, producing a more even blend at the expense of some of the highly saturated colors.
There are also a lot of other applications for intersecting lines with the gamut to explore.

My next post will be about creating HSV and HSL like color pickers with improved uniformity, utilizing the gamut intersection algorithm described here.

Source code

All the source code on this page is provided under the MIT license:

Copyright (c) 2021 Björn Ottosson

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Oklab to linear sRGB conversion

Linear sRGB to Oklab conversion, identical to the the Oklab blog post.

This is using linear sRGB values in the range 0.0 to 1.0. To compute linear sRGB values, see my previous post.

struct Lab {float L; float a; float b;};
struct RGB {float r; float g; float b;};

Lab linear_srgb_to_oklab(RGB c)
{
	float l = 0.4122214708f * c.r + 0.5363325363f * c.g + 0.0514459929f * c.b;
	float m = 0.2119034982f * c.r + 0.6806995451f * c.g + 0.1073969566f * c.b;
	float s = 0.0883024619f * c.r + 0.2817188376f * c.g + 0.6299787005f * c.b;

	float l_ = cbrtf(l);
	float m_ = cbrtf(m);
	float s_ = cbrtf(s);

	return {
		0.2104542553f * l_ + 0.7936177850f * m_ - 0.0040720468f * s_,
		1.9779984951f * l_ - 2.4285922050f * m_ + 0.4505937099f * s_,
		0.0259040371f * l_ + 0.7827717662f * m_ - 0.8086757660f * s_,
	};
}

RGB oklab_to_linear_srgb(Lab c)
{
    float l_ = c.L + 0.3963377774f * c.a + 0.2158037573f * c.b;
    float m_ = c.L - 0.1055613458f * c.a - 0.0638541728f * c.b;
    float s_ = c.L - 0.0894841775f * c.a - 1.2914855480f * c.b;

    float l = l_ * l_ * l_;
    float m = m_ * m_ * m_;
    float s = s_ * s_ * s_;

    return {
        +4.0767416621f * l - 3.3077115913f * m + 0.2309699292f * s,
        -1.2684380046f * l + 2.6097574011f * m - 0.3413193965f * s,
        -0.0041960863f * l - 0.7034186147f * m + 1.7076147010f * s,
    };
}

Intersection with sRGB gamut

Implementations of the sRGB gamut intersection algorithm. The code has been written to be easy to follow. Consider optimizing for your use case.

// Finds the maximum saturation possible for a given hue that fits in sRGB
// Saturation here is defined as S = C/L
// a and b must be normalized so a^2 + b^2 == 1
float compute_max_saturation(float a, float b)
{
    // Max saturation will be when one of r, g or b goes below zero.

    // Select different coefficients depending on which component goes below zero first
    float k0, k1, k2, k3, k4, wl, wm, ws;

    if (-1.88170328f * a - 0.80936493f * b > 1)
    {
        // Red component
        k0 = +1.19086277f; k1 = +1.76576728f; k2 = +0.59662641f; k3 = +0.75515197f; k4 = +0.56771245f;
        wl = +4.0767416621f; wm = -3.3077115913f; ws = +0.2309699292f;
    }
    else if (1.81444104f * a - 1.19445276f * b > 1)
    {
        // Green component
        k0 = +0.73956515f; k1 = -0.45954404f; k2 = +0.08285427f; k3 = +0.12541070f; k4 = +0.14503204f;
        wl = -1.2684380046f; wm = +2.6097574011f; ws = -0.3413193965f;
    }
    else
    {
        // Blue component
        k0 = +1.35733652f; k1 = -0.00915799f; k2 = -1.15130210f; k3 = -0.50559606f; k4 = +0.00692167f;
        wl = -0.0041960863f; wm = -0.7034186147f; ws = +1.7076147010f;
    }

    // Approximate max saturation using a polynomial:
    float S = k0 + k1 * a + k2 * b + k3 * a * a + k4 * a * b;

    // Do one step Halley's method to get closer
    // this gives an error less than 10e6, except for some blue hues where the dS/dh is close to infinite
    // this should be sufficient for most applications, otherwise do two/three steps 

    float k_l = +0.3963377774f * a + 0.2158037573f * b;
    float k_m = -0.1055613458f * a - 0.0638541728f * b;
    float k_s = -0.0894841775f * a - 1.2914855480f * b;

    {
        float l_ = 1.f + S * k_l;
        float m_ = 1.f + S * k_m;
        float s_ = 1.f + S * k_s;

        float l = l_ * l_ * l_;
        float m = m_ * m_ * m_;
        float s = s_ * s_ * s_;

        float l_dS = 3.f * k_l * l_ * l_;
        float m_dS = 3.f * k_m * m_ * m_;
        float s_dS = 3.f * k_s * s_ * s_;

        float l_dS2 = 6.f * k_l * k_l * l_;
        float m_dS2 = 6.f * k_m * k_m * m_;
        float s_dS2 = 6.f * k_s * k_s * s_;

        float f  = wl * l     + wm * m     + ws * s;
        float f1 = wl * l_dS  + wm * m_dS  + ws * s_dS;
        float f2 = wl * l_dS2 + wm * m_dS2 + ws * s_dS2;

        S = S - f * f1 / (f1*f1 - 0.5f * f * f2);
    }

    return S;
}

// finds L_cusp and C_cusp for a given hue
// a and b must be normalized so a^2 + b^2 == 1
struct LC { float L; float C; };
LC find_cusp(float a, float b)
{
	// First, find the maximum saturation (saturation S = C/L)
	float S_cusp = compute_max_saturation(a, b);

	// Convert to linear sRGB to find the first point where at least one of r,g or b >= 1:
	RGB rgb_at_max = oklab_to_linear_srgb({ 1, S_cusp * a, S_cusp * b });
	float L_cusp = cbrtf(1.f / max(max(rgb_at_max.r, rgb_at_max.g), rgb_at_max.b));
	float C_cusp = L_cusp * S_cusp;

	return { L_cusp , C_cusp };
}

// Finds intersection of the line defined by 
// L = L0 * (1 - t) + t * L1;
// C = t * C1;
// a and b must be normalized so a^2 + b^2 == 1
float find_gamut_intersection(float a, float b, float L1, float C1, float L0)
{
	// Find the cusp of the gamut triangle
	LC cusp = find_cusp(a, b);

	// Find the intersection for upper and lower half seprately
	float t;
	if (((L1 - L0) * cusp.C - (cusp.L - L0) * C1) <= 0.f)
	{
		// Lower half

		t = cusp.C * L0 / (C1 * cusp.L + cusp.C * (L0 - L1));
	}
	else
	{
		// Upper half

		// First intersect with triangle
		t = cusp.C * (L0 - 1.f) / (C1 * (cusp.L - 1.f) + cusp.C * (L0 - L1));

		// Then one step Halley's method
		{
			float dL = L1 - L0;
			float dC = C1;

			float k_l = +0.3963377774f * a + 0.2158037573f * b;
			float k_m = -0.1055613458f * a - 0.0638541728f * b;
			float k_s = -0.0894841775f * a - 1.2914855480f * b;

			float l_dt = dL + dC * k_l;
			float m_dt = dL + dC * k_m;
			float s_dt = dL + dC * k_s;

			
			// If higher accuracy is required, 2 or 3 iterations of the following block can be used:
			{
				float L = L0 * (1.f - t) + t * L1;
				float C = t * C1;

				float l_ = L + C * k_l;
				float m_ = L + C * k_m;
				float s_ = L + C * k_s;

				float l = l_ * l_ * l_;
				float m = m_ * m_ * m_;
				float s = s_ * s_ * s_;

				float ldt = 3 * l_dt * l_ * l_;
				float mdt = 3 * m_dt * m_ * m_;
				float sdt = 3 * s_dt * s_ * s_;

				float ldt2 = 6 * l_dt * l_dt * l_;
				float mdt2 = 6 * m_dt * m_dt * m_;
				float sdt2 = 6 * s_dt * s_dt * s_;

				float r = 4.0767416621f * l - 3.3077115913f * m + 0.2309699292f * s - 1;
				float r1 = 4.0767416621f * ldt - 3.3077115913f * mdt + 0.2309699292f * sdt;
				float r2 = 4.0767416621f * ldt2 - 3.3077115913f * mdt2 + 0.2309699292f * sdt2;

				float u_r = r1 / (r1 * r1 - 0.5f * r * r2);
				float t_r = -r * u_r;

				float g = -1.2684380046f * l + 2.6097574011f * m - 0.3413193965f * s - 1;
				float g1 = -1.2684380046f * ldt + 2.6097574011f * mdt - 0.3413193965f * sdt;
				float g2 = -1.2684380046f * ldt2 + 2.6097574011f * mdt2 - 0.3413193965f * sdt2;

				float u_g = g1 / (g1 * g1 - 0.5f * g * g2);
				float t_g = -g * u_g;

				float b = -0.0041960863f * l - 0.7034186147f * m + 1.7076147010f * s - 1;
				float b1 = -0.0041960863f * ldt - 0.7034186147f * mdt + 1.7076147010f * sdt;
				float b2 = -0.0041960863f * ldt2 - 0.7034186147f * mdt2 + 1.7076147010f * sdt2;

				float u_b = b1 / (b1 * b1 - 0.5f * b * b2);
				float t_b = -b * u_b;

				t_r = u_r >= 0.f ? t_r : FLT_MAX;
				t_g = u_g >= 0.f ? t_g : FLT_MAX;
				t_b = u_b >= 0.f ? t_b : FLT_MAX;

				t += min(t_r, min(t_g, t_b));
			}
		}
	}

	return t;
}

Gamut clipping

Implementations of the gamut clipping methods in this article. The code has been written to be easy to follow. Consider optimizing for your use case.


float clamp(float x, float min, float max)
{
	if (x < min)
		return min;
	if (x > max)
		return max;

	return x;
}

float sgn(float x)
{
	return (float)(0.f < x) - (float)(x < 0.f);
}

RGB gamut_clip_preserve_chroma(RGB rgb)
{
	if (rgb.r < 1 && rgb.g < 1 && rgb.b < 1 && rgb.r > 0 && rgb.g > 0 && rgb.b > 0)
		return rgb;

	Lab lab = linear_srgb_to_oklab(rgb);

	float L = lab.L;
	float eps = 0.00001f;
	float C = max(eps, sqrtf(lab.a * lab.a + lab.b * lab.b));
	float a_ = lab.a / C;
	float b_ = lab.b / C;

	float L0 = clamp(L, 0, 1);

	float t = find_gamut_intersection(a_, b_, L, C, L0);
	float L_clipped = L0 * (1 - t) + t * L;
	float C_clipped = t * C;

	return oklab_to_linear_srgb({ L_clipped, C_clipped * a_, C_clipped * b_ });
}

RGB gamut_clip_project_to_0_5(RGB rgb)
{
	if (rgb.r < 1 && rgb.g < 1 && rgb.b < 1 && rgb.r > 0 && rgb.g > 0 && rgb.b > 0)
		return rgb;

	Lab lab = linear_srgb_to_oklab(rgb);

	float L = lab.L;
	float eps = 0.00001f;
	float C = max(eps, sqrtf(lab.a * lab.a + lab.b * lab.b));
	float a_ = lab.a / C;
	float b_ = lab.b / C;

	float L0 = 0.5;

	float t = find_gamut_intersection(a_, b_, L, C, L0);
	float L_clipped = L0 * (1 - t) + t * L;
	float C_clipped = t * C;

	return oklab_to_linear_srgb({ L_clipped, C_clipped * a_, C_clipped * b_ });
}

RGB gamut_clip_project_to_L_cusp(RGB rgb)
{
	if (rgb.r < 1 && rgb.g < 1 && rgb.b < 1 && rgb.r > 0 && rgb.g > 0 && rgb.b > 0)
		return rgb;

	Lab lab = linear_srgb_to_oklab(rgb);

	float L = lab.L;
	float eps = 0.00001f;
	float C = max(eps, sqrtf(lab.a * lab.a + lab.b * lab.b));
	float a_ = lab.a / C;
	float b_ = lab.b / C;

	// The cusp is computed here and in find_gamut_intersection, an optimized solution would only compute it once.
	LC cusp = find_cusp(a_, b_);

	float L0 = cusp.L;

	float t = find_gamut_intersection(a_, b_, L, C, L0);

	float L_clipped = L0 * (1 - t) + t * L;
	float C_clipped = t * C;

	return oklab_to_linear_srgb({ L_clipped, C_clipped * a_, C_clipped * b_ });
}

RGB gamut_clip_adaptive_L0_0_5(RGB rgb, float alpha = 0.05f)
{
	if (rgb.r < 1 && rgb.g < 1 && rgb.b < 1 && rgb.r > 0 && rgb.g > 0 && rgb.b > 0)
		return rgb;

	Lab lab = linear_srgb_to_oklab(rgb);

	float L = lab.L;
	float eps = 0.00001f;
	float C = max(eps, sqrtf(lab.a * lab.a + lab.b * lab.b));
	float a_ = lab.a / C;
	float b_ = lab.b / C;

	float Ld = L - 0.5f;
	float e1 = 0.5f + fabs(Ld) + alpha * C;
	float L0 = 0.5f*(1.f + sgn(Ld)*(e1 - sqrtf(e1*e1 - 2.f *fabs(Ld))));

	float t = find_gamut_intersection(a_, b_, L, C, L0);
	float L_clipped = L0 * (1.f - t) + t * L;
	float C_clipped = t * C;

	return oklab_to_linear_srgb({ L_clipped, C_clipped * a_, C_clipped * b_ });
}

RGB gamut_clip_adaptive_L0_L_cusp(RGB rgb, float alpha = 0.05f)
{
	if (rgb.r < 1 && rgb.g < 1 && rgb.b < 1 && rgb.r > 0 && rgb.g > 0 && rgb.b > 0)
		return rgb;

	Lab lab = linear_srgb_to_oklab(rgb);

	float L = lab.L;
	float eps = 0.00001f;
	float C = max(eps, sqrtf(lab.a * lab.a + lab.b * lab.b));
	float a_ = lab.a / C;
	float b_ = lab.b / C;

	// The cusp is computed here and in find_gamut_intersection, an optimized solution would only compute it once.
	LC cusp = find_cusp(a_, b_);

	float Ld = L - cusp.L;
	float k = 2.f * (Ld > 0 ? 1.f - cusp.L : cusp.L);

	float e1 = 0.5f*k + fabs(Ld) + alpha * C/k;
	float L0 = cusp.L + 0.5f * (sgn(Ld) * (e1 - sqrtf(e1 * e1 - 2.f * k * fabs(Ld))));

	float t = find_gamut_intersection(a_, b_, L, C, L0);
	float L_clipped = L0 * (1.f - t) + t * L;
	float C_clipped = t * C;

	return oklab_to_linear_srgb({ L_clipped, C_clipped * a_, C_clipped * b_ });
}

Two new color spaces for color picking - Okhsv and Okhsl

2021-09-08T02:00:00+02:00

Okhsv and Okhsl

Two new color spaces for color picking

This post has an accompanying interactive comparison of color pickers.

I would recommend checking out the interactive demo first, then return if you are interested in the background and technical details.

Picking colors is a common operation in many applications and over the years color pickers have become fairly standardized. Ubiquitous today are color pickers based on HSL and HSV. They are simple transformations of RGB values to alternative coordinates chosen to better correlate with perceptual qualities.

Here are two common variants of color pickers built on HSL and HSV:

A HSV color picker.

A HSL color picker.

Despite color picking playing a big role in a lot of applications, the design of color pickers isn��t a particularly well researched topic. While some variation exist in the widgets themselves, the choice of HSL or HSV is mostly taken for granted, with only a few exceptions.

Is their dominance well deserved or would it be possible to create better alternatives? I at least think that this question deserves to be explored and that color picker design should be an active research topic. With this post I hope to contribute to the exploration of what a better color picker could and should be, and hopefully inspire others to do the same!

The main focus here will be on the choice of color space, rather than the design of the UI widget used for navigating the color space.

This rest of this post is organized as follow:

A brief history of color picking

What are meaningful properties of color spaces for color picking?

What options exist already and how do they perform?

Introducing two new color spaces: Okhsl and Okhsv

Ideas for future work

Color picking before computers

Munsell Color System. Photo by Hannes Grobe, license CC BY-SA 4.0

Example of NCS color picker. Screenshot from NCS Navigator in the Colourpin app.

Categorizing, describing and picking colors is an old problem and predates computers by many centuries. Over the years countless artists and scientists have worked to understand how humans perceive colors and used that knowledge to try and create practical systems for describing colors. Many different color ordering systems have been created over the years based on mixing properties of paints, light or on perceptual qualities.

During the 20th century two important color systems emerged. The Munsell Color System and the Natural Color System (NCS). Both of them are based on human perception and were derived using experiments, but with different approaches. The two systems are used in many practical applications today still.

In the Munsell color system, colors are described with three parameters, designed to match the perceived appearance of colors: Hue, Chroma and Value. The parameters are designed to be independent and each have a uniform scale. This results in a color solid with an irregular shape. Modern color spaces and models, such as CIELAB, Cam16 and my own Oklab, are very similar in their construction.

The Natural Color System takes a different approach, and is designed to make it easy to describe colors, rather than to match perceptual qualities. It does this by describing colors by their similarity to six primary colors: white, black, yellow, red, green and blue. The yellow, red, green and blue colors are used to determine the hue. The final color is described by a color triangle with the corners white, black and the most saturated color of the given hue. A position in the triangle is described with the parameters whiteness, blackness, chromaticness. Any two of those parameters are sufficient, since they sum to one.

For more information about historical color systems, this is a great resource: colorsystem.com.

What makes a good color picker?

Before diving into how color pickers have evolved in the digital era, let’s look a bit further at what considerations can be relevant when designing a color space for color picking. This part assumes familiarity with color appearance concepts such as lightness, chroma, saturation and hue.

In this post the focus will be on what is today the most common case, picking colors in the sRGB gamut. Wide gamut and HDR displays are becoming more common and will be increasingly important, so wide gamut and HDR color picking is definitely a topic for further research and development, but it will not be considered here.

Here’s an attempt at capturing useful properties for color spaces designed for picking colors:

Orthogonal Lightness - Hue/Chroma/Saturation can be altered, while keeping perceived Lightness constant
Orthogonal Chroma - Lightness/Hue can be altered, while keeping perceived Chroma constant
Orthogonal Saturation - Lightness/Hue can be altered, while keeping perceived Saturation constant
Orthogonal Hue - Lightness/Chroma/Saturation can be altered, while keeping perceived Hue constant
Simple Geometrical Shape - Fit the target gamut into a cylinder or other simple shape, so that parameters can be altered independently without resulting in colors outside the target gamut. Could also be a swept triangle like NCS, since it is simple to map back and forth to a cylinder.
Max Chroma at edge - Make it easy to find the strongest color of a given hue, by placing the strongest color on edge of the color volume.
Varies Smoothly - Vary smoothly with each parameter. No discontinuous or abrupt changes.
Varies Evenly - The perceived magnitude of the change in color caused by changing a parameter should be uniform for all values of the parameter.

Note: These properties are in conflict, so designing a color space for color picking is a about finding which tradeoffs to make. In particular, independent control of hue, lightness and chroma can not be achieved in a color space that also maps sRGB to a simple geometrical shape.

Color spaces for color picking

By far the most used color spaces today for color picking are HSL and HSV, two representations introduced in the classic 1978 paper “Color Spaces for Computer Graphics”. HSL and HSV designed to roughly correlate with perceptual color properties while being very simple and cheap to compute.

Worth noting is that HSL and HSV are not quite color spaces on their own, they are transformations from a source RGB color space. For each set of RGB primaries and transfer functions, the transformation to HSL and HSV produces unique color spaces. Today HSL and HSV are most commonly used together with the sRGB color space, so that is what we will look at here and we will here use HSL and HSV to refer to HSL and HSV for the sRGB color space.

Also useful to note is that HSL and HSV are not continuously differentiable, so that limits their use with numerical optimization and machine learning.

HSV

HSV describes colors with three parameters:

"Hue" - Roughly corresponds to perceived hue, but it has quite severe distortions.
"Saturation" - Roughly corresponds to saturation relative to maximum possible saturation in sRGB of the same hue.
"Value" - A bit hard to define. Can be seen as how much to mix the color with black, with 100% being no black and 0% completely black. “Value” is sometimes also referred to as Brightness.

HSV is quite similar to the Natural Color System in its structure and it’s possible to transform it to have parameters more similar to NCS, then referred to as hue, whiteness and blackness (HWB). After that transformation the largest difference compared with NCS are:

NCS is derived based on research into the appearance of colors and does a good job at matching human perception
HWB/HSV has a simple construction, not taking research into color appearance into account and is not matching perception closely. Hue is the most problematic.
NCS has a gamut designed to contain pigments realizable in paint/print
HWB/HSV has a gamut based on the RGB color space it is constructed from (most commonly sRGB)

Example of hue distortion for deep blue colors. Notice the purple shift as saturation decreases.

HSL

HSV describes colors with three parameters:

"Hue" - Identical to “hue” in HSV, with the same issues.
"Saturation" - Roughly the chroma of the color relative to the most colorful color with the same “lightness” and “hue”. Confusingly referred to as saturation, which it is not comparable to. In the original paper it was referred to as “relative chroma”, which is more accurate. Not the same as “saturation” in HSV.
"Lightness" - Some correlation with the perception of lightness, with 0% corresponding to black and 100% to white. Does not match the perception of lightness well at all for saturated colors. Referred to as “Intensity” in the original paper.

Example of hue distortion for deep blue colors.

Example of colors HSL considers to have the same lightness.

HSLuv

HSLuv is a recent development to tackle some of the shortcomings of HSL. It is based on CIELChuv, a cylindrical form of 1976 CIE color space CIELUV. CIELChuv is constructed so that for a given hue, all colors of that hue can be constructed by additive blending of white and a saturated color of that hue (and in general, additive blending of light forms straight lines in CIELuv).

HSLuv describes colors with three parameters:

"Hue" - Same as hue in CIELChuv. Does not match the perception of hue fully due to the Abney effect: the perception of hue does not correspond to additive blending.
"Saturation" - Based on chroma as defined in CIELChuv, but rescaled to be relative to the most saturated sRGB color of the same “lightness” and “hue”.
"Lightness" - Same as lightness in CIELChuv. Does a good job at matching perceived lightness.

Two drawback with HSLuv are:

Does not match perception of hue. This is particularly obvious for deep blue and purple colors.
The way “Saturation” is defined, it does not vary smoothly due to the uneven shape of the sRGB gamut. E.g. by keeping “Saturation” constant and changing hue, the perceived chroma can change drastically and abruptly.

Example of hue distortion for blue colors. The distortion in HSLuv is different from that in sRGB and is caused by the Abney effect.

Example of constant lightness in HSLuv, with low “saturation” close to the center of the circle and maximum “saturation” at the edge. Notice how the blue and red hues are much more saturated than surrounding colors.

Slice of colors with constant “saturation” in HSLuv. The scaling to match the uneven shape of the sRGB gamut makes the perceived chroma vary unevenly.

Color spaces modelling color appearance

While there is a limited amount of research done regarding color picking, a lot of work has been done to create color models that are able to predict color appearance. These continue in the tradition of the Munsell color model discussed above, but use more modern color science and mathematical models to better model the appearance of color. One of the most famous of these is CIELab, but there are today several new models that perform better.

Comparing all the color models is beyond the scope of this post, the important conclusion here is that these models can model the perception of Lightness, Hue and Chroma much better than all the previously discussed options. For a brief overview of some of the more recent models, see my previous post “A perceptual color space for image processing”. Since then another color model has also appeared: ZCAM. For a much deeper overview of modern color science and different attempts at modeling color appearance, I recommend the book “Color Appearance Models” by Mark D. Fairchild.

The main drawback of using these models directly for color picking is that the sRGB gamut has a quite irregular shape in these color spaces. As a result, changing one parameter, such as hue, can easily create a color outside the target gamut, making them quite tedious to use. Several color pickers have been made using either CIELab or more modern lab-like color spaces. From what I can tell they have only seen limited use compared with the more common HSV and HSL color pickers however.

I would think that the reason that they haven’t caught on is that their drawbacks outweigh their benefits: using a space with parameters that don’t match the our perception of hue, lightness and chroma is easier than using one with an irregular shape. That is certainly my personal experience.

For the more advanced models an additional complication is that they have several parameters meant to be adjusted based on the viewing conditions. When used for color picking they seem to mostly be set to match some kind of average viewing conditions though.

Here are a couple of examples of the irregular shape of the sRGB gamut in a perceptual color space:

A slice of the gamut with a constant blue hue.

A slice of the gamut with a constant yellow hue.

It is unfortunately also common to see CIELab based color pickers showing colors outside the target gamut and often they are mapped back by simply clamping individual RGB components. This creates severe distortions in hue, lightness and chroma, in would would otherwise be a fairly uniform color space.

Summary

These are the color spaces I am aware of that are relevant, but please reach out if you are aware of any more color spaces useful for color picking.

To summarize, here is a table of the different color spaces discussed and how they match the different desirable properties. This is definitely a bit subjective, but will hopefully give a decent overview.

	HSV	HSL	HSLuv	Lab-like*	NCS
Orthogonal Lightness	no	no	yes	yes	no
Orthogonal Chroma	no	no	no	yes	partial
Orthogonal Saturation	partial	no	no	no**	no
Orthogonal Hue	partial	partial	partial	yes	yes
Simple Geometrical Shape	yes	yes	yes	no	no***
Max Chroma at Edge	yes	no	no	yes	no
Varies Smoothly	yes	yes	no	yes	yes
Varies Evenly	no	no	no	yes	partial

*) This of course depends on which Lab-like color space. This is the best possible an appearance modelling color space could achieve.
**) If desirable, saturation can be used instead of Chroma, and then this would be a yes and “Orthogonal Chroma” a no.
***) NCS has a simple geometrical shape, but it does not match the sRGB gamut.

Finding a better tradeoff

One of the main advantages of HSL and HSV over the different Lab color spaces is that they map the sRGB gamut to a cylinder. This makes them easy to use since all parameters can be changed independently, without the risk of creating colors outside of the target gamut.

The main drawback on the other hand is that their properties don’t match human perception particularly well. Reconciling these conflicting goals perfectly isn’t possible, but given that HSV and HSL don’t use anything derived from experiments relating to human perception, creating something that makes a better tradeoff does not seem unreasonable.

We will attempt to do just that by creating new color spaces similar to HSL and HSV but that better match perception. This will be done by leveraging the Oklab color space. Using Oklab here over more advanced models such as CAM16 is useful because working out the math becomes a lot simpler. It also means that it won’t be a full color model able to adapt to different viewing conditions, but that is probably also desirable here since it is more practical.

For consistency with the naming of Oklab, these new color spaces will be called Okhsl and Okhsv. The parameters will also be referred to as $h$ , $s$ and $l$ and $h$ , $s$ and $v$ respectively. Those names are a bit confusing but I think making the new spaces easy to adopt for someone used to HSL and HSV is more important than trying to establish new names.

Intermission - a new lightness estimate for Oklab

One design decision with Oklab is to use a design that is scale independent. That is, Oklab has no concept of reference white, unlike CIELab for example. In many cases this is an advantage, since it makes dealing with larger dynamic ranges easier.

However, in the context of a color picker with well defined dynamic range and a clear reference white luminance it reduces Oklab’s ability to predict lightness. Therefore, an additional lightness estimate is needed to better handle these cases. With a reference white luminance of $Y=1$ , the new lightness estimate $L_r$ is defined as:

$k_1 = 0.206 ,\qquad k_2 = 0.03 ,\qquad k_3 = \frac{1+k_1}{1+k_2}$

$L_r = \frac{k_3 L - k_1 + \sqrt{(k_3 L - k_1)^2 + 4 k_2 k_3 L}} 2$

With the inversion:

$L = \frac{L_r (L_r + k_1)}{k_3 (L_r + k_2)}$

This new lightness estimate closely matches the lightness estimate of CIELab overall and is nearly equal at 50% lightness (Y for CIELab L is 0.18406, and $L_r$ 0.18419) which is useful for compatibility. Worth noting is that it is not possible to have a lightness scale that is perfectly uniform independent of viewing conditions and background color. This new lightness function is however a better tradeoff for cases with a well defined reference white.

From top to bottom: CIELab $L$ , Oklab $L_r$ , Oklab $L$ .

Introducing two new color spaces: Okhsv and Okhsl

With this new lightness estimate, we are ready to look into the construction of Okhsv and Okhsl. Here is a rough overview of the general idea behind Okhsv and Okhsl and their construction. Some details are glossed over here, for all the details check out the source code below.

Okhsv

To derive Okhsv, we will start with OkLCh, use its estimate for hue, $h$ , as is and introduce $s$ and $v$ parameters that are calculated based on lightness, $L_r$ , and chroma, $C$ . To keep the triangular shape when using $L_r$ we also scale $C$ by $L_r/L$ .

Here is the sRGB gamut plotted for set of hues, with $L_r$ on the y-axis and $C L_r/L$ on the x-axis:

To create a HSV-like color space, we want to find a mapping so that the cusp of the triangle is in $s=1$ and $v=1$ . We also want to change the triangle shape into a square, by stretching the lower part of the triangle.

To find the cusp we can use the same method as in my previous post about sRGB gamut clipping.

If we perform this remapping we get the following result:

Remaining now is a small curve at the top, that we also have to remove. This is done by scaling $v$ to compensate. This step makes the space less uniform perceptually, but is needed to fit the sRGB gamut to a cylinder exactly. The change is quite small however. This gives the following result:

As an additional step we adjust saturation to be more uniform for low saturation colors. This makes it easier to compare saturation values for different colors, when saturation is low. The effect of this is subtle.

This gives us a new model with a simple geometrical shape and a hue parameter that closely matches perception. Overall the space will be very familiar to someone who is used to HSV, but with improved perceptual uniformity.

Okhwb

If desired, Okhsv can also be converted to a HWB (hue, whiteness and blackness) form.

$w = (1-s) v$

$b = 1-v$

With the inverse:

$s = 1-\frac{w}{1-b}$

$v = 1-b$

Okhsl

To derive Okhsl we also start with OkLCh. $L_r$ and $h$ are kept as is, with $L_r$ referred to as $l$ instead for consistency.

For $s$ we want to somehow remap $C$ so that the sRGB gamut nicely fits into a cylinder.

The simplest way to do this is to simply scale it by the maximum chroma inside the sRGB gamut for a given value of $l$ and $h$ , $C_{max}(h, l)$ , which is what HSLuv does. As we have seen with HSLuv though, the unevenness of the shape of the gamut will affect the interior of the entire space resulting in an uneven scale for the $s$ component.

Instead it would be good if we could find a way to keep the unevenness local to colors close to the edge of the gamut, leaving the interior less affected. This is the key idea behind Okhsl.

One way to solve this would be to solve it as a boundary value problem, finding $C = f(h, s, l)$ , with a boundary condition that $C = C_{max}(h, l)$ and some set of differential equation that keeps the interior smooth. This approach could definitely give a good result and would be interesting to explore, but is likely to only have a numerical solution, which would make it hard to use practically to construct a color space.

Instead Okhsl uses a fairly ad-hoc approach to create a smoothly varying interior, since that makes it efficient to run and easy to invert.

Instead of scaling $s$ by a single value for $C$ , the max possible value in the gamut, three different values are used, one for low values of $s$ , $C_0$ , one for midrange values of $s$ , $C_{mid}$ and one for large values, $C_{max}$ . These are constructed the following way:

$C_0(l)$ is constructed to be independent of hue, this way creating continuity for colors close to the $s=0$ axis.
$C_{mid}(h, l)$ is constructed to be closer in shape to $C_max$ , but still much smoother and has been constructed using an optimization process. See the source code for more details and the exact computation.
$C_{max}(h, l)$ is the maximum possible value for $C$ in the sRGB gamut for the given values of $l$ and $h$

To get an understanding of $C_0$ , $C_{mid}$ and $C_{max}$ , here are a few hue slices where $C$ is computed $C = s C_0$ , $C = s C_{mid}$ and $C = s C_{max}$ respectively.

$C_0$

$C_{mid}$

$C_{max}$

To create the full Okhsl model, the values are interpolated so that:

At $s=0$ : $\frac{\partial C}{\partial s} = C_0$ , $C=0$
At $s=0.8$ : $C=C_{mid}$
At $s=1.0$ : $C=C_{max}$

This gives the final Okhsl model:

Altogether this gives a model with a simple geometrical shape that has parameters for lightness and hue that closely match perception. The model is quite different from regular HSL, in order to achieve a better lightness estimate. I believe Okhsl delivers a better overall compromise, and keeps many of the benefits of Lab-like color spaces, without the complexity of an irregular shape.

Here are a few examples of slices Okhsl, with constant lightness and saturation:

Example of constant “lightness” in Okhsl.

Slice of colors with constant “saturation” in Okhsl. While not matching perceived chroma fully it is smoothly varying.

For 100% ‘saturation’ the variation in perceived chroma is larger, due to the shape of the sRGB gamut.

Summary

For completeness, here is a table of how Okhsv and Okhsl match the desired properties from earlier. Again, this is definitely a bit subjective. A better way to judge the performance is to just try the results yourself.

	Okhsv	Okhsl
Orthogonal Lightness	no	yes
Orthogonal Chroma	no	no
Orthogonal Saturation	partial	no
Orthogonal Hue	yes	yes
Simple Geometrical Shape	yes	yes
Max Chroma at Edge	yes	no
Varies Smoothly	yes	yes
Varies Evenly	no	no

Ideas for future work

Okhsv and Okhsl are my attempts at making better color pickers for the sRGB gamut. I would love to see more experimentation overall with color picker design and in the next few years, color pickers for wide gamut and HDR will be more and more important and need a lot of research. They both offer their own new challenges.

Wide gamut is challenging since we are seeing an increased variety of different gamuts. At least for a while, target color spaces will be much more varied and applications for authoring colors will have to either settle for common subset or have to deal with this complexity. This of course will have a big impact on what color pickers look like and how they behave.

One interesting avenue to pursue would be to more automatically create color spaces like Okhsv and Okhsl for a given color gamut. This would likely need to use a bit of a different approach, maybe using lookup tables and numerical solutions in order to not need as much handcrafted logic.

HDR also has the issue of not being quite standardized, but an added complexity is the increased dynamic range and variation is absolute brightness. In the past color pickers have been able to mostly ignore how the eye adapts to different luminance levels, but this does not work as well with HDR. So far the approaches I’ve seen are to use regular SDR color pickers, but with and added exposure/intensity control. Is this the best approach or are there new ways we should be working with HDR color pickers?

An additional thing to explore is what spacing of hues would be the best. Okhsl and Okhsv simply inherit their spacing from Oklab. A different option could be to do a remapping similar to NCS, which would make the parameter vary less evenly, but could make it easier to use by mapping the different axes to more familiar colors.

Source Code

Here is the Source Code for conversion between sRGB, HSL and HSV. This code depends on the code from my previous post sRGB gamut clipping, which is not included here. You can find the source for both posts combined here as a C++ header.

The interactive comparison of color pickers also has an implementation of this in JavaScript. The source is available here.

License

All the source code on this page is provided under the MIT license:

Copyright (c) 2021 Björn Ottosson

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Common code

struct HSV { float h; float s; float v; };
struct HSL { float h; float s; float l; };
struct LC { float L; float C; };

// Alternative representation of (L_cusp, C_cusp)
// Encoded so S = C_cusp/L_cusp and T = C_cusp/(1-L_cusp) 
// The maximum value for C in the triangle is then found as fmin(S*L, T*(1-L)), for a given L
struct ST { float S; float T; };

// toe function for L_r
float toe(float x)
{
	constexpr float k_1 = 0.206f;
	constexpr float k_2 = 0.03f;
	constexpr float k_3 = (1.f + k_1) / (1.f + k_2);
	return 0.5f * (k_3 * x - k_1 + sqrtf((k_3 * x - k_1) * (k_3 * x - k_1) + 4 * k_2 * k_3 * x));
}

// inverse toe function for L_r
float toe_inv(float x)
{
	constexpr float k_1 = 0.206f;
	constexpr float k_2 = 0.03f;
	constexpr float k_3 = (1.f + k_1) / (1.f + k_2);
	return (x * x + k_1 * x) / (k_3 * (x + k_2));
}

ST to_ST(LC cusp)
{
	float L = cusp.L;
	float C = cusp.C;
	return { C / L, C / (1 - L) };
}

HSV

This code converts between sRGB (not linear) and Okhsv.

struct HSV { float h; float s; float v; };
RGB okhsv_to_srgb(HSV hsv)
{
	float h = hsv.h;
	float s = hsv.s;
	float v = hsv.v;

	float a_ = cosf(2.f * pi * h);
	float b_ = sinf(2.f * pi * h);
	
	LC cusp = find_cusp(a_, b_);
	ST ST_max = to_ST(cusp);
	float S_max = ST_max.S;
	float T_max = ST_max.T;
	float S_0 = 0.5f;
	float k = 1 - S_0 / S_max;

	// first we compute L and V as if the gamut is a perfect triangle:

	// L, C when v==1:
	float L_v = 1     - s * S_0 / (S_0 + T_max - T_max * k * s);
	float C_v = s * T_max * S_0 / (S_0 + T_max - T_max * k * s);

	float L = v * L_v;
	float C = v * C_v;

	// then we compensate for both toe and the curved top part of the triangle:
	float L_vt = toe_inv(L_v);
	float C_vt = C_v * L_vt / L_v;

	float L_new = toe_inv(L);
	C = C * L_new / L;
	L = L_new;

	RGB rgb_scale = oklab_to_linear_srgb({ L_vt, a_ * C_vt, b_ * C_vt });
	float scale_L = cbrtf(1.f / fmax(fmax(rgb_scale.r, rgb_scale.g), fmax(rgb_scale.b, 0.f)));

	L = L * scale_L;
	C = C * scale_L;

	RGB rgb = oklab_to_linear_srgb({ L, C * a_, C * b_ });
	return {
		srgb_transfer_function(rgb.r),
		srgb_transfer_function(rgb.g),
		srgb_transfer_function(rgb.b),
	};
}

HSV srgb_to_okhsv(RGB rgb)
{
	Lab lab = linear_srgb_to_oklab({
		srgb_transfer_function_inv(rgb.r),
		srgb_transfer_function_inv(rgb.g),
		srgb_transfer_function_inv(rgb.b)
		});

	float C = sqrtf(lab.a * lab.a + lab.b * lab.b);
	float a_ = lab.a / C;
	float b_ = lab.b / C;

	float L = lab.L;
	float h = 0.5f + 0.5f * atan2f(-lab.b, -lab.a) / pi;

	LC cusp = find_cusp(a_, b_);
	ST ST_max = to_ST(cusp);
	float S_max = ST_max.S;
	float T_max = ST_max.T;
	float S_0 = 0.5f;
	float k = 1 - S_0 / S_max;

	// first we find L_v, C_v, L_vt and C_vt

	float t = T_max / (C + L * T_max);
	float L_v = t * L;
	float C_v = t * C;

	float L_vt = toe_inv(L_v);
	float C_vt = C_v * L_vt / L_v;

	// we can then use these to invert the step that compensates for the toe and the curved top part of the triangle:
	RGB rgb_scale = oklab_to_linear_srgb({ L_vt, a_ * C_vt, b_ * C_vt });
	float scale_L = cbrtf(1.f / fmax(fmax(rgb_scale.r, rgb_scale.g), fmax(rgb_scale.b, 0.f)));

	L = L / scale_L;
	C = C / scale_L;

	C = C * toe(L) / L;
	L = toe(L);

	// we can now compute v and s:

	float v = L / L_v;
	float s = (S_0 + T_max) * C_v / ((T_max * S_0) + T_max * k * C_v);

	return { h, s, v };
}

HSL

This code converts between sRGB (not linear) and Okhsl.

struct HSL { float h; float s; float l; };

// Returns a smooth approximation of the location of the cusp
// This polynomial was created by an optimization process
// It has been designed so that S_mid < S_max and T_mid < T_max
ST get_ST_mid(float a_, float b_)
{
	float S = 0.11516993f + 1.f / (
		+7.44778970f + 4.15901240f * b_
		+ a_ * (-2.19557347f + 1.75198401f * b_
			+ a_ * (-2.13704948f - 10.02301043f * b_
				+ a_ * (-4.24894561f + 5.38770819f * b_ + 4.69891013f * a_
					)))
		);

	float T = 0.11239642f + 1.f / (
		+1.61320320f - 0.68124379f * b_
		+ a_ * (+0.40370612f + 0.90148123f * b_
			+ a_ * (-0.27087943f + 0.61223990f * b_
				+ a_ * (+0.00299215f - 0.45399568f * b_ - 0.14661872f * a_
					)))
		);

	return { S, T };
}

struct Cs { float C_0; float C_mid; float C_max; };
Cs get_Cs(float L, float a_, float b_)
{
	LC cusp = find_cusp(a_, b_);

	float C_max = find_gamut_intersection(a_, b_, L, 1, L, cusp);
	ST ST_max = to_ST(cusp);
	
	// Scale factor to compensate for the curved part of gamut shape:
	float k = C_max / fmin((L * ST_max.S), (1 - L) * ST_max.T);

	float C_mid;
	{
		ST ST_mid = get_ST_mid(a_, b_);

		// Use a soft minimum function, instead of a sharp triangle shape to get a smooth value for chroma.
		float C_a = L * ST_mid.S;
		float C_b = (1.f - L) * ST_mid.T;
		C_mid = 0.9f * k * sqrtf(sqrtf(1.f / (1.f / (C_a * C_a * C_a * C_a) + 1.f / (C_b * C_b * C_b * C_b))));
	}

	float C_0;
	{
		// for C_0, the shape is independent of hue, so ST are constant. Values picked to roughly be the average values of ST.
		float C_a = L * 0.4f;
		float C_b = (1.f - L) * 0.8f;

		// Use a soft minimum function, instead of a sharp triangle shape to get a smooth value for chroma.
		C_0 = sqrtf(1.f / (1.f / (C_a * C_a) + 1.f / (C_b * C_b)));
	}

	return { C_0, C_mid, C_max };
}

RGB okhsl_to_srgb(HSL hsl)
{
	float h = hsl.h;
	float s = hsl.s;
	float l = hsl.l;

	if (l == 1.0f)
	{
		return { 1.f, 1.f, 1.f };
	}

	else if (l == 0.f)
	{
		return { 0.f, 0.f, 0.f };
	}

	float a_ = cosf(2.f * pi * h);
	float b_ = sinf(2.f * pi * h);
	float L = toe_inv(l);

	Cs cs = get_Cs(L, a_, b_);
	float C_0 = cs.C_0;
	float C_mid = cs.C_mid;
	float C_max = cs.C_max;

    // Interpolate the three values for C so that:
    // At s=0: dC/ds = C_0, C=0
    // At s=0.8: C=C_mid
    // At s=1.0: C=C_max

	float mid = 0.8f;
	float mid_inv = 1.25f;

	float C, t, k_0, k_1, k_2;

	if (s < mid)
	{
		t = mid_inv * s;

		k_1 = mid * C_0;
		k_2 = (1.f - k_1 / C_mid);

		C = t * k_1 / (1.f - k_2 * t);
	}
	else
	{
		t = (s - mid)/ (1 - mid);

		k_0 = C_mid;
		k_1 = (1.f - mid) * C_mid * C_mid * mid_inv * mid_inv / C_0;
		k_2 = (1.f - (k_1) / (C_max - C_mid));

		C = k_0 + t * k_1 / (1.f - k_2 * t);
	}

	RGB rgb = oklab_to_linear_srgb({ L, C * a_, C * b_ });
	return {
		srgb_transfer_function(rgb.r),
		srgb_transfer_function(rgb.g),
		srgb_transfer_function(rgb.b),
	};
}

HSL srgb_to_okhsl(RGB rgb)
{
	Lab lab = linear_srgb_to_oklab({
		srgb_transfer_function_inv(rgb.r),
		srgb_transfer_function_inv(rgb.g),
		srgb_transfer_function_inv(rgb.b)
		});

	float C = sqrtf(lab.a * lab.a + lab.b * lab.b);
	float a_ = lab.a / C;
	float b_ = lab.b / C;

	float L = lab.L;
	float h = 0.5f + 0.5f * atan2f(-lab.b, -lab.a) / pi;

	Cs cs = get_Cs(L, a_, b_);
	float C_0 = cs.C_0;
	float C_mid = cs.C_mid;
	float C_max = cs.C_max;

    // Inverse of the interpolation in okhsl_to_srgb:

	float mid = 0.8f;
	float mid_inv = 1.25f;

	float s;
	if (C < C_mid)
	{
		float k_1 = mid * C_0;
		float k_2 = (1.f - k_1 / C_mid);

		float t = C / (k_1 + k_2 * C);
		s = t * mid;
	}
	else
	{
		float k_0 = C_mid;
		float k_1 = (1.f - mid) * C_mid * C_mid * mid_inv * mid_inv / C_0;
		float k_2 = (1.f - (k_1) / (C_max - C_mid));

		float t = (C - k_0) / (k_1 + k_2 * (C - k_0));
		s = mid + (1.f - mid) * t;
	}

	float l = toe(L);
	return { h, s, l };
}

Björn Ottosson

How software gets color wrong

How software gets color wrong

Introduction to color perception and color management

Representing color in software

Color gamut

Non-linear transfer function

Color management

Desirable operations with color

Color processing in software today

Not all bad

Comparisons

What about HSV and HSL?

What can we do?

Conclusion

A perceptual color space for image processing

From personal project to industry standard

A perceptual color space for image processing

The Oklab color space

Comparing Oklab to HSV

Implementation

Converting from XYZ to Oklab

Table of example XYZ and Oklab pairs

Converting from linear sRGB to Oklab

Motivation and derivation of Oklab

What about existing models?

How Oklab was derived

Comparison with other color spaces

Munsell data

Luo-Rigg dataset and full gamut

Blending colors

Conclusion

sRGB gamut clipping

sRGB gamut clipping

Gamut Clipping

Intersecting lines with the gamut

Choosing which line to project along

Keep lightness constant, only compress chroma

Projection towards a single point, hue independent

Projection towards a single point, hue dependent

Can we do better?

Adaptive L0L_0L​0​​, hue independent

Adaptive L0L_0L​0​​, hue dependent

Results

Flowers

Lamp

Tulip

Boat

Conclusions and future work

Source code

Oklab to linear sRGB conversion

Intersection with sRGB gamut

Gamut clipping

Two new color spaces for color picking - Okhsv and Okhsl

Okhsv and Okhsl

Two new color spaces for color picking

Color picking before computers

What makes a good color picker?

Color spaces for color picking

HSV

HSL

HSLuv

Color spaces modelling color appearance

Summary

Finding a better tradeoff

Intermission - a new lightness estimate for Oklab

Introducing two new color spaces: Okhsv and Okhsl

Okhsv

Okhwb

Okhsl

Summary

Ideas for future work

Source Code

License

Common code

HSV

HSL

Adaptive $L_0$ , hue independent

Adaptive $L_0$ , hue dependent