Image undistortion using undistort_image from kornia.geometry.calibration

[CharalamposTzamos]

I was trying to set a pipeline for radial distortion and I wanted to use the undistortion function in kornia.geometry.calibration for images.

I didn't find a distortion function for images, so I implemented a simple one with only k1 and k2 parameters (sharing the simple code below).

def forward_radial_distortion_model(input_image, fx, fy, cx, cy, k1, k2):
    h, w = input_image.shape[:2]

    # Generate grid of coordinates
    y, x = np.mgrid[0:h, 0:w]
    x = (x - cx) / fx
    y = (y - cy) / fy
    r = np.sqrt(x**2 + y**2)

    r_prime = 1 + k1 * r**2 + k2 * r**4

    x_prime = fx * x * r_prime + cx
    y_prime = fy * y * r_prime + cy

    # Remap using the distorted coordinates
    remapped_image = cv2.remap(input_image, x_prime.astype(np.float32), y_prime.astype(np.float32), interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT)

    return remapped_image

So, having the distorted image by my function, when trying to undistort it using the the undistort_image from kornia, with the same distortion parameters (using only k1 and k2), I am not getting the original image, but an even more distorted one. It looks like it is distorting instead of undistorting.

Am I doing something wrong or is there a bug on the undistort function?

[edgarriba]

@CharalamposTzamos i see a couple of things:

• the method implemented in kornia uses 6 distortion parameters which is based on the opencv implementation: https://github.com/opencv/opencv/blob/master/modules/calib3d/src/undistort.dispatch.cpp#L384
• maybe there's a bug as suggest here: https://github.com/kornia/kornia/blob/master/kornia/geometry/calibration/undistort.py#L172 /cc @jhacsonmeza

[jhacsonmeza]

• The minimum number of distortion parameters supported by the kornia implementation is 4 (2 radial and 2 tangential). But this is just the minimum length of the array of distortion parameters. If you have 2 distortion parameters you can pass [k1, k2, 0, 0] where the tangential values are 0 and this should work. This is how is done with OpenCV.
• This https://stackoverflow.com/a/21960181 can help to better understand the whole image undistortion process. I know it's a bit confusing and very counterintuitive, but to undistort an image we actually need to apply the distortion model to the points and not the undistortion. That's why we use distort_points inside undistort_image. You can also see how the points are distorted in the OpenCV implementation.

As @CharalamposTzamos mentioned, he wants to distort an image and indeed kornia doesn't have a function for that. I don't know how the process is like in this case, but I'm going to suppose that is the opposite to the image undistortion. So, if to undistort an image you need to distort the points, you should try to undistort the points to distort your image.

In your forward_radial_distortion_model function you're applying the radial distortion to the points, so this function is actually undistorting the image. Then when you use the kornia undistort_image function you're undistorting again the image. That's why your image looks worse. I tested the undistort_image function with some real datasets and it worked exactly the same like the OpenCV version.

@CharalamposTzamos could you then try using cv2.undistortPoints in your forward_radial_distortion_model function before the cv2.remap? If it works, please let us know!

[edgarriba]

In the direction of supporting more distortion / undistortion models. I put two tasks i'm gonna work to implement a couple common used models, Brown-Conrady and Kannala-Brandt. This will also align well with Sophus (friend project we support at my team)

[edgarriba]

@CharalamposTzamos would be great to expand the existing functionality for the case of only two parameters too