Distorsion correction on Images

[JBlefebvre]

Hi!

I try to correct distorsion on my Parrot Sequoia images.

I saw on the following links the exif tags related to distorsion correction: Parrot Announcement - Release of application notes (pdf n°4)

Then, I read the theorical explications about distorsion corrections on the Pix4d website: https://support.pix4d.com/hc/en-us/articles/202559089-How-are-the-Internal-and-External-Camera-Parameters-defined-#label101

These (very) theorical explications are very unclear to me, and I have a lot of questions that need to be solved.

1. First of all, I have read the exif tag “ModelType” for a few photos. I realised that RGB have the value “Perspective” whereas Monochrome images have the value “Fisheye”. Moreover tags that seem to be relative to perspective are only present in RGB images whereas tags relative to Fisheye correction are only present in Monochrome images. My conclusion is that we have to make only a perspective dist’ correction for RGB and a Fisheye dist’ correction for Monochromes. Can you confirm it to me? It’s absolutely not precised in the above links…

2. Normally, distorsion correction consists in deplacing pixels on a image to an other position. So, we should have a mathematical formula to compute the corrected position of the pixel as a function of the initial (uncorrected) position of the pixel . I didn’t see that, even on forums, So if you have an idea, please let me know.

3. If I try to follow the methodology shared by Pix4d, the steps seem to me :
   -We have a pixel (xu, yu) on our uncorrected image
   -I try to obtain the 3D-point coordinates(X’,Y’,Z’) of the pixel (with Pix4D formula n°(3)?) and I can take Z’=1 if I want?
   -According to the fact it is a RGB or a Monochrome images, the correction will not be the same.

For a fisheye correction(Monochrome images) :

• I recuperate informations in the EXIF (FisheyeAffineMatrix, Principal point, FisheyePolynomial, Focal plane(X,Y) resolution, Focal length)
  -I convert Principal point coordinates from mm to pixels (It should be explained in the tutorial…, the method is detailled in this topic: Fish eye correction)
  -I directly apply the Formulas (9),(10),(11) , (12) to obtain my new coordinates (xd,yd)
  -I do an interpolation to fill the gap in the new image.

For a perspective correction (RGB):
-I recuperate informations in the EXIF (like PerspectiveDistortion coefficient R1,R2,R3,T1,T2)
-And I apply the formulas (4) to (7)
-What should be considered as an homogeneous point? what is it?
-I do an interpolation just like for previous correction…

Is that exact?

4. I absolutely didn’t understand why the Pix4D tutorial talks about Camera rig at the end of the tutorial. What is the link with distorsion correction?? Do we have to use these notions for this?
   I saw @muzammil360 matlab code for distorsion (see Fisheye correction topic) but I didn’t understand everything. Why using camera rigs data to perform distortion?
   What is the use of CameraMatrix in the code? This is not in the Pix 4D tutorial…

Thanks for helping!

Have a good day!

JB

[muzammil360]

First of all, I have read the exif tag “ModelType” for a few photos. I realised that RGB have the value “Perspective” whereas Monochrome images have the value “Fisheye”. Moreover tags that seem to be relative to perspective are only present in RGB images whereas tags relative to Fisheye correction are only present in Monochrome images. My conclusion is that we have to make only a perspective dist’ correction for RGB and a Fisheye dist’ correction for Monochromes. Can you confirm it to me? It’s absolutely not precised in the above links…

Yes. It is true.

Normally, distorsion correction consists in deplacing pixels on a image to an other position. So, we should have a mathematical formula to compute the corrected position of the pixel as a function of the initial (uncorrected) position of the pixel . I didn’t see that, even on forums, So if you have an idea, please let me know.

Yes. true

-According to the fact it is a RGB or a Monochrome images, the correction will not be the same.

Yes. both RGB and monochrome(multispectral) sensors have different camera parameters.

For a fisheye correction(Monochrome images) :

• I recuperate informations in the EXIF (FisheyeAffineMatrix, Principal point, FisheyePolynomial, Focal plane(X,Y) resolution, Focal length)
  -I convert Principal point coordinates from mm to pixels (It should be explained in the tutorial…, the method is detailled in this topic: Fish eye correction)
  -I directly apply the Formulas (9),(10),(11) , (12) to obtain my new coordinates (xd,yd)
  -I do an interpolation to fill the gap in the new image.

For a perspective correction (RGB):
-I recuperate informations in the EXIF (like PerspectiveDistortion coefficient R1,R2,R3,T1,T2)
-And I apply the formulas (4) to (7)
-What should be considered as an homogeneous point? what is it?
-I do an interpolation just like for previous correction…

Is that exact?

I think you have the bigger picture right. I would recommend you to study pinhole camera model and fisheye distortion.
As for exact, nothing is exact in engineering. But it is good for making things work.

I absolutely didn’t understand why the Pix4D tutorial talks about Camera rig at the end of the tutorial. What is the link with distorsion correction?? Do we have to use these notions for this?
I saw @muzammil360 matlab code for distorsion (see Fisheye correction topic) but I didn’t understand everything. Why using camera rigs data to perform distortion?
What is the use of CameraMatrix in the code? This is not in the Pix 4D tutorial…

Rectifyinig sequoia images is a combination of two processes: fisheye undistortion + image registration.

You would have noticed that images are misaligned. Aligning them is called image registration. You need CamerMatrix for that.

The way it works, you first undistort the images and then apply image registration w-r-t one reference band (NIR in case of my code you mentioned above).

[JBlefebvre]

Ok thanks for answering.
I run your Matlab code for two Sequoia images and the results seem quite good!!
I’m going to try to understand it more precisely.

Your code is performing fisheye distorsion and image registration, but not perspective distorsion, right?

Have a good day!
JB

[clement.fallet]

Hi @JBlefebvre

I strongly suggest that you read some references about distortion in general.
This way it will be easier to understand the underlying concepts and then the code itself.

Basic optical design books and Pix4D knowledge base and its references should be a good start.

Cheers