Fish eye correction


#1

I’m having difficulties understanding the description of the Fisheye model on the Pix4D website.
Specifically, I’m unclear as to what X’, Y’ and Z’ represent.
Also, are xd and yd the actual pixel coordinates on the photo (1 to 1280 and 1 to 960)? So we need to use those to solve for X’ and Y’?

any help would be welcome!
thanks


#2

Look a bit further up at section 2.1.1 “Camera without distortion model” and you’ll see that X’, Y’, Z’ are the 3D coordinates of a point being projected.

Xd and Yd are the pixel coordinates, I’m pretty sure the d subscript refers to ‘distortion’ since this is a model with distortion, versus the Xu, Yu above in section 2.1.1 with the u subscript referring to undistorted.


#3

I’m definitely no expert but reading through the Pix4d document my initial understanding is that you could perform the fisheye correction in the following way.

First of all take a look at section 2.1.1 Camera without distortion model. From this for every pixel Xu (0 – 1279) Yu (0 – 959) for the Sequoia multispectral sensor you can determine the 3D world point (X’ Y’ Z’) assuming a particular Z’, e.g. assume Z’ is 1m.

So using equation (3) with Z’ as 1 you can calculate a world coordinate pair (X’, Y’) corresponding to a particular (Xu, Yu) coordinate pair.

And this would correspond to a perspective transform without any distortion.

So now you can plug in (X’ Y’) and Z’ = 1 into equation (10), (12) and (11) in that order and get out (Xd, Yd) which will be the pixel location after the fisheye projection for the world 3D coordinate of (X’ Y’ 1).

And so to generate your fisheye corrected image you would fetch the pixel value from pixel location (Xd, Yd) and store it in pixel location (Xu, Yu).


#4

@seanmcleod, I think you are right. But I would like to add that we will need equation (9) instead of equation (10) directly. Equation (10) needs to be used for computing equation (9) actually.

Please correct me if I am wrong.

Further, how do we know that Parrot has defined the starting pixel index with (0, 0). Maybe they fit the model with (1,1) as the top left pixel. @clement.fallet what is your opinion on that?

Similar confusion exists in my mind for vignette model as well.


#5

@muzammil360 you’re right, it was late at night and I was in a bit of a rush so I left out equation (9).

So after using equation (3) you’ll then use (10) followed by (9) then (12) and finally (11).

Not sure on the starting pixel index.


#6

@muzammil360 in terms of the location of pixel index (0, 0) for the fisheye correction glancing at the Pix4d document again it looks to me like it’s at the lower left corner.

Take a look at section 2.1.1 and take a look at figure 2 and the following comment:

“the origin of the image coordinate system is at the lower left corner of the image.”

Although the image in section 2.2 showing an example for the Sigma lens is confusing since the origin appears to be the top left given the x y axes labels.

Also I don’t follow the use of the minus sign in equation (3).

The following diagram I think makes things much clearer. In this diagram the origin is the top left corner and the Yc axis is positive downwards.

So to match the comment about the origin being the lower left corner you just need to flip the direction of Yc and remove the minus sign from equation (3).

In terms of the origin for the vignetting case looking at the application note again it’s not stated at all where the origin is so Parrot need to clear that up.

Parrot and Pix4d haven’t really been clear enough.


#7

@seanmcleod, thanks a lot.

You are right all of this is confusing. I don’t understand why Parrot would not like to follow the general convention of starting with the top left pixel.


#8

Following is a snap shot from Pix4d reference page for fisheye distortion removal:

In exif, we get coefficient vector as following:
Fisheye Polynomial : 0,1,0.009290101,-0.145406216

The question is where does p1 go in the equation? Its value in Fisheye Polynomial seem to be zero but eq (9) above seem to fix it at 1. Is it a typo or something else?

Any suggestions?


#9

for the following tag Fisheye Polynomial : 0,1,0.009290101,-0.145406216
p0 = 0
p1 = 1
p2 = 0.009290101
p3 = -0.145406216
for i>3, pi = 0

We tag the coefficients of the polynomial \sum_i p_i \theta_i and if p_0 = 0 and p_1 = 1 then obviously you get the equation stated


#10

@clement.fallet, thanks a lot for clearing up this confusion.

However, I must say that the above equation should be edited and made more clear on the relevant page.

For those who still might have some issue with it, I am explaining it here. @clement.fallet, please correct me if I am wrong.

Please note that p is 0 for powers of theta greater than 3.


#11

A diagram I found useful while looking at the Pix4d equations.


#12

@seanmcleod, good share. This diagram explains all those fisheye correction equations say in a brief overview. :slight_smile:


#13

Does anyone have functional python code to get the fisheye correction process working? I have some code going but the output doesn’t seem completely correct. I can throw a quick document together if anyone cares to have a look and offer assistance!


#14

You can share. I will have a look. I had developed the correction model in C++. But i really doesn’t matter what language you use.


#15

Hi, I’m experimenting w/ available Sequoia dataset and have a couple of questions.

  1. Could anyone explain ‘Fisheye Affine Symmetric’ tag meaning?
  2. Question about CIR Composite: Should I fuse reflectance map values or ‘raw’ pixel values?
    Thanks in advance.

#16

Data in this tag is used in fish eye model.

Your point 2 is not very clear.


#17

Thank you for the quick response

  1. About tag: what model should be used if this flag is false?
  2. CIR Composite: what channel values should I use to obtain CIR? Reflectance maps values, ‘raw’ pixel values or something else? Reference earticle

#18

CIR could not be use because sequoia doesn’t have blue channel
There is blue channel in rolling shutter RGB 16Mpix camera but it’s not usable !


#19

There’s no Blue in CIR, only NIR, Red an Green. I’ve found CIR example here


#20

Sorry, I made confusion with another one
R = NIR band
G = red band
B = green band