I try to align red and nir mosaic. But I can’t. Photos have different pixel number after generate mosaic. Also photos are misalignment. How can I solve this problem ?
So your question relates to processing outputs of your processing?
I think it would be more efficient to ask your software provider as the answer will be more on point.
If you want to align pictures before mosaicing , I suggest you read this application note
@blackman, red and nir can’t be aligned by normal image sticking methods as they are multimodal images. You will need to align them by relative orientation of two respective cameras.
I would suggest that you study Pinhole camera model and Epipolar geometry first to understand the concept of camera rig relatives which describe the orientation of two cameras in space relative to each other.
Once you know the basic equations and model, you will be able to under the document pointed to in the above comment.
I’m implementing image registration in Python, referencing application note 3.
But my results may be wrong.
Does anyone improve my code?
My source code:
import cv2 import exiftool import numpy as np DEBUG = True rig_relatives_tag = "XMP:RigRelatives" # T_rel NIR_T_rel = [15, -15, 0] RED_T_rel = [0, -15, 0] def image_registration(img_path, type): # Master Camera parameter T_m = [0, 0, 0] R_m = rotM([0, 0, 0]) forcal_length = 4 img = cv2.imread(img_path, -1) height, width = img.shape[:2] with exiftool.ExifTool() as et: rig_relatives = et.get_tag(rig_relatives_tag, img_path) rig_relatives_list = rig_relatives.split(",") rig_relatives_array = np.array(rig_relatives_list, dtype=np.float32) # Calculate Rotation Matrix R_rel = rotM(rig_relatives_array) if type == "NIR": T_rel = NIR_T_rel elif type == "RED": T_rel = RED_T_rel else: print("Error: Image type error, NIR or RED") if DEBUG: print("Rig Relatives:", rig_relatives_array) print("R_rel:\n", R_rel) print("T_rel: ", T_rel) print("det(R_rel)", np.linalg.det(R_rel)) # matrix for remap map_x = np.empty((height, width), dtype=np.float32) map_y = np.empty((height, width), dtype=np.float32) for y in range(height): for x in range(width): # camera coodinate to world coodinate u, v = inv_pp(x, y, width, height, forcal_length) X = np.array([u, v, 1]) X_d = R_rel.T.dot(R_m.T.dot(X - T_m) - T_rel) # world coodinate to camera coodinate x_d, y_d = perspective_projection(X_d, X_d, width, height, forcal_length) map_x[y, x] = x_d map_y[y, x] = y_d regi_img = cv2.remap(img, map_x, map_y, cv2.INTER_LINEAR) return regi_img def perspective_projection(u, v, width, height, forcal_length): f_x = forcal_length / width f_y = forcal_length / height delta_x = width / 2 delta_y = height / 2 x = f_x * u + delta_x y = f_y * v + delta_y return x, y # inverse perspective projection def inv_pp(x, y, width, height, forcal_length): f_x = forcal_length / width f_y = forcal_length / height delta_x = width / 2 delta_y = height / 2 u = (x - delta_x) / f_x v = (y - delta_y) / f_y return u, v # calculate Rotation Matrix def rotM(p): px = p py = p pz = p Rx = np.array([[1, 0, 0], [0, np.cos(px), -np.sin(px)], [0, np.sin(px), np.cos(px)]]) Ry = np.array([[np.cos(py), 0, np.sin(py)], [0, 1, 0], [-np.sin(py), 0, np.cos(py)]]) Rz = np.array([[np.cos(pz), -np.sin(pz), 0], [np.sin(pz), np.cos(pz), 0], [0, 0, 1]]) R = Rx.dot(Ry).dot(Rz) return R if __name__ == '__main__': img_nir_path = "src/IMG_170704_002438_0036_NIR.TIF" img_red_path = "src/IMG_170704_002438_0036_RED.TIF" img_regi_nir = image_registration(img_nir_path, "NIR") img_regi_red = image_registration(img_red_path, "RED") cv2.imwrite("img_regi_nir.png", img_regi_nir) cv2.imwrite("img_regi_red.png", img_regi_red)
This is a simple flowchart for image-registration:
Does anyone know where a mistake of the flowchart is?
Are you sure you are using the focal length in pixels?
Also, the transformation you use in betwen image and 3D projection looks different from the one in:
@domenzain thank you for reply.
I don’t know how to evaluate the focal length in pixels well.
Could you tell me it?
I found this page: http://answers.opencv.org/question/17076/conversion-focal-distance-from-mm-to-pixels/
Is this way correct?
By the way, I also found error in my python script.
I didn’t use radian instead of degree when I evaluate
The conversion looks fine.
As @muzammil360 mentions above to the original poster, you should read up on the base concepts to understand the language and conventions of the Application Notes and other supporting documents. They are complete but are not intended as introductory materials.
For most applications Parrot recommends using a photogrammetry solution like Pix4D. Only people with very specific applications should prefer implementing everything themselves as it is highly non-trivial and error-prone.