Question about 3d landmarks

[qlihua]

Hi, thank you very much for your excellent work in this paper! While going through the paper, I had a question regarding the SBA formulation
​
In the paper, p_k is described as follow figure, which I understand to be a 3D point in the world coordinate system.

However, when reviewing the code, I found that p_k is computed as follows:

camera_coords = np.linalg.inv(K) @ pixel_coords
camera_coords = camera_coords.T * d[:, np.newaxis]

This seems that camera_coords represents 3D points in the camera coordinate system, and later these are used as landmarks to construct the Q matrix.

So I was wondering：p_k mentioned in paper the same as landmarks used in the code？If not, what is the relationship between them?
I’d really appreciate any clarification. Thanks again!

[hyhan0118]

Hi @qlihua ,

Sorry for the misleading, the camera_coords is the $\tilde{u}_{ik}$ in the math equation. And p is recovered by

# recover p and t
R_real, s_real, p_est, t_est = recover_XM(Q, R, s, Abar, lam)

Best,
Haoyu

[qlihua]

Thank you for your explanation @hyhan0118 . I have tested it on my own dataset, processed the input according to the sample, and want to perform pose optimization. Is this running result correct?

After using this SBA solver, the pose visualized is incorrect, and all of them seem to be the same point. What could be the reason for this? Below are the visualizations of the original poses and the optimization. I hope you can give me some suggestions.

[hyhan0118]

Hi, sorry for the late reply, the solver seems to finish (the min-eig is small). Maybe you should check input? Note the observation is in camera frame.