Dibyendu Mondal

This project involves three components:

1. Camera Projection Matrix
2. Fundamental Matrix Estimation
3. Fundamental Matrix with RANSAC

Part 1: Camera Projection Matrix

We need to solve this non-homogeneous linear system of equations:

We solve it using linear least squares method.

				M = A\Y;
				M = [M;1];
				M = reshape(M,[],3)';
			

After solving this, we get the projection matrix (M):

				The projection matrix is:
				    0.7679   -0.4938   -0.0234    0.0067
				   -0.0852   -0.0915   -0.9065   -0.0878
				    0.1827    0.2988   -0.0742    1.0000
			

The total residual is: 0.0445
Next, we need to calculate the camera center. We do this by using these equations:

				Q = M(:,1:3);
				m4 = M(:,4);
				Center = -Q\m4;
			

On solving these we get:

				The estimated location of camera is: <-1.5126, -2.3517, 0.2827>
			

Part 2: Fundamental Matrix Estimation

We need to solve this homogeneous linear system of equations:

We solve it using Singular Value Decomposition method. We also enfore det(F) = 0 since F has rank 2.

				[U,S,V] = svd(A);
				f = V(:,end);
				F = reshape(f,[3,3])';

				[U,S,V] = svd(F);
				S(3,3) = 0;
				F_matrix = U*S*V';
			

Results:

Fundamental matrix for the given test images:

				F_matrix =

				    0.0000    0.0000   -0.0018
				   -0.0000    0.0000   -0.0054
				    0.0008   -0.0009    1.0000
			

Part 3: Fundamental Matrix with RANSAC

It involves the following steps:

1. Creating a sample with 'x' number of points and calling the estimate_fundamental_matrix function on those samples.

   
   						index = randsample(size(matches_a,1),x);
   						samples_a = matches_a(index,:);
   						samples_b = matches_b(index,:);
   						F_matrix = estimate_fundamental_matrix(samples_a,samples_b);
   					

2. Finding the number of inliers based on a threshold.

   
   						inliers = zeros(size(matches_a,1));
   						count = 0;
   						for j = 1:size(matches_a,1)
   							res = [matches_a(j,:) 1]*F_matrix*[matches_b(j,:),1]';
   							if(abs(res) < threshold)
   								count = count + 1;
   								inliers(j) = 1;
   							end
   						end
   					

3. Finding the best matrix based on the maximum number of inliers.

   
   						confidence_ = count;
   						if(confidence_ > confidence)
   							in = find(inliers);
   							inliers_a = matches_a(in,:);
   							inliers_b = matches_b(in,:);
   							Best_Fmatrix = F_matrix;
   							confidence = confidence_;
   						end
   					

I repeat these steps for 1000 iterations to find the best fundamental matrix.

Results:

iterations = 1000; sample points = 9; threshold = 0.01;

Extra Credit: Fundamental Matrix with RANSAC with normalized points

It involves the following steps:

1. Using linear transformations to normalize the coordinates of image a by making the mean of the points zero and the average magnitude 1.

   
   						mean_a = mean(Points_a);
   						Ta = [1 0 -1*mean_a(1);0 1 -1*mean_a(2);0 0 1];
   						temp = Points_b - repmat(mean_a,size(Points_a,1),1);
   						s = 1/std2(temp);
   						Sa = [s 0 0;0 s 0;0 0 1];
   
   						temp = zeros(size(Points_a,1),3);
   						for i = 1:size(Points_a,1)
   							temp(i,:) = Sa*Ta*[Points_a(i,1);Points_a(i,2);1];
   						end
   						Points_a = temp(:,1:2);
   					

2. Repeating the same for coordinates of image b.

   
   						mean_b = mean(Points_b);
   						Tb = [1 0 -1*mean_b(1);0 1 -1*mean_b(2);0 0 1];
   						temp = Points_b - repmat(mean_b,size(Points_b,1),1);
   						s = 1/std2(temp);
   						Sb = [s 0 0;0 s 0;0 0 1];
   
   						temp = zeros(size(Points_b,1),3);
   						for i = 1:size(Points_b,1)
   							temp(i,:) = Sb*Tb*[Points_b(i,1);Points_b(i,2);1];
   						end
   						Points_b = temp(:,1:2);
   					

3. Estimating the fundamental matrix as was done previously.

   
   						[U,S,V] = svd(A);
   						f = V(:,end);
   						F = reshape(f,[3,3])';
   
   						[U,S,V] = svd(F);
   						S(3,3) = 0;
   						F_matrix = U*S*V';
   					

4. Adjusting the fundamental matrix so that it can operate on the original coordinates.

   
   						F_matrix = (Sb*Tb)'*F_matrix*(Sa*Ta);
   					

Results:

iterations = 1000; sample points = 9; threshold = 0.01;

Conclusion

It's quite clear from the above results that we get a lot better correspondences if we use normalized coordinates.