Project 2: Local Feature Matching

Feature Matching

This involves three major steps:

  1. Finding Interest Points
  2. Finding Local Features
  3. Finding Feature Matches

Finding Interest Points

I used the Harris Corner Detection algorithm for finding interest points. It involves the following steps:

  1. Getting the Image derivatives 
    
					filter = fspecial('Gaussian',3,0.5);
					[Gx,Gy] = imgradientxy(filter,'sobel');
					Ix = imfilter(image,Gx,'symmetric','same','conv');
					Iy = imfilter(image,Gy,'symmetric','same','conv');
  2. Squaring the derivatives 
    
					Ix2 = Ix.*Ix;
					Iy2 = Iy.*Iy;
					Ixy = Ix.*Iy;
  3. Applying the gausian filter 
    
					filter = fspecial('Gaussian',3,1);
					GIx2 = imfilter(Ix2,filter,'symmetric','same','conv');
					GIy2 = imfilter(Iy2,filter,'symmetric','same','conv');
					GIxy = imfilter(Ixy,filter,'symmetric','same','conv');
  4. Getting the cornerness scores 
    
					alpha = 0.05;
					Har = (GIx2.*GIy2 - GIxy.*GIxy) - alpha*((GIx2 + GIy2).*(GIx2 + GIy2));
  5. Non-maximum supression 
    
					threshold = 1.54e-3;
					Har_th = Har>threshold;
					Har_th1 = Har.*Har_th;

					Har_supp = imregionalmax(Har_th1);
					Har_supp1 = Har.*Har_supp;

					Har_th_supp = (Har_th1==Har_supp1).*Har_supp1;
					[y,x,confidence] = find(Har_th_supp>0);

Norte Dame Interest Points

Finding Local Features

I used the SIFT feature detection algorithm for finding local features around the interest points. It involves the following steps:

  1. Getting the Image derivatives 
    
						filter = fspecial('gaussian',3,0.5);
						image_filtered = imfilter (image, filter,'symmetric','same','conv');
						[~ , imgrad_orient] = imgradient(image_filtered, 'sobel');
  2. Finding orientations of patches and assigning them to one of the 8 bins 
    
						impatch_or(:,:,i) = imgrad_orient((y(i)-7):(y(i)+8),(x(i)-7):(x(i)+8));    
					    	hist_temp = blockproc(impatch_or(:,:,i),[feature_width/4 feature_width/4], fun);
					    	features(i,:) = reshape(hist_temp',[1 ((feature_width/4)^2)*8]);
  3. Normalizing the features 
    
						features(i,:) = features(i,:)/norm(features(i,:),2);

Norte Dame Features

Finding Feature Matches

I used the Nearest Neighbour Distance ratios to find the matches. It involves the following steps:

  1. Finding Nearest Neighbour distance ratios 
    
						d = pdist2(features1,features2);
						[dist,index] = sort(d,2,'ascend');
						nn(:) = dist(:,1)./dist(:,2);
  2. Finding the matches and confidences by putting a threshold on the ratios 
    
						threshold = 0.9991;
						nn1 = nn < threshold;
						nn1index = find(nn1);
						matches = [nnindex(nn1index,1),nnindex(nn1index,2)];
						confidences = nn(nn1index);

Results:

Image = Norte Dame
alpha = 0.05
threshold = 1.54e-3
Accuracy = 100%

Norte Dame Matches

Image = Mount Rushmore
alpha = 0.05
threshold = 1.54e-3
Accuracy = 97%

Mount Rushmore Matches

Image = Episcopal Gaudi
alpha = 0.04
threshold = 1.54e-3
Accuracy = 5%

Episcopal Gaudi Matches

Extra Credit

Interest points using multiple scales

I used scales 1, 0.5, 0.25 and 0.125. It involves the following steps:

  1. Creating an array of scales 
    
				scale_val=0.5.^[0;3];
  2. Resizing the image based on the current scale value 
    
				image1 = imresize(image,scale_val(i));
  3. Calling the get_interest_ponts() function 
    
				[x1,y1] = get_interest_points(image1,feature_width,scale_val(i));
  4. Returning the correct x, y and scale values 
    
				scale1=scale_val(i)*ones(size(y1,1),1);
			    	if(i==1)
					x=x1;
					y=y1;
					scale=scale1;
			   	else
					x=[x;x1];
					y=[y;y1];
					scale=[scale;scale1];
			    	end

Features using multiple scales

I used scales 1, 0.5, 0.25 and 0.125. It involves the following steps:

  1. Creating an array of scales 
    
				scale_val=0.5.^[0;3];
  2. Resizing the image based on the current scale value 
    
				image1 = imresize(image,scale_val(i));
  3. Calling the get_features() function with correct parameters 
    
				[index1,~]=find(scale==scale_val(i));
				x1=x(index1);
				y1=y(index1);
				features1=get_features(image1,x1,y1,feature_width);
  4. Returning the correct features 
    
				if(i==1)
					features=features1;
				else
					features=[features;features1];
				end

Results:

Image = Mount Rushmore
alpha = 0.05
threshold = 1.54e-3
Accuracy = 97%

Mount Rushmore Matches