+
+
+
+
+
Part A
+
Shoot the Pictures
+
To prepare for this project, I have shot several sets of pictures for later steps. In each set, I setup several
+ correspondence points for each pair of pictures.
+ The first set is the beautiful San Francisco skyline during a clear night. I took the set with a camera on a long
+ focal length lens.
+
+
San Francisco Skyline
+
+
+
+
+ 
+ SF 1
+ |
+
+ 
+ SF 2
+ |
+
+ 
+ SF 3
+ |
+
+
+
+
+
+ Correspondence 1-2
+
+
+ 
+ |
+
+ 
+
+ |
+
+
+
+
+
+ Correspondence 2-3
+
+
+
+ 
+ |
+
+ 
+
+ |
+
+
+
+
+
+
Village under Snow Mountain
+
The second set is a village under the beautiful mountain. I took this set with a wide angle lens.
+
+
+
+
+ 
+ MT 1
+ |
+
+ 
+ MT 2
+ |
+
+ 
+ MT 3
+ |
+
+
+
+
+
+ Correspondence 1-2
+
+
+ 
+ |
+
+ 
+
+ |
+
+
+
+
+
+ Correspondence 2-3
+
+
+
+ 
+ |
+
+ 
+
+ |
+
+
+
+
+
+
My Living Room
+
I took the third set randomly in my living room with my phone.
+
+
+
+
+ 
+ RM 1
+ |
+
+ 
+ RM 2
+ |
+
+ 
+ RM 3
+ |
+
+
+
+
+
+ Correspondence 1-2
+
+
+ 
+ |
+
+ 
+
+ |
+
+
+
+
+
+ Correspondence 2-3
+
+
+
+ 
+ |
+
+ 
+
+ |
+
+
+
+
+
Recover Homographies
+
To recover homographies, I need to find a matrix to perform projective mapping between correspondence points. The
+ matrix has 8 degrees of freedom and the lower right corner is set to 1 since all my images are taken with the same
+ focal length within the same set.
+
+ $$
+ H = \begin{bmatrix}
+ h_1 & h_2 & h_3 \\
+ h_4 & h_5 & h_6 \\
+ h_7 & h_8 & 1
+ \end{bmatrix}
+ $$
+
+ For each correspondence points $p'$ and $p$, I will have $p' = Hp$.
+ $$
+ \begin{bmatrix}
+ x' \\
+ y' \\
+ 1
+ \end{bmatrix}
+ =
+ \begin{bmatrix}
+ h_1 & h_2 & h_3 \\
+ h_4 & h_5 & h_6 \\
+ h_7 & h_8 & 1
+ \end{bmatrix}
+ \begin{bmatrix}
+ x \\
+ y \\
+ 1
+ \end{bmatrix}
+ $$
+
+ Expanding the matrix I have:
+ $$
+ \begin{aligned}
+ x' &= h_1 x + h_2 y + h_3 \\
+ y' &= h_4 x + h_5 y + h_6 \\
+ w' &= h_7 x + h_8 y + 1
+ \end{aligned}
+ $$
+
+ To set the scaling factor to 1, I normalized the result by the third row:
+ $$
+ x' = \frac{h_1 x + h_2 y + h_3}{h_7 x + h_8 y + 1}
+ $$
+ $$
+ y' = \frac{h_4 x + h_5 y + h_6}{h_7 x + h_8 y + 1}
+ $$
+ Simplify the result we have:
+ $$
+ h_1 x + h_2 y + h_3 - x' (h_7 x + h_8 y + 1) = 0
+ $$
+ $$
+ h_4 x + h_5 y + h_6 - y' (h_7 x + h_8 y + 1) = 0
+ $$
+ Write this result back to matrix form:
+ $$
+ \begin{align}
+
+ \begin{bmatrix}
+ -x & -y & -1 & 0 & 0 & 0 & x' x & x' y & x' \\
+ 0 & 0 & 0 & -x & -y & -1 & y' x & y' y & y'
+ \end{bmatrix}&\begin{bmatrix}
+ h_1 \\
+ h_2 \\
+ h_3 \\
+ h_4 \\
+ h_5 \\
+ h_6 \\
+ h_7 \\
+ h_8 \\
+ 1
+ \end{bmatrix} &= &0 \\
+
+ A&h &= &0
+ \end{align}
+ $$
+
+ To eliminate error, I set more correspondence points than the equation needed. When solving the matrix, I try to
+ get the minimum error solution. I use Singular Value Decomposition for $H$.
+
+ In SVD, we decomposite $A$ into three components.
+ $$
+ A = U \Sigma V^T
+ $$
+ The solution for $H$ is the last column of $V^T$. I reshape the last column to a $3\times3$ matrix. And normalized
+ the matrix based on the bottom right corner to retrive $H$.
+
+
+
+
Warp the Images
+ After I get the recovery matrix from correspondence points, I can simply use forward warpping to transform one image
+ into another by multiplying the coordinates with $H$. To map the original image into the new transformed image, I
+ use
scipy.interpolate.griddata to interpolate the color of the new coordinates. I also calculate the
+ the size of output image by the four corners of the new image.
+
+ To test the result of the warping function, I rectify some images to make parts of the images into a shape of
+ square.
+
+
+
+ TV into square
+
+
+ 
+ |
+
+ 
+
+ |
+
+
+
+
+
+ Road Sign into square
+
+
+ 
+ |
+
+ 
+ |
+
+
+
+
+
Blend Images into a Mosaic
+ Here comes the most interesting part of the project. I can finally blend the images into a single panorama. I use
+ the warping function to warp the images into the result shape. Then I compute their bounding box to decide their
+ position in the final image. Then I align each image based on their position.
+ At first I use a hard blending that directly put the three images into the final image.
+
+
+
+
+
+ 
+ SF Pano
+ |
+
+
+
+
+
+
+
+
+
+
+
+ 
+ MT Pano
+ |
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 
+ RM Pano
+ |
+
+
+
+
+
+
+ The direct blending works fine for the first set of images. However, it's obvious that the panorama of the second
+ and third set did not blend smoothly. To create a smooth blending between images, I use the Laplacian multilayer
+ blending method from project 2. I use a gaussian filter to create a mask that smoothly merge the edge of two images
+ together.
+
+
+
+
+ 
+ SF Pano Smooth
+ |
+
+
+
+
+
+
+
+
+ 
+ MT Pano Smooth
+ |
+
+
+
+
+
+
+
+
+ 
+ RM Pano Smooth
+ |
+
+
+
+
+
+
+
+
\ No newline at end of file
+ 
+ 
+ 
+ 
+ 
+ 
+ 
+ 
+