Activity 19: Retoration of Blurred Images

Have you ever taken a picture where the your object is blurred because of its apparent motion while the image was being taken? This kind of blurring is termed as motion blur. Although it is a kind of blurring which might be unwanted for some instances, motion blur is a favorite style in photography. Here are some motion blurred images I've searched around the web.

images are taken from: http://www.smashingmagazine.com/2008/08/24/45-beautiful-motion-blur-photos/

Motion blur is often caused by the long exposure time of the camera. Exposure time is the duration of time that the camera's shutter is open. In this case, the object moved within the exposure time causing the blur in the image captured.

For activity, we restored motion blurred images. In doing this we used degradation modeling. to be able to blur a given image. Here, we consider the case where in the image represented by f(x,y) has been blurred by the linear movement between the object and the image acquisition device used. Assuming that the opening and closing of the shutter happens instantaneously, the blurred image is given by

Equation 1

where x_o and y_o represents the time varying components of motion in the x and y directions, and T represents the exposure time.

Consequently, the Fourier transform of the bulrred image is given by,

Equation 2

where G(u,v) and F(u,v) blurred and original images in frequency space while H(u,v) is the degradation function in Fourier space given in the following equation.

Equation 3

where a and b are the total distances for which the image has been displaced both in the x and y directions.

After blurring the image, gaussian noise was also added into it. Thus G(u,v) can now be expressed as

Equation 4

where N(u,v) is the added noise in frequency space.

Now to be able to recover the original image from its blurred counterpart we used the Minimum mean square error or Weiner filtering. In this method, the image and the noise added are considered as random processes. The orginal image in frequency space is then given by,

Equation 5

Where S_{n and} S_{f are the power spectrum of the noise and the original image both in frequency space.} However, in reality the power spectrum of the ungraded or the original image is unknown. Thus the equation for the restored image can be written as,

Equation 6

where K is a constant.

Using Equations 2 and 3, blurred image of the text below with a and b equal to 0.1 and T = 1 were simulated. This blurred image was then used to recover the original image using equations 5 and 6. These images are shown below.

Original image

Using equation 6, the original image was recovered for varying values of K. What I used here are: 5, 3, 0.01 and 0.00001.


As can be seen from the figure above as K decreses, the recovered image looks more like the original. The values of a and b (x and y displacements) were also varied. The recovered images are shown below. The First column of the suceeding 4 sets of images are the blurred images, the second column are the recovered images using Equation 6 and the third column are the recovered images using Equation 5.

a & b = 0.9

a & b = 0.9a & b = 0.01a & b = 0.001

As can be observed in the images above, as the value of a and b decreases the recovered images looks more like the original image. This can be seen especially in the last set of images where a and b are equal to 0.001. This behaviour is expected since a and b are the displacements of the object in the x and y directions. Small values of a and b only mean that the object moved a small distance from its original position while its image is being captured. Thus as the value of a and b increases the generated blurred image becomes more blurred.

The values of the exposure time, T, was also varied. The recovered images are shown below. The First column of the suceeding 4 sets of images are the blurred images, the second column are the recovered images using Equation 6 and the third column are the recovered images using Equation 5.

T = 0.001
T = 0.01

T = 3
T = 5

As can be observed from the four sets of images above, as the value of T increases, the recovered images look more like the original one. Also, observe that the generated blurred images for small value of T are dark. If we will recall the definition of T, it is the duration of the exposure time or the duration of time that the shutter of the imaging device is opened. For small values of T, small amount of light will be able to enter the imaging system. Thus, the generated dark blurred images.

A possible extension of this activity is the recovery of the "unblurred" image of a nactual blurred image taken. However to be able to do this, one must have the knowledge of the degradation function of the blurred image.

For this activity I thank Irene Crisologo, Thirdy Buno, Jaya Combinido and Miguel Sison for the helpful discussions.

I give myself a grade of 10 for I was able to do all the required tasks for this activity.

References:
[1] Activity 19: Retoration of Blurred Images Manual

Activity 18: Noise Models and Basic Image Restoration

In reality the images that we are able to capture does not exactly replicate the object of our desire. Often than not, the images that we capture contains noise that can be due to the environment or background while acquiring the images, the object itself or even the imaging device that we use.

In this activity we played with different noise: exponential, gamma, gaussian, rayleigh, salt and pepper and uniform noise. These noises were added to an image. Same as adding noise to the image, we tried to restore the original image using four image restoration techniques - arithmetic, contraharmonic, geometric and harmonic.

Noise is composed of random variables that are characterized by a probability distribution function or PDF [1]. The PDF of exponential, gamma, gaussian, rayleigh, salt and pepper and uniform noise are given in the following equations.

For Exponential noise, it's PDF is expressed by,

where the mean and variance are given by,
The PDF of Gamma noise is given by,

where the mean and variance are given by
The PDF of Gaussian noise is given by,
where µ is the mean value of z and σ is its standard deviation. Meanwhile the PDF of the salt and pepper noise is given by,
The PDF of the Rayleigh noise is expressed by the following equation,
where the mean and variance are given by,
The PDF of Uniform noise is given by,
where the mean and variance are given by,
These six noise were generated and added to the image shown below.

Its histogram is given by,

The addition of the six different noise to the image above are shown in the following image.

GENERATED IMAGE WITH NOISE
clockwise direction: exponential noise, gamma noise, gaussian noise, rayleigh noise, salt and pepper noise, uniform noise.

HISTOGRAM OF GENERATED IMAGE WITH NOISE
clockwise direction: exponential noise, gamma noise, gaussian noise, rayleigh noise, salt and pepper noise, uniform noise.

The generated images with noise were restored using four image restoration techniques - aritmetic filtering, contraharmonic filtering, geometric filtering and harmonic filtering. Given that the corrupted version of an image is described by g(x, y), a retangular window of size m x n centered can be defined such that this window g(s, t) is centered at point (x, y) . Following this, the arithmetic mean filter is given by the following equation,

On the other hand geometric mean filter is given by,The harmonic mean filter is given by,The contaharmonic mean filter is given by,where Q is called the order of the filter. For positive values of Q, the filter pepper noise. On the other hand for negative values of Q, salt noise are eliminated.

The four filters described above were applied to the images with noise described above. The results are shown below.

row(top to bottom): exponential noise, gamma noise, gaussian noise, rayleigh noise, salt and pepper noise, uniform noise.
column(left to right): type of noise, arithmetic filter, contraharmonic filter (Q = 1), geometric filter, harmonic filter

It can be observed that indeed for Q = 1, the contraharmonic mean filter eliminates pepper noise. For Q = -1, the result of applying this filter to an image with salt and pepper noise is shown in the following image.

Observe that since a negative value for Q was used, the resulting image has no pepper noise. However it can be observed that the image has more pepper noise than the previously noisy image. It seems like part of the image with salt noise were converted into pepper noise. This is also observed to the image of the restored image when Q is positive. However instead of having several pepper noise, the image is full of salt noise.

The same process was applied to a different grayscale image.

image taken from: http://incisors.files.wordpress.com/2008/06/gs.jpg

when noise was added to this image,

IMAGE WITH NOISE
clockwise direction: exponential noise, gamma noise, gaussian noise, rayleigh noise, salt and pepper noise, uniform noise.

Applying the filters resulted to the following,

row(top to bottom): exponential noise, gamma noise, gaussian noise, rayleigh noise, salt and pepper noise, uniform noise.
column(left to right): type of noise, arithmetic filter, contraharmonic filter (Q = 1), geometric filter, harmonic filter

Comparing values of Q (positive and negative) for the contraharmonic filter,

(left-right): Q = 1, Q = -1.

Again it can be observed that at positive values of Q, pepper noise is filtered out from the image and at negative value of Q, salt noise is filtered out.


For this activity I'll give myself a grade of 10 for I was able to do all the required tasks.

I thank Jica Monsanto, Irene Crisologo, Thirdy Buno and Janno Vergara.


References:
[1] Activity 18: Noise Models and Basic Image Restoration Manual

Acitivity 17: Photometric Stereo

3D imagery is of increasing importance in many fields such as biomedicine and computer vision. There are different ways of obtaining the 3D reconstruction of an object. Some of these are stereo vision, structured illumination, 3D from motion and photometric stereo. In photometric stereo, depth is calculated from shadow cues. This is done by capturing images of the objects with the source at different locations. Let V be the matrix describing the x, y and z component of the light source and I be the matrix containing the images of the object at different light source location. The relationship between the source location and a specific location at the surface of the object as seen by the camera is given by,

I = Vg

g can be solved using the least square method and is given by,

Using g, the normal vector can be calculated by normalizing g. Once the surface normals are estimated, they are related to the surface elevation or the depth z which is here expressed as the function f,
The depth z is at a point (u,v) in th image of the object is evaluated by a line integral given by,


In this activity, the 3D reconstruction of a sphere was rendered. This was done by capturing images of the sphere at different light source locations. The following figure shows the images of the sphere used.Using the equations discussed earlier, the surface normals of the sphere were calculated. The depth z of the object was then calculated and visualized using the plot3d function in Scilab. The obtained reconstruction is shown below.
As can be observe, the reconstruction resembles a sphere. However, it can be observed that there are visible grooves.

For this activity I give myself a grade of 9. Although I was able to reconstruct the sphere, still I fail to explain the reason for the visible grooves.

I thank Jica Monsanto and Irene Crisologo for useful discussions.


References:
[1] Acitivity 17: Photometric Stereo Manual

Activity 16: Neural Networks

Similar to the previous activity, Activity 16 is an extension of Activity 14. Here, we again use the features that we were able to extract in activity 14. However, unlike activity 14 and 15 where we used minimum distance classification and least discriminant analysis to determine class membership, in this activity what we'll use is neural networks.

A neural network is a computational model of how neurons in the brain work. It is one of the preferred method in pattern recognition for one does not need heuristics and recognition rules to perform classification. In this method, neural network "learn" the rules of mapping by example. [1]

For this activity what we want to do is to train the neural network with the training set for each class. The neural network should learn the "characteristic" feature of a specific class. To be able to perform this, we used the Artificial Neural Network toolbox for Scilab. We utilized the code posted in Cole Fabros' AP186 blog. Assigning the learning rate be equal to 0.1, 0 for the error tolerated by the network run for 1000 cycles the results are the following..

The first column is the expected class of each test object, the second column is the assigned output assignment. This means that if the input test object belong say to the class of flower, then output should be near the assigned output assignment. As can be observed in the table below, indeed the neural network was able to classify the test objects correctly.



For this activity I give myself a grade of 10 for I was able to obtain 100% accuracy in class classification.


I thank Thirdy Buno, Miguel Sison for helping me install the Artificial Neural Network Toolbox and Cole Fabros and Jeric Tugaff, former students of AP 186, forcode utilized in this activity.


References:
[1] Activity 16: Neural Networks Manual

Activity 15: Probabilistic Classification

This activity is an extension of the previous activity, Pattern Recognition. However instead of using minimum distance classification to determine class membership what we used is Linear Discriminant Analysis or LDA. As defined in [1], Linear Discriminant Analysis finds a linear transformation of two predictors that yields a new set of transformed values that provide a more accurate discrimination than either predictor alone.

To fully understand the math of LDA, I recommend to visit the site in [2]. So applying LDA as described in [2] to the data gathered in the previous activity resulted to the following.

The table above shows the classification obtained for each test object. Each table contains 5 test objects belonging to a specific class. For example the first table above shows the result in classifying 5 test objects which actually belong to the class of long leaves. Same follows for the second table - rectangular leaf, third table - flower, fourth table - 25 centavo coin.

Class membership is determined by the largeness in value of the calculated discriminant function. As can be seen from the table above, all the test objects were correctly classified resulting to 100% accuracy of the method used.

For this activity, I will give myself a grade of 10 for I was able to do all the required task.

I thank Jica Monsanto for the useful discussions.


References:

[1] http://www.dtreg.com/lda.htm
[2] http://people.revoledu.com/kardi/tutorial/LDA/LDA.html#LDA

Activity 14: Pattern Recognition

With today's technology, cameras with smile recognition and laptops having face recognition security are not just devices seen in a science fiction movie but are available to ordinary peoples' use. But the big question is, how was a camera able to detect that a person is smiling? In the same way that how was a laptop able to recognize the owner's face?

In this activity, we studied pattern recognition. In image processing, features are defined as quantifiable properties such as color, shape, size etc. A pattern is a set of features. A class, on the other hand, is a set of patterns that share a common property. The aim in pattern recognition is to decide if a given feature belongs to one of several classes [1].

To understand the concept further, let's consider an image of a plant with its flower and leaves. Our task is to identify a leaf from a flower. The leaf and flower are both classes that are composed of patterns that make these classes unique from one another. For this example, we may use color, size, eccentricity and even shape as features to identify a flower from a leaf. After which we are now ready to classify the classes. We have to know to if a class belongs to a group of another class. To be able to do such decisions, classifiers are needed.

Classifiers attempt to find decision boundaries that separate the classes. Depending on the features employed the decision boundaries may be a plane, a convex or concave surface, or arbitrary closed regions in feature space [1].

For this activity, the classifier used is the minimum distance classification. With this type of classifier class membership is calculated by computing for the distance d expressed in the following equation.
where m is the mean pattern for each class. A class x belongs to class j with mean m if the calculated distance d is maximum.

In this activity, four different objects were classified: long leaf, rectangular leaf, flower and 25 centavo coin.
Each of these groups of objects were cropped from the image shown above. For each set of objects, 5 were used as training while the other five were used as test objects. The features used are: quotient of the area and the squared of the perimeter, and the quotient of the perimeter and the width of the object(length of in the y-direction). As can be observed the features used are both unit-less. This is because a much as possible, it is desired that the feature used be scale-invariant. This means that the setting and other condition (height of the camera from the object) won't affect classification.

The graph above shows the plot of the classes extracted from the training set. As can be noticed the class of the flower and 25 cents are far from the rest of the classes. However, it can be observed that the class of the long leaf and the rectangular leaf are near from one another.

After applying the minimum distance classification in order to classify to what class the remaining 20 test objects (five for each kind of four objects) belongs. The following results were obtained.
The tables above shows the calculated distances for each of the test objects. Class membership is determined by the largeness of the calculated value. In the first table, the five test objects are from the long leaf class. As can be observed, the calculated distance was maximum at the long leaf column for all test objects. This means that the test objects were correctly classified as long leaves. This is the same for the remaining three graphs: rectangular leaf, flower and 25 cents test objects.

For this activity I give myself a grade of 10 for I was able to use features that resulted to 100% accuracy of the classification.

I would like to thank Irene Crisologo, Jica Monsanto and Thirdy Buno for useful discussions.


References:
[1] Activity 12: Pattern Recognition Manual

Activity 13: Correcting Geometric Distortion

Have you ever observed bent lines that should actually look straight in images? Image distortions are introduced by the spherical lens used in capturing the image. There are two common kind of image distortion: barrel and pincushion. In barrel distortion, the image seems bloated in the middle and pinches at the sides where as in pincushion distortion, the image seems pinched in the middle and expanded in the boundaries [1]. Figure 1 shows an image of the Petronas Twin tower that exhibits barrel distortion.

Figure 1. http://www.photos-of-the-year.com/barrel-distortion/1.jpg

In this activity, we corrected distortions found in images. To be able to do the correction, the pixel locations and grayscale values of the image would be altered.

The first step in generating the undistorted image is to find the ideal pixel locations of the grids.

Let g(x,y) be the function describing the distorted image while the undistorted image is represented by the function f(x,y) . The transformation from the distorted to the undistorted image can be expressed by the equations,
From this equations, the distorted pixel locations is expressed by,
To be able to calculate the values of all the c's, four points are needed. These points should be near one another (corner points). Thus a set of c's for each set of corner points should be calculated.

After looking for ideal pixel locations, gray level interpolation techniques should be applied. For this purpose, we used the bilinear interpolation. In this method, the graylevel v at an arbitrary location (x, y) in the ditorted image can be found using the equation,

The four equations needed to compute for the four constants (a, b, c, d) maybe formed using the four nearest neighbor pixels encompassing the pixel point (x, y).

For this activity, we use the distorted image of a grid,

image taken from: http://images.trustedreviews.com/images/article/inline/7690-FujiJ10barrel.jpg

The ideal grid point of this image was calculated using one of the grids that seems to be undistorted. Using the dimensions of this grid, the ideal grid points were recovered.

Applying the algorithm discussed above, the undistorted image was recovered.


The undistorted image still seems a little bit distorted. This maybe due to the fact that the corners(four neighbor points) used for the computation are relatively far from one another. In this case, the corner points of one hollow block compose a set.

For this activity I give myself a grade of 9.3. This is because I was able to recover the undistorted image. However it seems like the recovered undistorted image still have a little distortion.

I thank Jaya Combinido, Thirdy Buno, Miguel Sison and Jica Monsanto for useful discussions.

References:
[1] Activity 13: Correcting Geometric Distortion Manual