Activity 12:Color Image Segmentation

Image segmentation is used in various applications such as in medical imaging, face recognition, fingerprint recognition and machine vision [1]. It can be thought as the process of locating an object within an image.

There are different ways in implimenting image segmentation, one of which is by thresholding grayscale images. However, this method is sometimes not plausible because there are cases in which the object to be segmented has the same grayscale value as that of the background. In cases such as this, Color image segmentation can be implimented.

Basically in color image segmentation, the color of the object of interest is used in order for it to be segmented in the image. However, the color space is not represented by the RGB values. This is because 3D objects have shading variations that cannot be represented alone by RGB values. Thus for this purpose we use the normalized chromaticity coordinates. That is per pixel,

I = R + G + B;
r = R/I ; g = G/I ; b = B/I

We note that r + g + b = 1, thus b can be represented by,

b = 1 - r - g;

This only means that the chromaticity coordinates can be represented by only two coordinates, r and g. The normalized chromaticity space is therefore represented by the following graph.

Normalized Chromaticity space. The x-axis is r while the y-axis is g.

There are two methods in image segmentation: parametric and non-parametric. In the parametric method, tagging pixels that are similar to those found in the region of interest is done by calculating the probability that it is found in the ROI. This is done by assuming a gaussian distribution independently along the normalized chromaticity coordinates of red and green, the probability for r is therefore given by the equation.

where µ_r and σ_r are the mean and standard deviation of r, respectively. An equation of the similar form is applied to calculate the p(q). The joint probability of obtaining r and q is the product of p(q) and p(r).

While parametric segmentation uses gaussian probability, non-parametric segmentation uses histogram backprojection. In this method, the histogram of the ROI is calculated and this is used as a look-up table in backprojection. That is a pixel location is given a value equal to its corresponding value in chromaticity space.

To test this two methods, the image below was used.

http://www.babygadget.net/pics/kitty-crayons.jpg

A patch of this image was cropped and was used as the region of interest.

The histogram of this patch was calculated using the code given in the exercise protocol and is presented in the image below.

The obtained histogram when compared to the plot of the normalized chromaticity space indeed represents a color in the shade of the green region.

Using the green patch, the green kitten in the original image is segmented first by using the parametric distribution estimation.

As can be observed from the image above, the method was able to locate and segment the object having the same color as that of the patch. Also, notice that the method was able to detect (some amount) portions of the light green kitten. This is because the two kittens detected (green and light green) are of the same color, however differ in brightness.

Using the same patch and its calculated histogram, image segmentation was implemented by using the non-parametric probability distribution and histogram backprojection. The result of applying this method is give by the image below.

Observe that this method was also able to detect both of the kittens just like of the previous one. However, notice that the from the image above it looks like the method was able to locate more portions of the light green kitten than the green one despite the fact that the patch was obtained from the green kitten.

For the second time the same process was applied using a brown patch to segment the brown kitten shown in the reference image.
The images above shows the patch used and the histogram of the patch. As can be observed the brown patch is composed of the colors, yellow orange and red.

After applying parametric and non-parametric image segmentation to the image,

Parametric Distribution EstimationNon-parametric Distrbution Estimation

As can be observed in the obtained results for both of the method, 5 colors were spotted. These are dark orange, orange, dark yellow, yellow and brown. Orange and yellow color of varying brightness (light and dark) were spotted because these colors makes up brown. This is clearly shown in the calculated histogram of the brown patch.

Comparing the two methods, the parametric distribution estimation resulted to more accurate results. I think this is because the non-parametric method is highly dependent on the patch chosen. Remember that in the non-parametric distribution, the histogram of the patch is used for backprojection. On the other hand, in the parametric distribution probability of the occurence of the normalized chromaticity red and green are the ones calculated. A wider range of values is present in the parametric approach as compared to the non-parametric.

For this activity, I will give myself a grade of 10 for I was able to do all the required tasks while enjoying the activity.

I thank Thirdy Buno and Irene Crisologo for discussing with me informations regarding this activity.

References:
[1]http://en.wikipedia.org/wiki/Segmentation_(image_processing)
[2] Activity 12: Color Image Segmentation Manual

Activity 10: Preprocessing Text

For this activity, we were asked to extract handwritten text from an imaged document with lines. The given image is shown below.

Observe that the image is rotated (the lines are not horizontal and is tilted by some angle). To be able to tilt the image such that horizontal lines are horizontal, the function mogrify was used. This resulted to the image below.

After rotating the image, a small portion containing of it was cropped.

Since we want to extract the text, what should be done first is to remove the lines in the image. To be able to do that the Fourier transform of the cropped image was calculated.

Equipped with the knowledge of the previous activities (activity 6 and 7), a filter was created using Gimp to block the frequencies of the lines.

We know from Fourier optics that the frequencies of horizontal lines can be found in a vertical region in the Fourier space. Knowing this, a filter as shown above was created. Notice that the center of the Fourier transform was not blocked. This is because this region contains large amount of information, not only of the lines but the texts as well.

Applying the the created filter and inverting the image resulted to..

The image was then binarized using thresholding,

Notice that the image contains noise, for it to be "clean" morphological operations must be applied. This is done by applying erosion and dilation to the image.


Then afterwards it was thinned to be a pixel thick. This was done by applying the thin function in Scilab.


Lastly, the occurrence of the the word DESCRIPTION was located using the correlation. This was done by creating a binarized image of the same size of the text with the word correlation as the object.The word DESCRIPTION was created with font Arial and a fontsize of 11.


In the image above white signifies high correlation. As can be observe, the locations of the word description was located however locations of other text as well.

For this activity, I'm giving myself a grade of 8. This is because I know I could have done more image processing to obtain better results. However, I was not able to do so because of the time constraint.

I thank Irene and all my classmates who discussed with me this activity.

Activity 11: Color Camera Processing

Almost all cameras today including built-in ones found in regular cellphones have white balancing options. Common white balancing options include cloudy, daylight, fluorescent, tungsten and AWB (Auto White Balance). Before white balancing was explained to me by a senior in the laboratory I belong to, I thought that it was a camera option that was there for color enhancement depending on the scene to be captured. Somehow I was correct, but I actually did not know its deeper definition.

An image is composed of three channels, namely: red, green and blue. These three channels can be described by the following equations.

whereIn these equations, S(λ) represents the spectral power distribution of the incident light, p(λ) is the surface reflectance while n(λ) for each channel are the spectral sensitivity of the camera. Observe that in the RGB equations, a factor K is present. This factor is termed as the white balancing constant that is equal to the inverse of the camera output when shown a white object.

White balancing can be thought as the process of finding and applying the right white balancing constant such that a white patch in an image is visually seen as white.

There are two known methods in achieving automatic white balance: the White patch Algorithm and the Gray World Algorithm. In the White patch algorithm, given an unbalanced image, the RGB values of a known white in the image is used as the coeffient K. On the other hand, the Gray world algorithm assumes that the average color of the world is gray. Thus in this algorithm, the balancing constant is equal to the average R, G and B multiplied by some constant since gray is a family of white.

For this exercise, the two methods: White Patch algorithm and Gray World Algorithm were applied to images of varying white balancing conditions.

The resulting images when the White patch algorithm was implemented. Each column in the images above denotes a white balancing condition. The images in the first column has an auto white balance setting, second column has a cloudy setting, third column has a daylight setting and finally the last column was taken with a fluorescent white balancing setting.

When the gray world algorithm was applied, the resulting images are as follows.

The order of the images is the same as descried in the previous one. It can be obseved that for the gray world algorithm, the result shows that the rendered images are generally whitish. Also notice that in the images rendered using gray world, part of the image is somewhat saturated. This maybe due to the possibility that the cropped white object in the image is saturated. On the other hand, it can be observed that for both of the algorithm, images with white objects appearing as white were able to be reconstructed.

Images of varying brightness were also captured with a white balancing setting of daylight.

The image rendered using the White Balancing Patch algorithm,
On the other hand, that of the gray world algorithm,

Again, it can be observed that the rendered image using the WPA is better than that rendered by the GWA.

For this activity I give myself a grade of 8. This is because The patch that I used for the GWA is saturated.

Acitivity 9: Binary Operations

In biomedical fields, cells from sample slides are classified through its size or area. For this purpose, image processing can be applied. In this activity, we were asked to estimate the area of a punched paper given an image.

Area estimation was done by first cutting the image into 256 x 256 subimages. To be able to get a more accurate estimated area, I decided to cut the image into 30 subimages since image cutting was programed in Scilab. After cutting the image, the next step was to convert the grayscale images (the original image was converted to grayscale first before cutting was done) into binary images. This was done by thresholding. A single treshold value was used for all the images since a uniform illumination throughout the original image was assumed. Here is one of the 30 subimages used.
After binarization..
As can be observed, the binarized image is not that "clean". To be able to "clean" it morphological operations must be used. Equipped with the knowledge of the previous activity, erosion and dilation are applied to the images. This was done by creating a opening function in Scilab. Opening can be described as the process of erotion followed by dilation. The structuring element used is a disk that is larger than the noise but must be smaller than the size of the punched papers.

The image above is the resulting image after applying opening into the binarized image. As can be observed the pepper noise found in the first binarized image is now gone. This is because in opening, the image is first eroded then afterward dilated. Since the size of the structuring element used is bigger than the noise and is smaller than the object, all objects with size smaller than the structuring element will be turned into background. The process was done for all the remaining 29 images.

The goal in this actiity is to estimate the area of a single blob, to be able to do this, the function bwlabel was used. The input of this fuction is a binary image and what it does is it labels an object(region of 1). Using this function, the area of each object in the 30 images were calculated through pixel count. The historgram of the areas calculated is given below.
From the histogram, the outliers can be easily distinguised. These outliers are the area of the blobs that are close to each other that they were seen as a single object, or the blobs that became smaller after opening was implemented. These outliers can be filtered. In this case, only areas ranging from 500 to 580 were considered.

From the filtered histogram, it was estimated that the area of a blob is 513.17 ± 23.33 pixels.

For this activity, I will give myself a grade of 10 for I was able to do all the required tasks.

I thank Jica Monsanto for discussing with me this activity.

Activity 8: Morphological Operations

For this activity, we were able to explore different morphological operators and functions. Morphological operations were applied to five objects, namely: circle, square, hollow square, cross and triangle.

For the first part, the five images were eroded using four different structuring element. The first is a 4 X 4 ones, followed by 2 x 2 ones, 4 x 2 ones and lastly a cross that is 5 pixel long and a pixel thick.

SE = 4 x 4 ones

SE = 2 x 2 ones
SE = 4 x 2 onesSE = cross

The same objects were also dilated using the same structuring elements

SE = 4 x 4 ones

SE = 2 x 2 ones

SE = 4 x 2 onesSE = cross

As can be observed in the obtained dilated and eroded images, dilation pads-up the object depending on the dimensions of the structuring element used. On the contrary, erosion erodes the image as dictated by again the structuring element.

Before programming the erosion and dilation of the objects used, we were first asked to predict the results. Almost all of the predicted results for the dilation and erosion of images were the same as the results when programmed.

Other morphological operations like skel and thin were also explored.

The first column of the image above is the image original image, the second is the image after applying the function skel and lastly in the third column is the image after applying thin. The skel function skeletonizes the image. The thin function is also the same as the skel, however the method in which it thins the object is different to that of scale. As can be observed in the image above, the result of the skel is better than that of the thin function. Discontinuities are present in the thin function.

For this activity, I give myself a grade of 9 for I was able to do all the required tasks. However I was not able to fully discuss the process of morphology.

I thank Ms Herminia Balgos for discussing with me this activity.