Month: June 2020

While considering and changing individual pixels isn’t the most efficient way of modifying an image, it can certainly help with a person’s understanding of what is truly happening with an image at the pixel level. In this code, we will modify the pixels in such a way to make the image look more “cartoonish”, with solid colors representing all colors within a given range. In this implementation I chose numbers that were suitable for my particular picture, but may not be best for your picture, so I encourage people reading this to change numbers as they see fit, and learn how this changes the image modification.

In the above code, we define a new function called color_pic, taking as input the image that you would like to cartoonify, evaluating each pixel by itself, modifying it, and showing the final image at the end. The first thing we do is copy the image into a separate variable, so that the original values of the image could still be accessed by other parts of the code later. In many cases, this step isn’t necessary, but if you are not constrained for memory it may be worth it to be safe rather than ending up with unexpected values. After this, the for loops begin.

In this code, we make use of 2 “for loops” to consider each pixel in the x and y directions, respectively, which are also the first and second dimensions of the image. The third dimension, again, is to tell which of the three colors (blue, green, or red) you are currently considering. I should note that for loops are generally pretty inefficient, but here they can do a better job of . The for loops are constructed by using the range function, which is native to python, and creates a list of numbers within a range you define. If you type print(range(5)), it will return a list: [0, 1, 2, 3, 4]. If you type print(range(2, 5)), it will return a list: [2, 3, 4]. There are additional options for range, but they aren’t important for the purposes of this exercise. Using the “for loop” essentially just means that it will consider all of the listed cases for a variable. For example, if you type for i in range(5): print(i), the program will print lines listing 0, 1, 2, 3, and 4, as these are the different values that i will represent before the for loop is complete. In our case, we will have i represent all of the numbers between 0 and the height of your input image (which is the first dimension of the image, as can be found by typing my_im.shape[0], or im.shape[0], if you do it within a function. The second dimension of the image (the horizontal direction) can similarly be found by typing my_im.shape[1] or im.shape[1].

So, within the for loops, calling im[i, j] indicates that you are considering a single pixel which is i pixels down from the top of the image and j pixels right from the left of the image. Typing print(im[i, j]) will return 3 values which correspond to the blue, green, and red values that are present in the pixel you are currently considering. Each of these values can additionally be individually called using im[i, j, 0], im[i, j, 1], and im[i, j, 2], respectively. The rest of the code uses these constructions to determine how to modify the image.

Line 38, for example, determines if the currently considered pixel contains B, G, and R values that are all above 200. As the maximum value of each is only 255, this would mean that these pixels are almost completely white (if you’re confused about this, please consult this website again). In line 39, it then makes any pixel meeting this requirement to be exactly white, with maximum values for each of B, G, and R for that pixel. Similarly, line 40 determines if the B value is greater than 165. If so, then it changes this pixel to be a pre-defined color with B, G, and R values of 230, 150, and 20, respectively. Each pixel goes through many similar considerations in lines 38-54, each considering whether it falls within a pre-specified range of B, G, or R values. If the pixel doesn’t meet any of these conditions, it is finally assigned to be a sort of dark grey color with B, G, and R values each equal to 50.

The result of this code (you will notice it is slow compared to the others, due to the inefficiency if the for loop) is shown below. You can imagine how a similar technique (with some tweaking and polishing) could achieve something similar to the previously very popular Obama “HOPE” photo, also shown below:

The next technique that we will discuss is edge detection, which is one of the most simple ways of detecting “features” in the image. When talking about image features, what we really mean is something in the image that can be mathematically defined which distinguishes one part of an image from the rest of the image. The pixel values themselves can be considered features, but it is often helpful to extract different types of features that can allow to generalize about different regions of an image. This method of edge detection, similar to image blurring, moves a kernel throughout an entire image, and highlights the pixels where the values change dramatically when compared to their neighbors. For this technique, we will define our own filter (albeit a common one called the Sobel operator) and apply it to the entire image.

Looking at the functions defined below, you should notice that they are almost exactly the same as the blur_image function we previously defined. The only difference comes in the first line (lines 15-17, and lines 25-27, respectively), where we define the kernel. I should note that while this definition takes up 3 lines in reality, the program treats this as only one line because lines 15 and 16 never properly end the statements that come before it. You can see that the function we use (np.array) has a “([[“ right after the function name, while the end of the line only contains “],”. These first three lines could have similarly been typed out: “hor_kernel = np.array([[1, 0, -1], [2, 0, -2], [1, 0, -1]])”, but I chose to separate them into 3 different lines in order to emphasize that we are, in fact, defining a 2-dimensional 3×3 array to serve as the filter. Notice, as well, that in defining the array, we have called the NumPy library (shortened to np).

A Sobel filter is always oriented with negatives on one side of the filter, positives on the other, and zeros in the middle. Applying any filter to a group of pixels is actually done via a process called convolution, which is demonstrated on the right below. Here, the Sobel filter is being applied to 2 pixels, the orange and the blue. In convolution, you place the center pixel of the filter over the pixel you are currently evaluating, then multiply each overlapping element, and add everything together. This results in a value of -23 for the orange square and -310 for the blue square. However, in reality only the magnitude of these values will be considered, as the edge strength is considered as one thing (the magnitude), while the edge direction determines the positive or negative sign. When additionally using a vertical edge filter (as is defined in our function vert_edge below), the overall edge strength can be calculated, as well as a specific angle, which can be calculated based on the ratio of these two edge values for a particular pixel. It should be noted that detecting the edges of a color image requires running through the 2-dimensional image 3 times, once for each the blue, green, and red values of a given pixel. This results in edge strengths for each color, which can again be combined to form the final edge-detected image.

After typing the above functions into your Image_Process.py file, you can test them in the same way as demonstrated by the blur_image function above. You should see images that look similar to the below pictures. Notice the highlighted areas for the horizontal and vertical filters, respectively. The horizontal kernel results in mostly vertical lines, while the vertical kernel results in horizontal lines, because they are horizontally and vertically comparing pixels, respectively:

We can also write a function to combine these two edge detectors together through addition. This may not be the most effective way of combining them, but it can achieve some interesting results. Note that the only reason that direct addition works is because of the datatype of the number. These numbers are represented as int8, meaning that they are an integer with 8 bits. As each bit is binary, that means that this number can only hold up to 2⁸ = 256 possible numbers, matching the values between 0 and 255 which are used for the GBR values of each pixel. You can think of this by saying that any number higher than 256 (or any multiple of 256) subtracts 256 until the given number fits within these constraints. In the actual computer, it just means that any bit past the 8^th bit (or any bits beyond this) isn’t considered, as it hasn’t been allocated in memory. The code and results of this can be seen below: