Batman vs Superman - Image Classification

This is a simple demo to illustrate Feature Extraction and Image classification. While Batman vs Superman is yet to kick off, here is my take on the problem.

In [38]:
#necessary imports
from skimage import io
from skimage.color import rgb2gray, rgb2hsv
from skimage.util import img_as_ubyte, img_as_int
import os
import numpy as np
import warnings
from matplotlib import pyplot as plt
warnings.filterwarnings("ignore")
from sklearn.neighbors import KNeighborsClassifier

%pylab inline

#custom function to diplsya images, handlers grayscale and size is bigger
def show_img(img):
    width = 10
    height = img.shape[0]*float(width)/img.shape[1]
    plt.figure(figsize = (width,height))
    ax = plt.imshow(img, cmap='gray',aspect='auto')
Populating the interactive namespace from numpy and matplotlib

The difference in Brightness

  • Superman and Bartman photos exhibit a stark contrast of brightness, Superman is usually bright and shiny, while batman lingers in the dark ( they don't call him Dark Knight for nothing )
  • To take advantage of this fact, we compute the mean grayscale value over the entire image
In [39]:
#Calculates, average grayscale , if beguf is True, displays intermediate image
def mean_gray(img, debug = False):
    img = rgb2gray(img)
    img = img_as_ubyte(img)
    if debug:
        show_img(img)
    return np.mean(img)
  • Superman image shows a moderate average gray value
In [40]:
img = io.imread('images/1.jpg')
print mean_gray(img, True)
127.355975057

  • Batman image shows low average grayscale value
In [41]:
img = io.imread('images/2.jpg')
print mean_gray(img, True)
16.2633492477

The Blueness

  • Superman photos are filled with blue color, either from his suit, or from the sky behind.
  • Batman photos are filled with, ....... well darkness
  • To exploit this fact we can compute, the proportion of pixels with a color similar to Blue
In [42]:
#converts image to HSV, to compare with blue color
def mean_blue_diff(img, debug = False):
    img = rgb2hsv(img)
    img = img_as_ubyte(img)
    hue = img[:,:,0]
    diff = hue - 140 # hue value of blue
    # pixels within a threshold of blue color
    diff = np.abs(diff) < 10
    if debug :
        show_img(diff)
    
    return np.mean(diff*255)
In [43]:
img = imread('images/6.jpg')
print mean_blue_diff(img, True)
155.595042735

In [44]:
img = imread('images/2.jpg')
print mean_blue_diff(img, True)
42.0993489583

  • Load images and specify labels
In [45]:
images = [ io.imread('images/' + name) for name in sort(os.listdir('images')) ]
training_data = [ (mean_gray(i),mean_blue_diff(i)) for i in images ]
training_data = np.array(training_data)

Plotting the training data

  • Superman examples are towards the upper right, whereas batman examples are towards the bottom left
In [46]:
#plt.scatter(training_data[:,0], training_data[:,1])
expected_values = [1,0,0,0,0,1,1,1]  # superman=1, batman=0 as class labels

for i in range(len(training_data)):
    a,b = training_data[i]
    if expected_values[i] == 0:
        plt.scatter( [a], [b] , color='#FF0000')
    else :
        plt.scatter( [a], [b] , color='#0000FF')

Use a KNN Classifier to do the classification

In [47]:
neighbours = KNeighborsClassifier()
neighbours.fit(training_data, expected_values)
Out[47]:
KNeighborsClassifier(algorithm='auto', leaf_size=30, n_neighbors=5, p=2,
           warn_on_equidistant=True, weights='uniform')

Assign labels to Training Examples

In [48]:
batman_logo = imread('logos/batman.jpg')
brows,bcols = batman_logo.shape[0:2]
superman_logo = imread('logos/superman.jpg')
srows,scols = superman_logo.shape[0:2]


batman_points = []
superman_points = []
for image in images:
    
    a,b = mean_gray(image), mean_blue_diff(image)
    
    prediction = neighbours.predict((a, b))
    
    if prediction[0] == 1:
        superman_points += [(a,b)]
        image[:srows,:scols] = superman_logo
    
    else:
        batman_points += [(a,b)]
        image[:brows, :bcols] = batman_logo
        
    show_img(image)