A while ago we’ve shown how to get predictions from a Pylearn2 model. It is a little tricky, partly because of splitting data into batches. If you’re able to fit your data in memory, you can strip the batch handling code and it becomes easier to see what’s going on. We exercise the concept to distinguish cats from dogs again, with superior results.

Step by step

You have a pickled model from Pylearn2. Let’s load it:

from pylearn2.utils import serial

model_path = 'model.pkl'
model = serial.load( model_path )

Next, some Theano weirdness. Theano is a compiler for symbolic expressions and with these expressions we deal when predicting. We need to define expressions for X and Y:

X = model.get_input_space().make_theano_batch()
Y = model.fprop( X )

Mind you, these are not variables, but rather descriptions of how to get variables. Y is easy to understand: just feed the data to the model and forward-propagate. X is more of an idiom, the incantations above make sure that the dimensionality fits.

In case of classification, you need to throw in argmax:

Y = T.argmax( Y, axis = 1 )

The next step is to define a link between X and Y. This link is a function that takes X and returns Y. It’s not an ordinary function, but a theano.function: the interface for compiling graphs into callable objects.

f = theano.function( [X], Y )

The final step is what you’d expect: you provide some data to the function and it returns predictions:

y = f( x_test )

Both x_test and y are numpy arrays. That’s it, folks. Go predict.

Pugs on a road trip. Credit: @CuteEmergency

A practical example

In the previous article we learned how to get 88% accuracy with a pre-trained network. The notion of using such net comes from Kyle Kastner, one of the top contenders. Kyle has been publishing his code for the contest all along. The code scores around 96% and uses decaf.

The general idea is to use a pre-trained network not to classify, but to extract features from images. kaggle_dataset.py transforms one image into 4096 float features. Then you train a custom classifier, here a biggish perceptron with two hidden layers, rectified linear units and dropout. The resulting network is able to distinguish between classes much better than the original model because it’s trained specifically for the task at hand.

Initially there was no prediction script so you had to write one yourself. Now it’s available, but still - here’s our simpler version without batches.

UPDATE: Ian Goodfellow merged the slightly more polished script into Pylearn2, as scripts/mlp/predict_csv.py.