Trained Tensorflow CNN giving a random output when tested on an image

I have trained a CNN to classify images of cats/dogs. (It works to an accepted standard for the project I'm working on)

I load in my trained tensor flow model as follows:

sess = tf.Session()

saver = tf.train.import_meta_graph('temp/dogs-cats-model.meta', clear_devices=True)

saver.restore(sess, tf.train.latest_checkpoint('temp/./'))

graph = tf.get_default_graph()

y_pred = graph.get_tensor_by_name("y_pred:0")

there are some other lines in there to load in the image and call the place holders for the the inputs of the image and the labels. I believe that all works fine.

But when an image is tested using the current setup the output I get is a bit random as in I could test the same image 10 times and get a slightly different probability for each class on every run.

I am led to believe it is because the model is still training on these new images and thus I need to know how to make it just test the model on the image and not train at all. Is there a way to make the loaded Tensorflow model just test the image and return the same prediction every time?

Any help is much appreciated.