Convolution Neural Network application Python

I have a trained CNN where the input is a 5x5 grid, and the output is a single floating point value. The basics of the model is such that the 5x5 grid variables (var1, var2, var3) are centered over the truth value which are scattered throughout an entire dataset, where the entire dataset is typically a 540 x 596 grid. I have named the model 'mymodel.h5', so the weights should be saved from the training. I have used a large amount of training / validation data, while the independent testing data shows good performance (correlation ~ 0.97, MAE ~ 2.3, and mean bias ratio ~ 0.98).

The idea now is to run this 5x5 CNN model on an entire dataset, where the full grid has dimensions 540 x 596. I want to extract every 5x5 grid possible and then run this through the CNN model, and recreate a grid of the results. However, the resultant grid does not seem to look like how I expected the results to look, and am wondering if there is a bug in the code.

 # load in the data
 f1 = netCDF4.Dataset('/path/to/data.netcdf')
 # How big is the grid size?
 gsize = 5
 # Get the values from the netcdf file
 Z1 = f1.variables('VariableName')
 # Get the dimensions of the data
 lenx = Z1.shape[0]
 leny = Z1.shape[1]
 # Pre-allocate the matrix
 allZ = np.zeros([(lenx-gsize) * (leny-gsize), gsize, gsize])
 cnt = 0

 # Now extract the gsize x gsize grids from the data within Z1
 for x in range(gsize, lenx-gsize):
     for y in range(gsize, leny-gsize):
         cnt +=1
         allZ[cnt,:,:] = Z1[x:x+gsize,y:y+gsize]

 # Load model
 model = load_model('/path/to/mymodel.h5')
 sim_predict = model.predict(allZ, verbose = 1)

 # Reshape the sim_predict variable to be the same dimensions as Z1
 reshape_sim = np.reshape(sim_predict,(lenx-gsize, leny-gsize))

allZ should be a matrix that loops through each of the gsize x gsize (in this case, 5 x 5) grids available within the domain, while sim_predict is the output value from each 5 x 5 grid. I have padded the edges by 5 to remain within the domain as well, yet the results look weird. Any thoughts on the structure of the code?