Image Convolution

How the model works

For each output pixel, the kernel K is centred on the matching input pixel, each of the 9 neighbours is multiplied by the kernel weight on top of it, and the products are added:

out(x, y) = Σᵢ Σⱼ K(i, j) · in(x + i, y + j) / divisor + offset, i, j ∈ {−1, 0, 1}

(Strictly, flipping the kernel first makes this a convolution rather than a correlation; for the symmetric kernels used here it makes no difference.) Results are clipped to the range 0 (black) to 255 (white). At the image border, missing neighbours copy the nearest edge pixel.

The sum of the weights tells you what happens in flat areas, where all neighbours are equal: a sum of 1 keeps the brightness (blur, sharpen), a sum of 0 turns flat areas to 0 and keeps only changes (edge detectors), which is why an offset of 128 is often added to show negative values as darker than mid-grey.

Blur kernels average neighbours. Sharpen adds the difference between a pixel and its neighbours, exaggerating edges. The Laplacian responds to edges in every direction; Sobel kernels respond to brightness changing in one direction, the basis of many edge detectors in computer vision and the first layer of convolutional neural networks.

Greyscale 32 × 32 test image, one 3 × 3 kernel at a time, edge pixels replicated, and integer rounding of the output.

Guided experiments

1. Box blur. All nine weights 1, divisor 9. Prediction: sharp edges become 3-pixel ramps; the one-pixel diagonal line fades to about a third of its contrast. Box blur
2. Sharpen. Centre 5, four neighbours −1 (sum 1). Prediction: flat areas keep their brightness exactly, but each edge gets a bright rim on the light side and a dark rim on the dark side, often clipped to white or black. Sharpen
3. Laplacian edges. Centre −4, four neighbours 1 (sum 0), offset 128. Prediction: every flat area becomes mid-grey (128), and only the outlines of the shapes stand out. Laplacian
4. Sobel, one direction. Detects left–right changes only. Prediction: the vertical sides of the square light up (one side bright, the other dark), while its top and bottom edges vanish. The background, which brightens gently from left to right, shows as a slightly lighter grey than 128./span> Sobel x

Challenge

A pixel of brightness 100 sits in a flat region of value 100, except that its right neighbour is 200. What does the sharpen kernel (centre 5, four neighbours −1) output there? What does the box blur output?

References

• Gonzalez, R. C. and Woods, R. E., Digital Image Processing, 4th ed. (2018), ch. 3.
• Sobel, I. and Feldman, G. (1968), "A 3×3 isotropic gradient operator for image processing", presented at the Stanford Artificial Intelligence Project.
• LeCun, Y., Bengio, Y. and Hinton, G. (2015), "Deep learning", Nature 521: 436–444.