Calibrating for distortion:
each lens has distortions. There are pincushion type of distortions – concave image. Barrel – convay. The most obvious is the fish-eye lens distortion. In the case of real lenses the distortions are even more complicated – not pure pincushion and not pure barrel – the shape is more complicated. We have to compensate for the lens distortions to be able to perform pixel-mapping in panorama stitching. Once the image is straightened we can assign each pixel it's ray in space from the center of the lens to the pixel on the image.