TL DR
I tested other image processing libraries (scikit-image, Pillow and Matlab), and none of them returned the expected result.
Most likely, this behavior is associated with the method of performing two-line interpolation in order to obtain effective results or somehow rather a convention, and not an error, in my opinion.
I posted a code sample for resizing an image using two-line interpolation (check if everything is okay, I'm not sure how to handle image indices correctly ...), which displays the expected result.
Partial answer to the question.
What is the output of some other image processing libraries?
scikit image
The Python scikit-image module contains many image processing algorithms. Here are the outputs of the skimage.transform.resize method ( skimage.__version__: 0.12.3 ):
mode='constant' (default)
code:
import numpy as np from skimage.transform import resize image = np.array( [ [0., 1.], [0., 1.] ] ) print 'image:\n', image image_resized = resize(image, (5,5), order=1, mode='constant') print 'image_resized:\n', image_resized
Result:
image: [[ 0. 1.] [ 0. 1.]] image_resized: [[ 0. 0.07 0.35 0.63 0.49] [ 0. 0.1 0.5 0.9 0.7 ] [ 0. 0.1 0.5 0.9 0.7 ] [ 0. 0.1 0.5 0.9 0.7 ] [ 0. 0.07 0.35 0.63 0.49]]
Result:
image: [[ 0. 1.] [ 0. 1.]] image_resized: [[ 0. 0.1 0.5 0.9 1. ] [ 0. 0.1 0.5 0.9 1. ] [ 0. 0.1 0.5 0.9 1. ] [ 0. 0.1 0.5 0.9 1. ] [ 0. 0.1 0.5 0.9 1. ]]
Result:
image: [[ 0. 1.] [ 0. 1.]] image_resized: [[ 0. 0.1 0.5 0.9 1. ] [ 0. 0.1 0.5 0.9 1. ] [ 0. 0.1 0.5 0.9 1. ] [ 0. 0.1 0.5 0.9 1. ] [ 0. 0.1 0.5 0.9 1. ]]
Result:
image: [[ 0. 1.] [ 0. 1.]] image_resized: [[ 0.3 0.1 0.5 0.9 0.7] [ 0.3 0.1 0.5 0.9 0.7] [ 0.3 0.1 0.5 0.9 0.7] [ 0.3 0.1 0.5 0.9 0.7] [ 0.3 0.1 0.5 0.9 0.7]]
Result:
image: [[ 0. 1.] [ 0. 1.]] image_resized: [[ 0.3 0.1 0.5 0.9 0.7] [ 0.3 0.1 0.5 0.9 0.7] [ 0.3 0.1 0.5 0.9 0.7] [ 0.3 0.1 0.5 0.9 0.7] [ 0.3 0.1 0.5 0.9 0.7]]
As you can see, the default resize mode ( constant ) produces a different output, but the edge mode returns the same result as OpenCV. None of the resize modes gives the expected result.
Learn more about Interpolation: Edge Modes .
This snapshot summarizes all the results in our case:
Pillow
Pillow
is a friendly PIL fork by Alex Clark and Contributors. PIL is the Python Image Library by Fredrik Lundh and Contributors.
What about PIL.Image.Image.resize ( PIL.__version__: 4.0.0 )?
code:
import numpy as np from PIL import Image image = np.array( [ [0., 1.], [0., 1.] ] ) print 'image:\n', image image_pil = Image.fromarray(image) image_resized_pil = image_pil.resize((5,5), resample=Image.BILINEAR) print 'image_resized_pil:\n', np.asarray(image_resized_pil, dtype=np.float)
Result:
image: [[ 0. 1.] [ 0. 1.]] image_resized_pil: [[ 0. 0.1 0.5 0.89999998 1. ] [ 0. 0.1 0.5 0.89999998 1. ] [ 0. 0.1 0.5 0.89999998 1. ] [ 0. 0.1 0.5 0.89999998 1. ] [ 0. 0.1 0.5 0.89999998 1. ]]
Pillow image resizing corresponds to the output of the OpenCV library.
Matlab
Matlab offers a toolbox called Image Processing Tool . The imresize function in this toolbar allows you to resize the image.
code:
image = zeros(2,1,'double'); image(1,2) = 1; image(2,2) = 1; image image_resize = imresize(image, [5 5], 'bilinear')
Result:
image = 0 1 0 1 image_resize = 0 0.1000 0.5000 0.9000 1.0000 0 0.1000 0.5000 0.9000 1.0000 0 0.1000 0.5000 0.9000 1.0000 0 0.1000 0.5000 0.9000 1.0000 0 0.1000 0.5000 0.9000 1.0000
Again, this is not the expected result with Matlab, but the same result with the two previous examples.
Custom bilinear image resizing method
The basic principle
See the Wikipedia article on Bilinear Interpolation for more information.
This figure should basically illustrate what happens when scaling from a 2x2 image to a 4x4 image:
When interpolating the nearest neighbor, the target pixel at (0,0) will receive the value of the original pixel at (0,0) , as well as the pixels at (0,1) , (1,0) and (1,1) .
With bilinear interpolation, the destination pixel at (0,0) will receive a value that is a linear combination of 4 neighbors in the original image:
The four red dots show the data points, and the green dot the point at which we want to interpolate.
R1 calculated as: R1 = ((x2 – x)/(x2 – x1))*Q11 + ((x – x1)/(x2 – x1))*Q21 .
R2 calculated as: R2 = ((x2 – x)/(x2 – x1))*Q12 + ((x – x1)/(x2 – x1))*Q22 .
Finally, P calculated as the weighted average of R1 and R2 : P = ((y2 – y)/(y2 – y1))*R1 + ((y – y1)/(y2 – y1))*R2 .
Using coordinates normalized between [0, 1] simplifies the formula.
C ++ implementation
This blog post ( Resizing Images with Bicubic Interpolation ) contains C ++ code to perform image resizing using two-line interpolation.
This is my own adaptation (some changes regarding indexes compared to the source code, not sure if this is correct ) of the code for working with cv::Mat :
#include <iostream> #include <opencv2/core.hpp> float lerp(const float A, const float B, const float t) { return A * (1.0f - t) + B * t; } template <typename Type> Type resizeBilinear(const cv::Mat &src, const float u, const float v, const float xFrac, const float yFrac) { int u0 = (int) u; int v0 = (int) v; int u1 = (std::min)(src.cols-1, (int) u+1); int v1 = v0; int u2 = u0; int v2 = (std::min)(src.rows-1, (int) v+1); int u3 = (std::min)(src.cols-1, (int) u+1); int v3 = (std::min)(src.rows-1, (int) v+1); float col0 = lerp(src.at<Type>(v0, u0), src.at<Type>(v1, u1), xFrac); float col1 = lerp(src.at<Type>(v2, u2), src.at<Type>(v3, u3), xFrac); float value = lerp(col0, col1, yFrac); return cv::saturate_cast<Type>(value); } template <typename Type> void resize(const cv::Mat &src, cv::Mat &dst) { float scaleY = (src.rows - 1) / (float) (dst.rows - 1); float scaleX = (src.cols - 1) / (float) (dst.cols - 1); for (int i = 0; i < dst.rows; i++) { float v = i * scaleY; float yFrac = v - (int) v; for (int j = 0; j < dst.cols; j++) { float u = j * scaleX; float xFrac = u - (int) u; dst.at<Type>(i, j) = resizeBilinear<Type>(src, u, v, xFrac, yFrac); } } } void resize(const cv::Mat &src, cv::Mat &dst, const int width, const int height) { if (width < 2 || height < 2 || src.cols < 2 || src.rows < 2) { std::cerr << "Too small!" << std::endl; return; } dst = cv::Mat::zeros(height, width, src.type()); switch (src.type()) { case CV_8U: resize<uchar>(src, dst); break; case CV_64F: resize<double>(src, dst); break; default: std::cerr << "Src type is not supported!" << std::endl; break; } } int main() { cv::Mat img = (cv::Mat_<double>(2,2) << 0, 1, 0, 1); std::cout << "img:\n" << img << std::endl; cv::Mat img_resize; resize(img, img_resize, 5, 5); std::cout << "img_resize=\n" << img_resize << std::endl; return EXIT_SUCCESS; }
He produces:
img: [0, 1; 0, 1] img_resize= [0, 0.25, 0.5, 0.75, 1; 0, 0.25, 0.5, 0.75, 1; 0, 0.25, 0.5, 0.75, 1; 0, 0.25, 0.5, 0.75, 1; 0, 0.25, 0.5, 0.75, 1]
Conclusion
In my opinion, it is unlikely that the OpenCV resize() function is incorrect, since none of the other image processing libraries I can test gives the expected result and, in addition, can produce the same OpenCV result with a good parameter.
I tested two Python modules (scikit-image and Pillow), because they are easy to use and image-oriented. I was also able to test Matlab and the image processing toolbar.
A rough user-defined implementation of bilinear interpolation for resizing an image gives the expected result. Two possibilities for me can explain this behavior:
- Is the difference inherent in the method that these image processing libraries use, and not an error (maybe they use the method to efficiently resize images with some loss compared to a strict bilinear implementation?)?
- does it somehow conditionally interpolate properly the exception of the border?
These libraries are open source, and you can examine their source code to understand where the discrepancy comes from.
A related answer shows that interpolation only works between two source blue dots, but I cannot explain why this behavior.
Why is this answer?
This answer, even if it partially answers the OP question, is a good way for me to summarize a few things that I found on this topic. I also believe that this can help some people who can find it.