The way to solve this problem is to find a way to combine information from A and B so that filtering itself becomes easy.
The first thing I thought was to link A and B in the third dimension and pass in a filter mask that will take 8 elements from the “A-slice” and the central element from the “B-slice”. Unfortunately, this is not supported by Matlab.
While nlfilter only works with 2D images, it allows you to specify any function for filtering. So you can create a function that somehow can find the correct values for A and B. So, I came to my first decision.
You create a new C array that contains the index of the element for each element, i.e. the first element is 1, the second element is 2, etc. Then you start nlfilter, which takes a 3x3 sliding window and passes the C values inside the window to the filter function, ffn. ffn is an anonymous function that calls crazyFilter, and it was initialized so that A and B are passed on every call. CrazyFunction takes values from the sliding window C, which are nothing more than the indices in A and B, and collects the values from A and B from them.
The second solution is exactly the same, except that instead of moving the sliding window, you create a new array that in each column has the contents of the sliding window in any possible place. With an overlapping window, the column array becomes larger than the original array. Again, you just need to use the values of the array of columns C, which are the indices in A and B, to look up the values of A and B in the appropriate places.
EDIT If you have enough memory, im2col and col2im can significantly speed up the process
%
Old version (uses less memory, filling may be required)
%
Jonas source share