Use dstack :
>>> np.dstack((A, B)).shape (480, 640, 4)
This handles cases where arrays are of different sizes and stack arrays along the third axis.
Otherwise, to use append or concatenate , you need to make B three-dimensional on your own and specify the axis you want to join:
>>> np.append(A, np.atleast_3d(B), axis=2).shape (480, 640, 4)
source share