Oh my God. Oh dear, dear, dear.
I looked through the documentation twice, and only after posting this question did I find the historyApiFallback
settings, as described here.
Add this to your settings:
devServer: { historyApiFallback: true, proxy: { ... } }
I keep this question here, hoping this will help others save time in the future.
source share