From the main branch, you can simply reinstall the new b3 , saving the merge using the --preserve-merges (or -p short) option:
git rebase -p feature
That way, when Git rebases, it will not try to smooth out the merge, but instead recreates it on top of the new base commit. So your story will look like this:
master ↓ a1 -- a2 -- a3 --------- a4' -- a5' \ / \ / b1 -- b2 -- b3 ↑ feature
Compared to the following when the --preserve-merges flag is not used:
master ↓ a1 -- a2 a3' -- a4' -- a5' \ / \ / b1 -- b2 -- b3 ↑ feature
source share