To show your opinion:
Minor incremental changes that leave the code in a better state than was found
Definitely Yes: “Cosmetological” changes that are not directly related to the functions (i.e. they are not paid as a change request).
Definitely No: Rewriting large chunks clearly violates the "small, incremental" part. Refactoring is often used as the opposite of rewriting: instead of doing it again, improve the existing one.
Certainly Maybe: Replacing data structures and algorithms is a bit of a borderline case. The decisive difference here is IMO - these are small steps: to be ready for delivery, to be ready to work on another case.
Example: Imagine that you have a report randomizer module that slows down its use of the vector. You have profiled that inserting vectors is a bottleneck, but unfortunately the module uses many-sided memory in many places, so when using the list everything will break quietly.
Rewriting would mean throwing a module out of a building better and faster from scratch, simply by selecting some parts from the old one. Or write a new kernel, and then paste it into an existing dialog.
Refactoring means taking small steps to remove pointer arithmetic, so that switch. Perhaps you even create a utility function that wraps pointer arithmetic, replaces the direct pointer manipulation with calls to that function, and then switches to an iterator so that the compiler complains about places where pointer arithmetic is still used, then switch to list and then delete the function ultility.
The idea is that the code gets worse on its own. When fixing errors and adding functions, the quality breaks up in small steps - the value of the variable changes subtly, the functions receive an additional parameter that breaks the isolation, the cycle becomes a bit complicated, etc. None of them is a real mistake, you can’t tell the number of lines, which makes the cycle difficult, but you will damage readability and maintenance.
Similarly, changing a variable name or extracting a function are not significant improvements. But all together they fight slow erosion.
Like a pebble wall where every day falls to the ground. And every day, one passerby takes it and returns it.