I am developing a WP 8.1 application that also uses background audio assignment. Everything you explained is also with me.
If you see “Resume ...” for a few seconds, it means that your application was interrupted by the OS after it was suspended. In this case, the resume event will not fire because your application has been completely killed and must resume. Resume usually means that a paused (not completed) application resumes execution.
When your application is completed and then “resumed” from the application switch, your override of the Application.OnLaunched() method will be called using e.PreviousExecutionState == ApplicationExecutionState.Terminated . In this method, you must check if the previous execution state was “Completed”, and if so, restore the application to the state that was before the suspension. This gives the user the illusion that the application never stopped, and they can resume what they were doing at that time.
If you create a new Pivot App Windows Phone 8.1 project (for example), you will see that the application life cycle events are correctly executed in App.xaml.cs
The Resume App event fires as expected when I move quickly and then return.
This is the correct behavior. It will take several seconds after the application was programmed before the OS was suspended, so if you resume the application before the OS suspends it, it will simply resume from memory.
I'm not sure why background audio applications are more prone to termination. I even found that the Xbox Music app behaves the same way. We hope that in the next version of Windows Phone this problem will be addressed.
FYI here is the application lifecycle diagram from MSDN (I recommend you read this page for more information on the application lifecycle):

Resume occurs only from paused states.
source share