I am a fan for individual controllers, as they remove the connection between Users and Contacts (for example) - what if later you want to use Contacts in a separate context? (i.e. regardless of User they belong)
If they are separated, the paths will look very similar to what you have:
Users
/users GET, POST /users/{user-id} PATCH, GET
Contacts
/contacts GET, POST /contacts/{contact-id} GET, PATCH
If there is a relationship between Contacts and Users (and in this case, it looks like it is), you can have it like this:
/contacts/for-user/{user-id} GET, POST /contacts/for-user/{user-id}/{contact-id} GET, PATCH
This allows you to add contacts to other things (and not just to users) and to other contexts (regardless of their contacts).
For example, coffee makers
/coffee-maker/users GET
Or if the coffee maker fails, who will we contact for repairs?
/coffee-maker/contacts GET
Perhaps the contact for repairing the coffee machine is also a user (but they are not necessary)
One more thing, you can rotate it and say that the contact is for the device (i.e. coffee maker)
/contact/for-appliance/{appliance-id} GET
My point is that if you separate contacts from users, you can assign contacts to other objects instead of formatting the relationship User -> Contacts - unless, of course, you want to separate them due to business rules or according to the reason
In any case, beware that there are many ways to make a comparison.