Not the best answer, but once you have UsbManager, you can figure it out. Usually this UsbManager is created using Context, which I suppose, but it looks like you are switching modes, so you hopefully can get an instance of UsbManager, m in this case:
UsbManager m = createManagerSomehow
In accessory mode, this has only one callback. If this returns one, then you know that it is an accessory.
m.getAccessoryList()
So, I think this might work:
if(m.getAccessoryList().size() > 0) accessoryMode = true;
And for the host, if you have a UsbDevice device or String deviceName , you can use the same UsbManager m function to find out if it contains this device.
if(m.getDeviceList().containsValue(device)) hostMode = true;
or
if(m.getDeviceList().containsKey(deviceName)) hostMode = true;
and I donโt know what a device , but if none of the above is true, you know that itโs just a device. You do not need this boolean variable below, because you have two others. It's just here to help with my explanation through the state logic.
if(!hostMode && !accessoryMode) deviceMode = true
Hope this helps. Go to UsbManager for more documentation and just find the page for the host and accessories.
http://developer.android.com/reference/android/hardware/usb/UsbManager.html
NOTE. I'm a little confused when you say you switch modes in your comment. It conflicts with the Accessory call, which I am doing above, and may not work, but what would I do then (if you do not need deviceMode ), then just check if the USB device is HOST mode, and if you do not know what he is in application mode ...