From online books OPENROWSET (Transact-SQL)
User requires ADMINISTER BULK OPERATIONS permission.
And here is the entry for GRANTing. This is a server-level resolution, so yes, it's pretty high.
To try lower resolutions, you can create a standard connection to a connected server and add a login using
EXEC sp_addlinkedsrvlogin 'LINKSERVERNAME', 'false', 'localuser', 'rmtuser', 'rmtpass'
It does not seem to require any specific permissions, so if you have configured a linked server, it is impractical to configure it using a common linkedsrvlogin that maps to each local user. Configure specific locally-remote mappings to control local user access through the linked server on the remote server (using rmtuser login).
source share