According to the article HTTP Access Control (CORS) by MDN:
All requests must be set to the Origin header to work correctly under the CORS (Cross-origin) resource sharing mechanism).
The Origin request header is part of RFC 6454 and describes it as part of the CORS mechanism and is compatible with all browsers according to MDN.
MDN Description:
The Origin request header indicates where the fetch comes from. It does not contain any path information, but only the server name. this is sent with CORS requests as well as with POST requests. This is similar to the Referer header, but, unlike this header, it does not expand all the way.
Source: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Origin
MDN example: 
So, to get the original XHR request with PHP, you can use:
$_SERVER['HTTP_ORIGIN']
And, in the case of a direct request, you can combine HTTP_REFERER and REMOTE_ADDR as:
if (array_key_exists('HTTP_REFERER', $_SERVER)) { $origin = $_SERVER['HTTP_REFERER']; } else { $origin = $_SERVER['REMOTE_ADDR']; }
So, a possible final solution:
if (array_key_exists('HTTP_ORIGIN', $_SERVER)) { $origin = $_SERVER['HTTP_ORIGIN']; } else if (array_key_exists('HTTP_REFERER', $_SERVER)) { $origin = $_SERVER['HTTP_REFERER']; } else { $origin = $_SERVER['REMOTE_ADDR']; }
MDN Mozilla Developer Network .
Thanks so much for helping @trine, @ waseem-bashir, @ p0lt10n and others.