Royal Mail Shipping API - SOAP request and pem / certificate request

Question

I am trying to set up the Royal Mail delivery API (if anyone has experience, I would be grateful if you could help).

In the documentation that they provide, I need to download the certificate (.p12 file) and import it into my Windows machine - it's quite simple using the "Certificate Import Wizard". Once it gets to "Set Security Level", I have to select High , and it will ask for permission with a password every time it is used.

In Internet Explorer, in the "Internet Options" section of the "Content" tab, I can view the certificates and clearly see that this certificate has been imported and has not expired.

The next step is to extract the certificate components , here I have to run the following three commands using OpenSSL to generate .pem files.

$ openssl pkcs12 -in mycert.p12 -cacerts -nokeys -out cacert.pem $ openssl pkcs12 -in mycert.p12 -clcerts -nokeys -out mycert.pem $ openssl pkcs12 -in mycert.p12 -nocerts -nodes -out mykey.pem 

The documentation states that the cacert.pem file can directly link to the application using the file itself , which I believe I made in my PHP script, but it is not clear where I should put the other mycert and mykey pem files .

The documentation states the following: -

How the application transmits the SSL certificate for the issued client when establishing the SSL network connection depends on the application and the environment, but essentially it needs to access the file "mycert.pem" and "mykey.pem" or, in some cases, one combined file containing both cert and key.

Therefore, it does not say anywhere how these two files are used by the application, at the moment I just left them in the same directory as the cacert.pem file.

If I try to access the url https://api.royalmail.com/shipping/onboarding directly from the browser, it will ask me to select a certificate, I select this and then enter the correct password when it asks “Grant or deny permission to use this key. " After entering the correct password, the following page appears - someone can confirm whether this means that the problem is ending or something that Royal Mail did not configure correctly at the end.

In addition to this, the actual PHP script that I have that is used to send SOAP requests to the delivery API does not work (probably related to all of the above).

In my PHP script, the parameters of soapclient are configured as follows:

 $soapclient_options['cache_wsdl'] = 'WSDL_CACHE_NONE'; $soapclient_options['local_cert'] = 'certs/cacert.pem'; $soapclient_options['passphrase'] = $api_certificate_passphrase; $soapclient_options['trace'] = true; $soapclient_options['ssl_method'] = 'SOAP_SSL_METHOD_SSLv3'; $soapclient_options['location'] = 'https://api.royalmail.com/shipping/onboarding'; $client = new SoapClient('SAPI/ShippingAPI_V2_0_8.wsdl', $soapclient_options); $client->__setLocation($soapclient_options['location']); 

When I run the PHP script (this is basically the same code as Royal Mail, which contains my personal data for entering the API), I get the following message in the browser:

 Could not connect to host REQUEST: [e-mail] rngfJ+4dt4Gt855a5pr6u38i3B4= ODcwMTE5Nzc3 2015-10-13T11:02:20Z 2015-10-13T11:02:201.00526348001DeliveryDSD12015-10-13bobSS23, Some AvenueLondonE10g1000000 

Obviously, this cannot be associated with the host for some unknown reason, the latter is just the request that was sent. The rest of the PHP script is exactly the same as the Royal Mail script they sent me and confirmed that they are being used by others and are working fine.

I work in a WAMP environment, although the final code will be in a Linux environment. Someone can help . I really am confused, and Royal Mail themselves could not provide any solid technical support.

UPDATE

This is a complete error message displayed in the browser (I changed the email address for security purposes)

 Invalid Request REQUEST: [e-mail] dgCW98Vqw3ladYgPPpNialODhvI= MTMzMjE1NjM4 2015-10-13T13:25:30Z 2015-10-13T13:25:302.00526348001DeliveryDSD12015-10-13Jon DoeSS23, Some RoadLondonE10g1000000 

I combined the two pem files into a single file named "bundle.pem" and referenced this in the "local_cert" variable for SoapClient and BINGO, which are now connected. This now longer shows Failed to connect , but instead indicates "Invalid request", so at least now this connection gives me another error.

My whole PHP script is below:

 <?php ini_set('default_socket_timeout', 120); ini_set('soap.wsdl_cache_enabled',1); ini_set('soap.wsdl_cache_ttl',1); $api_password = "xxxxxxxxxxxxxx!"; $api_username = " [e-mail] "; $api_application_id = "xxxxxxxxxxxx"; $api_service_type = "D"; $api_service_code = "SD1"; $api_service_format = ""; $api_certificate_passphrase = 'xxxxxxxxxx'; $api_service_enhancements = ""; $data = new ArrayObject(); $data->order_tracking_id = ""; $data->shipping_name = "Jon Doe"; $data->shipping_company = "SS"; $data->shipping_address1 = "23, Some Road"; $data->shipping_address2 = ""; $data->shipping_town = "London"; $data->shipping_postcode = "E1"; $data->order_tracking_boxes = "0"; $data->order_tracking_weight = "1000"; $time = gmdate('Ymd\TH:i:s'); $created = gmdate('Ymd\TH:i:s\Z'); $nonce = mt_rand(); $nonce_date_pwd = pack("A*",$nonce) . pack("A*",$created) . pack("H*", sha1($api_password)); $passwordDigest = base64_encode(pack('H*',sha1($nonce_date_pwd))); $ENCODEDNONCE = base64_encode($nonce); $soapclient_options = array(); $soapclient_options['cache_wsdl'] = 'WSDL_CACHE_NONE'; $soapclient_options['local_cert'] = 'royalmail/cert/bundle.pem'; $soapclient_options['passphrase'] = $api_certificate_passphrase; $soapclient_options['trace'] = true; $soapclient_options['ssl_method'] = 'SOAP_SSL_METHOD_SSLv3'; $soapclient_options['exceptions'] = true; $soapclient_options['location'] = 'https://api.royalmail.com/shipping/onboarding'; //launch soap client $client = new SoapClient('royalmail/ShippingAPI_V2_0_8.wsdl', $soapclient_options); $client->__setLocation($soapclient_options['location']); //headers needed for royal mail $HeaderObjectXML = '<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"> <wsse:UsernameToken wsu:Id="UsernameToken-000"> <wsse:Username>'.$api_username.'</wsse:Username> <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">'.$passwordDigest.'</wsse:Password> <wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">'.$ENCODEDNONCE.'</wsse:Nonce> <wsu:Created>'.$created.'</wsu:Created> </wsse:UsernameToken> </wsse:Security>'; //push the header into soap $HeaderObject = new SoapVar( $HeaderObjectXML, XSD_ANYXML ); //push soap header $header = new SoapHeader( 'http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd', 'Security', $HeaderObject ); $client->__setSoapHeaders($header); //build the request $request = array( 'integrationHeader' => array( 'dateTime' => $time, 'version' => '1.0', 'identification' => array( 'applicationId' => $api_application_id, 'transactionId' => $data->order_tracking_id ) ), 'requestedShipment' => array( 'shipmentType' => array('code' => 'Delivery'), 'serviceOccurence' => '1', 'serviceType' => array('code' => $api_service_type), 'serviceOffering' => array('serviceOfferingCode' => array('code' => $api_service_code)), 'serviceFormat' => array('serviceFormatCode' => array('code' => $api_service_format)), 'shippingDate' => date('Ym-d'), 'recipientContact' => array('name' => $data->shipping_name, 'complementaryName' => $data->shipping_company), 'recipientAddress' => array('addressLine1' => $data->shipping_address1, 'addressLine2' => $data->shipping_address2, 'postTown' => $data->shipping_town, 'postcode' => $data->shipping_postcode), 'items' => array('item' => array( 'numberOfItems' => $data->order_tracking_boxes, 'weight' => array( 'unitOfMeasure' => array('unitOfMeasureCode' => array('code' => 'g')), 'value' => ($data->order_tracking_weight*1000) //weight of each individual item ) ) ) ) ); //if any enhancements, add it into the array if($api_service_enhancements != "") { $request['requestedShipment']['serviceEnhancements'] = array('enhancementType' => array('serviceEnhancementCode' => array('code' => $api_service_enhancements))); } //try make the call try { $response = $client->__soapCall( 'createShipment', array($request), array('soapaction' => 'https://api.royalmail.com/shipping/onboarding') ); } catch (Exception $e) { //catch the error message and echo the last request for debug echo $e->getMessage(); echo " REQUEST:\n" . $client->__getLastRequest() . "\n"; die; } //check for any errors if(isset($response->integrationFooter->errors)) { $build = ""; //check it wasn't a single error message if(isset($response->integrationFooter->errors->error->errorCode)) { $build .= $output_error->errorCode.": ".$output_error->errorDescription."<br/>"; } else { //loop out each error message, throw exception will be added ehre foreach($response->integrationFooter->errors->error as $output_error) { $build .= $output_error->errorCode.": ".$output_error->errorDescription."<br/>"; } } echo $build; die; } print_r($response); echo "REQUEST:\n" . $client->__getLastRequest() . "\n"; die; ?> 

Just for added clarity, I added a dump to the $ request variable just before it reaches the try / catch block (note that this is quite long).

 Array ( [integrationHeader] => Array ( [dateTime] => 2015-10-13T13:34:44 [version] => 1.0 [identification] => Array ( [applicationId] => 0526348001 [transactionId] => ) ) [requestedShipment] => Array ( [shipmentType] => Array ( [code] => Delivery ) [serviceOccurence] => 1 [serviceType] => Array ( [code] => D ) [serviceOffering] => Array ( [serviceOfferingCode] => Array ( [code] => SD1 ) ) [serviceFormat] => Array ( [serviceFormatCode] => Array ( [code] => ) ) [shippingDate] => 2015-10-13 [recipientContact] => Array ( [name] => Jon Doe [complementaryName] => SS ) [recipientAddress] => Array ( [addressLine1] => 23, Some Road [addressLine2] => [postTown] => London [postcode] => E1 ) [items] => Array ( [item] => Array ( [numberOfItems] => 0 [weight] => Array ( [unitOfMeasure] => Array ( [unitOfMeasureCode] => Array ( [code] => g ) ) [value] => 1000000 ) ) ) ) 

)

Answer 1

Firstly, access to https://api.royalmail.com/shipping/onboarding will not work directly, because it is only available through the API.

With Royal Mail, do you have all the CDM files and WSDL files? Make sure the CDM files are in the same directory as the WSDL files.

Here is what I did when working with the API;

 $client = new SoapClient("/royalmail/ShippingAPI_V2_0_8.wsdl", array( 'trace' => 1, 'location' => $location, //https://api.royalmail.com/shipping 'soap_version' => SOAP_1_1, 'local_cert' => '/royalmail/cert/cert.pem', 'passphrase' => 'xxx', 'exceptions' => true )); 

Then, when it came to the actual connection, I did something like this:

  $password = 'xxx'; $date = gmdate('Ymd\TH:i:s\Z'); $nonce = mt_rand(); $nonce_date_pwd = pack("A*",$nonce) . pack("A*",$date) . pack("H*", sha1($password)); $encoded_password = base64_encode(pack('H*',sha1($nonce_date_pwd))); $encoded_nonce = base64_encode($nonce); $HeaderObjectXML = '<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"> <wsse:UsernameToken wsu:Id="UsernameToken-0000"> <wsse:Username>Username</wsse:Username> <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">' . $encoded_password . '</wsse:Password> <wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">' . $encoded_nonce . '</wsse:Nonce> <wsu:Created>'.$date.'</wsu:Created> </wsse:UsernameToken> </wsse:Security>'; $HeaderObject = new SoapVar( $HeaderObjectXML, XSD_ANYXML ); $header = new SoapHeader( 'http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd', 'Security', $HeaderObject ); $client->__setSoapHeaders( $header ); $request = array('the shipment request'); try { $client->__soapCall( 'createShipment', array($request) ); } catch (SoapFault $soapFault) { print_r($soapFault); } 

Hope this helps.

Edit

Try this request

I think you need to use version 2 for this Service event, if I remember correctly. Also, just ... formatting. Easier to debug.

 $request = Array( 'integrationHeader' => array( 'dateTime' => date('Ymd\TH:i:s'), 'version' => '2', 'identification' => array( 'applicationId' => $api_application_id, 'transactionId' => $data->order_tracking_id ), ), 'requestedShipment' => array( 'shipmentType' => array( 'code' => 'Delivery' ), 'serviceOccurrence' => 1, 'serviceType' => array( 'code' => $api_service_type ), 'serviceOffering' => array( 'serviceOfferingCode' => array( 'code' => $api_service_code ) ), 'serviceFormat' => array( 'serviceFormatCode' => array( 'code' => $api_service_format ) ), 'shippingDate' => gmdate('Ym-d'), 'recipientContact' => array( 'name' => $data->shipping_name, 'complementaryName' => $data->shipping_company ), 'recipientAddress' => array( 'addressLine1' => $data->shipping_address1, 'addressLine2' => $data->shipping_address2, 'postTown' => $data->shipping_town, 'postcode' => $data->shipping_postcode ), 'items' => array( 'item' => array( 'numberOfItems' => $data->order_tracking_boxes, 'weight' => array( 'unitOfMeasure' => array( 'unitOfMeasureCode' => array( 'code' => 'g' ) ), 'value' => ($data->order_tracking_weight*1000) ) ) ) ) ); 

Answer 2

^{! Although you’ve already mentioned that you have completed most of these steps, I’ll still give them details to follow the instructions}

Install CA on the server

• Upload the p12 file to the server (e.g. /root/Desktop )
• Run the following 3 commands to install

 openssl pkcs12 -in mycert.p12 -cacerts -nokeys -out cacert.pem openssl pkcs12 -in mycert.p12 -clcerts -nokeys -out mycert.pem openssl pkcs12 -in mycert.p12 -nocerts -nodes -out mykey.pem 

• Now copy the *.pem files to /etc/ssl/certs
  • I like to create a subdirectory here ( mkdir royalmail )
  • And move the *.pem files ( mv *.pem /etc/ssl/certs/certificates )
• Now create a new file and copy the contents of mycert.pem and mykey.pem into it (only from -----BEGIN .... ------- to EOF)

You must have the following files:

Using SOAPClient

Now that the certificates are installed, we can check the connection (if your certificates are in /etc/pki/tls/certs/certificates/royalmail/shippingv2 )

If you run

 wget https://api.royalmail.com/shipping/onboarding --private-key=/etc/ssl/certs/certificates/royalmail/shippingv2/rm_bundle.pem --private-key-type=PEM 

You should be able to connect to port 443 (perhaps in OpenSSL , a handshake may fail, although at least with my test only in our intermediate environment).

Now we can create an instance of SoapClient using a local copy of WSDL and specify a local certificate in the $options parameter.

 $objSoapClient = new \SoapClient('lib/wsdl/royalmail/shipping/ShippingAPI_V2_0_8.wsdl', array( 'soap_version' => SOAP_1_1, 'trace' => 1, 'uri' => 'http://www.royalmailgroup.com/api/ship/V2', 'location' => 'https://api.royalmail.com/shipping/onboarding', 'local_cert' => '/etc/ssl/certs/certificates/royalmail/shippingv2/rm_bundle.pem', 'passphrase' => '', //Your passphrase when doing step 1 'ssl_method' => 'SOAP_SSL_METHOD_TLS', 'exceptions' => 1, 'trace' => 1 )); 

Authentication

In your documentation, you should find a file called rm_password_digest.php or something similar that details how to create authentication headers.

 /* The value below should be changed to your password. If you store the password */ /* as hashed in your database, you will need to change the code below to remove hashing */ $password = 'just_my_royalmail_api_password'; /* CREATIONDATE - The timestamp. The computer must be on correct time or the server you are * connecting may reject the password digest for security. */ $CREATIONDATE = gmdate('Ymd\TH:i:s\Z'); /* NONCE - A random word. The use of rand() may repeat the word if the server is * very loaded. */ $nonce = mt_rand(); /* PASSWORDDIGEST This is the way to create the password digest. As per OASIS standard * digest = base64_encode(Sha1(nonce + creationdate + password) * however note that we use a SHA1(password) instead of the password above */ $nonce_date_pwd = pack("A*",$nonce) . pack("A*",$CREATIONDATE) . pack("H*", sha1($password)); $PASSWORDDIGEST = base64_encode( pack('H*', sha1($nonce_date_pwd))); /* ENCODEDNONCE - Now encode the nonce for security header */ $ENCODEDNONCE = base64_encode($nonce); /* Now Print all the values - so we can use it for testing with tools like soapui */ print "WS Security Header elements \n"; print "--------------------------- \n"; print 'Nonce = ' . $nonce; print "\n"; print 'PASSWORDDIGEST= ' . $PASSWORDDIGEST; print "\n"; print 'ENCODEDNONCE= ' . $ENCODEDNONCE; print "\n"; print "CREATIONDATE= " . $CREATIONDATE; 

This will help you create the following in SOAPHeader

 <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"> <wsse:UsernameToken wsu:Id="UsernameToken-0000"> <wsse:Username>[...]</wsse:Username> <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">[...]</wsse:Password> <wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">[...]</wsse:Nonce> <wsu:Created>[...]</wsu:Created> </wsse:UsernameToken> </wsse:Security>