All geek questions in one place

New-WebServiceProxy cannot authenticate with NTLM

I am dealing with a rather peculiar problem. We need to get into the Lists service on our SharePoint farm. Web authentication combined through a single repository of Oracle, but we have accounts configured for automation that can fulfill web requests. Using AAM, we have an “internal” URL configured for server-side automation that bypasses AD directly and everything else is redirected to SSO.

Here's the code (sanitized) that I use to try to get a list.

$username = "DOMAIN\username" $password = "somepassword" $site = "https://sp.biz.com/sites/SiteCollection" $credentials = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $username, (ConvertTo-SecureString $password -AsPlainText -Force) $proxy = New-WebServiceProxy -Uri "$site/_vti_bin/Lists.asmx" -Credentials $credentials $proxy.GetListCollection()

I use this code 403 when I use this code.

Throw an exception "GetListCollection" with arguments "0": "The server could not process the request. ---> Access denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))"

If I modify $ site to use the internal URL (set via AAM) and run it at one of the front ends, I get the list collection successfully. Now at first I thought that there was a problem with the account and permissions, but after starting Fiddler capture, I see that this is not authentication at all.

When I run the following cURL command, it authenticates and returns a collection of lists. Soap.xml is just the basic GetListCollection package, copied directly from WDSL.

 curl -v -u 'username':'pass' --ntlm -X POST -H "Content-Type: text/xml" --data-binary @soap.xml https://sp.biz.com/sites/SiteCollection/_vti_bin/Lists.asmx

This is where the cleared output from cURL is executed.

 * STATE: INIT => CONNECT handle 0x600056190; line 1029 (connection #-5000) * Hostname was NOT found in DNS cache * Trying <IPv6>... * STATE: CONNECT => WAITCONNECT handle 0x600056190; line 1082 (connection #0) % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to sp.biz.com (<IPv6>) port 443 (#0) * successfully set certificate verify locations: * CAfile: /usr/ssl/certs/ca-bundle.crt CApath: none * SSLv3, TLS handshake, Client hello (1): } [data not shown] * STATE: WAITCONNECT => PROTOCONNECT handle 0x600056190; line 1222 (connection #0) * SSLv3, TLS handshake, Server hello (2): { [data not shown] * SSLv3, TLS handshake, CERT (11): { [data not shown] * SSLv3, TLS handshake, Server finished (14): { [data not shown] * SSLv3, TLS handshake, Client key exchange (16): } [data not shown] * SSLv3, TLS change cipher, Client hello (1): } [data not shown] * SSLv3, TLS handshake, Finished (20): } [data not shown] * SSLv3, TLS change cipher, Client hello (1): { [data not shown] * SSLv3, TLS handshake, Finished (20): { [data not shown] * SSL connection using TLSv1.2 / DES-CBC3-SHA * SSL certificate verify ok. * STATE: PROTOCONNECT => DO handle 0x600056190; line 1241 (connection #0) * Server auth using NTLM with user 'DOMAIN\username' > POST /sites/SiteCollection/_vti_bin/Lists.asmx HTTP/1.1 > Authorization: NTLM <snip> > User-Agent: curl/7.39.0 > Host: sp.biz.com > Accept: */* > Content-Type: text/xml > Content-Length: 0 > * STATE: DO => DO_DONE handle 0x600056190; line 1314 (connection #0) * STATE: DO_DONE => WAITPERFORM handle 0x600056190; line 1441 (connection #0) * STATE: WAITPERFORM => PERFORM handle 0x600056190; line 1454 (connection #0) * HTTP 1.1 or later with persistent connection, pipelining supported < HTTP/1.1 401 Unauthorized * Server Microsoft-IIS/7.5 is not blacklisted < Server: Microsoft-IIS/7.5 < SPRequestGuid: <snip> < WWW-Authenticate: NTLM <snip> < X-Powered-By: ASP.NET < MicrosoftSharePointTeamServices: 14.0.0.7006 < X-MS-InvokeApp: 1; RequireReadOnly < Date: Fri, 16 Jan 2015 01:02:56 GMT < Content-Length: 0 < Set-Cookie: BIGipServerserver_pool=<snip>; expires=Sat, 17-Jan-2015 01:02:56 GMT; path=/ < 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 * Connection #0 to host sp.biz.com left intact * Issue another request to this URL: 'https://sp.biz.com/sites/SiteCollection/_vti_bin/Lists.asmx' * STATE: PERFORM => CONNECT handle 0x600056190; line 1601 (connection #-5000) * Found bundle for host sp.biz.com: 0x60006aef0 * Re-using existing connection! (#0) with host sp.biz.com * Connected to sp.biz.com (<IPv6>) port 443 (#0) * STATE: CONNECT => DO handle 0x600056190; line 1075 (connection #0) * Server auth using NTLM with user 'DOMAIN\username' > POST /sites/SiteCollection/_vti_bin/Lists.asmx HTTP/1.1 > Authorization: NTLM <snip> > User-Agent: curl/7.39.0 > Host: sp.biz.com > Accept: */* > Content-Type: text/xml > Content-Length: 353 > } [data not shown] * upload completely sent off: 353 out of 353 bytes * STATE: DO => DO_DONE handle 0x600056190; line 1314 (connection #0) * STATE: DO_DONE => WAITPERFORM handle 0x600056190; line 1441 (connection #0) * STATE: WAITPERFORM => PERFORM handle 0x600056190; line 1454 (connection #0) * HTTP 1.1 or later with persistent connection, pipelining supported < HTTP/1.1 200 OK < Cache-Control: private, max-age=0 < Content-Type: text/xml; charset=utf-8 * Server Microsoft-IIS/7.5 is not blacklisted < Server: Microsoft-IIS/7.5 < SPRequestGuid: <snip> < Set-Cookie: FedAuth=<snip>; expires=Fri, 16-Jan-2015 08:36:07 GMT; path=/; secure; HttpOnly < X-SharePointHealthScore: 0 < X-AspNet-Version: 2.0.50727 < Persistent-Auth: true < X-Powered-By: ASP.NET < MicrosoftSharePointTeamServices: 14.0.0.7006 < X-MS-InvokeApp: 1; RequireReadOnly < Date: Fri, 16 Jan 2015 01:02:56 GMT < Content-Length: 104088 < Vary: Accept-Encoding < { [data not shown] * STATE: PERFORM => DONE handle 0x600056190; line 1626 (connection #0) 100 101k 100 101k 100 353 219k 762 --:--:-- --:--:-- --:--:-- 219k * Connection #0 to host sp.biz.com left intact

Any help is appreciated. I am not against a C # solution over PowerShell if there are no cmdlets.

01-16-2015 12:13 PM EST Update . I updated the question to reflect the HighlyUnavailable suggestion and included the headers from the Fiddler capture.

Here are the sanitized headers from the PowerShell script:

 CONNECT sp.biz.com:443 HTTP/1.1 Host: sp.biz.com Connection: Keep-Alive HTTP/1.1 200 Connection Established FiddlerGateway: Direct StartTime: 12:14:46.372 Connection: close ------------------------------------------------------------------ GET https://sp.biz.com/sites/SiteCollection/_vti_bin/Lists.asmx HTTP/1.1 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.5485) Host: sp.biz.com Connection: Keep-Alive HTTP/1.1 200 OK Cache-Control: private, max-age=0 Content-Type: text/html; charset=utf-8 Server: Microsoft-IIS/7.5 SPRequestGuid: <snip> X-SharePointHealthScore: 0 X-AspNet-Version: 2.0.50727 X-Powered-By: ASP.NET MicrosoftSharePointTeamServices: 14.0.0.7006 X-MS-InvokeApp: 1; RequireReadOnly Date: Fri, 16 Jan 2015 17:14:46 GMT Connection: keep-alive Content-Length: 9066 Set-Cookie: BIGipServerserver_pool=<snip>; expires=Sat, 17-Jan-2015 17:14:46 GMT; path=/ Vary: Accept-Encoding ------------------------------------------------------------------ GET https://sp.biz.com/_vti_bin/Lists.asmx?disco HTTP/1.1 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.5485) Host: sp.biz.com HTTP/1.1 200 OK Cache-Control: private Content-Type: text/xml; charset=utf-8 Server: Microsoft-IIS/7.5 SPRequestGuid: <snip> X-SharePointHealthScore: 0 X-AspNet-Version: 2.0.50727 X-Powered-By: ASP.NET MicrosoftSharePointTeamServices: 14.0.0.7006 X-MS-InvokeApp: 1; RequireReadOnly Date: Fri, 16 Jan 2015 17:14:46 GMT Connection: close Content-Length: 747 ------------------------------------------------------------------ CONNECT sp.biz.com:443 HTTP/1.1 Host: sp.biz.com Connection: Keep-Alive HTTP/1.1 200 Connection Established FiddlerGateway: Direct StartTime: 12:14:47.505 Connection: close ------------------------------------------------------------------ GET https://sp.biz.com/_vti_bin/Lists.asmx?wsdl HTTP/1.1 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.5485) Host: sp.biz.com HTTP/1.1 200 OK Cache-Control: private Content-Type: text/xml; charset=utf-8 Server: Microsoft-IIS/7.5 SPRequestGuid: <snip> X-SharePointHealthScore: 0 X-AspNet-Version: 2.0.50727 X-Powered-By: ASP.NET MicrosoftSharePointTeamServices: 14.0.0.7006 X-MS-InvokeApp: 1; RequireReadOnly Date: Fri, 16 Jan 2015 17:14:46 GMT Connection: close Content-Length: 72672 Set-Cookie: BIGipServerserver_pool=<snip>; expires=Sat, 17-Jan-2015 17:14:47 GMT; path=/ Vary: Accept-Encoding ------------------------------------------------------------------ CONNECT sp.biz.com:443 HTTP/1.1 Host: sp.biz.com Connection: Keep-Alive HTTP/1.1 200 Connection Established FiddlerGateway: Direct StartTime: 12:14:48.727 Connection: close ------------------------------------------------------------------ POST https://sp.biz.com/_vti_bin/Lists.asmx HTTP/1.1 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.5485) Content-Type: text/xml; charset=utf-8 SOAPAction: "http://schemas.microsoft.com/sharepoint/soap/GetListCollection" Host: sp.biz.com Content-Length: 321 Expect: 100-continue HTTP/1.1 500 Internal Server Error Cache-Control: private Content-Type: text/xml; charset=utf-8 Server: Microsoft-IIS/7.5 X-AspNet-Version: 2.0.50727 X-Powered-By: ASP.NET MicrosoftSharePointTeamServices: 14.0.0.7006 X-MS-InvokeApp: 1; RequireReadOnly Date: Fri, 16 Jan 2015 17:14:48 GMT Content-Length: 459 Set-Cookie: BIGipServerserver_pool=686493706.47873.0000; expires=Sat, 17-Jan-2015 17:14:48 GMT; path=/ ------------------------------------------------------------------

Here are the headers for the cURL command.

 CONNECT sp.biz.com:443 HTTP/1.1 Host: sp.biz.com:443 User-Agent: curl/7.39.0 Connection: Keep-Alive Content-Type: text/xml HTTP/1.1 200 Connection Established FiddlerGateway: Direct StartTime: 12:21:07.928 Connection: close ------------------------------------------------------------------ POST https://sp.biz.com/sites/SiteCollection/_vti_bin/Lists.asmx HTTP/1.1 Authorization: NTLM <snip>= User-Agent: curl/7.39.0 Host: sp.biz.com Accept: */* Content-Type: text/xml Content-Length: 0 HTTP/1.1 401 Unauthorized Server: Microsoft-IIS/7.5 SPRequestGuid: <snip> WWW-Authenticate: NTLM <snip> X-Powered-By: ASP.NET MicrosoftSharePointTeamServices: 14.0.0.7006 X-MS-InvokeApp: 1; RequireReadOnly Date: Fri, 16 Jan 2015 17:21:07 GMT Content-Length: 0 Set-Cookie: BIGipServerserver_pool=<snip>; expires=Sat, 17-Jan-2015 17:21:07 GMT; path=/ Proxy-Support: Session-Based-Authentication ------------------------------------------------------------------ POST https://sp.biz.com/sites/SiteCollection/_vti_bin/Lists.asmx HTTP/1.1 Authorization: NTLM <snip> User-Agent: curl/7.39.0 Host: sp.biz.com Accept: */* Content-Type: text/xml Content-Length: 417 HTTP/1.1 200 OK Cache-Control: private, max-age=0 Content-Type: text/xml; charset=utf-8 Server: Microsoft-IIS/7.5 SPRequestGuid: <snip> Set-Cookie: FedAuth=<snip>; expires=Sat, 17-Jan-2015 03:20:50 GMT; path=/; secure; HttpOnly X-SharePointHealthScore: 0 X-AspNet-Version: 2.0.50727 Persistent-Auth: true X-Powered-By: ASP.NET MicrosoftSharePointTeamServices: 14.0.0.7006 X-MS-InvokeApp: 1; RequireReadOnly Date: Fri, 16 Jan 2015 17:21:07 GMT Content-Length: 66628 Vary: Accept-Encoding ------------------------------------------------------------------
+6
source share
2 answers

I never got a solution for this, but I can explain why. In our environment, we use form-based authentication based on our Oracle Identity Foundation SSO with SAML v1.1.

When trying to authenticate, it redirects you to SSO, but the client tries to use NTLM for the actual web interfaces, not for SSO. To do this, you need to include the X-FORMS_BASED_AUTH_ACCEPTED: f header in your request so that it really authenticates using NTLM against WFE (not SSO).

Here's the problem: you cannot add headers to New-WebServiceProxy in PowerShell (until 4.0 - I won't show 5 yet). The only recommendation I can make for others with problems is to follow the HighlyUnavailable recommendations or use Invoke-WebRequest and manually create SOAP calls.

The only problem is that Invoke-WebRequest can chew on your encoding, so this is how I worked on it. If anyone has a suggestion on working with a coding problem, I'm all ears.

 # Set your credentials here. $UserName = 'BartSimpson' $Password = '3atmMySh0rtz!' $Domain = 'SF' $SecurePassword = ConvertTo-SecureString -String $Password -AsPlainText -Force $Credentials = New-Object System.Management.Automation.PSCredential (($Domain + "\" + $UserName), $SecurePassword) # SOAP request headers and body $BaseHeaders = @{"X-FORMS_BASED_AUTH_ACCEPTED" = 'f'; "SOAPAction" = "`"http://schemas.microsoft.com/sharepoint/soap/GetListCollection`""; "Content-Type" = "text/xml; charset=utf-8"} $SOAP = @" <?xml version="1.0" encoding="utf-8"?> <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body> <GetListCollection xmlns="http://schemas.microsoft.com/sharepoint/soap/" /> </soap:Body> </soap:Envelope> "@ # Gives us a random temp file to pipe output to $TmpFile = [System.IO.Path]::GetTempFileName() Invoke-WebRequest -Uri $URL -Headers $BaseHeaders -Credential $Credentials -Method POST -Body $SOAP -OutFile $TmpFile # Get the outfile with UTF8 encoding [xml]$Result = Get-Content -Raw -Path $TmpFile -Encoding UTF8 # Remove the temporary file Remove-Item $TmpFile

This seems to be a long way to go, but it works if you insist on using PowerShell.

I switched to python-suds and was able to do what I needed.

0
source

Here you mix two fundamentally different methods.

$proxy = New-WebServiceProxy -Uri "$site/_vti_bin/Lists.asmx" -UseDefaultCredential $proxy.PreAuthenticate = $TRUE $proxy.Credentials = $credentials

UseDefaultCredential will attempt to transfer your current Windows domain user to the site. However, you also set credentials. Usually you use -Credential $credentials (see http://technet.microsoft.com/en-us/library/hh849841.aspx )

The curl command you're working with is more akin to using -Credential : -u equivalent.

Instead, try using $proxy = New-WebServiceProxy -Uri "$site/_vti_bin/Lists.asmx" -Credential $credentials .

If that doesn't work, edit your question to include the headers returned from the Oracle SSO connection - maybe it just doesn't even ask for credentials.

+4
source

Source: https://habr.com/ru/post/981085/

More articles:


All Articles