I had a strange problem sending data to a web API service hosted in Azure, and I would like to evaluate any information.
I wrote a fairly simple web API service that has a controller that receives a list through a submit method. I send data to this API through a console application using a client created using HttpClientFactory .
When I debug it locally, everything works fine, and I successfully posted a list containing 100 items and wrote this to my local database. If I switch my local web API code to point to my Azure database, it still works fine, and I can get 100 elements without problems and write an instance of Azure DB.
When I deploy the API service in my Azure account and connect the console application for publishing data - it works fine with a list of up to 45 items, and PostAsync waits about 6 minutes, and then crashes with an internal server error.
So my question is why the service cannot send 43 items per second so as not to execute 44 items, but it takes 6 minutes?
I must point out that my list contains 100 duplicate elements as I am generating data for testing, so there is nothing wrong (or other) with a specific element inside the set.
T is a simple class with 30 properties and no children, so I do not place a massive structure.
My thinking is that it should be a size limit somewhere that doesn't work on my local machine, but when I deploy Azure. I'm by no means a web API specialist, so I'm not sure where to start looking.
Client code (works fine when i = 43):
List<Foo> results = new List<Foo>(); for (int i = 1; i < 100; i++) { results.Add(new Foo() { // Populated with data - removed for brevity }); } StringContent content = new StringContent(JsonConvert.SerializeObject(results), Encoding.UTF8, "application/json"); HttpResponseMessage postDataResponse = await postClient.PostAsync(myWebApiServiceURL, content);
Controller Method:
[HttpPost] public List<Foo>Post([FromBody]List<Foo> value) { return value; }
Web.config settings I tried that has no effect:
<httpRuntime targetFramework="4.6.1" maxRequestLength="200000000" requestLengthDiskThreshold="16384" />
and
<add key="aspnet:MaxJsonDeserializerMembers" value="20000000"/>
I just checked the test and does not work with 44 elements content length = 53278
The maximum length it works on is when my totals are 43 content lengths = 48361.
NOTE. I just ran a few more tests, and if I reduce the data length in some of my row properties (thereby reducing the size of each element), I can place more elements. Therefore, it should be a message size limit somewhere !!
UPDATE - HMAC Authentication
OK reason is always the least expected thing that you are not documenting in the question. I have HMAC authentication on my API, and when I remove the attribute from my controller, I can publish 10,000 elements without problems. As soon as I turn it on, I will limit myself to 43.
On the client, I generate an MD5 hash of the request content as part of the header signature, this process is repeated in the HMAC attribute in the web API service. If I delete this part of the signature from both my client and the service, I can place as much data as I want. The code I commented on on the client is shown below:
//Checking if the request contains body, usually will be null with HTTP GET and DELETE if (request.Content != null) { byte[] content = await request.Content.ReadAsByteArrayAsync(); MD5 md5 = MD5.Create(); //Hashing the request body, any change in request body will result in different hash, we'll incure message integrity byte[] requestContentHash = md5.ComputeHash(content); requestContentBase64String = Convert.ToBase64String(requestContentHash); }
The code that I commented on in the service is given below:
byte[] hash = await ComputeHash(req.Content); if (hash != null) { requestContentBase64String = Convert.ToBase64String(hash); }
ComputeHash :
private static async Task<byte[]> ComputeHash(HttpContent httpContent) { using (MD5 md5 = MD5.Create()) { byte[] hash = null; var content = await httpContent.ReadAsByteArrayAsync(); if (content.Length != 0) { hash = md5.ComputeHash(content); } return hash; } }
Thus, with the above code, requestContentBase64String always an empty string on both the client and server, so the MD5 hash of the content is not used as part of the authentication signature.
Now I have to investigate the root cause of this behavior.