Slow serialization of WCF response using DataContractSerializer

Question

I have a WCF service that searches and returns a list of fairly complex objects to the client. This is an EAV system, so each returned object has a list of attached values, the size of which depends on the size of the object.

In my test search, I confirmed through registration that the actual search took less than a second to complete in almost every case. The last thing I do before returning the response to the client is the record that completed processing.

Unfortunately, the client does not receive a response until 15-20 seconds after I finished with it. The total response size is about 250 kb, so it is rather small. This is transmitted over the local network for a minute, and I tried to disable the firewall and antivirus to make sure that they did not interfere.

I noticed, however, that if the answer was much smaller, for example, by removing the fields attached to each object, the answer was much faster. I also tried going through a locally hosted (IIS) copy of the service, and after passing the final return it still took another 15 seconds to reach the local client application.

I use basicHttpBinding, because the service will be consumed by both .NET and PHP clients.

Now, can anyone suggest a way that I can confirm that this is indeed the case? And how can I fix a maliciously slow serialization time?

Edit:

To clarify, I marked each class with the [DataContract] attribute and each property with [DataMember] - WCF handles serialization when I return the data. In this case, it is a list of Entity types (a custom class that contains a list of values.

Edit 2:

I tested the speed of a DataContractSerializer and it took about 15 seconds to write a list of 65 returned objects to a simple memory stream. It seems ridiculous, and I'm not sure what has changed to make it so painfully slow.

Answer 1

I figured it out, and it's rather awkward.

While testing the DataContractSerializer speed, I used a list of 65 products that came back from my search. I decided to load all the products (about 600) into the database and then serialize them in memory, but got a few exceptions from the memory, so instead I started to write the results to a text file.

It turns out that the text file was 1.5 GB, which is about 3 times the size of the entire database. It took 17 seconds to serialize. So actually it was a damn good job. What happens is that each product can have a list of attached objects, and they are also downloaded. Because they are serialized, these objects are duplicated where they exist in the database only once.

Along with deleting a ton of metadata that the client does not need, I managed to get a list of 50 products (from 194 MB) to only 3 MB. ( update: this weekend I got a list of 1.5 GB to 66 MB).

The moral of the story? Listen to the community. Everyone else told me how fast the DataContractSerializer was, so when it seemed to me that it was bothering me, I had to blame myself instead of the DataContractSerializer.

UPDATE:

I spent some time trying to understand why this was a sudden problem. The answer lies in the nature of EAV systems - the list of data associated with any object is dynamic. Initially, I downloaded only field data when a separate object was requested - several objects downloaded only the minimum for speed. After I did the caching, I changed this to independently load all the data in the field, but I did not expect how big the difference was in the amount of data, due to complex data models and a large number of objects. In fact, I should not have assumed anything about the amount of data I requested.

Answer 2

Do you need to use a DataContractSerializer ? I found DataContractSerializer when sending large amounts of data very slowly, performance loss not with serialization, but with deserialization.

We switched to binary serialization, which is much faster, but may not be compatible with you PHP clients. You can write a component that can be reused by the client, handles deserialization for you, but you hit it for all non.net clients.