All geek questions in one place

MSMQ consumes large amounts of memory when processing messages using NServiceBus

I have two windows services that use NserviceBus. One writes messages to the queue, while the other reads them and does some processing. All queues are transactional, and the NserviceBus endpoints are configured as follows.

.IsTransactional(true) .IsolationLevel(IsolationLevel.ReadCommitted) .MsmqTransport() .RunTimeoutManager() .UseInMemoryTimeoutPersister() .MsmqSubscriptionStorage() .DisableRavenInstall() .JsonSerializer()

The problem is that a large number of messages (170,000+) are queued, the MSMQ service (mqsvc.exe) is playing out quite a bit of memory (1.5 - 2.0 GB), and this memory is not freed up for at least 5 - 6 o'clock. The average message size is about 5 - 10 KB. And it looks like the more messages you queue the more memory it uses. The memory consumption of Windows Services based on NServiceBus is within acceptable limits (50 - 100 MB) and does not increase no matter how many messages they process.

Any ideas on why MSMQ will use this large memory and take it long enough? Thanks heaps.

+4
source share
2 answers

This is completely normal. MSMQ uses memory in 4 MB memory blocks that map to files in the Storage folder. 170,000 messages at 5-10kb each comprise 0.85-1.7 GB, so it’s not surprising that you allocate so much virtual memory. To reduce the overhead of deleting and creating files when deleting or receiving messages, the storage files are stored for 6 hours. After this period, empty files are deleted. You can customize this as described in my blog post:

Forcing MSMQ to clean its storage files

+7
source

Otherwise, it will help anyone - this is a message in Google groups according to the msmq legend. John Ramwell documents how to completely clear all messages in the repository, which is sometimes desirable / necessary

https://groups.google.com/d/msg/microsoft.public.msmq.performance/jByfXUwXFw8/i1hVP1WJpJgJ

I had 8 GB of files, but there were no messages in any queues, and it would take about 2 hours for the msmq service to enter the initial state. Clearing any queue will take 10 seconds and cause massive bursts of memory, which will then not be released for several days, if ever.

If you are ever in this situation, instead of reinstalling the message queue, you can simply follow these steps:

  • Message Queuing Service
  • Go to MSMQ storage (usually C: \ Windows \ System32 \ msmq \ storage)
  • Delete ONLY P * .MQ, J * .MQ, R * .MQ, and L * .MQ files
+1
source

Source: https://habr.com/ru/post/1446831/

More articles:


All Articles