Slow item search with advanced property on Exchange

Question

Problem at hand

Our C # Windows application uses the EWS Managed API 2.0 to create appointments in the user's calendar. Each meeting has an extended property with a unique meaning. He then makes an appointment using FindItems and ItemView .

The first time a search is performed, users experience significant delays. Subsequent response times are quite acceptable.

(“first time” is a bit vague here because users may experience a delay later that day)

 // locate ID of appointment where extended property value equals 1234: var filter = new Ews.SearchFilter.IsEqualTo(extendedPropertyDefinition, 1234); var view = new ItemView(1, 0); view.PropertySet = BasePropertySet.IdOnly; var folder = new FolderId(WellKnownFolderName.Calendar, new Mailbox("...")); var result = service.FindItems(folder, filter, view); 

Remote Server - Exchange Server 2007 Service Pack 1 (SP1) .

Study

MSDN links some comments to search folders and limited views , however I'm not sure if they apply to our situation.

The act of applying a view to a folder creates search folders in the store . When a search folder is created, it is cached for later use. If the user tries to create a search folder that already exists, the cached search folder is used. This allows future observations to be valid quickly. By default, Exchange does not cache all search folders indefinitely.

In particular with regard to EWS :

It is also important to be aware of the fact that for the first time a query is issued to search for Exchange storage, it will work very slowly and possibly time out, while in future runs it will answer without a question. This is caused by internal processes that occur on the Exchange server when a store search is performed.

They propose creating search folders for non-changing, non-dynamic queries, which in our case does not seem appropriate, since the query is different for each meeting.

If an application requires a specific request with a fixed set of immutable parameters, you can use the search folders. [...] folder searches are only useful for immutable non-dynamic queries.

We essentially need to create an “index” —in terms of a database — on a property, ensuring that all searches for that particular property are fast, regardless of time or frequency.

Is it possible to "index" this property? Can the client or server side be configured to remove this initial delay?

Answer 1

I faced the same problem with an integration project. I wish there was a good solution ...

You cannot create an index for a property that has not yet been indexed by Exchange. Creating a search folder for each of them is not viable if the number of destinations grows high enough. Too many search folders in the same folder will cause additional problems, since they all need to be updated when a new item is added to the folder. This is my understanding, at least. In addition, Exchange 2007 is limited to 11 dynamic search folders per parent folder, so it may be even less viable depending on the number of destinations and how often they are available. Using existing indexed properties may not be practical because they can be modified by the user outside of your application. If you have a way to make sure that the meetings you created can only be accessed or modified from your application, then this is a different story.

A database table is a good way, but there is a potential error that some people don’t see until it is too late. ItemId is the obvious choice to reference your extended property, but ItemId is not a constant. This is a calculated property based on several others. It can change if the item is moved to another folder, and it can also change with the installation of the service pack or with enough time, or I heard it. I can confirm at least the first one. ItemId is not viable for long-term storage, at least without additional checks. You can potentially store the ItemId and your extended property. If the binding using ItemId fails, return to the advanced property search. If the binding is successful, check for an extended property in the database to make sure it matches. Update ItemId if you have an item if it does not match. Do you need to work with anything outside the Appointment objects, that is, respond with answers, forward notifications, etc., Or is it just the calendar?

This is ugly, but it should be a somewhat reasonable compromise. You can still have a random search, but they should be few and far apart until the user decides to move the assignments to different folders or plans some appointments in advance, and even then synchronization should help mitigate this, just be prepared to re-fill this table if there are updates for Exchange.

Of course, if Microsoft either added the ability to index additional properties, or even added an empty string field or two to the index in Exchange Search for this purpose, we would not have this problem. Heck, the GlobalObjectId property index in Assignments and related objects would help, but alas ... no. I'm not a fan of repurposing existing indexed fields. Not all of them apply to Assignments and to those that are usually either required or edited by the user. If you don’t know exactly what you are doing, reprofiling these fields could potentially have unforeseen consequences in the future.

In any case, I do not pretend to be an expert on all issues of EWS / Exchange, so there may be a better way than this. Take it with salt.

Answer 2

Cannot enable indexing for your property. I am not familiar with what properties are indexed in Exchange 2007. Since your application seems to use appointments, you can probably redistribute one of the other non-assignment properties to preserve your unique value. Perhaps use the AssistantName property through an extended property (to get around the restrictions imposed by the schema and EWS). Thus, most customers will not use this property for calendar items.

In accordance with this topic http://technet.microsoft.com/en-us/library/jj983804(v=exchg.150).aspx this property is indexed (in 2013). This property has existed for a long time, so it can be indexed for 2007.

Hey, this is a long shot and not optimal in any way, but maybe it can work for your scenario.

Answer 3

After reading this topic, I see that you are not looking for all the elements with an extended property, but a specific element. Sorry, I did not notice this in my first answer. I agree that the search folder will not work for you, as you will need to update the filter every time you search for your item. Obviously, it will be quite expensive (perhaps worse than your current approach). One of my ideas is to create a view that is sorted by your advanced property. I could be wrong, but I believe that you can apply this view to the above search folder (note that I'm talking about explicitly creating a search and viewing folder and storing it in a mailbox, they can be hidden or opened in the OL interface in the tree search folders). Only destinations that have an extended property will be filtered in the search folder. Then View will sort the folder by property value. In some readings that I did on internal ESEs, I saw some comments that show that sorting by property will force Exchange to create an index in ESE (I would like for me to find it now). The B-Tree ESE Indexes section seems to confirm this: http://books.google.com/books?id=12VMxwe3OMwC&pg=PA73&lpg=PA73&dq=how+to+create+exchange+ese+indexes&source=bl&ots=D5hJyJIEo5&sig=hZJFJFJRJFJRJFJRJFJFJRJFJRJFJFJrJFJRJFJRJFJFJRJFJFJRJFJFZRFJFJRJFJFJFJrJfJrfge = en & sa = X & ei = QQ7HUtgggvTbBdjcgfAP & ved = 0CFwQ6AEwBQ # v = onepage & q = how% 20to% 20create% 20exchange% 20ese% 20indexes & f = false

Then you will need to use the same approach that you used above in the search folder to find a specific item that matches your criteria. One problem, of course, is that Exchange throws your index (which is probably happening in your current approach). Perhaps you could programmatically touch the search folder periodically to prevent this from happening? This link is also useful for understanding the performance impact of creating a search / view folder: http://technet.microsoft.com/en-us/library/cc535025%28EXCHG.80%29.aspx

If you find a good solution (or it works), I am very interested to hear about it (and I am sure that many others too). Oh, the joy of Exchange development :-)

Answer 4

Creating a search folder with an extended property as criteria is the way to go. You will pay the price, while the search folder will be created initially, but after creating the index, while the folder exists and works, it is automatically updated by Exchange. We use this technique quite successfully to find the proverbial "needle in a haystack."

http://msdn.microsoft.com/EN-US/library/dd633687(v=exchg.80).aspx