Required Free Space for Lucene IndexWriter.ExpungeDeletes ()

Question

I am writing an application that uses Lucene indexes where the user must frequently update or delete indexed documents.

I understand that if someone wants to permanently delete documents marked for deletion from the Lucene index, he has two options:

• Indexwriter.Optimize () OR

• Indexwriter.ExpungeDeletes ()

Because optimizing the index for each delete session is expensive and time consuming, and it requires 2X free disk space , so in situations where disk space is running, Optimize () did not seem to be a good choice.

So obviously I had to go to ExpungeDeletes (), which allegedly takes less time, and since it does not force the merging of all segments, so I hope it does not require this 2x free space.

But this operation requires some free disk space to successfully merge certain segments.

So my question is:

How much free disk is required for ExpungeDeletes () to successfully complete its task?

I ask this question because I tested it with some indexes that were already optimized and combined into one segment ... and I found that in this case the expungedeletes () function succeeds only when the free disk space is less than the same the size of the index, otherwise it throws an exception: MergePolicy-MergeException

Furthermore, is there any way to prevent the exclusion of this exception from the above case?

Thanks in advance.

Answer 1

You might be interested in the question JIRA 3577 . To summarize: expungeDeletes is a bad name because it doesn't just remove deletions and leaves the rest of the index separate.

If you want to know its exact performance: expungeDeltes just causes a merge on every segment that deletes. Which in your case sounds like it could be every segment that ... is roughly equivalent to calling optimize .

How about your searches not working, what do you consider necessary for optimization?