How to scale out something like Google alerts?

Question

The requirement is that we have many saved searches, and when new documents appear, we want to find which saved searches are matched by these new documents and be able to notify the creators of these saved searches in real time.

Is something like this already supported in any of the popular search engines and free software libraries, either built-in or as a plugin? If not, what is the best way to do this in an efficient and scalable way?

I think that the general idea should be to index stored queries as if they were documents, and use incoming documents as queries to find them. But I did not understand how to do this.

It is also better if we can put as few restrictions on supported query operators as possible.

Thanks in advance.

Answer 1

How to do the opposite -

• Maintain a search in another database using user-defined information instead of solr.
• Index of the new document so that it is available for all subsequent searches.
• Repeat all saved queries in the document after adding to check if it matches the document.

In addition, Solr supports a wide range of query support, so you can check it out.