Search Strategies for Different Data Sources

Question

I am creating a tool that searches for people based on several attributes. The values ​​of these attributes are scattered across several systems.

As an example, dateOfBirth is stored in the SQL Server database as part of the ABC system. The assignment of this person’s sales area is stored in some terrible old database. Other attributes are stored on a system accessible only through the XML web service.

Even worse, an outdated database and web service can be very slow.

What strategies and tips should be considered when implementing a search in all of these systems?

Note. Although I posted a response, I'm not sure if this is a great answer. I am not going to accept my own answer if no one else gives a better understanding.

Answer 1

You can use the indexing engine to retrieve and index data locally on all systems, and then perform an index search. Searches would be much faster and more reliable.

Of course, this simply transfers the problem from one part of your system to another - now your indexing mechanism should handle failures and heterogeneous systems, but it may be easier to solve.

Another factor is how often the data changes. If you need to request real-time data that is out of date very quickly, then indexing can be impractical.

Answer 2

, , . , .

, .

Answer 3

, . - , , , // .. Mule, Enterprise Service Bus, . , , , Mule. , , . , SQL , , -, , , piecess , . , , XML .

. , - , , . , , , , , .

, , , . 4 5 , , 10 .

Answer 4

?

, Lucene . , Lucene. , , .

http://lucene.apache.org/java/docs/ ( )

Answer 5

YQL? , .

Answer 6

, . , .

, .

, , . , .

/ / . , , .

EDIT: , . , , , . , (, ), . , , / , / - .

Answer 7

Pentaho/Kettle, , MySQL
http://www.pentaho.com/products/data_integration/

Create a script package to run overnight and update the local copy. Maybe even every hour. Then write your query in the local MySQL database and show the results.