Most effective website architecture using Cassandra and Solr?

Question

I am developing a website that will use Cassandra to store a database and Solr to index and search for some of the data contained in this database (only some of the data that I want to search). I intended to use PHP for server-side scripting, interacting with the Cassandra database and providing dynamic HTML content based on the contents of the database.

When a user commits something in the database, I foresaw that PHP would issue a record in Cassandra, and if it was the data that needed to be searched, the same data could be written to the Solr index. The thing is, I don’t need the searchable data in the Solr index, and I don’t want the process of adding it to the index through PHP to consume valuable resources, especially during peak hours. Is there a way to have asynchronous updates to the Solr index in the background, passing data directly from Cassandra? Perhaps a queue of searchable data could be created that is used to update the Solr index during downtime by some background process?

I'm new to this, but somehow I would like the connection between Cassandra and Solr to be isolated from the main PHP scripts. I'm not sure that Cassandra and Solr can be effectively connected with Java, and only for a higher level of access to Cassandra (for reading / writing to the database) and Solr (for requesting data that can be searched), the creation of the web will be supported in PHP content. I appreciate any suggestions.

Answer 1

Instead of managing Solr and Cassandra separately, you should consider Solandra, the cassandra backend for solr.

Read more about this here: http://github.com/tjake/Lucandra

Answer 2

You have many options.

One simple one is to have a scheduled task that captures all of your updates since the last time you started the task and inserts the package into solr.

Or you could make your cassandra post and then send the async message to solr. as described here: How to make an asynchronous GET request in PHP?

Since you do not need a real-time search, you can set the default commit size and be large enough.