Integrate MONGODB and SOLR

Question

I tried integrating MONGODB and SOLR using MONGO CONNECTOR provided by mongodb, which works in replica set configuration.

**python2.7 mongo_connector.py -m localhost:27017 -t http://localhost:8983/solr -u_id -d ./doc_managers/solr_doc_manager.py** 

My conclusion

 2013-06-19 16:19:10,943 - INFO - Finished 'http://localhost:8983/solr/update/?commit=true' (post) with body 'u'<commit ' in 0.012 seconds. 

But I could not configure SOLR to receive documents from MONGODB. Please help me configure SOLR to get documents from MONGODB. Should I use SolrMongoImporter?

Answer 1

I had the same problem. I could not solve it, so I found an interesting link: http://derickrethans.nl/mongodb-and-solr.html It connects mongo with solr via php script. A

Answer 2

Step 1 : Install the Mongo Connector

To install mongo connector start

  Pip install mongo-connector 

Step 2 : creating the Solr core

  ./bin/solr create -c <corename>-p 8983 -s 3 -rf 3 

Step 3 : Solr setup

Fields in mongodb indexed files are specified in the schema.xml configuration file. open the schema.xml file in vi. how

 vi/solr/solr-6.6.2/server/solr/configsets/data_driven_schema_configs/ conf/schema.xml 

Step 4 : Mongo Connector also saves metadata associated with each mongodb document that it indexes in the ns and _ts fields. Also add the ns and _ts fields to schema.xml.

 <schema> <?xml version="1.0" encoding="UTF-8" ?> <schema name="example" version="1.5"> <field name="time_stamp" type="string" indexed="true" stored="true" multiValued="false" /> <field name="category" type="string" indexed="true" stored="true" multiValued="false" /> <field name="type" type="string" indexed="true" stored="true" multiValued="false" /> <field name="servername" type="string" indexed="true" stored="true" multiValued="false" /> <field name="code" type="string" indexed="true" stored="true" multiValued="false" /> <field name="msg" type="string" indexed="true" stored="true" multiValued="false" /> <field name="_ts" type="long" indexed="true" stored="true" /> <field name="ns" type="string" indexed="true" stored="true"/> <field name="_version_" type="long" indexed="true" stored="true"/> </schema> 

Step 5 We also need to configure the org.apache.solr.handler.admin.LukeRequestHandler request handler in the solrconfig.xml file.

Open solrconfig.xml in vi.

  vi ./solr-5.3.1/server/solr/configsets/basic_configs/conf/solrconfig.xml 

Specify the request handler for the Mongo connector.

 *<requestHandler name="/admin/luke" class="org.apache.solr.handler.admin.LukeRequestHandler" />* 

Also set auto commit to true so that Solr automatically commits data from MongoDB after the configured time.

 <autoCommit> <maxTime>15000</maxTime> <openSearcher>true</openSearcher> </autoCommit> 

Step 6 : Solr Must Be Restarted

 Bin/solr restart -force 

Starting the MongoDB server Mongo Connector requires that the MongoDB replica set be launched to index MongoDB data in Solr. A replica set is a cluster of MongoDB servers that implements replication and automatically switches to another resource. A replica set can consist of only one server, and the port is specified as 27017, the data directory for MongoDB is specified as / data / db, and the name of the replica set is specified as rs0 with the -replSet parameter.

 Sudo mongod --port 27017 --dbpath /data/db --replSet rs0 

Step 7 : launch the MongoDB shell Run the Mongodb shell with the following command

 Mongo 

The MongoDB shell starts. We need to initiate a set of replicas. Run the following command to start the replica set.

  rs.initiate() 

Step 8 : launch the MongoDB connector and index the MongoDB database using Solr Run the Mongo-connector command as shown below

 mongo-connector --unique-key=id –n solr.wlslog -m localhost:27017 -t http://xx.xxx.xxx.xx:8983/solr/wlslog -d solr_doc_manager 

In the above statement, solr.wlslog -> solr is databasename wlslog is the name of the collection Solr / wlslog -> wlslog is the name of CORE

For future reference, use the link below https://www.toadworld.com/platforms/nosql/b/weblog/archive/2017/02/03/indexing-mongodb-data-in-apache-solr