Why is my index missing documents with Thinking Sphinx?

Question

I have a simple Thinking Sphinx index defined on my Account model:

 define_index do indexes display_name indexes email_addresses.email_address has created_at set_property :delta => :datetime, :threshold => 2.minutes end 

(Ignore the delta for now, I am generating the full index and searching for account_core .)

But I get unexpected results:

 >> Account.count # => 885138 >> Account.search.total_entries # => 260795 >> Account.search(" [e-mail] ") # => [] 

However, on the command line, using the search utility, I can find Lenny:

 $ search -c /etc/sphinx/water.sphinx.conf -i account_core [e-mail] index 'account_core': query ' [e-mail] ': returned 2 matches of 2 total in 0.759 sec displaying matches: 1. document=3543432, weight=4, sphinx_internal_id=442101, sphinx_deleted=0, class_crc=0, created_at=Mon Apr 11 12:18:08 2011 2. document=5752816, weight=2, sphinx_internal_id=719552, sphinx_deleted=0, class_crc=0, created_at=Tue Dec 27 12:01:12 2011 

Indeed, these are Drew account identifiers.

Why can't I find Lenny when searching with Thinking Sphinx? Why is the number of total_entries so much smaller than the full rows of the accounts table?

Answer 1

It turns out the problem is with the way Thinking Sphinx handles single-page inheritance. TS returns only records with type that correspond to one of the subclasses of the parent class. If type NULL , the document is not included in the search results. We had many entries in the accounts table with type=NULL . After fixing the data, the search now works as expected.

Thanks to roman3x in #sphinxsearch for pointing this out.