Hello, i've just installed and configured Sphinx and so far it works ok. Thanks for module, btw.
And here's my question from the title:
Can this module implement feature for Live updae index? This thing described in Sphinx documentation and it allows Sphinx to index not a whole site every time, but only new added content (and occasionally whole index, in case of update or whatever). It uses main/delta indexes for this stuff and while configuring this module i've found delta index in conf file, so i thought why not? Maybe it's implemented and i just don't know how to use it.

So can it?

Again, many thanks for module.

Comments

markus_petrux’s picture

Status: Active » Fixed

main+delta index scheme is supported, but this is a complex issue if you're new to Sphinx. Best I can tell is please, read the README shipped with this module. It is all explained there.

Kane’s picture

Thanks, i've done all like in readme, but got stuck while reindexing, here's a problem:

ERROR: index 'index_drupal_example_delta': xmlpipe: expected '', got 'XMLPipe for delta index failed: Could not obtain list of main i'.

No idea what's that.
Thanks.

markus_petrux’s picture

Status: Fixed » Active

Could you please post your sphinx.conf (attached as a plain text file) and the output of the "Check connection" option of the Sphinx search module (next to Sphinx search settings)?

markus_petrux’s picture

Status: Active » Postponed (maintainer needs more info)
markus_petrux’s picture

Title: Can this module live update index? » XMLPipe for delta index failed: Could not obtain list of main indexes from Sphinx

hmm... I guess you see this error in the Drupal log, and that usually means your Sphinx search daemon is not started, or your Drupal installation has not been able to connect (settings? firewall?).

Kane’s picture

I believe i resolved my problem, changed some settings here and there. Thanks to README file and contributed manuals everything works fine, even delta index works (almost live update, heh).
Thanks again.

markus_petrux’s picture

Status: Postponed (maintainer needs more info) » Fixed

Ok, so issue fixed.

Live updates in Sphinx works only for certain attributes, and it is used here to flag deleted documents. It cannot be used to re-index documents yet.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

quotesbro’s picture

Status: Closed (fixed) » Active

@Kane: Can you say what you changed? I have the same problem (ERROR: index 'index_delta': xmlpipe: expected '', got 'XMLPipe for delta index failed: Could not obtain list of main i'.) and I don't know what to do with it.

P.S. I use Sphinx 0.9.9-rc2 (r1785)

upd: i resolved this problem (reduced the number of nodes in the index)

quotesbro’s picture

Status: Active » Closed (fixed)
Kane’s picture

I've updated drupal (6.16?) today in the morning (night) and after that sphinx stopped working.
Settings and config file were intact, so i've logged via ssh and tried to run indexer
first, there were problem with main index - nothing indexed, i thought it must be php memory limit, so i've increased it and main index was done.
Then i checked delta index and here's where fun began.
Good old "xmlpipe: expected '', got 'XMLPipe for delta index failed: Could not obtain list of main i'." returned.

Tried everything but in the end decided to use clean config file. That took about 5 minutes to delete all old indexes, stop daemon and other stuff.
After configuration i started sphinx daemon. Some problems occurred but it was about port.
Anyway, during that part there were some alerts about missing delta.sph (or something similar) file.
So, i've created clean delta.sph and put it in index folder.
Then i started daemon again. Worked.
Then i started index. Worked.

Basically, i haven't changed anything at all.

Could it be that "xmlpipe: expected '', got 'XMLPipe for delta index failed: Could not obtain list of main i'." error occurs when it can't find that delta.sph file?

dark_kz’s picture

Hello!
Debian Squezze 6
sphinxsearch-2.0.5
sphinxsearch-2.0-dev version
I have a site with ~30 000 nodes.
When I start indexer
/usr/local/sphinx/bin/indexer --config /home/dark/domains/example.com/html/sphinx/config/sphinx.conf --all

using config file '/home/dark/domains/example.com/html/sphinx/config/sphinx.conf'...
indexing index 'index_m0'...
WARNING: source 'sm0': unexpected string 'Fatal error: Allowed memory size of 268435456 bytes exhausted (t' (line=189085, pos=0) outside of <sphinx:schema> and <sphinx:document>
ERROR: index 'index_m0': source 'sm0': XML parse error: no element found (line=189086, pos=0, docid=999).
total 907 docs, 49881942 bytes
total 22.785 sec, 2189221 bytes/sec, 39.80 docs/sec
indexing index 'index_m1'...
WARNING: source 'sm1': unexpected string 'Fatal error: Allowed memory size of 268435456 bytes exhausted (t' (line=176829, pos=0) outside of <sphinx:schema> and <sphinx:document>
ERROR: index 'index_m1': source 'sm1': XML parse error: no element found (line=176830, pos=0, docid=13302).
total 2985 docs, 33767141 bytes
total 43.420 sec, 777680 bytes/sec, 68.74 docs/sec
indexing index 'index_m2'...
WARNING: source 'sm2': unexpected string 'Fatal error: Allowed memory size of 268435456 bytes exhausted (t' (line=114584, pos=0) outside of <sphinx:schema> and <sphinx:document>
ERROR: index 'index_m2': source 'sm2': XML parse error: no element found (line=114585, pos=0, docid=24023).
total 1707 docs, 18383971 bytes
total 31.265 sec, 587992 bytes/sec, 54.59 docs/sec
indexing index 'index_m0_delta'...
ERROR: index 'index_m0_delta': xmlpipe: expected '<document>', got 'XMLPipe for delta index failed: Could not obtain list of main i'.
total 0 docs, 0 bytes
total 0.000 sec, 0 bytes/sec, 0.00 docs/sec
skipping non-plain index 'index_example_join'...
total 0 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
total 0 writes, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg

Any ideas about "xmlpipe: expected '< document >', got 'XMLPipe for delta index failed" error?

dark_kz’s picture

StatusFileSize
new29.66 KB
new3.01 KB

I divide indexes, and then it works. But index delta still has an error
ERROR: index 'index_m0_delta': xmlpipe: expected '<document>', got 'XMLPipe for delta index failed: Could not obtain list of main i'.

Connection status:
Testing Sphinx searchd daemon connection.
OK - Your Sphinx searchd daemon is up and running.
Testing search query using index index_example_join.
Query asks for a list of all indexed documents to retrieve the last indexed node.
MySQL equivalent would look something like:
SELECT nid FROM {node} ORDER BY nid DESC LIMIT 1;
OK - Test passed successfully. Sphinx results summary follow:
Query execution time: 0.033 seconds
Total results found: 25932
Total results available: 1000
Last indexed node: 39838 (2012-10-05 11:24:53)
Testing excerpts builder using index sm1.
ERROR - Unable to build excerpts.
Sphinx message: searchd error: unknown local index 'sm0' in search request