;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Sphinx module for Drupal 5.x
;; $Id$
;; Original author: markus_petrux at drupal.org (July 2008)
;; This project and donation to the community is sponsored by gamefilia.com
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

Requirements:

  - Drupal 5.x
  - PHP 4
  - Sphinx 0.9.8
  - MySQL, but it also may work with PostgreSQL.
  - Shell access to the box where sphinx is installed.


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

Installation:

  1) Install sphinx.

     It is recommended to install sphinx on separate box, but it may also work
     on any other server of your farm, or even in the same box your web server,
     mysql or whatever is installed.

     For more details, additional requirements, etc. please, read sphinx
     documentation. Here's just a quick start guide. You need root access
     to the box.

     # move to a temp directory.
     cd /opt
     # download and untar sphinx source.
     wget http://www.sphinxsearch.com/downloads/sphinx-0.9.8-rc2.tar.gz
     tar xzf sphinx-0.9.8-rc2.tar.gz
     cd sphinx-0.9.8-rc2
     # download and un tar libstemmer.
     wget http://snowball.tartarus.org/dist/libstemmer_c.tgz
     tar xzf libstemmer_c.tgz
     # you may need to adjust file ownerships.
     chown -R root.root *
     # build, compile and install sphinx + libstemmer.
     ./configure --with-mysql --with-libstemmer --prefix=/usr/local/sphinx
     make
     make install


  2) See sphinx/contrib folder for sample sphinx.conf and sphinx start/stop
     script.

     ***** IMPORTANT *****
     Files in contrib folder are just samples. They are provided in order to
     help you get into sphinx, but without warranties of any kind. Note that
     I just started to learn it just recently. Also, my environment and needs
     may differ a lot from yours. You cannot use them as-is. If you do, it is
     at your own risk.
     *********************

  3) Install sphinx Drupal module as usual.

     Copy to modules/sphinx all files and directories.
     Goto admin/build/modules to install the module.
       (creates a table used to control node index queue)
     Goto admin/user/access to adjust permissions.
       (use sphinx search, administer sphinx)
     Goto admin/settings/sphinx to configure module options.
       (see below)

  4) Customization:

     Check module settings and adjust to your environment.
     Initialize index queue and build main index.
       (see contrib/sphinx script for details).
     Setup cron to build delta indexes at intervals.

     ***** IMPORTANT *****
     There are options in the module settings panel that require you to
     rebuild main index. Otherwise, you may get errors when searching.
     *********************

  - Logging:

    Index processing generates watchdog records with information on memory
    used and execution time, to help you adjust module settings to suit
    your needs.


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

A few notes about Sphinx index generation:

- This module suppots xmlpipe index type so we can index preprocessed node
  content, which allow us to pass on to sphinx text generated by any kind of
  Drupal module. ie. node itself, but also comments, taxonomies, CCK, etc.
  However, it is MUCH slower than indexing content using mysql/pgsql sphinx
  index types.

- At this time, Sphinx does not support index updates or deletions in real time.
  In this scenario, you need to work with the so called main+delta scheme. See
  sphinx documentation for more details.

- I had a lot of problems related to resource consumption in Drupal to generate
  the XMLPipe stream used by sphinx to build indexes. It all depends on the
  complexity of your Drupal intallation, modules installed, overall size of
  your nodes, available infraestructure, etc.
  In order to minimize these problems, I implemented in the XMLPipe generation
  script a few checks that will terminate stream generation, just before memory
  limit or max execution time is reached. It can also restart DB server
  connection to avoid getting timeouts from here (see module settings panel).

- There may be other implemetation techniques to fight with this problem. You
  may take a look here for details:

  http://www.sphinxsearch.com/forum/view.html?id=1215


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

A few notes on the user interface for searching:

- This module does NOT index users and no other kind of content. Only nodes. So,
  the URL path does not have index type in it. ie. to search content the URL with
  Drupal search looks like 'search/node'. With this module it is just 'sphinx'.
  There is a constant definition on top of sphinx.module, that allows you to
  change it to whatever else you may need. If you change this constant, then you
  may need to clear drupal menu cache.

- Sphinx has several modes of matching results. This module works with extended
  method, which offer a wide range of options. Please, read sphinx docs.


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

TODO:

- Well, it depends... I created this module for a particular installation. This
  made me take a few decissions on some implementation details that may vary
  from yours, or to get something that can be widely used by the Drupal
  community.

  I don't have the time at this moment to use Drupal CVS, and project management
  resources, nor time to maintain module project, support requests, etc.
  However, I will be more than happy if this piece of work can be of any help or
  useful to others.

  So, I will contact someone at drupal.org, maybe through the infraestructure
  mailing list, as suggested by moshe weitzman here:

  http://groups.drupal.org/node/9795#comment-43190

- That being said, I suggest adding tagadelic functionality to this module,
  faceted search capabilities from additional blocks, advanced search form,
  or something in that direction.
  I made some tests already and sphinx grouping is a lot faster than all those
  heavy SQL joins that are often a performance problem for many. Though, I have
  to leave this project now to work on a lot of other things my employer wants
  me to.

  Just hope some of the experience I collected while working on this module
  can make things easier for someone that is able to implement shpinx integration
  module into Drupal.
