Migrating from Sphinx

In this article we talk about how to upgrade from Sphinx.
Manticore Search is a fork from Sphinx Search 2.3. While many portions of the code have been refactored and improved, there is no breaking change that require a major effort to upgrade from Sphinx. In most cases, the upgrade procedure is the same as upgrading between minor Sphinx versions, which mostly consist in just replacing the binary executables.

Indexes

Manticore uses the same index format as Sphinx 2.x and indexes created with Sphinx will be loaded by the Manticore daemon. The only situation when reindexing is required is if you use pre Sphinx 2.2 indexes that use str2ordinal/str2wordcount attributes which have been deprecated in Sphinx 2.2.

As a general consideration it’s best to test first if upgrade pass without issues before performing upgrade on production instances. What to follow in testing is to check if indexes load without issue and common queries performed works as expected.

In case of distributed indexes with remote nodes, the master should be upgraded first. The API protocol used between master and nodes received some updates, but the master can still communicate with nodes using an older version.

Configuration

There are settings deprecated in Manticore compared to Sphinx 2.3.2. As a reminder in 2.3.1 the prefork and fork worker modes have been removed. Also during Sphinx 2.2 options like charset_type, enable_star and max_matches have been deprecated from sphinx.conf ( max_matches is still a query setting, just not global anymore). It’s worth to mention that crc dictionary mode is still supported in Manticore as there are cases where it can be more advantageous than ‘keywords’ mode.

Querying

On the client communication, if you’re using SphinxQL there is no nothing needed on the client side.

If you’re using SphinxAPI, we didn’t made any change in the client library since the fork, however if you’re coming from older Sphinx versions you should upgrade to the latest API client library.

The SphinxSE MySQL plugin haven’t receive updates in a while, but in Manticore we introduced support for FEDERATED engine, which is a better choice as FEDERATED is included in all MySQL variants (unlike SphinxSE which is only distributed with MariaDB).

Compiling

Manticore switched from  autotools to cmake . The code requires newer versions of compilers. For more information consult the compiling docs.

Upgrading on Windows

Stop the Sphinx service.
Replace the all old executables and libraries with the new ones.
The existing service set by Sphinx can start Manticore without issue. If you want to replace it, just run searchd.exe --delete and searchd.exe --install.

Upgrading from Linux packages

Manticore packages uses the same location for the configuration file : /etc/sphinxsearch/sphinx.conf or /etc/sphinx/sphinx.conf, depending on distribution.

What changed is the user under which searchd runs and the folders used. Sphinx ran under a sphinx/sphinxsearch user and use /var/lib/sphinxsearch, /var/log/sphinxsearch and /var/run/sphinx folder.

Manticore run under a manticore user and default folders are /var/lib/manticore, /var/log/manticore and /var/run/manticore.

First stop the Sphinx service, with service sphinxsearch stop or systemctl stop sphinx, depending on the distribution used.

You can upgrade directly using dpkg -i package or rpm -Uvh package and choose to keep the existing sphinx.conf.

As the Manticore service uses /var/run/manticore to check the existence of the pid file, the only change needed in sphinx.conf is to change the location of pid_file to /var/run/manticore/searchd.pid. We recommend changing the location of the pid file in sphinx.conf rather then editing the manticore.service file as future upgrades can overwrite changes made in the service file and re-patching will be required.

Next is to set  permission on existing Sphinx folders to manticore user.

At this point you can start the Manticore service with service manticore start or systemctl start manticore.

If you need help with upgrading from Sphinx,don’t hesitate to ask on official slack channel or forums.

2 thoughts on “Migrating from Sphinx

  • >> in current version update of MVA needs exclusive lock of index
    > It is a blocking regression.

    Not exactly. Sphinx has MVA wlock as well and the issue reported in ticket we couldn’t reproduce it only with the data provided.
    Provide more information requested by developer in the ticket.

Leave a Reply