Manticore Search 2.8.0 GA release

We are pleased to announce the release of Manticore Search 2.8.0 GA.

Prebuilt packages available for Debian,Ubuntu, Red Hat/CentOS, Windows and OSX can be downloaded from here.

New features and improvements

Major feature introduces in this release is the support of clustering for percolate indexes.

For percolate indexes, DESCRIBE command has now extension TABLE which allows viewing the schema of the percolate documents.

ANY/ALL functions can be used on string attributes or expressions as well on PQ tags. SELECT over pq indexes have same functionality as on regular indexes except MATCH clause.

New CALL PQ options:

  • skip_bad_json – if enabled, broken json documents will be skipped and will not terminate the call in progress
  • mode – for distributed setups. Default value is ‘sparsed’ –  the engine will split the documents among the percolate agents. Alternative option is ‘sharded’ in which case all documents are sent to every percolate agent
  • shift – an offset to document ids if ‘docs_id’ not specified. Useful for distributed ‘sparsed’ mode

charset_table can use alias ‘non_cjk’ which offer a collection of common characters of most non-cjk languages. The set is inspired from ICU tokenizer.

A similar alias for ngram_chars ‘cjk’  can be used for a ready-to-go collection of CJK characters used in unsegmented texts.

Manticore now ships a collection of 50 stopwords language-based files, which can be declared in stopwords directive by using the short code of the language.

JSON fields in expression no longer require explicit conversion. For example in case of GEODIST it was required to explicit cast on geo coordinates stored as  floats inside a JSON  object: GEODIST(DOUBLE(json.lat),DOUBLE(json.long),x,y).

RPM packages come with a systemd generator that detects if jemalloc can be used instead of the standard malloc. jemalloc needs to be already installed on the system and transparent huge pages to be disabled. Using jemalloc showed improvements in RAM usage and performance.

Fixes

18 bug fixes are also included in the release, to highlight a few:

  • bf4e9ea fixed save of empty meta for RT index
  • 6b95d48 fixed string attrs >4M use saturate instead of overflow
  • 8868b20 fixed cut SphinxQL error message larger 512 chars
  • de9deda fixed crash on save percolate index without binlog

For a full report please consult the Release note.

Leave a Reply

Training

Personal and team training will maximize them performance. 

Custom development

Need cone custom or individual features?

Fill the form and don’t forget to make the description of what you need.

Free config review

There are often optimizations that can be made to a Sphinx / Manticore setup by changing some simple directives in the configuration or making quick changes to an index definition.

Some common mistakes and issues can include:

  • doing main+delta without kill-lists, even if the delta does include updated records found in the main
  • using wildcarding with very short prefix/infix which can hammer performance in some cases
  • disabled (unintentional) seamless rotates and getting stalls on index rotations
  • adding texts as string attributes even if they are not using for any kind of operation (filtering, grouping, sorting) or mandatory to be present in results
  • using deprecated settings 

Having a quick look on the configuration can show issues or potential issues, this is why we want to offer a gift to our growing community!

When uploading your configuration file, we recommend to remove any database credentials first.

We suggest also you give as many possible details about your setup: how big is the data you have, how typical queries look and what issues you experience.

Contact us