Manticore Search 3.5.2: optimized OPTIMIZE, standalone NOT, faceting via HTTP and more

We are excited to announce that Manticore Search 3.5.2 is out!

Besides 19 bug fixes it introduces few nice new features and changes:

Smarter OPTIMIZE

If you use Manticore Search real-time indexes you know that there's command OPTIMIZE INDEX which does index compaction. Historically it could only merge index chunks down to a single chunk. In 3.5.2 this is changed and is now configurable:

  • by default, the final number of chunks equals to 2 * # of CPU cores which allows to utilize your multiple CPUs efficiently
  • to fine-tune the behavior there's a new option cutoff which makes it keep merging until the number of chunks is what you want

👷 Work in progress: even smarter OPTIMIZE and which you won't have to run externally at all!

Standalone NOT operator

NOT operator can be now used standalone. By default, it is disabled since accidental single NOT queries can be slow. But if you know what you are doing you can enable it by setting a new searchd directive not_terms_only_allowed to 0 . This solves the task of full-text blacklisting, when you want to find all documents except those that contain some bad words. This is especially useful when used as a percolate query in Manticore Percolate Queries as it lets you just save your "black list" as a set of percolate queries and then filter out all documents containing them.

max_threads_per_query and OPTION threads

In 3.5.0 we introduced a new multitasking mode which is based on option searchd.threads limiting max number of threads Manticore Search can allocate for different jobs. But if you have too many chunks in a real-time index or way too distributed index + high QPS you might not want to let each query take all your threads. In 3.5.2 there are 2 new options:

  • max_threads_per_query to limit it globally
  • and SELECT ... OPTION threads=N for per-query limiting

Aggregations in HTTP protocol

In Manticore 3.5.2 HTTP API /search receives basic support for faceting and grouping.

👷 Work in progress: add this functionality to HTTP-based Manticore clients (php, python, javascript, java, elixir)

Default replication ports

Configuring replication has become easier. In most cases, you don't have to think about what ports to dedicate to be used by replication and how to enable replication in configuration at all. If you want to sync an index between multiple Manticore nodes it's now this easy:
Easy replication

New packaging schema

As of 3.5.2 it's now possible to install/upgrade Manticore Search server and auxiliary tools (indexer, indextool etc.) separately. It can be especially useful for those who use plain indexes and before rolling out a new indexer (which perhaps changes index format) want to try out just the searchd (which understands older formats).

The new package names are "manticore-server" and "manticore-tools" and can be installed via package managers.
Package manticore is supported as previously and if you don't need the above just install/upgrade manticore as you are used to.

Read about more changes happened in 3.5.2 in Manticore Manual and stay tuned!

Sincerely yours,
Manticore team

Leave a Reply