Sphinx UDF example

Hi

Many databases and search engines allow you to customize your queries using your own so called “user defined functions” or UDF. Sphinx and Manticore are not exceptions. There’s a long section in documentation about this – https://docs.manticoresearch.com/latest/html/extending.html#udfs-user-defined-functions

Here I want to give just a quick example of how you can make a UDF which enables some fucntionality which can be really useful in some cases, but missing out of the box – sleep() function.

First of all create the code of your UDF in C:

[snikolaev@dev01 ~]$ cat sleep.c
#include </home/snikolaev/./manticore_gitlab/src/sphinxudf.h>

int sleep_ver() {
	return SPH_UDF_VERSION;
}

sphinx_int64_t sleep ( SPH_UDF_INIT * init, SPH_UDF_ARGS * args, char * error_flag ) {
	usleep(*args->arg_values[0] * 1000000);
}

As you can see it has only 2 functions:

  • sleep_ver() returning just current SPH_UDF_VERSION from the included sphinxudf.h
  • and sleep() which does the job – sleeps the number of seconds provided in argument #1

HINT: Make sure your file has extension “.c”, not “.cpp”, otherwise it may not work.

Next you need to build your library:

[snikolaev@dev01 ~]$ gcc sleep.c -shared -o sleep.so -g -fPIC
[snikolaev@dev01 ~]$

You now have sleep.so which includes your function:

[snikolaev@dev01 ~]$ ls -la sleep.so
-rwx--x--x 1 snikolaev snikolaev 8209 Sep 10 04:37 sleep.so

Next make sure you specify in your config where searchd should look for UDFs using plugin_dir directive:

common
{
plugin_dir = /home/snikolaev
}

Now run your searchd and create the function:

mysql> create function sleep returns int soname 'sleep.so';
Query OK, 0 rows affected (0.00 sec)

HINT: if you do not want to do it every time you start searchd you can put the command to a file:

[snikolaev@dev01 ~]$ cat state.sql
CREATE FUNCTION sleep RETURNS INT SONAME 'sleep.so';

and use

sphinxql_state = state.sql

in the searchd section of your config so it runs the command on start.

Once you have the function registered in the searchd you can use it:

mysql> select sleep(1);
+----------+
| sleep(1) |
+----------+
| 0 |
+----------+
1 row in set (1.00 sec)

If you have any questions about Sphinx UDFs don’t hesitate to ask us on our forum or in our public slack channel.

Leave a Reply

Training

Personal and team training will maximize them performance. 

Custom development

Need cone custom or individual features?

Fill the form and don’t forget to make the description of what you need.

Free config review

There are often optimizations that can be made to a Sphinx / Manticore setup by changing some simple directives in the configuration or making quick changes to an index definition.

Some common mistakes and issues can include:

  • doing main+delta without kill-lists, even if the delta does include updated records found in the main
  • using wildcarding with very short prefix/infix which can hammer performance in some cases
  • disabled (unintentional) seamless rotates and getting stalls on index rotations
  • adding texts as string attributes even if they are not using for any kind of operation (filtering, grouping, sorting) or mandatory to be present in results
  • using deprecated settings¬†

Having a quick look on the configuration can show issues or potential issues, this is why we want to offer a gift to our growing community!

When uploading your configuration file, we recommend to remove any database credentials first.

We suggest also you give as many possible details about your setup: how big is the data you have, how typical queries look and what issues you experience.

Contact us