MySQL source wasn't initialized. Wrong name in dlopen?

In this article we talk about a frequent issue we are seeing when users try to import data from MySQL.

It's well know Manticore can index data from MySQL very easy using the index building tool indexer. Before getting into the issue that makes the title of this article let's first
make an introduction of the relation between Manticore and MySQL.

Manticore's most powerful API right now is SphinxQL which uses the MySQL transport protocol and makes very easy to perform searches or handle other tasks as the query language syntax is very similar with MYSQL one. MySQL protocol commands required for SphinxQL are implemented native in the searchd. For this reason, Manticore doesn't have a minimum requirement for MySQL libraries or MySQL to run on the box where Manticore is installed.

indexer tool can connect to a MySQL db and get data to build a Manticore index. Connecting to a MYSQL db is done using the official C++ client provided to MySQL developers. When compiling Manticore, the MySQL client library is not embedded in Manticore binaries. The same goes for PostgreSQL, XML pipes and ODBC. In case of indexer if the user wants to connect to PostgreSQL, MySQL or a database supporting ODBC, the driver libraries needs to be installed on the box where indexer is installed.

So why do we get the 'MySQL source wasn't initialized. Wrong name in dlopen?' error?

There are several causes of it:

- libmysqlclient is not installed on the system. The library is provided by the linux distribution under a package which can have different names:

  • On CentOS/RHEL it's mysql-libs
  • On Debian/Ubuntu you can use default-libmysqlclient-dev.
  • Alpine includes a mariadb-connector-c-dev packages.

- You do have libmysqlclient, but it's not the right one. Indexer is compiled against a specific client library version, the one included on the default MySQL of the Linux distribution. The version of MySQL is not the same across linux distributions: some use the official MySQL,others use MariaDB as default.

The problem you can get is when you upgrade to a new major version of your MySQL or you are switching to another MySQL variant. In both cases, the client library they ship can provide a different (in most cases newer) version of client library.

MySQL distributions offer a compatibility package which install older versions of the client library for applications designed to work with those. For example if you are switching on a distribution that uses MariaDB by default to the latest official MySQL, you should install from official MySQL the compatibility package  named mysql-community-libs-compat.

- you installed a wrong Manticore package. For example you are using Ubuntu, but installed Manticore Debian package. While searchd may work, the indexer you install was made for a different version than the one provided by the current distribution you use.

- a lesser case is when MySQL is manually installed or the mysql library client for some reason is not included in the shared libraries path. While Manticore doesn't care how MySQL is installed, it needs to be able to find the client library using the operating system's shared library path, which usually is /usr/lib or /usr/lib/x86_64-linux-gnu/. In this case libmysqlclient should be added (or symlinked)  in the default ld path or if you are using a custom path, you need to add that path in /etc/ld.so.conf.d/ (for example in some distributions the client libraries are added in /usr/lib/mysql folder, but the package that installs the files also adds an entry in /etc/ld.so.conf.d/). Alternative the ld library path can be set to include the mysql library when starting daemon like LD_LIBRARY_PATH=/path/to/mysql searchd or if you use a systemd unit file to set it in Environment, but recommended is to resolve the path to the library client system wide (as unit files may be changed by future updates).

Leave a Reply

Training

Personal and team training will maximize them performance. 

Custom development

Need cone custom or individual features?

Fill the form and don’t forget to make the description of what you need.

Free config review

There are often optimizations that can be made to a Sphinx / Manticore setup by changing some simple directives in the configuration or making quick changes to an index definition.

Some common mistakes and issues can include:

  • doing main+delta without kill-lists, even if the delta does include updated records found in the main
  • using wildcarding with very short prefix/infix which can hammer performance in some cases
  • disabled (unintentional) seamless rotates and getting stalls on index rotations
  • adding texts as string attributes even if they are not using for any kind of operation (filtering, grouping, sorting) or mandatory to be present in results
  • using deprecated settings 

Having a quick look on the configuration can show issues or potential issues, this is why we want to offer a gift to our growing community!

When uploading your configuration file, we recommend to remove any database credentials first.

We suggest also you give as many possible details about your setup: how big is the data you have, how typical queries look and what issues you experience.

Contact us