In this article we talk about a frequent issue we are seeing when users try to import data from MySQL.
It’s well know Manticore can index data from MySQL very easy using the index building tool
indexer. Before getting into the issue that makes the title of this article let’s first
make an introduction of the relation between Manticore and MySQL.
Manticore’s most powerful API right now is SphinxQL which uses the MySQL transport protocol and makes very easy to perform searches or handle other tasks as the query language syntax is very similar with MYSQL one. MySQL protocol commands required for SphinxQL are implemented native in the
searchd. For this reason, Manticore doesn’t have a minimum requirement for MySQL libraries or MySQL to run on the box where Manticore is installed.
indexer tool can connect to a MySQL db and get data to build a Manticore index. Connecting to a MYSQL db is done using the official C++ client provided to MySQL developers. When compiling Manticore, the MySQL client library is not embedded in Manticore binaries. The same goes for PostgreSQL, XML pipes and ODBC. In case of
indexer if the user wants to connect to PostgreSQL, MySQL or a database supporting ODBC, the driver libraries needs to be installed on the box where indexer is installed.
So why do we get the ‘MySQL source wasn’t initialized. Wrong name in dlopen?’ error?
There are several causes of it:
- libmysqlclient is not installed on the system. The library is provided by the linux distribution under a package which can have different names:
On CentOS/RHEL it’s
On Debian/Ubuntu you can use
Alpine includes a mariadb-connector-c-dev packages.
You do have libmysqlclient, but it’s not the right one. Indexer is compiled against a specific client library version, the one included on the default MySQL of the Linux distribution. The version of MySQL is not the same across linux distributions: some use the official MySQL,others use MariaDB as default.
The problem you can get is when you upgrade to a new major version of your MySQL or you are switching to another MySQL variant. In both cases, the client library they ship can provide a different (in most cases newer) version of client library.
MySQL distributions offer a compatibility package which install older versions of the client library for applications designed to work with those. For example if you are switching on a distribution that uses MariaDB by default to the latest official MySQL, you should install from official MySQL the compatibility package named
- you installed a wrong Manticore package. For example you are using Ubuntu, but installed Manticore Debian package. While
searchdmay work, the indexer you install was made for a different version than the one provided by the current distribution you use.
- a lesser case is when MySQL is manually installed or the mysql library client for some reason is not included in the shared libraries path. While Manticore doesn’t care how MySQL is installed, it needs to be able to find the client library using the operating system’s shared library path, which usually is
/usr/lib/x86_64-linux-gnu/. In this case libmysqlclient should be added (or symlinked) in the default ld path or if you are using a custom path, you need to add that path in
/etc/ld.so.conf.d/(for example in some distributions the client libraries are added in
/usr/lib/mysqlfolder, but the package that installs the files also adds an entry in
/etc/ld.so.conf.d/). Alternative the ld library path can be set to include the mysql library when starting daemon like
LD_LIBRARY_PATH=/path/to/mysql searchdor if you use a systemd unit file to set it in
Environment, but recommended is to resolve the path to the library client system wide (as unit files may be changed by future updates).