# Manticore Search 29.0.2: Better sharded-table operations and safer maintenance Manticore Search 29.0.2 adds native distributed and sharded table optimization, sharded-table settings inspection, faster columnar KNN rescoring, and fixes for long writes, RT optimization, backups, JDBC, highlighting, and distributed queries. [Manticore Search 29.0.2](/install/) has been released. The main theme this time is operational: sharded and distributed tables are easier to inspect and maintain, long-running shard writes are more reliable, and several nasty failure paths around RT optimization and backups are fixed. This post covers everything shipped after 28.6.6, from **28.6.7 through 29.0.2**. --- ## Upgrade Notes Most installations can upgrade normally. There is no table-format change, and existing data and configuration remain compatible. One note matters for mixed-version deployments that use native sharded-table writes or replication state transfer (SST): 29.0.0 changed the internal `SHARD_WRITE` API and cluster protocol to carry heartbeat data. Upgrade remote shard agents before the coordinators that write to them. For replication, upgrade all peers before initiating SST. Standalone nodes, and deployments that use neither native sharded-table writes nor replication SST, need no special handling. Downgrade is also possible without rebuilding data, but coordinators should be downgraded before agents, and SST should not run while replication peers use mixed versions. --- ## Native OPTIMIZE for Distributed and Sharded Tables [`OPTIMIZE TABLE`](https://manual.manticoresearch.com/Securing_and_compacting_a_table/Compacting_a_table#OPTIMIZE-TABLE) can now compact distributed and sharded tables directly: ```sql OPTIMIZE TABLE products OPTION sync=1, cutoff=1; ``` For a distributed table, Manticore applies the requested cutoff to every local RT component and every configured remote mirror. Native sharded tables use the same synchronous fan-out across their physical RT tables. Previously, operators had to address those physical tables separately. Now the logical table is enough. The command is synchronous by design for these table types, so success means the requested work has completed across the selected targets; failures from remote targets are reported instead of being hidden behind a successful local response. ## Sharded-table Settings Are Visible [`SHOW TABLE SETTINGS`](https://manual.manticoresearch.com/Node_info_and_management/Table_settings_and_status/SHOW_TABLE_SETTINGS) now works for native sharded tables: ```sql SHOW TABLE products SETTINGS; ``` It returns the unified tokenizer, dictionary, and table settings stored by the sharded-table wrapper. Besides being useful when debugging a table by hand, this also gives Buddy-backed features a reliable way to inspect how the logical table is configured. --- ## Long Shard Writes No Longer Depend on Oversized Timeouts A vector table may spend seconds building HNSW data while a RAM chunk is flushed. On a sharded table, that could exceed `agent_query_timeout`: the coordinator would give up even though the remote node was still doing valid work. Manticore now sends heartbeat replies over the same connection during long shard writes. Each valid heartbeat renews the timeout as a rolling lease, so the connection stays alive while the operation is making progress. You no longer need to hide normal flush time behind an excessively large query timeout. The same transport is used for long replication SST operations and for synchronous distributed optimization. This is also the reason for the mixed-version upgrade note above. --- ## More Control in Vector Search This release adds `access_columnar_attrs` and `access_secondary`, allowing columnar and secondary-index files to use buffered `file` access or `mmap`. That gives operators a direct way to choose the access pattern that fits the host and workload instead of relying on one fixed behavior. Columnar [KNN search](https://manual.manticoresearch.com/Searching/KNN#KNN-vector-search) also does less per-candidate work during rescoring. Full-precision distance calculations are now batched before the final sort. The result set does not change; the final ranking stage simply has less overhead. Conversational search received a smaller reliability improvement as well: failed LLM route-selection tool calls are retried up to three times before an error is returned. --- ## Reliability Fixes ### RT optimization and KNN A concurrent update that grew a blob-backed attribute could remap a source disk chunk while `OPTIMIZE TABLE` was merging it. Merge workers could then continue through a stale memory address and crash `searchd`. Source addresses are now refreshed after merge scheduling points, covering both regular attribute copying and KNN construction. ([Issue #4780](https://github.com/manticoresoftware/manticoresearch/issues/4780)) Explicit [`optimize_cutoff`](https://manual.manticoresearch.com/Server_settings/Searchd#optimize_cutoff) values now also control the KNN compaction threshold. Vector tables no longer keep using a separate default when a server-wide or per-table cutoff was set. ([Issue #4738](https://github.com/manticoresoftware/manticoresearch/issues/4738)) ### Backups and client compatibility [Manticore Backup](https://github.com/manticoresoftware/manticoresearch-backup) 1.10.3 fixes another freeze-related failure path. If a backup fails after the per-table `FREEZE`—for example, because a frozen file disappears before it is copied—both freeze levels are now released and the RT table remains writable. ([Backup PR #140](https://github.com/manticoresoftware/manticoresearch-backup/pull/140)) Buddy 4.4.1 restores MySQL Connector/J initialization by supporting `@@query_cache_size` and numeric system-variable values in the initialization query. ([Issue #4434](https://github.com/manticoresoftware/manticoresearch/issues/4434)) ### Query inspection, highlighting, and distributed memory - [`CALL SNIPPETS` and `HIGHLIGHT()`](https://manual.manticoresearch.com/Searching/Highlighting) now preserve existing HTML element boundaries with `html_strip_mode=retain`, producing well-formed nested markup. ([Issue #4792](https://github.com/manticoresoftware/manticoresearch/issues/4792)) - [`SHOW THREADS`](https://manual.manticoresearch.com/Node_info_and_management/SHOW_THREADS) no longer risks a use-after-free while inspecting concurrent HTTP SQL queries. ([Issue #4678](https://github.com/manticoresoftware/manticoresearch/issues/4678)) - [`EXPLAIN QUERY`](https://manual.manticoresearch.com/Searching/Full_text_matching/Profiling#Profiling-without-running-a-query) no longer hangs on template tables when a full-text query contains field selectors. Invalid selectors now return a syntax error. ([PR #4777](https://github.com/manticoresoftware/manticoresearch/pull/4777)) - Completed distributed-agent operations now release large request and reply buffers after parsing instead of retaining them until `agent_query_timeout` expires. This reduces master-daemon memory use after large distributed requests. ([Issue #4771](https://github.com/manticoresoftware/manticoresearch/issues/4771)) For the complete list, see the [Version 29.0.2 changelog](https://manual.manticoresearch.com/Changelog#Version-29.0.2). --- ## Get Manticore Search 29.0.2 Install or upgrade Manticore Search with the [installation guide](/install/). If you operate native sharded tables or a replication cluster, follow the rollout order in the upgrade notes before changing production traffic. ## Need Help or Want to Connect? - Join our [Slack](https://slack.manticoresearch.com) - Visit the [Forum](https://forum.manticoresearch.com) - Report issues or suggest features on [GitHub](https://github.com/manticoresoftware/manticoresearch/issues) - Email us at `contact@manticoresearch.com`