deduplicate_insert
Enables or disables block deduplication ofINSERT INTO (for Replicated* tables).
The setting applies to both synchronous and asynchronous inserts, and it supersedes the insert_deduplicate and async_insert_deduplicate settings.
That setting has three possible values:
- disable — Deduplication is disabled for
INSERT INTOquery. - enable — Deduplication is enabled for
INSERT INTOquery. - backward_compatible_choice — Deduplication is enabled if
insert_deduplicateorasync_insert_deduplicateare enabled for specific insert type.
INSERT SELECT queries, deduplicate_insert_select is consulted first. Deduplication also requires the destination table to keep a deduplication log, see replicated_deduplication_window and non_replicated_deduplication_window.
deduplicate_insert_select
Enables or disables block deduplication ofINSERT SELECT (for Replicated* tables).
For INSERT SELECT queries this setting is consulted before deduplicate_insert.
A SELECT is stable when it carries ORDER BY ALL and its pipeline ends in a single stream. A non-empty insert_deduplication_token is an equivalent substitute for stability.
That setting has four possible values:
- disable — Deduplication is disabled for
INSERT SELECTquery. - force_enable — Deduplication is enabled for
INSERT SELECTquery, regardless ofdeduplicate_insert. If select result is not stable and no token is provided, exception is thrown. - enable_when_possible — Deduplication is enabled if it is enabled by
deduplicate_insertand the select result is stable, otherwise disabled. - enable_even_for_bad_queries - Deduplication is enabled if it is enabled by
deduplicate_insert, regardless of whether the select result is stable. If select result is not stable, a message is logged, but query is executed with deduplication. This option is for backward compatibility. Consider to use other options instead as it may lead to unexpected results.