WIP: MDEV-40032 promote wide VARCHAR to BLOB inside the HEAP engine - #5655
Draft
arcivanov wants to merge 2 commits into
Draft
WIP: MDEV-40032 promote wide VARCHAR to BLOB inside the HEAP engine#5655arcivanov wants to merge 2 commits into
arcivanov wants to merge 2 commits into
Conversation
arcivanov
marked this pull request as draft
September 9, 2026 22:27
arcivanov
force-pushed
the
MDEV-40032-v2
branch
from
September 9, 2026 23:22
6f08f32 to
33adbcb
Compare
gkodinov
added
the
External Contribution
arcivanov
force-pushed
the
MDEV-40032-v2
branch
from
September 10, 2026 20:18
33adbcb to
f7c8e95
Compare
A VARCHAR whose declared width exceeds a threshold is stored out of line by the HEAP engine itself, in the blob continuation records that MDEV-38975 already built. The SQL layer is never told: the field stays a `Field_varstring`, so the declared type, the metadata sent to the client, and VARCHAR comparison and key semantics are all unchanged. This applies to user `ENGINE=MEMORY` tables as well as to internal temporary tables. Heap records are fixed width, so an inline `VARCHAR(N)` reserves its full declared width in every row whether the row uses it or not. N counts characters, so that width is between N and 4N bytes depending on the character set. Stored out of line the row pays a length prefix, a chain pointer, and only the bytes actually present. `HP_BLOB_DESC` carries the stored-side geometry and a `promoted` flag, `HP_SHARE` gains a `stored_reclength` and the byte ranges that are identical between the two layouts, and `hp_pack_record()` / `hp_unpack_record()` replace the whole-record memcpys. The three key functions that read a stored record use a parallel keyseg array. With nothing promoted every new pointer aliases the old one and the code path is what it was. Those two are `static inline` in `heapdef.h` rather than functions in `hp_blob.c`, because every row read and every row write of every HEAP table goes through one of them, including the tables with nothing promoted. There the span list is a single span covering the whole record and the body is the memcpy the call sites used to do inline, so out of line it costs a call and two dependent loads to reach its own size. Measured at `-O2` on a keyless table with a 128-byte record and nothing promoted, a scan reads a row in 5.5 ns inline against 6.8 out of line. The chain pointer a stored record holds for an out-of-line column is read, written and cleared through `hp_blob_get_chain()`, `hp_blob_set_chain()` and `hp_blob_clear_chain()`. The offset arithmetic behind them is what moves when a promoted column moves, and it was written out at seventeen sites across four files. The engine's limits are split three ways, each enforced in its own unit. Rows are limited by the new `HP_SHARE::max_rows`, tested in `heap_write()` where rows are counted; `share->records` had been incremented and compared against nothing, so HEAP had no row limit at all. Bytes are limited by the existing `max_table_size` test in `hp_alloc_from_tail()`, now the only ceiling there. `max_records` survives only as `HP_CREATE_INFO`'s block-sizing estimate: it was being tested against a slot counter, which a row using many continuation records exhausts long before its memory budget. With nothing left reading them, `HP_SHARE::max_records`, `HEAPINFO::max_records` and `NO_LIMIT_RECORDS` go rather than stay beside the row limit that replaced them, where two sentinels of the same value would have differed only in name. This is also the first time LIMIT pushdown is applied as the row count it always was. That changes what a table accepts, which `heap.blob` records. Under `max_heap_table_size=65536` a table with a declared BLOB took one 30000-byte row and refused the second; it now takes two and refuses the third. The refusal came from the `max_records` test, which compared a slot counter against an estimate derived from the SQL row width, and only where a new block was about to be allocated. A row whose value spans many continuation records exhausts that count while using a fraction of the bytes it stands for, so the table filled at half its budget. With bytes limited where bytes are counted, it fills when it has spent them. `hp_clear_dark_records()` clears a short record whole rather than at a stride. Measured on Zen 4, Broadwell and Denverton: at a 16-byte record the contiguous clear is 2.6 to 4.7 times faster, at 32 it is 1.3 to 2.3 times faster, and the two cross over above that, so the switch is at 32. `heap.promotion_transparent` asserts that storing a VARCHAR out of line changes nothing. Such a test would pass just as well if the engine never promoted anything, so it opens by proving promotion is in effect: an inline `VARCHAR(60000)` holds about 17 rows under a 1MB ceiling while the promoted form holds 200. `heap.row_limit` covers the row limit on a promoted VARCHAR, a declared BLOB and a plain INT. `heap_info()` now reports the ceiling `data_length` counts toward rather than the expected record count times `reclength`. That product is a record count times a row width, which is a byte ceiling only while one record holds one row; once a column is stored out of line it overstates the ceiling by the promotion ratio, so a `VARCHAR(3000)` table under a 1MB limit advertised 196MB and read as 0.2% full at the point it refused a row. The number is `max_table_size`, or `max_rows * reclength` where a row limit admits fewer bytes than that. `heap.max_data_length` asserts a table reaches the ceiling it reports before it refuses a row. `ha_heap::scan_time()` prices the free records a scan steps over at one step per free list entry rather than one per free record. heap_scan() reads a coalesced block's length from its first record and skips the block in a single step, so a row whose out-of-line data freed a run of a thousand records costs a later scan what a row that freed one record costs it; charging per record priced such a table at its promotion ratio above what it is worth. `HP_SHARE` counts the entries alongside the records, maintained in the five functions that own the free list -- only a block's last record ends an entry, and a coalescing push extends one rather than adding another. `heap_check_heap()` counts both while walking the list and rejects a share whose counters disagree. Two existing tests observed the old storage layout. `versioning.partition` filled a `VARCHAR(45000)` MEMORY table with rows left at their default and relied on each one reserving its full width anyway. Stored out of line those rows occupy almost nothing and the table never fills, so the `ER_RECORD_FILE_FULL` the test is built around never arrives. The rows now carry their full width as data, which restores the fullness the test needs and holds whether the column is stored inline or not. `perfschema.memory_table_io` records one more fetch on the scan that follows a delete. A row stored out of line leaves a hole behind when it is updated, and a scan that lands in one reports a deleted record to the caller, which costs a further `rnd_next`. A declared BLOB produces exactly the same event sequence, so this is the existing behaviour of out-of-line storage rather than anything promotion introduces.
Promoting inside the engine is not zero-copy. `record[0]` keeps its full declared width, so the engine compacts the record on the way in and expands it again on the way out, and every read of a promoted column copies the payload back into the record. Promote the field instead, and the payload is not copied at all: a read is handed the address of the engine's own bytes, and a record copied wholesale copies that address rather than the value behind it. `Field_varstring` gains a `promoted` flag that changes only where the payload lives. The record slot becomes the column's own length prefix followed by a pointer, which is the shape a blob already has, and `pack_length()` answers for it. `type()`, `type_handler()` and `sql_type()` keep answering VARCHAR, so nothing above the Field can tell the difference. `Create_tmp_table::add_field()` takes the decision, before the record layout is measured. It may read the column list and nothing else: two temporary tables built from one column list are written from each other's record buffer -- a recursive CTE fills its increment table that way -- and they agree on the layout only while every column decides the same way in both. `insert_all_rows_into_tmp_table()` asserts that the two record lengths match, the same thing `select_union_recursive::send_data()` already asserts for the other direction of that copy. A column that disagrees costs at least the width that put it over the threshold, so the record lengths cannot match if any column does. Not every `Field_varstring` can be moved, and the decision cannot be taken from `type()`. `Field_vector` reports `MYSQL_TYPE_VARCHAR`, because `Type_handler_vector` derives from `Type_handler_varchar` and does not override `field_type()`, while its `reset()` writes the whole declared width into the length prefix and the copy functions `get_copy_func()` returns address the value at a fixed offset in the record. A `VECTOR(10)` is over the threshold, so a decision taken on type and width alone moved one: a row storing no value then advertised its full declared width in the prefix while the slot held only a pointer, and the engine read the value through it. `can_store_data_out_of_line()` asks the field instead and every `Field_varstring` subclass answers for itself, a compressed VARCHAR saying no for the same reason. Only the SQL layer asks, because only it rewrites a field's record layout; the engine route moves the bytes itself and leaves `record[0]` as the SQL layer laid it out, so a field that cannot represent the moved layout is not affected by it. `Field::data_is_out_of_line()` is the question the rest of the server means where it asks `flags & BLOB_FLAG` today, and the two are not the same question. BLOB_FLAG says the column is declared as a blob, and `sql_select.cc` asserts that it agrees with `type()`, so it cannot be set on anything reported as a VARCHAR. Where the payload sits is a separate property, and a promoted VARCHAR now has it too. The sites that meant the latter are switched over: the `blob_field[]` array, join buffer sizing and its cache fields, the wholesale copy and free of out-of-line values, `Item_copy_string` substitution, `Cached_item` selection, and the key segment setup for both temporary table engines. Where a caller instead means "declared as a blob" and was reading `s->blob_fields`, which conflates the two, `TABLE::has_unbounded_blob_field()` answers the question it meant: duplicate removal by hash and the subquery expression cache both need a bounded width rather than an inline one, and a wide VARCHAR has it. Reaching an out-of-line payload becomes a set of `Field` virtuals rather than a cast. `out_of_line_data()` and `out_of_line_length()` read the length and pointer pair out of any image of the field's record slot, and `set_out_of_line_image()` writes such a pair back, pointing it at the payload where it already lies. With `copy()` and `free()` also virtual, the join buffer, the window function row remapper and the `GROUP_CONCAT` cut check no longer cast to `Field_blob` to reach a field that is not one. Keys are where the two questions diverge most. A declared blob has no maximum width, so a unique index over one has to be a hash, and its key is marked `HA_BLOB_PART_KEY`, which stops it being a key at all once the table converts to Aria. A promoted VARCHAR is still as wide as it was declared, so it keeps an ordinary bounded key part: a VARTEXT of that width, whose `HA_BLOB_PART` tells the engine to follow the pointer. It has to keep one, because the optimizer planned a lookup on it from the declared type. `Create_tmp_table` therefore tracks whether any distinct column is a declared blob separately from the columns that are merely out of line, and only the former reaches `HA_UNIQUE_HASH`. Carrying both flags also decides `key_restore()`, where the bounded question has to be asked first: it is `HA_VAR_LENGTH_PART` that says how to put the value back, and only a column declared as a blob may be cast to `Field_blob`. A one-byte-prefix VARCHAR key segment used to have `HA_BLOB_PART` stripped in `heap_create()` unconditionally, a real blob always entering as VARTEXT4 or VARBINARY4 and the flag therefore only ever being a stray one. A promoted VARCHAR sets it deliberately and keeps its own one or two byte prefix, so the strip is narrowed to the segments with no descriptor behind them rather than dropped for an assertion: `key_part_flag` reaches the engine from a `.frm` byte that nothing masks, and the key code reads the record as a pointer wherever the flag is set. A value stored through a promoted field needs somewhere to live. `Field_varstring` gains the buffer `Field_blob` has, with the same rule of one per field, and `copy()` gives the record a set of bytes it owns when the value has to outlive the buffer it was read from. `store()` takes the source somewhere else first when it lies inside that buffer, the way `Field_blob::store()` does, so that `UPDATE t SET c = c` does not read a buffer it is writing. Where many rows are alive at once and sorted afterwards, as they are for `GROUP_CONCAT` with `ORDER BY` or `DISTINCT`, the value goes to the table's `Blob_mem_storage` instead, so each row has bytes of its own. It is stored whole there: a blob is cut to `group_concat_max_len` on the way in because it has no declared width, and a VARCHAR has one that the copy has already applied. `unpack()` points the slot at the row being read, the way `Field_blob::unpack()` does, and `max_packed_col_length()` answers from the declared width, since its callers ask with `pack_length()` and that now describes the pointer. The engine reads a promoted column exactly as it reads a declared blob, both being a length prefix and a pointer, so `heap_prepare_hp_create_info()` builds one descriptor for either and does not promote again what the SQL layer has already moved. Two of the heap key functions capped a VARCHAR segment at the segment's declared width; for an out-of-line column that width describes the descriptor in the record rather than the value, so the value is used whole, as the blob segments already are. Two places in the engine still counted records as though one held a row. `heap_info()` reports free space as the free record count times the record stride rather than times the SQL row width, which is a byte count only while nothing is stored out of line: a `VARCHAR(3000)` table reported more free space than it had ever allocated. And `hp_rectest()`, which answers whether the record a caller read is still what the table holds, compares each out-of-line column through its length and then its data. A promoted column's payload is not in the stored record to be compared at all, and the pointer beside a declared blob's is the continuation chain rather than wherever the read handed the value out, so comparing the two records byte for byte both missed a change and reported one that had not happened. It reaches that data through `hp_materialize_one_blob()`, the call the key code already makes, which hands back a pointer into the chain where the run layout has the bytes contiguous and copies only where it does not. Four tests observed the old storage layout. `main.gconcat_distinct_walk_fail` starved the duplicate filter with 600 rows of a `VARCHAR(100)`. Out of line the filter holds a pointer per value rather than the value itself, so it takes more distinct values to fill it. The row count and the number of distinct values go up, which is what fills the filter either way. `heap.count_distinct_blob_convert` described its last case as having no blob argument. Its `VARCHAR(64)` is now stored out of line and takes the same path a declared blob does, so that case is relabelled and a narrow column carrying the same values is added beside it as the control the old case used to be. `heap.tmp_table_convert_dedup` needs the write that finds the table full to be a duplicate of a row the conversion has already copied. Its two wide columns held values, and a row storing an out-of-line value costs a record slot and a place for the value, so the overflowing write was as likely to be the store as the duplicate that follows it. The wide columns are left empty, which is all they are needed for, and a narrow column carries what makes the rows distinct. A third run puts values back into them and asserts only what does not depend on where the overflow lands: that the conversion happens, and that deduplication over columns stored out of line still returns each distinct row once. Without it the storage layout never meets the conversion path. All three runs read the conversion counter through `heap/count_distinct_converted.inc`, which the suite already had. `heap.blob_update_overflow` records different counters for the same overflow. The aggregate's own temporary tables are unchanged. What moved is the `information_schema.session_status` query the test reads the counters with: `VARIABLE_VALUE` is a `varchar(2048)`, 6144 bytes per row in utf8mb3, and under the shrunken ceiling the test sets it used to spill on its own. Six tests are added, each written against the question it answers rather than against the fix. `heap.promotion_layout_agreement` reads one recursive CTE three ways, one of them alongside a fulltext match, which is what can build the two tables of a recursion with different options. `heap.promotion_keeps_optimizations` measures the rows read by `SELECT DISTINCT` and the subquery cache hits for a narrow VARCHAR, a wide one and a declared TEXT. A wide column that lost an optimization to the BLOB_FLAG question reads the declared column's row count. `heap.group_concat_cut_reporting` pins which row each of the three reports as cut, a wide VARCHAR having a declared width to cut against where a TEXT has none. `heap.promotion_data_free` asserts that a table never reports more free space than it has allocated, which holds whatever the rows look like and does not pin a byte count. `heap.promotion_vector_column` reads a `VECTOR` column through both routes, the internal temporary table where the SQL layer decides and a user `ENGINE=MEMORY` table where the engine does. The fatal row is the one storing no value, so each case includes one, and the first reads `Created_tmp_tables` and `Created_tmp_disk_tables` to show that the temporary table it needs was built and stayed in memory rather than passing on a query that materialized nothing. `hp_test_rectest-t` is the only heap unit test that leaves the read check on, the server turning it off for every table it opens. It runs each case at both a single-record and a multi-run chain, because which one a read hands out decides whether the record buffer's pointer happens to equal the stored one, and a case run at only one of them cannot tell a right answer from that coincidence. `main.information_schema` and `main.log_slow_innodb` each asked for an order that does not determine one, and were stable only by accident. Keeping their temporary table in `MEMORY` where it previously converted to `Aria` takes the accident away: `filesort` tie-breaks equal sort keys on the rowid, and a `MEMORY` table's rowid is the address the record was allocated at, so tied rows do not come back in the same order from one server start to the next. Both tests are given the tie-break they were missing rather than re-recorded -- a `GROUP_CONCAT` ordered by the columns it concatenates, which is what the bug that case covers is about, and a `LIMIT` window ordered by the grouping column as well as by the count that is equal across every group. Neither is what the case checks, so pinning it costs the tests nothing.
arcivanov
force-pushed
the
MDEV-40032-v2
branch
from
September 11, 2026 03:13
f7c8e95 to
aeeef63
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft, pushed for CI. Not proposed for review yet.
Transparent VARCHAR to BLOB promotion for HEAP tables. A VARCHAR whose declared width exceeds
HEAP_CONVERT_IF_BIGGER_TO_BLOBhas its payload stored outside the row, while the SQL layer continues to see an unchanged VARCHAR:type(),type_handler(),sql_type(), the metadata sent to the client, and VARCHAR comparison and key semantics are all unaffected. This applies to userENGINE=MEMORYtables as well as to internal temporary tables.Heap records are fixed width, so an inline
VARCHAR(N)reserves its full declared width in every row whether the row uses it or not, and N counts characters, so that width is between N and 4N bytes depending on the character set. Stored out of line the row pays a length prefix, a chain pointer, and only the bytes actually present.Two commits, on
bb-blob-main-montyat5bfa1a2cdbe.1.
MDEV-40032 Promote wide VARCHAR to BLOB inside the HEAP engineThe engine stores the column out of line itself, in the blob continuation records MDEV-38975 already built.
record[0]keeps its full declared width, so the engine compacts the record on the way in and expands it again on the way out.HP_BLOB_DESCcarries the stored-side geometry and apromotedflag,HP_SHAREgains astored_reclengthand the byte ranges that are identical between the two layouts, andhp_pack_record()/hp_unpack_record()replace the whole-record memcpys. The three key functions that read a stored record use a parallel keyseg array. With nothing promoted every new pointer aliases the old one and the code path is what it was.Not every column reporting
MYSQL_TYPE_VARCHARhas a width to reclaim. AVECTORreports it too, becauseType_handler_vectorderives fromType_handler_varcharwithout overridingfield_type(), and aVECTOR(10)is 40 bytes, already over the threshold. But aVECTORholds one length and no other --store()rejects anything that is not exactly the declared width -- andreset()writes that width into the length prefix even of a row that stored nothing, so storing one out of line ships every byte into a continuation run and pays that run's per-record overhead on top of what the column already cost inline. Under a 1MB ceiling aVECTOR(16383)table holds 30 rows inline against 17 out of line. The decision therefore asksField::can_store_data_out_of_line(), which aFieldanswers no to unless it says otherwise, rather than asking the type. A compressed VARCHAR says no for its own reason: its header and compressed bytes sit where the value would be, and it addresses them directly.Two changes independent of promotion are folded into this commit:
max_recordswas fed a row count and tested against a record counter, which a row using many continuation records exhausts long before its memory budget, so such a table filled at a fraction of its budget. Rows are now limited by the newHP_SHARE::max_rows, tested inheap_write()where rows are counted --share->recordshad been incremented and compared against nothing, so HEAP had no row limit at all. Bytes are limited by the existingmax_table_sizetest inhp_alloc_from_tail(), andmax_recordssurvives only asHP_CREATE_INFO's block-sizing estimate. That changes what a table accepts, whichheap/blob.resultrecords.bzeroof the whole record beats the strided loop by 1.3x to 4.7x, measured on Zen 4, Broadwell and Denverton; above it the two cross over.Also here:
heap_info()reports the ceilingdata_lengthcounts toward rather than the expected record count timesreclength, that product being a byte ceiling only while one record holds one row; andha_heap::scan_time()prices the free records a scan steps over at one step per free list entry rather than one per free record, sinceheap_scan()skips a coalesced block in a single step.2.
Make VARCHAR promotion zero-copyPromoting inside the engine is not zero-copy. Promote the field instead and the payload is not copied at all: a read is handed the address of the engine's own bytes, and a record copied wholesale copies that address rather than the value behind it.
Field_varstringgains apromotedflag that changes only where the payload lives. The record slot becomes the column's own length prefix followed by a pointer, which is the shape a blob already has, andpack_length()answers for it.Create_tmp_table::add_field()takes the decision, before the record layout is measured, and it may read the column list and nothing else: two temporary tables built from one column list are written from each other's record buffer -- a recursive CTE fills its increment table that way -- and they agree on the layout only while every column decides the same way in both.Both routes ask the one question the first commit introduced,
Field::can_store_data_out_of_line(), whichheap_wants_out_of_line()now carries for both. Here it is stricter than a matter of what moving a column costs. This route rewrites the field's record layout, so a field whose accessors address the payload at a fixed offset in the record would be handed a layout it cannot read: aVECTORrow storing no value advertises its declared width in the length prefix while the slot holds only a pointer, and the value is read through it.Field::data_is_out_of_line()is the question the rest of the server means where it asksflags & BLOB_FLAGtoday, and the two are not the same question: BLOB_FLAG says the column is declared as a blob, while where the payload sits is a separate property that a promoted VARCHAR now has too. The sites that meant the latter are switched over -- theblob_field[]array, join buffer sizing, the wholesale copy and free of out-of-line values,Item_copy_stringsubstitution,Cached_itemselection, and the key segment setup for both temporary table engines. Where a caller instead means "declared as a blob" and was readings->blob_fields,TABLE::has_unbounded_blob_field()answers the question it meant.Keys are where the two diverge most. A declared blob has no maximum width, so a unique index over one has to be a hash, and its key is marked
HA_BLOB_PART_KEY, which stops it being a key at all once the table converts to Aria. A promoted VARCHAR is still as wide as it was declared, so it keeps an ordinary bounded key part -- it has to, because the optimizer planned a lookup on it from the declared type.Create_tmp_tabletherefore tracks whether any distinct column is a declared blob separately from the columns that are merely out of line.Reaching an out-of-line payload becomes a set of
Fieldvirtuals rather than a cast toField_blob, and a value stored through a promoted field gets somewhere to live:Field_varstringgains the bufferField_blobhas, with the same rule of one per field, and where many rows are alive at once and sorted afterwards the value goes to the table'sBlob_mem_storageinstead.Tests
Nine tests are added. In the first commit:
heap.promotion_transparent(which opens by proving promotion is in effect, since the assertions would pass just as well if nothing were promoted),heap.row_limit,heap.max_data_length, andheap.promotion_vector_column, which closes on the capacity that separates a column left inline from one stored out of line, with a VARBINARY of the same declared width beside it as the control that gives that reading its meaning. In the second:heap.promotion_layout_agreement,heap.promotion_keeps_optimizations,heap.group_concat_cut_reporting,heap.promotion_data_free, the SQL layer's half ofheap.promotion_vector_column, and thehp_test_rectest-tunit test.Six existing tests observed the old storage layout and are adjusted:
versioning.partition,perfschema.memory_table_io,main.gconcat_distinct_walk_fail,heap.count_distinct_blob_convert,heap.tmp_table_convert_dedupandheap.blob_update_overflow.main.information_schemaandmain.log_slow_innodbeach asked for an order that does not determine one and were stable only by accident; both are given the tie-break they were missing rather than re-recorded.Status
heap,perfschema,versioningandmain: all 1975 tests successful. Heap unit tests 11/11, and the first commit builds and passes its 10 unit tests standalone.