Repository navigation
Conversation
Contributor
Author
|
Initial independent self-review completed (one round).
|
Caideyipi
marked this pull request as ready for review
November 13, 2026 04:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Aligned BatchData reads without record filters box primitive values only to check whether a row is entirely null. Tablet stream serialization also creates temporary byte arrays for each numeric value. This change reduces those allocations while preserving filtering, deletion handling, and the Tablet wire format.
TsPrimitiveTypepresence directly when no record filters are installed. For example, tree-model rows[42, null]and[null, null]still produce only the first row; table-model reads still retain both rows.ReadWriteIOUtils.writeoverloads forDataOutputStreamthat write primitive values and Binary lengths directly.Tablet.serialize()write to a precisely sized ByteBuffer and addserialize(ByteBuffer)for writing at the caller's current position. For example, a caller can reserve an 8-byte prefix and append a Tablet without an intermediate serialized buffer:The new overload writes the same big-endian bytes as
serialize(DataOutputStream)and preserves the caller's byte order. Float/double array buffer serialization canonicalizes NaNs to match the stream format.Performance measurements from the earlier snapshot-base experiment (baseline
8f665d113dbb5257b901c528c31c8bc24da2a018, before rebasing ontodevelop):Tablet.serialize(DataOutputStream)Tablet.serialize()BatchData-Xms512m -Xmx512m -XX:+UseParallelGC -XX:ActiveProcessorCount=2.ThreadMXBean.getThreadAllocatedBytes.8f7c77d267d547928aad66c1357432098c682cd32af413e058beb74d2f98f2fb.These earlier microbenchmarks measure the listed operations. Performance at the final rebased PR head and end-to-end Pipe throughput still require separate measurements.
Validation on Linux x86_64 with Java 17 and Maven 3.9.12: clean compilation and all 110 selected tests passed (zero failures, errors, or skipped tests).
mvn -o clean test -P with-java -pl java/tsfile -am \ -Dtest=TabletTest,ReadWriteIOUtilsTest,AlignedPageReaderPushDownTest,TypeTest,PageReaderTest \ -Dsurefire.failIfNoSpecifiedTests=falseVerification uses an isolated offline dependency repository with build scan extensions disabled in the verification source copy. Tests cover stream/buffer byte equivalence, supported data types, nulls, Unicode schemas and properties, NaNs, heap/direct/sliced buffers, nonzero positions, byte-order restoration, filters, deletions, and tree/table all-null row behavior.