Repository navigation
feat(🖼️): SkiaImageView and <Canvas static>, presented from one bitmap per image - #4328
YevheniiKotyrlo wants to merge 33 commits into
Conversation
MeasurementsRelease builds on a Galaxy S23 (Android 14, 120 Hz), a Mi MIX 3 (Android 10, 60 Hz), a RedMagic NX809J (Android 16, 144 Hz) and an iPad Pro 12.9" (4th generation, iOS 27, 120 Hz); the specs of the last section also ran on an iPhone 12 Pro Max (iOS 27). The drives below ran on earlier cuts of this branch, the phones on Mounting static content120 views of 40 points mounted and unmounted every 800 ms, each showing one image, or one of 120, through a
Memory, On the iPad, Instruments' Animation Hitches over the mount drive counts 13 hitches in the canvas arm and 2 in the static canvas arm, each diagnosed "3 offscreen passes", a pass per frame rather than per view. The image view arm has 1 hitch. Regular canvases against
|
| Drive | main, round 1 |
main, round 2 |
this branch, round 1 | this branch, round 2 |
|---|---|---|---|---|
| mount, one image | 123 (121–126) | 123 (103–123) | 123 (100–123) | 120 (120–122) |
| mount, 120 images | 122 (100–123) | 122 (102–124) | 104 (101–121) | 101 (100–121) |
| mounted, the parent animating | 3 (0–4) | 3 (0–3) | 0 (0–1) | 0 |
| a new image every 800 ms | 2 (2–4) | 2 (2–3) | 2 (2–3) | 2 (2–4) |
The passes of one round spread over about 20 frames per window in the mount drives, so the rounds agree within that spread. With 120 canvases mounted, EGL mtrack reads 79,464 KB on both builds. Once they unmount, main stays at 79,464 KB, and still reads 78,288 KB through the plain-views arm after them. This branch is back at 25,704 KB, as before they mounted.
Time to first pixel
One view at a time, every frame captured from the commit that mounts it until its pixels show, the median of twelve at 40 / 240 points; the phones on c729429dd:
| Galaxy S23 | Mi MIX 3 | RedMagic | iPad Pro | |
|---|---|---|---|---|
<Canvas> |
40.0 / 36.5 ms | 64.5 / 64.8 ms | 30.5 / 29.4 ms | 52.2 / 85.6 ms |
<Canvas static> |
32.5 / 42.0 ms | 64.8 / 66.3 ms | 33.7 / 36.5 ms | 52.2 / 85.6 ms |
SkiaImageView |
17.6 / 21.1 ms | 32.5 / 34.1 ms | 13.8 / 30.2 ms | 39.5 / 52.6 ms |
SkiaImageView, a 5200 × 5200 image |
116.6 / 142.9 ms | 225.2 / 210.3 ms | 145.2 / 113.5 ms | 77.3 / 110.7 ms |
Every trial's pixels showed within the frames captured. On the iPad both canvases' medians fall on the same frames. On Android a view of the 5200 × 5200 image draws a smaller level of its bitmap, which it waits for (ImageLevels). The bitmap is scaled to the 100 MB Android draws in one bitmap; on the S23 its first pixel came at 105.1 / 71.5 ms before the levels. Those levels were made with a scaled Canvas draw; the head makes them natively, the rows across up to four threads, and on the Mi MIX 3 the 5200 × 5200 image's first pixel reads 216.5 / 248.0 ms that way against 216.9 / 260.3 ms with createScaledBitmap's halving, in one run each. Reading the image back at the level its views draw it at, with those levels made before it is shown, is the follow-up the description names.
A static canvas's frame and an image view's bitmap stay through the app going to the background and back, and through a GPU device loss, on every device.
Images drawn smaller than they are
Android's renderer samples a bitmap without mipmaps. Here a zone plate, 4096 × 4096 pixels of rings that grow finer toward its edge, is drawn into the S23's screen at about a fifth of its size, as captured on 1ee02f26c, whose drawing the head keeps:
No levels (levelFor returning 0) |
With the levels |
|---|---|
 |
 |
Without levels, the rings too fine for the screen alias into false rings across the whole image. With the level the view draws, they average out to grey, as a GPU's mipmaps average them, and the coarse rings stay. The false rings that are left are faint, along the axes.
The ImageView spec reads the same through one-pixel black and white stripes drawn several times smaller, which average to an even grey when smoothed. With Bitmap.setHasMipMap(true) alone, the middle's luminance deviated by 73.25 on all three phones against the 30 the case allows. With the levels it passes on all three, and so do stripes with odd sides, at every level. A level made with createScaledBitmap would scale an odd side by (n / 2) / n, landing its samples on the stripes' centres in the middle; there the odd-sided case reads a mean of 223 against the 200 it allows. The middle 480 × 240 pixels of those screens on the S23:
| Stripes | Before | After, an even grey |
|---|---|---|
| 8,000 × 16 |  |
 |
| 7,999 × 15 |  |
 |
"Before" is setHasMipMap(true) for the even stripes and createScaledBitmap's halving for the odd ones.
Force dark
Force dark (Android 10 and later) darkens what a view draws, leaves a canvas's window as it is, and inverts a bitmap it judges light. On the Mi MIX 3 with night mode on and debug.hwui.force_dark set, a canvas above and a static canvas or an image view below, each drawing white on its left half over a white background:
| Below the canvas | Before | This branch |
|---|---|---|
<Canvas static> |
 |
 |
SkiaImageView |
 |
 |
With nothing opted out, force dark inverts the static canvas's white frame. With the whole image view opted out, the image keeps its colours, but the view's background stays white beside the canvas's darkened one. On this branch the bitmap is drawn through a render node force dark leaves alone, so the frame and the image keep their colours and the backgrounds darken.
The memory the static frames leave
A static canvas leaves a texture with each frame. The texture goes with its frame, and Dawn keeps its memory for reuse until it is asked, as for any texture Skia frees. On 57c74f385, after 1,000 mount cycles per arm on the S23, Dawn's allocator holds (allocated / used), as held and once asked to give back what it caches (dawn::native::ReduceMemoryUsage):
| Arm | As held | Once asked |
|---|---|---|
| plain views | 48.6 / 24.7 MB | 48.6 / 24.7 MB |
| static canvases | 80.8 / 31.9 MB | 49.5 / 25.5 MB |
| image views of one image | 49.5 / 25.5 MB | 49.5 / 25.5 MB |
| image views of 120 images | 49.5 / 25.6 MB | 49.5 / 25.6 MB |
The textures alive in Dawn come to 17.5 MB after every arm, as after the plain views: the static frames' textures are gone, and what the static arm adds is the freed memory Dawn keeps for reuse. EGL memory stays at 40.3 MB throughout. GL mtrack, which counts the driver's memory too, reads 147.3 MB after the static arm against 51.3 MB after the plain views: Dawn giving back its cache takes it to 114.4 MB, and Android's renderer trimming its own to 91.2 MB, against 50.3 MB for the plain views trimmed. The 41 MB left is about the same with #4352's release (93.1 MB, a584e067b), so neither Dawn's account nor the renderer's holds it. #4352 asks Dawn for its cache once Skia is idle; with that release, which this branch carried at ddf204a3e, the arms read (allocated / used):
| Arm | Galaxy S23 | Mi MIX 3 | RedMagic |
|---|---|---|---|
| plain views | 48.6 / 24.7 MB | 31.9 / 23.6 MB | 53.9 / 36.2 MB |
| static canvases | 49.7 / 25.7 MB | 32.5 / 24.2 MB | 57.4 / 39.7 MB |
| image views of 120 images | 49.7 / 25.8 MB | 34.5 / 26.2 MB | 58.2 / 40.5 MB |
Dawn's own reduce, asked after each of those arms, finds nothing left to give back. On the iPad, Dawn on Metal keeps no suballocator account; its textures and buffers come to 29.0 MB after the static canvases against 26.3 MB after plain views, and its reduce finds nothing either.
Web
Chromium, the 120 cells of the example's image views page, six passes with the arms rotated through three places. An image view of a GPU drawing misses 0 (0–6) frames per window, a static canvas 2 (1–11) and a canvas 2 (1–50). Of a decoded 5214 × 3222 photo, each first view reads the pixels on the page thread and the browser encodes them asynchronously: 90 (11–107) missed frames per window, with 61 long tasks of 53–198 ms.
At the head
98fb335d7 is e112a4973 with main (3184116c9), #4346's head (3748cf8d0) and #4357's head (496100885) merged in. main makes readbacks asynchronous (#4368): readPixels() and the encodes return promises, and DawnContext follows each readback until its callback runs. The presenter's readback into its bitmap's SkImageInfo is followed like any other, and its rows are still copied inside the callback, where a device loss cannot free them. On the web, the image view awaits the pixels before the browser encodes them, as the e2e specs' pixel readers now do. #4357's head changes the native files as the 5c7a8b359 this branch carried does; its cases now mount the harness's canvas again after them, and wait 5 s before each loss they force, since DawnContext counts a device lost within 5 s of being made as a replacement that did not take and puts the next replacement off, 1 s at first and twice as long for each such loss in a row. Each run below is one pass of local Release builds:
- The S23, on
98fb335d7.StaticCanvasandGpu, run in both orders, pass 11 of 12 in each order, fix(🧠): destroy a lost GPU device when it is replaced #4357's two cases among them, andImageViewandClippedSubviewspass 17 of 18, the skipped cases being the force-dark ones;GraphiteView, whose spec and code the merges leave as they were, did not run again. Every lossloseGpuDeviceand fix(🧠): destroy a lost GPU device when it is replaced #4357's cases forced was replaced within 0.07 s;Gpu's own first losses, each within 5 s of its device being made, were replaced after 1 and 2 s, as that rule has it, where one112a4973they chained up to 8 s. No logcat holdsVK_ERROR_NATIVE_WINDOW_IN_USE_KHR. - CI's jobs pass on
98fb335d7: cpplint, lint, tsc, test, test-cpp, test-web, build-library and the JVM unit tests. Run locally too, clang-format 21 is clean on the whole ofRNDawnContext.h, the one native file whose lines the merges resolved. The macOS job, the SwiftPM build and the iPhone last ran one112a4973, below. - The web view's spec, whose test doubles of
readPixels()now return promises asmain's does, fails 8 of its 10 cases on the merge ofmainwithout the view's change, each image warning that its pixels could not be read, and passes 10 of 10 with it.
Before that, e112a4973 was 197e6cf1b with #4357's head (5c7a8b359) merged in, then this branch's commits on it: an include the merge took out, a window's surface and the generation of its device kept as one value in Window, loseGpuDevice watching until a replacement took, the static canvas case drawing before its first device loss, a presented image kept in Display P3 on Apple whatever the main screen, and a comment in setColorSpace:. DawnContext::recover lets go of every window's surface on a lost device, then destroys the device as it replaces it, so retireUnheldDevice, with which this branch destroyed a replaced device no window held, is gone; the static presenter holds no surface, so recover has nothing of it to let go of, and it takes the device in use at its next frame. Each run below is one pass of local Release builds:
- The S23 and an iPhone 12 Pro Max (iOS 27), on
e112a4973.StaticCanvasandGpu, run in both orders, pass 11 of 12 on each device in each order, fix(🧠): destroy a lost GPU device when it is replaced #4357's two cases among them, andImageView,ClippedSubviewsandGraphiteViewpass 21 of 22 on each, the skipped cases being the force-dark ones, which run only where the device darkens apps.DawnContextcounts a device lost within 5 s of being made as a replacement that did not take, and puts the next replacement off;loseGpuDevicenow watches 5 s after each replacement, so the loss forced after it finds a device that took; in either order, each loss it forced was replaced at once.Gpu's own losses, which do not go throughloseGpuDevice, come sooner and chain, their replacements waiting up to 8 s, within the harness's 30 s timeout. With the 3 sloseGpuDevicewatched before,StaticCanvasthenGpuchained six such losses, the last replacement 32 s away, past that timeout, and fix(🧠): destroy a lost GPU device when it is replaced #4357's cases failed on both devices. No S23 logcat holdsVK_ERROR_NATIVE_WINDOW_IN_USE_KHR, the error Vulkan returns for a surface made over a window the lost device still holds. - CI's jobs pass on
e112a4973: cpplint, lint, tsc, test, test-cpp, test-web, build-library, the JVM unit tests, the macOS job and the SwiftPM build. Run locally too, clang-format 21 is clean on the lines the branch changes; it flags one line ofRNDawnContext.h, line 1122, which ismain's.
Before that, 197e6cf1b was d73ee1b4a with main (d4a36ca4a) and #4346's new head (a47ce50d5) merged in, then the colorSpace case and clang-format's wrapping of two statements. Since #4367 brought colorSpace back, the static presenter renders in the color space the prop asks for, as a canvas's window does. Its runs, one pass of local Release builds each:
- The S23, on
197e6cf1b.StaticCanvas,ClippedSubviews,Gpu,GraphiteViewandImageViewpass, 30 of 32, the 2 force-dark cases skipped with force dark off. There thecolorSpacecase checks that each static canvas reads as the canvas beside it and that the prop changes nothing, Android rendering in sRGB whatever it says. - An iPhone 12 Pro Max (iOS 27), on
c6d9ad0c9, the head but for clang-format's wrapping of two statements.StaticCanvas,ClippedSubviews,GpuandGraphiteViewpass, 22 of 23, the force-dark case skipped. ThecolorSpacecase fills a canvas and a static canvas, each in sRGB and in Display P3, with one runtime shader's output, which Skia does not colour-manage, so the same values look more saturated in Display P3; the e2e snapshot is in sRGB, where the two color spaces read apart. Each static canvas reads within 4 of 255, in every channel, of the canvas in its color space. A build ofc6d9ad0c9whose static presenter takes the display's color space instead, as it did before the merge, reads its static sRGB canvas 37 away from the sRGB canvas and fails the case. - The checks of what the merge touched pass on
197e6cf1b, run locally: tsc, ESLint, the docs' snippets and site, the jest specs of the changed modules, fix(🏞️): log a worker job that throws instead of ending the app #4346'sWorkerTest, cpplint, clang-format, the Android build, the macOS job and the SwiftPM build.
Before main's merge, d73ee1b4a was this branch with #4324's split head, #4349 and #4346 merged in, and the idle trim and the memory release left to #4347 and #4352:
- The S23.
StaticCanvas,ImageView,ClippedSubviewsandGpupass three times, 47 of 49, the 2 force-dark cases skipped with force dark off, through the 9 GPU device losses the specs force each round. The first memory table above ran on57c74f385,d73ee1b4abut for fix(🏞️): log a worker job that throws instead of ending the app #4346's catch and a comment. - CI's jobs pass on
d73ee1b4a: cpplint, lint, tsc, test, test-cpp, test-web, build-library, the JVM unit tests, the macOS job and the SwiftPM build. The example also builds for macOS and an iOS device with1e0a2a29b's sources,d73ee1b4abut for fix(🏞️): log a worker job that throws instead of ending the app #4346's catch, a comment and two lines joined.
Before the split, on the commits 1e5a72ae0 folds:
- The phones. The changed specs pass three times on the S23 and the RedMagic on
1ee02f26c, 64 of 66, the 2 force-dark cases skipped. On the Mi MIX 3,ImageViewandStaticCanvaspass three times onf57d3beac, 29 of 31 with the same 2 skipped. There the static canvases' churn lasts 1.4–1.7 s, with the image view's render node or without it, and the oversized images' readback longer, both past the 1.5 s a capture waits from the mount. Under force dark there, the force-dark cases pass, and they fail with the image view opting nothing out or opted out whole. The background probe passes on all three phones, andImageViewpasses on the RedMagic on1e5a72ae0, 24 of 25 with the force-dark case skipped. - The iPad, on the same iOS code at
b56f2b7bb: the ImageView spec passes 23 of 23, the background probe 2 of 2, and the macOS and SwiftPM builds succeed.
…rview On iOS and macOS, SkiaView destroyed its native view in removeFromSuperview and built a new one in willMoveToSuperview:, while the registry gives a view's properties to its first native view only. A canvas, a SkiaPictureView or a SkiaGraphiteView taken out of its superview while mounted, as a scroll view with removeClippedSubviews does to the rows it scrolls away, came back blank. On Android since wcandillon#4318, a SkiaPictureView re-rendered while scrolled away came back blank too: its redraw ran without a window and kept its request (below). The native view now lives from the view's first superview until the view is recycled, as on Android, where only the window goes. Out of its superview, the view lets go of the window's surface and of the layer behind it; back in one, it makes a new layer, attaches the surface to it and presents its last frame as the layer is made, so the Core Animation commit that carries the layer carries the frame. A first mount attaches at layout, and a frame scheduled meanwhile waits for that attach. The attach's redraw presents the queued recordings, or else the last frame, which a window attached since shows until the next frame lands, on Android too. A recycled view unregisters before it lets go of its native view, and forgets its id; a new display link asks for the display's full rate again. setSkiaView and unregisterAll go. A redraw takes its request before it reads the window, and decides on one read of it (RNSkRedrawRequest): what no window or GPU device can show is dropped, as attaching a window and replacing the device each ask again, and what a window that cannot present misses is owed until it can. A redraw that ran while the view had no window, whose description detach() leaves in place, or no GPU device used to keep the request, which then swallowed every request after it, the attach's and the device replacement's included. The producer frees a slot for each of its own recordings once the view is done with it (RNSkRecordingSlots): the last frame shown again frees none, a recording the queue dropped frees its own, and a batch presented in part frees what it presented (RNSkPresentOrKeep). The e2e screens take a canvas, a picture view and a graphite view through a scroll view that clips its subviews, also while the GPU device is lost and replaced; the screen registry says which are snapshotted once they say their last trip back is done, and such a screen requested again is mounted again. The specs name and count colours through one set of thresholds, Resize's included. Host tests cover the redraw request, wherever the window can present again during it, the slots and the present loop.
…p per image
A drawing that never changes still made a native view, a backing view and a
swapchain. <SkiaImageView image={image} /> shows an SkImage as a platform
bitmap. On iOS and Android the image is read back once, off the JS thread, and
every view of it shares one bitmap, an android.graphics.Bitmap a React Native
view draws, or a CGImage as a layer's contents. Android's renderer samples a
bitmap without mipmaps, so an Android view that shows its image at half its
size or less draws a smaller level of the bitmap, each the two by two mean of
the one before, made once off the main thread for every view that needs it
and uploaded ahead of its first draw (ImageLevels); the view shows nothing
until its level is made, and a level there is no memory for leaves its views
the deepest one made. halvePixels makes a level natively, its rows split
across up to four threads for a large level, a band whose thread cannot start
halved on the calling thread; a level a view took to draw is halved from a
copy, since reading a bitmap natively marks its pixels changed and drops the
texture its views uploaded, and any other is read in place. The platform makes
the bitmap first: a CPU image is scaled into it, and a GPU image's rows are
copied into it inside the readback's callback, where a device loss cannot free
them, the one thing done there under the GPU context's lock. Android's bitmaps are made, published and
discarded through SkiaPresentedImages, a static store the native side calls
directly. On the web the view reads the pixels and the browser encodes them
into a PNG asynchronously, once per image, for an <img>, whose URL stays until
the next one is shown. An image larger than the platform draws in one bitmap
(100 MB on Android) is presented scaled down to fit, and a readback that
fails, or that no GPU can run, is logged. On Android the image view's props
reach it through their @ReactProp setters, ReactViewManager's among them: the
codegen delegate drops overflow, the borders and the border radius. Force dark
(Android 10 and later) would invert a light bitmap where the canvas the image
view stands in for keeps its window's colours, so the view draws its bitmap
through a render node force dark leaves alone, and its background and border
darken as a canvas's do.
<Canvas static>, and the static prop of SkiaGraphiteView,
read when the native view is made, presents its frames through the same
presenter: RNSkView::presentAsImages makes RNSkWorker the view's present
thread, with no frame scheduler, so its frames are presented as they are
queued, and RNSkWindowSurface takes an image presenter in place of a window,
each frame replayed into a texture of the layout size, which
DawnContext::MakeSurfaceFromTexture now wraps for a given recorder and color
space. What a window surface has attached is a std::variant of a window and a
presenter's image frames, each with its own state, which the presents and the
device check visit. The presenter holds no surface, so it takes no part in the
sweep that retires a replaced GPU device; a replaced device no window holds a
surface on is destroyed as it is replaced.
A static canvas leaves a texture with each frame, whose memory Dawn keeps for
reuse until it is asked for it. A render thread that runs its idle task idle
asks, and the context's poller, renamed from the readback poller, asks Dawn
once neither the device nor the GPU context was in use for a second, in rounds
10 ms apart while Dawn says more remains, 20 at most
(RNSkMemoryReleaseSchedule): a view still drawing would allocate it again. A
render thread that ran a job between an idle request and its due time, such as
the last job of a view just dropped, ran the task busy and was never asked
again: RNSkIdleTaskSchedule, which decides when each thread runs it, owes such
a thread an idle run once it ran no job for the delay.
RNSkWindowSurface::isAttached() reads what the last attach or detach left,
without waiting for a present in flight: the platform view asks on every
layout. presentThroughTexture skips a recording no intermediate texture can be
made for rather than throwing, which on a present thread ends the process and
on the worker would lose the frames. On Android a view registers its id once
its prop transaction is applied, so that a static canvas has its present
thread before it is bound. On iOS a SkiaView presents through its window or as
images, a variant each lifecycle method visits, and a static canvas keeps its
native view and its presented frame across a change of superview until the
view is recycled. A job on RNSkWorker that throws anything is logged rather
than ending the process. The docs gain an image view page and the example app
an image views screen.
64e3b59 to
1e5a72a
Compare
|
The documentation in this PR looks compelling. You seem to suggest that main as a memory issue where things don't get released on unmount. Do we need to look at this separately? My current hypothesis is that now that we use a pool of recorder, the gains here might be very marginal but let's have a look at what the data says. Happy to run benchmark on my side too. |
|
Thanks, and sorry: that comment packed far too much into one place. Here is the short version. Device measurement — before / after
This PR —
Each cell is the median of the 9.6 s windows with their range, and of twelve mounts of one view for the first pixel. A window holds six mounts and six unmounts of 120 views of 40 × 40 pt showing one image, while Reanimated turns a square on the UI thread. A missed frame is a 120 Hz frame in which Reanimated's frame callback didn't run, the longest stall is the longest gap between two of those callbacks, and 8.3 ms is one frame. The time to first pixel runs from the commit that mounts the view to the first frame that shows its pixels: a static canvas's comes within a frame of a canvas's, an image view's in about half the time. Device evidence — recorded
Left to right: Your questionsThe recorder pool. Both builds have it: Memory on unmount. Yes, separately. On To run it. The example's Image Views screen in this PR mounts the same 120 views of 40 pt every 800 ms: switch between Canvas, Static canvas and SkiaImageView and watch the square. The numbers come from the same scene with a frame counter, git apply static-content-bench.patch
cd apps/example
npx react-native bundle --entry-file static-content-bench.js --platform android --dev false --minify true \
--bundle-output dist/main.android.jsbundle --assets-dest dist/res
cd android && ./gradlew assembleRelease -PreactNativeArchitectures=arm64-v8a
adb install -r app/build/outputs/apk/release/app-release.apk && adb shell monkey -p com.reactnativeskia.example 1
adb logcat -s ReactNativeJS | grep "BENCH probe arm=mount-shared"In those lines Every deviceThe same scene, UI-thread frames missed per 9.6 s and the longest stall, on this PR's release builds:
The parent animating over the views, a new image every 800 ms, memory, the first pixel on every device, images drawn smaller than they are, force dark and the web are in the earlier comment. SceneEach of the 120 cells, as the example's Image Views screen renders it: const Cell = ({ image, presenter }: { image: SkImage; presenter: Presenter }) => {
switch (presenter) {
case "SkiaImageView":
return <SkiaImageView image={image} style={{ width: 40, height: 40 }} />;
case "Canvas":
case "Static canvas":
return (
<Canvas style={{ width: 40, height: 40 }} static={presenter === "Static canvas"}>
<Image image={image} x={0} y={0} width={40} height={40} fit="fill" />
</Canvas>
);
}
}; |
Eight e2e specs read a snapshot's pixels with a readPixels call of their own, named its colours with their own copy of the same thresholds, or both. setup.tsx now exports readAllPixels, nameColor, readScreenColor and countColors over one classifier, and each spec reads through them. The thresholds, and so the colour each pixel is read as, are unchanged.
…rview On iOS and macOS, SkiaView destroyed its native view in removeFromSuperview and built a new one in willMoveToSuperview:, while the registry gives a view's properties to its first native view only. A canvas or a SkiaGraphiteView taken out of its superview while mounted, as a scroll view with removeClippedSubviews does to the rows it scrolls away, came back blank. On Android since wcandillon#4318, a canvas asked for a redraw while scrolled away came back blank too: the redraw found the window's description, which detach() leaves in place, kept its request at canPresent(), and swallowed the request the next attach made. The native view now lives from the view's first superview until the view is recycled, as on Android, where only the window goes. Out of its superview, the view lets go of the window's surface and of the layer behind it; back in one, it makes a new layer, attaches the surface to it and presents its last frame as the layer is made, so the Core Animation commit that carries the layer carries the frame. A first mount attaches at layout, and a frame scheduled meanwhile waits for that attach. The attach's redraw presents the queued recordings, or else the last frame, which a window attached since shows until the next frame lands, on Android too; redraw does not count that frame among the producer's recordings. A recycled view unregisters before it lets go of its native view, and forgets its id. A redraw takes its request before it reads the window, and decides on one read of it (RNSkRedrawRequest): what no window or GPU device can show is dropped, as attaching a window and replacing the device each ask again, and what a window that cannot present misses is owed until it can. Read the other way round, an attach landing between the read and the clear on Android's present thread lost its redraw, and a redraw during a device loss swallowed the replacement's. The e2e screens take a canvas, a canvas drawing a picture and a graphite view through a scroll view that clips its subviews, also while the GPU device is lost and replaced; the screen registry says which are snapshotted once they say their last trip back is done, and such a screen requested again is mounted again. RedrawRequestTest covers the request.
RNSkWorker ran each job with nothing around it on a detached thread, so an exception a job let out left the thread and std::terminate ended the app. The worker now logs it, with what() for a std::exception, and runs the next job. WorkerTest checks the jobs behind a std::exception and behind anything else. RNSkLog.h, which the worker logs through, includes <cstdarg> and <cstdio> for what it uses, so the test can build it with nothing included before it, and run.sh adds React Native's JSI headers, which it includes.
…ic-canvas-image-view wcandillon#4324 now only keeps a view's native view, and what it shows, out of its superview: its recording slots, colour helpers, display link reset and view registry cleanup became wcandillon#4348, wcandillon#4349, wcandillon#4350 and wcandillon#4351. This branch carries wcandillon#4324 as it stands, and wcandillon#4349, whose helpers the region readers in setup.tsx build on; stopDisplayLink leaves the reset to wcandillon#4350.
…release to their own PRs A worker job that throws is wcandillon#4346's, the idle trim owed after a busy one wcandillon#4347's, and the GPU memory Dawn keeps wcandillon#4352's: this PR drops their files, tests and hunks, renderPool()'s doc with them. cpp-tests/run.sh runs every host test again, as on main.
The worker presents static canvases and makes the presented bitmaps, whose calls into Java can throw: this PR needs wcandillon#4346's catch, so it stacks on wcandillon#4346 as it does on wcandillon#4324 and wcandillon#4349.
A replaced GPU device was destroyed only by the next frame of a window still on it: every window lets go of its surface on the device under its own mutex before the destroy, and that mutex comes before the context lock the replacement holds, so the presenting thread ran the sweep and the destroy before its present. With no window presenting on the device again, as on a screen without canvases, on Android while the app is in the background, or once the last window on it lets go of it first, the device was never destroyed, and Dawn, which runs the deletions it deferred on a lost device once it is destroyed, kept them for the life of the process. The replacement now sweeps the windows itself, before it takes the context lock, then destroys the lost device, which frees its memory for the new one, and makes and publishes the new device: no window makes a swapchain on the new device while the lost one still holds its native window. A window attached while the device was being replaced makes its surface again by its first frame on the new device. Two Gpu spec cases check it through a buffer mapped on the device before its loss, which a loss leaves mapped and Dawn unmaps when it destroys the device: it reads unmapped once Skia.gpu.lost resolves with no window on screen, and with a canvas on the device, which then draws on the new one. The e2e harness can unmount its own canvas until the next drawing for them.
A worker job that threw was logged and dropped, so makeRasterImage()'s promise stayed pending when the decode threw. RNSkWorker::post(make, deliver) runs make inside the catch every job runs in, then hands deliver what it returned, or null when it threw, and makeRasterImage() posts its decode that way: the promise rejects as it does for a decode that returns null. WorkerTest checks that a result is delivered once, and null once when make throws a std::exception or anything else. makeRasterImage()'s doc says it rejects when the readback or the decode fails.
main brings back the colorSpace prop (wcandillon#4367). A static canvas takes it as a canvas does: its image presenter is attached in the color space the prop asks for, and attached again when the prop changes. wcandillon#4349 is in main now, so the branch no longer carries it.
wcandillon#4346 now rejects makeRasterImage()'s promise when its decode throws, through RNSkWorker::post(make, deliver). The branch takes its head as it stands.
…for, as a canvas does
wcandillon#4357 destroys a lost GPU device as DawnContext::recover replaces it, once every window has let go of its surface on it. The branch takes that in place of retireUnheldDevice, which destroyed a replaced device no window held from the loss listener: the windows list and the sweep stay, the sweep lets go of a window's surface, and a static canvas's presenter, which holds none, takes the device in use at its next frame.
RNSkWindowSurface.h declares an std::atomic (_attached), and the merge took wcandillon#4357's removal of <atomic>, which went with its atomic _generation there. The comment the merge pointed at DawnContext::recover in RNSkPresentedImages.h is wrapped again.
Since the sweep runs in DawnContext::recover, the generation of a window's surface is read only under the window's mutex, so it no longer sits beside the attachment for a read without the lock. It moves into Window, as ImageFrames keeps the generation of its frames, and the sweep reads it from a window alone.
loseGpuDevice watched for 3 s after a replacement, so the next loss a test forced through it came about 4.6 s into the new device's life, inside the 5 s within which DawnContext counts a loss as a replacement that did not take. Run back to back, StaticCanvas's losses and Gpu's chained six of those, and the replacement after the sixth was 32 s away, past the harness's 30 s eval timeout. The watch now lasts the 5 s.
The case forced its first device loss before its first drawing, so where it ran after Gpu's cases, which unmount the harness's canvas, that loss found no canvas window on the lost device. It now draws first. It forces two losses rather than four: DawnContext::recover lets go of the windows' surfaces before it publishes a replacement, so the second loss is there to find the surface the first replacement made.
A window's Dawn surface and the generation of the device it was made on are set and let go of together, and the generation means nothing without the surface: after a make that failed, the window kept the current generation beside no surface. They are one optional value now, empty from a device's replacement or a failed make until a frame makes the surface on the device in use.
A presented image kept Display P3 only where the main screen has a wide color gamut (prefersP3ColorSpace). So on a Mac whose main display is sRGB, a static canvas whose colorSpace asks for Display P3 was read back into an sRGB bitmap, where a canvas with the same prop keeps its colors outside sRGB on a Display P3 display. A CGImage carries its color space on any screen, so on Apple platforms a presented image now keeps Display P3 wherever its colors are outside sRGB (canPresentP3Bitmaps); Android's bitmaps stay in sRGB.
Its comment said frames are recorded for the new color space until the window or the presenter is attached again, which holds only for a view that was never attached. It now says when that attach comes.
…b throws The encodes of encodeToBytes() and encodeToBase64(), and readPixels()' CPU read and its GPU read's conversion, run on the worker as jobs that complete their promise from inside, so a job that threw was logged and its promise stayed pending, and on a worklet runtime its tick kept re-arming. They post through RNSkWorker::post(make, deliver), as makeRasterImage()'s decode does: make returns the encoded data or the pixels read, null when there is none, and deliver completes with it, so a job that throws settles as one that fails without throwing: the encodes reject and readPixels() resolves to null.
A replaced GPU device was destroyed only by the next frame of a window still on it: every window lets go of its surface on the device under its own mutex before the destroy, and that mutex comes before the context lock the replacement holds, so the presenting thread ran the sweep and the destroy before its present. With no window presenting on the device again, as on a screen without canvases, on Android while the app is in the background, or once the last window on it lets go of it first, the device was never destroyed, and Dawn, which runs the deletions it deferred on a lost device once it is destroyed, kept them for the life of the process. The replacement now sweeps the windows itself, before it takes the context lock, then destroys the lost device, which frees its memory for the new one, and makes and publishes the new device: no window makes a swapchain on the new device while the lost one still holds its native window. A window attached while the device was being replaced makes its surface again by its first frame on the new device. Two Gpu spec cases check it through a buffer mapped on the device before its loss, which a loss leaves mapped and Dawn unmaps when it destroys the device: it reads unmapped once Skia.gpu.lost resolves with no window on screen, and with a canvas on the device, which then draws on the new one. Each forces its loss once the device in use has lived 5 s, so its replacement is a plain one, and the e2e harness can unmount its own canvas while they run.
main makes readbacks asynchronous (wcandillon#4368): readPixels() and the encodes return promises, and DawnContext follows each readback until its callback runs. The branch's readPixels() into a caller's SkImageInfo, which the image presenter reads its frames back through, takes that tracking. SkiaImageView on the web, the branch's e2e readers in setup.tsx and the specs that read through them, ClippedSubviews' own reader among them, await the pixels. JsiSkImage.h and Image.ts are wcandillon#4346's merge of the same main (7dfd058).
wcandillon#4346 now settles the promises of the encodes and of readPixels() when their job throws, through RNSkWorker::post(make, deliver), on main's asynchronous readbacks. The branch takes its head as it stands.
wcandillon#4357's head, one commit on main, runs its Gpu cases without the harness's canvas and mounts it again after them (ctx.showCanvas()), and lets the device take before each loss it forces. Its DawnContext and window surface changes are those the branch carries since ebd0c9e. The harness hands showCanvas to the specs beside the branch's hooks, and the static case's comment no longer counts on a spec before it leaving the canvas unmounted.
Problem
A drawing that never changes still mounts everything an animated canvas needs: every
<Canvas>and<SkiaGraphiteView>makes a native view, a backing view and a swapchain. Mounting and unmounting 120 small canvases every 800 ms misses 74 frames per 9.6 s window on a Galaxy S23 (120 Hz) and 72 on an iPad Pro (120 Hz), and 120 canvases hold 42–54 MB of EGL memory on Android phones while mounted. That is any screen of icons, badges, baked shadows or thumbnails.Solution
Two entry points over one presenter, a plain platform view that shows a finished image:
<SkiaImageView image={skImage} />shows anSkImagewith no canvas of its own. On iOS and Android the image is read back once, off the JS thread, and its views share one platform bitmap: anandroid.graphics.Bitmapa React Native view draws, or aCGImageas a layer's contents, in Display P3 wherever the image has colours outside sRGB. A GPU image's rows are copied into the bitmap inside the readback's callback, where a device loss cannot free them; a CPU image is scaled in with mipmaps; an image past what Android draws in one bitmap (100 MB) is scaled down to fit. Android's renderer samples a bitmap without mipmaps, so a view drawing its image at half its size or less draws a smaller level, each the two by two mean of the one before, made once off the main thread (ImageLevels,RNSkPixelHalving.h). On Android 10 and later its bitmap is drawn through a render node force dark leaves alone, so the image keeps its colours, as a canvas's drawing does, while its background and border darken. The Android view takes aView's props, the border radius andoverflowamong them, throughReactViewManager.<Canvas static>, andstaticonSkiaGraphiteView, presents the view's frames through the same presenter, in place of the backing view or theCAMetalLayer.RNSkView::presentAsImagesmakesRNSkWorkerits present thread through fix(🤖): present Android canvases off the main thread #4318'ssetPresentThread, andRNSkWindowSurfaceattaches either a window or the presenter's frames, astd::variantthe presents and the device check visit. Like a canvas, it renders in the color spacecolorSpaceasks for (feat(🎨): bring back the colorSpace prop of the Canvas #4367). The presenter holds no surface, so fix(🧠): destroy a lost GPU device when it is replaced #4357'sDawnContext::recoverhas nothing of it to let go of before it destroys a lost device, and its next frame takes the device in use. Each change is read back into a new bitmap, so the docs send an animating drawing to a regular canvas.canvas.toBlob), at most 4096 × 4096 pixels, Safari's largest canvas on iOS before WebKit's d1f63c061e;statichas no effect there. On iOS, which refuses GPU work in the background (Apple), an image view mounted there gets its bitmap once JavaScript makes the image again afterSkia.gpu.lost.With the same drive, static canvases miss a median of 1–2 frames per window on the S23 and 7–8.5 on the iPad, image views 0–1, and neither holds EGL memory beyond a plain view's.
Tests
SkiaImageViewretains and releases its image by id across views and under StrictMode;<Canvas static>hands the native viewstaticPresentation; the web view's encode, scaling and URL lifetime.PresentedImageLedgerTest,PresentedBitmapSizeTestandPixelHalvingTest(the box filter, the same bytes however many threads start).BackingViewKindTest,PresentedImageTestandImageLevelsTest.ImageView(GPU and CPU images, at 5200 × 5200 and 16,400 pixels wide, through a device loss, with rounded corners, drawn smaller with even and odd sides),StaticCanvas(frames through device losses, while 60 other canvases mount and drop, and in eachcolorSpace) and the static row of fix(🍏): keep a view's native view, and what it shows, out of its superview #4324's clipped screens. On a device that darkens apps (FORCE_DARK=true), both compare their image with a canvas's: its colours kept, the background darkened.Verification
98fb335d7(main3184116c9with fix(🏞️): log a worker job that throws instead of ending the app #4346's and fix(🧠): destroy a lost GPU device when it is replaced #4357's new heads), run locally:ci.yml's off-device jobs and clang-format on the PR's lines; in a Release build on the S23 (Android 14),StaticCanvasandGpuin either order, fix(🧠): destroy a lost GPU device when it is replaced #4357's cases among them,ImageViewandClippedSubviewspass. iOS and the macOS and SwiftPM builds did not run on it.e112a4973, beforemain's asynchronous readbacks (feat(🖼️): asynchronous readbacks #4368), which the presenter's readback now uses: the macOS and SwiftPM builds, and those four specs andGraphiteViewon the S23 and an iPhone 12 Pro Max (iOS 27).197e6cf1b, the earlier merge ofmain: tsc, the docs' snippets and site, andWorkerTest. There, thecolorSpacecase fails a static presenter in the display's color space.Earlier heads' runs, and each device's drives, memory and first pixel: the evidence comment.
Open questions
fit, and tell JavaScript when its bitmap arrives?AHardwareBuffer(API 29+) and anIOSurfacedrop the readback and the compositor's upload, as React Native WebGPU'sHardwareBufferViewdoes?Related
Stacked on #4324 (a view out of its superview keeps what it shows), #4346 (a throwing worker job is logged, not fatal, which the presenter's Java calls need) and #4357 (a lost device destroyed as it is replaced, which a screen of static canvases needs), whose commits this PR carries until they merge. #4347 (a busy render thread trimmed once idle) and #4352 (Dawn's kept GPU memory released once Skia is idle) were split out of this PR. #4318 (the present thread) · #4309 (the recorder pool) · #4264 (explicit backing views) · #4291 (device loss) · #4307 (window surfaces across device replacement) · #4221 (one WebGL context). #4350 (
stopDisplayLink), #4266 (SkiaView,Canvas,SkiaGraphiteView) and #4253 (RNSkView,RNSkWindowSurface,FrameScheduler) change what this PR changes; I'll merge whichever lands second.