GPU-accelerated UI toolkit (Vulkan)
git clone https://git.lucas.co/cce-ui.git
perf(vk): a streaming path for images that change every frame
The image registry was built for callers that upload a picture once and draw
it forever: a decoded PNG, a rasterized SVG, a thumbnail. cce-browser is not
that caller — its whole page is a readback of what the engine just painted,
so it uploaded a new full-window image every frame and freed last frame's.
Each frame therefore created a VkImage, allocated its memory, allocated and
wrote a descriptor set, made and destroyed a staging buffer, and — the
expensive part — destroyed the previous image behind a device_wait_idle,
which waits for every frame still in flight.
update_pixels is the streaming path: same id, same image, same descriptor,
contents replaced in place. Nothing is allocated and nothing is destroyed, so
none of that per-frame cost remains. A size or format change falls back to a
fresh image under the same id, which is what a window resize does, and an id
that does not exist yet is treated as an upload so callers need not sequence
the two.
Three supporting changes:
- PixelFormat, so a caller whose source is already BGRA (Wayland's and
WebKit's usual order) says so instead of swizzling on the CPU. The sampler
reads either order at no cost; that swizzle measured 7.4 ms per frame at
3840x2400. upload_rgba is unchanged, now a wrapper.
- One shared staging buffer instead of one per upload, grown to the largest
need. Uploads are serialized (each waits for its own copy), so one serves
them all. It is released after ~2s idle above 1 MB, so a one-shot caller
that uploads a 96 MB photograph does not leave 96 MB mapped for the life of
the process, while a streaming caller touches it every frame and keeps it.
- recycle_buffer, handing a finished pixel buffer back so a streaming caller
refills one instead of allocating a frame-sized Vec per frame: 7.4 ms
against 2.9 ms at 3840x2400, nearly all of it zeroing and page faults
rather than copying.
The copy is still its own submission followed by queue_wait_idle, and that
wait is load-bearing: the image may still be sampled by the previous frame,
and two submissions on one queue are not otherwise ordered. Lifting it needs
a second buffer per streaming image and the copy recorded into the frame's
own command buffer — a later step, noted at the call site.
Existing callers are untouched: upload_rgba keeps its signature and its
one-shot behaviour, and no other crate needs a rebuild to keep working.
Verified through cce-browser in a headless shadow at scale 2: exact colours
on a known-colour page (255,0,0 / 0,255,0 / 0,0,255 / 255,200,0 read back
from the screenshot), correct rendering across a floating/fullscreen resize
round trip which exercises the recreate path, clean scrolling, and no
validation errors. cce-ui's own suite passes (339, single-threaded; the
parallel run has a pre-existing flake in a group-layout test, present on a
clean tree 4 runs in 5, filed separately).
Co-Authored-By: Claude Opus 5 <[email protected]>
src/vk/image.rs | 389 +++++++++++++++++++++++++++++++++++++++++++++++++++-----
src/vk/mod.rs | 4 +-
2 files changed, 358 insertions(+), 35 deletions(-)
diff --git a/src/vk/image.rs b/src/vk/image.rs
index c4bf01c..97a2463 100644
--- a/src/vk/image.rs
+++ b/src/vk/image.rs
@@ -6,7 +6,19 @@
//! (the engine runner owns the frame): [`upload_rgba`] queues pixels from any
//! code and returns a stable id usable immediately in draws; the renderer
//! drains the queue at the next frame. [`free_image`] queues destruction the
-//! same way. Draw ordering comes from [`super::Frame2D::images`]: each
+//! same way.
+//!
+//! **Two shapes of caller.** Most upload an image once and draw it for the
+//! rest of the process: a decoded PNG, a rasterized SVG, a thumbnail. One
+//! uploads a *new* image every frame — cce-browser, whose whole page is a
+//! readback of what the engine just painted. The one-shot path allocated a
+//! fresh `VkImage` per upload and freed the old one behind a
+//! `device_wait_idle`, which for a streaming caller meant an allocation, a
+//! descriptor set and a full device stall per frame. [`update_pixels`] is the
+//! streaming path: same id, same image, same descriptor, contents replaced in
+//! place. [`recycle_buffer`] closes the loop on the CPU side by handing back
+//! the pixel buffer the renderer has finished with, so a streaming caller
+//! refills one buffer instead of allocating a frame-sized `Vec` per frame. Draw ordering comes from [`super::Frame2D::images`]: each
//! [`ImageQuad`] carries the vertex index it sorts before.
use std::collections::HashMap;
@@ -34,23 +46,106 @@ pub struct ImageQuad {
pub clip: Option<(u32, u32, u32, u32)>,
}
+/// Byte order of the pixels handed over.
+///
+/// Both are sRGB-encoded 8-bit-per-channel; the difference is only which
+/// channel comes first in memory, and the hardware handles it on sample. A
+/// caller whose source is already BGRA (Wayland's and WebKit's usual order)
+/// should say so rather than swizzle on the CPU: at a fullscreen 3840x2400
+/// that swizzle measured 7.4 ms per frame, which is most of a frame budget
+/// spent rearranging bytes the sampler can read either way.
+#[derive(Clone, Copy, PartialEq, Eq, Debug)]
+pub enum PixelFormat {
+ Rgba,
+ Bgra,
+}
+
+impl PixelFormat {
+ fn vk(self) -> vk::Format {
+ // SRGB, not UNORM — see the note in `upload`.
+ match self {
+ Self::Rgba => vk::Format::R8G8B8A8_SRGB,
+ Self::Bgra => vk::Format::B8G8R8A8_SRGB,
+ }
+ }
+}
+
enum Pending {
- Upload { id: u32, pixels: Vec<u8>, width: u32, height: u32 },
+ Upload { id: u32, pixels: Vec<u8>, width: u32, height: u32, format: PixelFormat },
+ /// Replace the contents of an image that already exists, keeping its
+ /// id, its `VkImage` and its descriptor set.
+ Update { id: u32, pixels: Vec<u8>, width: u32, height: u32, format: PixelFormat },
Free { id: u32 },
}
static PENDING: Mutex<Vec<Pending>> = Mutex::new(Vec::new());
static NEXT_ID: AtomicU32 = AtomicU32::new(1);
+/// Pixel buffers the renderer has finished with, waiting to be refilled.
+/// Bounded: a streaming caller needs one or two in flight, and holding more
+/// frame-sized buffers than that is just memory.
+static RECYCLED: Mutex<Vec<Vec<u8>>> = Mutex::new(Vec::new());
+const MAX_RECYCLED: usize = 3;
/// Queue an RGBA8 image for upload; the id is usable in [`ImageQuad`]s right
/// away (draws before the upload lands are skipped, not errors).
pub fn upload_rgba(pixels: Vec<u8>, width: u32, height: u32) -> u32 {
- assert_eq!(pixels.len(), (width * height * 4) as usize, "RGBA8 size mismatch");
+ upload_pixels(pixels, width, height, PixelFormat::Rgba)
+}
+
+/// Queue an image whose bytes are in `format`. [`upload_rgba`] is this with
+/// [`PixelFormat::Rgba`].
+pub fn upload_pixels(pixels: Vec<u8>, width: u32, height: u32, format: PixelFormat) -> u32 {
+ assert_eq!(pixels.len(), (width * height * 4) as usize, "8888 size mismatch");
let id = NEXT_ID.fetch_add(1, Ordering::Relaxed);
- PENDING.lock().unwrap().push(Pending::Upload { id, pixels, width, height });
+ PENDING.lock().unwrap().push(Pending::Upload { id, pixels, width, height, format });
id
}
+/// Replace what `id` holds, keeping the image itself.
+///
+/// For a caller that redraws the same picture over and over — a page, a video
+/// frame, a live preview. Nothing is allocated, no descriptor is rewritten and
+/// no image is destroyed, so none of the per-frame `device_wait_idle` that
+/// freeing one costs. The size or format changing is allowed and simply falls
+/// back to a fresh image under the same id, which is what a window resize
+/// does.
+///
+/// An `id` that does not exist yet is treated as an upload, so a caller can
+/// take an id from [`upload_pixels`] and update it from the next frame on
+/// without sequencing the two.
+pub fn update_pixels(id: u32, pixels: Vec<u8>, width: u32, height: u32, format: PixelFormat) {
+ assert_eq!(pixels.len(), (width * height * 4) as usize, "8888 size mismatch");
+ PENDING.lock().unwrap().push(Pending::Update { id, pixels, width, height, format });
+}
+
+/// A pixel buffer to fill, reusing one the renderer has finished with when
+/// there is one of at least `len` bytes.
+///
+/// The returned buffer is exactly `len` long and its contents are unspecified
+/// — a caller is expected to overwrite every byte, which a full-frame readback
+/// does by definition. Allocating a fresh frame-sized `Vec` instead measured
+/// 7.4 ms against 2.9 ms at 3840x2400: the cost is not the copy, it is the
+/// zeroing and the page faults on newly mapped memory.
+pub fn recycle_buffer(len: usize) -> Vec<u8> {
+ let mut pool = RECYCLED.lock().unwrap();
+ if let Some(index) = pool.iter().position(|b| b.capacity() >= len) {
+ let mut buf = pool.swap_remove(index);
+ buf.clear();
+ buf.resize(len, 0);
+ return buf;
+ }
+ vec![0u8; len]
+}
+
+/// Hand a finished buffer back to the pool.
+fn retire_buffer(mut buf: Vec<u8>) {
+ let mut pool = RECYCLED.lock().unwrap();
+ if pool.len() < MAX_RECYCLED {
+ buf.clear();
+ pool.push(buf);
+ }
+}
+
/// Queue an image's GPU resources for destruction.
pub fn free_image(id: u32) {
PENDING.lock().unwrap().push(Pending::Free { id });
@@ -89,9 +184,21 @@ struct GpuImage {
view: vk::ImageView,
allocation: Option<Allocation>,
descriptor_set: vk::DescriptorSet,
+ /// What the image was created as, so an update can tell "same picture,
+ /// new contents" from "different image under the same id".
+ width: u32,
+ height: u32,
+ format: PixelFormat,
}
const MAX_IMAGES: u32 = 256;
+/// Idle frames before the staging buffer is handed back — about two seconds
+/// at 60 Hz. Long enough that a burst of uploads reuses one buffer, short
+/// enough that a big one-shot upload does not hold its memory.
+const STAGING_IDLE_FRAMES: u32 = 120;
+/// Below this an idle staging buffer is simply kept; releasing and remaking a
+/// small one costs more than it saves.
+const STAGING_KEEP_BYTES: vk::DeviceSize = 1 << 20;
pub(crate) struct ImageStage {
pipeline: vk::Pipeline,
@@ -101,6 +208,19 @@ pub(crate) struct ImageStage {
shader_module: vk::ShaderModule,
sampler: vk::Sampler,
images: HashMap<u32, GpuImage>,
+ /// One host-visible staging buffer, grown to the largest upload and kept.
+ /// Uploads are serialized against each other (each waits for its own copy
+ /// before returning), so one buffer serves them all — and a streaming
+ /// caller stops paying an allocation and a free per frame.
+ ///
+ /// Kept only while it is being used: a one-shot caller that uploads a
+ /// 96 MB photograph should not leave 96 MB of host memory mapped for the
+ /// life of the process, so an idle buffer is released (see
+ /// `STAGING_IDLE_FRAMES`). A streaming caller touches it every frame and
+ /// never reaches that.
+ staging: Option<AllocatedBuffer>,
+ /// Frames since the staging buffer was last used.
+ staging_idle: u32,
/// Per frame in flight: this frame's quad vertices (6 per ImageQuad).
frame_buffers: Vec<AllocatedBuffer>,
}
@@ -287,6 +407,8 @@ impl ImageStage {
shader_module,
sampler,
images: HashMap::new(),
+ staging: None,
+ staging_idle: 0,
frame_buffers,
}
}
@@ -302,29 +424,99 @@ impl ImageStage {
command_pool: vk::CommandPool,
) {
let pending: Vec<Pending> = std::mem::take(&mut *PENDING.lock().unwrap());
+ if pending.is_empty() {
+ self.staging_idle = self.staging_idle.saturating_add(1);
+ if self.staging_idle > STAGING_IDLE_FRAMES {
+ if let Some(mut idle) = self
+ .staging
+ .take_if(|b| b.size > STAGING_KEEP_BYTES)
+ {
+ // Safe without a wait for the same reason `staging_for`
+ // needs none: every copy waits for itself before
+ // returning, so nothing is reading this buffer here.
+ destroy_cpu_buffer(device, allocator, &mut idle);
+ }
+ }
+ return;
+ }
+ self.staging_idle = 0;
for item in pending {
match item {
- Pending::Upload { id, pixels, width, height } => {
- self.upload(device, allocator, queue, command_pool, id, &pixels, width, height);
+ Pending::Upload { id, pixels, width, height, format } => {
+ self.upload(
+ device, allocator, queue, command_pool, id, &pixels, width, height, format,
+ );
+ retire_buffer(pixels);
}
- Pending::Free { id } => {
- if let Some(mut gpu) = self.images.remove(&id) {
- unsafe {
- let _ = device.device_wait_idle();
- device.destroy_image_view(gpu.view, None);
- device.destroy_image(gpu.image, None);
- let _ = device.free_descriptor_sets(
- self.descriptor_pool,
- &[gpu.descriptor_set],
- );
- }
- if let Some(a) = gpu.allocation.take() {
- let _ = allocator.free(a);
- }
+ Pending::Update { id, pixels, width, height, format } => {
+ // Same picture, new contents: copy into the image that is
+ // already there. Anything else about it changing (a window
+ // resize) falls back to building a fresh one under the
+ // same id.
+ let reusable = self.images.get(&id).is_some_and(|gpu| {
+ gpu.width == width && gpu.height == height && gpu.format == format
+ });
+ if reusable {
+ self.write_into(device, allocator, queue, command_pool, id, &pixels);
+ } else {
+ self.destroy_image(device, allocator, id);
+ self.upload(
+ device, allocator, queue, command_pool, id, &pixels, width, height,
+ format,
+ );
}
+ retire_buffer(pixels);
}
+ Pending::Free { id } => self.destroy_image(device, allocator, id),
+ }
+ }
+ }
+
+ /// Tear one image down. Destroying something the GPU may still be reading
+ /// needs the device idle, which is why this is not on the per-frame path
+ /// any more: a streaming caller updates in place and never gets here until
+ /// it is finished with the image for good.
+ fn destroy_image(&mut self, device: &ash::Device, allocator: &mut Allocator, id: u32) {
+ let Some(mut gpu) = self.images.remove(&id) else { return };
+ unsafe {
+ let _ = device.device_wait_idle();
+ device.destroy_image_view(gpu.view, None);
+ device.destroy_image(gpu.image, None);
+ let _ = device.free_descriptor_sets(self.descriptor_pool, &[gpu.descriptor_set]);
+ }
+ if let Some(a) = gpu.allocation.take() {
+ let _ = allocator.free(a);
+ }
+ }
+
+ /// The shared staging buffer, grown if this upload needs more room.
+ fn staging_for(
+ &mut self,
+ device: &ash::Device,
+ allocator: &mut Allocator,
+ bytes: usize,
+ ) -> &mut AllocatedBuffer {
+ let too_small = self
+ .staging
+ .as_ref()
+ .is_none_or(|b| (b.size as usize) < bytes);
+ if too_small {
+ if let Some(mut old) = self.staging.take() {
+ // No wait: every copy submitted from here is followed by
+ // `queue_wait_idle` before its caller returns, so no GPU work
+ // is referencing the old buffer by the time anything asks for
+ // a bigger one.
+ destroy_cpu_buffer(device, allocator, &mut old);
}
+ self.staging = Some(create_cpu_buffer(
+ device,
+ allocator,
+ bytes as vk::DeviceSize,
+ vk::BufferUsageFlags::TRANSFER_SRC,
+ "image-staging",
+ ));
}
+ self.staging.as_mut().expect("staging buffer")
}
#[allow(clippy::too_many_arguments)]
@@ -338,6 +530,7 @@ impl ImageStage {
pixels: &[u8],
width: u32,
height: u32,
+ format: PixelFormat,
) {
if self.images.len() as u32 >= MAX_IMAGES {
log::error!("image registry full ({MAX_IMAGES}); dropping upload {id}");
@@ -356,7 +549,11 @@ impl ImageStage {
// them a second time, lightening every midtone —
// a page's #101010 measured (71,71,71) on screen.
// Decoding on sample makes the round trip exact.
- .format(vk::Format::R8G8B8A8_SRGB)
+ //
+ // Which channel comes first is the caller's business
+ // (`PixelFormat`): the sampler reads either order at
+ // no cost, so a BGRA source never needs a CPU swizzle.
+ .format(format.vk())
.extent(vk::Extent3D { width, height, depth: 1 })
.mip_levels(1)
.array_layers(1)
@@ -381,15 +578,12 @@ impl ImageStage {
.bind_image_memory(image, allocation.memory(), allocation.offset())
.expect("Failed to bind user image memory");
- let mut staging = create_cpu_buffer(
- device,
- allocator,
- pixels.len() as vk::DeviceSize,
- vk::BufferUsageFlags::TRANSFER_SRC,
- "image-staging",
- );
- staging.allocation.as_mut().unwrap().mapped_slice_mut().unwrap()[..pixels.len()]
- .copy_from_slice(pixels);
+ let staging_buffer = {
+ let staging = self.staging_for(device, allocator, pixels.len());
+ staging.allocation.as_mut().unwrap().mapped_slice_mut().unwrap()[..pixels.len()]
+ .copy_from_slice(pixels);
+ staging.buffer
+ };
let range = vk::ImageSubresourceRange::default()
.aspect_mask(vk::ImageAspectFlags::COLOR)
@@ -429,7 +623,7 @@ impl ImageStage {
);
device.cmd_copy_buffer_to_image(
cmd,
- staging.buffer,
+ staging_buffer,
image,
vk::ImageLayout::TRANSFER_DST_OPTIMAL,
&[vk::BufferImageCopy::default()
@@ -466,14 +660,13 @@ impl ImageStage {
.expect("Image upload submit failed");
device.queue_wait_idle(queue).expect("Image upload wait failed");
device.free_command_buffers(command_pool, &cmds);
- destroy_cpu_buffer(device, allocator, &mut staging);
let view = device
.create_image_view(
&vk::ImageViewCreateInfo::default()
.image(image)
.view_type(vk::ImageViewType::TYPE_2D)
- .format(vk::Format::R8G8B8A8_SRGB)
+ .format(format.vk())
.subresource_range(range),
None,
)
@@ -507,7 +700,135 @@ impl ImageStage {
&[],
);
- self.images.insert(id, GpuImage { image, view, allocation: Some(allocation), descriptor_set });
+ self.images.insert(
+ id,
+ GpuImage {
+ image,
+ view,
+ allocation: Some(allocation),
+ descriptor_set,
+ width,
+ height,
+ format,
+ },
+ );
+ }
+ }
+
+ /// Replace an existing image's contents in place.
+ ///
+ /// The whole streaming path. Against `upload` it skips creating an image,
+ /// allocating its memory, allocating and writing a descriptor set, and —
+ /// the expensive one — destroying last frame's image, which needs the
+ /// device idle and so waits for every frame still in flight.
+ ///
+ /// The copy is still its own submission followed by `queue_wait_idle`,
+ /// and that wait is doing real work: the image is one the *previous*
+ /// frame may still be sampling, and two submissions on one queue are not
+ /// ordered against each other by anything weaker. Lifting it means giving
+ /// each streaming image a second buffer to alternate between and
+ /// recording the copy into the frame's own command buffer, which is the
+ /// next step rather than this one.
+ fn write_into(
+ &mut self,
+ device: &ash::Device,
+ allocator: &mut Allocator,
+ queue: vk::Queue,
+ command_pool: vk::CommandPool,
+ id: u32,
+ pixels: &[u8],
+ ) {
+ let Some(&GpuImage { image, width, height, .. }) = self.images.get(&id) else { return };
+ unsafe {
+ let staging_buffer = {
+ let staging = self.staging_for(device, allocator, pixels.len());
+ staging.allocation.as_mut().unwrap().mapped_slice_mut().unwrap()[..pixels.len()]
+ .copy_from_slice(pixels);
+ staging.buffer
+ };
+ let range = vk::ImageSubresourceRange::default()
+ .aspect_mask(vk::ImageAspectFlags::COLOR)
+ .level_count(1)
+ .layer_count(1);
+ let cmd = device
+ .allocate_command_buffers(
+ &vk::CommandBufferAllocateInfo::default()
+ .command_pool(command_pool)
+ .level(vk::CommandBufferLevel::PRIMARY)
+ .command_buffer_count(1),
+ )
+ .expect("Failed to allocate update command buffer")[0];
+ device
+ .begin_command_buffer(
+ cmd,
+ &vk::CommandBufferBeginInfo::default()
+ .flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT),
+ )
+ .unwrap();
+ // Unlike a fresh upload this image holds a picture already, and
+ // it is in the layout the shader reads. Every byte is about to be
+ // overwritten, so its old contents need not be preserved — but
+ // the layout transition still has to be spelled out both ways.
+ device.cmd_pipeline_barrier(
+ cmd,
+ vk::PipelineStageFlags::FRAGMENT_SHADER,
+ vk::PipelineStageFlags::TRANSFER,
+ vk::DependencyFlags::empty(),
+ &[],
+ &[],
+ &[vk::ImageMemoryBarrier::default()
+ .src_access_mask(vk::AccessFlags::SHADER_READ)
+ .dst_access_mask(vk::AccessFlags::TRANSFER_WRITE)
+ .old_layout(vk::ImageLayout::SHADER_READ_ONLY_OPTIMAL)
+ .new_layout(vk::ImageLayout::TRANSFER_DST_OPTIMAL)
+ .src_queue_family_index(vk::QUEUE_FAMILY_IGNORED)
+ .dst_queue_family_index(vk::QUEUE_FAMILY_IGNORED)
+ .image(image)
+ .subresource_range(range)],
+ );
+ device.cmd_copy_buffer_to_image(
+ cmd,
+ staging_buffer,
+ image,
+ vk::ImageLayout::TRANSFER_DST_OPTIMAL,
+ &[vk::BufferImageCopy::default()
+ .buffer_row_length(width)
+ .buffer_image_height(height)
+ .image_subresource(
+ vk::ImageSubresourceLayers::default()
+ .aspect_mask(vk::ImageAspectFlags::COLOR)
+ .layer_count(1),
+ )
+ .image_extent(vk::Extent3D { width, height, depth: 1 })],
+ );
+ device.cmd_pipeline_barrier(
+ cmd,
+ vk::PipelineStageFlags::TRANSFER,
+ vk::PipelineStageFlags::FRAGMENT_SHADER,
+ vk::DependencyFlags::empty(),
+ &[],
+ &[],
+ &[vk::ImageMemoryBarrier::default()
+ .src_access_mask(vk::AccessFlags::TRANSFER_WRITE)
+ .dst_access_mask(vk::AccessFlags::SHADER_READ)
+ .old_layout(vk::ImageLayout::TRANSFER_DST_OPTIMAL)
+ .new_layout(vk::ImageLayout::SHADER_READ_ONLY_OPTIMAL)
+ .src_queue_family_index(vk::QUEUE_FAMILY_IGNORED)
+ .dst_queue_family_index(vk::QUEUE_FAMILY_IGNORED)
+ .image(image)
+ .subresource_range(range)],
+ );
+ device.end_command_buffer(cmd).unwrap();
+ let cmds = [cmd];
+ device
+ .queue_submit(
+ queue,
+ &[vk::SubmitInfo::default().command_buffers(&cmds)],
+ vk::Fence::null(),
+ )
+ .expect("Image update submit failed");
+ device.queue_wait_idle(queue).expect("Image update wait failed");
+ device.free_command_buffers(command_pool, &cmds);
}
}
diff --git a/src/vk/mod.rs b/src/vk/mod.rs
index a40a82f..ddefa51 100644
--- a/src/vk/mod.rs
+++ b/src/vk/mod.rs
@@ -39,7 +39,9 @@ mod scene;
mod text;
pub use core::VkCore;
-pub use image::{free_image, upload_rgba, ImageQuad};
+pub use image::{
+ free_image, recycle_buffer, update_pixels, upload_pixels, upload_rgba, ImageQuad, PixelFormat,
+};
pub use renderer::{Batch2D, Frame2D, PlatePush, VkRenderer, MAX_PLATE_FEATURES};
pub use rt::{RtCamera, RtMaterial, RtOffscreen, RtTriangle};
pub use scene::{MeshId, SceneDraw, Vertex3D};