feat(windows): audio-context on Node-API - #155
Merged
Merged
Conversation
…t keeps its host - destroy() (and reading `lost` on a destroyed device) settled the weak lost promise through its threadsafe function. An unref'd function does not keep Node's loop alive, so an await on `lost` right after could see the loop end first (Node 22, CI). It now settles in place, and destroy() takes the pending promise before canvas-c's own lost callback can. - windows-host: the flush test held only the pointer-wrapped context; the NSCCanvas that owns it could be collected across the awaits, freeing the context under it (transparent reads or an access violation on Node 22).
- crates/audio-context-napi (audiocontext.node): AudioContext / OfflineAudioContext, every node
in index.d.ts, AudioParam automation, AudioBuffer, PeriodicWave and the listener over the
web-audio-api crate, played through WASAPI (cpal). web-audio-api panics on spec violations;
each call turns the panic into a JS error ("InvalidStateError: ...") instead of aborting the
host. Decoding (bytes, base64, files) and offline rendering run as async work; ended and
statechange arrive through weak threadsafe functions.
- index.windows.ts: the public classes over it, sharing common.ts with iOS and Android.
canvas-polyfill's probe now finds AudioContext on Windows. Not yet: MediaElementAudioSourceNode
(canvas-media has no Windows backend); a WaveShaperNode takes one curve.
- build-napi.sh audio-context-napi, `make windows-audio`, the MSBuild copy targets, a CI job
(x64 and arm64; the x64 job runs the Node-API suite offline and on the 'none' sink) and the
PR workflow's artifact placement. THIRD_PARTY_NOTICES.txt for web-audio-api, Symphonia
(MPL-2.0) and cpal.
- demo: an `audio` spec suite on Windows (offline graphs, decoding, realtime state and ended
events on the default device).
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…he device.lost fix
triniwiz
added a commit
that referenced
this pull request
Sep 27, 2026
* feat(windows): D3D12 and ANGLE backends in canvas-core, patched napi-sys
canvas-core gains the two Windows GPU backends the Node-API port needs:
- gpu::d3d: a per-thread Direct3D 12 device + queue shared by every 2D
canvas (Skia Ganesh D3D), with WARP (CANVAS_FORCE_WARP) and debug-layer
(CANVAS_D3D_DEBUG) switches.
- gpu::gl on Windows: a GLContext over a dynamically loaded ANGLE
(libEGL/libGLESv2 next to the binary, or CANVAS_ANGLE_DIR) on D3D11,
with the same API as the Android EGL context plus D3D11 client-texture
surfaces for presenting into a composition swapchain.
Examples d3d_offscreen and angle_offscreen render and read back on both.
Also vendors napi-sys 3.3.2 with a host-module lookup patch (Node-API
hosts that export napi_* from a DLL, e.g. NativeScript Windows' runtime),
and adds a release-napi profile that unwinds panics into JS exceptions.
* fix(windows): build canvas-2d, canvas-webgl and canvas-c for Windows
Platform branches that only knew Apple and Android:
- Pixel readback formats and the default WebGPU surface format now treat
Android as the RGBA exception rather than listing Apple as BGRA, so
Windows (and later Linux) default to BGRA surfaces / RGBA readback.
- The view-based WebGL/2D-GL constructors are Apple-only; Windows
renders GL offscreen on ANGLE and the host presents it.
- Skia's GL interface is loaded through ANGLE's eglGetProcAddress on
Windows instead of the native (WGL) interface.
* feat(canvas-napi): Node-API rewrite on napi-rs 3
canvas-napi becomes the Node-API build of the canvas native module: it
installs the same global.CanvasModule the V8 bindings do, so
packages/canvas can drive it on Node-API hosts (NativeScript Windows
first, later macOS/Linux). The standalone macOS Node/Deno host layer
and the napi-rs 2 sources are removed.
Classes are raw Node-API callbacks (util::class macros) that coerce
arguments like the V8 bindings/WebIDL and dispatch on a wrapped-object
header (util::native) instead of Node-API type tags. napi-rs 3.13
provides registration; util::task (pool + threadsafe functions) and
util::frame (dirty tracking, frame-end flush) replace the platform
run loops and display links the V8 bindings use.
Path2D and the shared types for 2D, images and WebGL are in place; the
remaining classes are ported against PORTING.md.
* feat(canvas-napi): restore the napi-rs bindings, migrate to napi-rs 3
Brings back the existing napi-rs binding code (c2d, WebGL, WebGL2,
WebGPU, images, text) in place of the raw-callback rewrite, migrated
to napi-rs 3.13 and the current canvas-c:
- napi-rs 3 idioms: class arguments by reference, classes returned by
value, Unknown/Object with lifetimes, ThreadsafeFunction parameters,
zero-copy typed arrays; crate::js::ToJs for argument-dependent
returns and AnyArrayBuffer for ArrayBuffers inside Either.
- canvas-c drift: colour spaces, has_current_texture, texture view
usage, pipeline constants, optional depth-stencil fields, wgpu 30
limits, new vertex/texture formats.
- Fixes: lineCap round/butt were swapped; ImageData wrapped a Box with
Arc::from_raw; fillStyle/strokeStyle getters leaked the style;
ImageAsset loaded encoded bytes as raw pixels and vice versa;
createImageBitmap decoded ImageData pixels as an encoded image and
ignored the source rect; drawImage(gpu canvas) panicked; GPU_INSTANCE
was a const (a new wgpu instance per use); getMappedRange buffers
dangled after unmap.
- Apple-only view/layer factories removed; surfaces come from the
desktop host.
The 2D context now matches what packages/canvas calls: int fill rules,
textBaseline and globalCompositeOperation; __toDataURL/__getPointer/
__makeDirty/__startRaf/__stopRaf/__resize/__createPatternWithNative;
drawPaint/drawPoint(s)/drawAtlas/fillOval; drawImage/createPattern take
the native source objects; create2DContext/create2DContextWithPointer
(the latter non-owning). Drawing marks the context dirty for the
desktop frame flush (crate::frame, CanvasModule.__flushAll). Exports are
installed as globalThis.CanvasModule.
* feat(canvas-napi): images, text, DOMMatrix and module functions match packages/canvas
Aligns the rest of the CanvasModule surface with what packages/canvas
calls and the V8 bindings register:
- DOMMatrix: a-f/m11-m44 accessors; translate/scaleNonUniform/rotate/
skewX/skewY (from the matrix passed last, or this) and their *Self
forms; multiplySelf/premultiplySelf take the native matrix or its
wrapper (MatrixArg).
- Path2D.addPath(path, transform?) applies the matrix (self-add copies
first); roundRect radii optional. CanvasPattern.setTransform(DOMMatrix).
- ImageAsset: fromUrlSync/fromUrlCb/fromFileCb/fromBytesCb/
fromEncodedBytesCb, premultiplied fromBytesSync, __addr/__getRef.
Callbacks run on a worker pool and return through threadsafe
functions; byte arguments are read in place and kept alive meanwhile.
- ImageBitmap: fromAsset (shares the image), __addr/__getRef, 0x0 after
close(); options read field by field like the C++ HandleOptions.
- TextEncoder/TextDecoder: lowercase encoding; decode reads any buffer or
view in place; decodeAsync.
- Module functions (module.rs): createImageBitmap in all four argument
forms, readFile, getMime, __addFontFamily, __addFontData, __base64*.
Fixed rather than copied from the C++: the crop rect is applied to
encoded bytes; failed decodes report an error; zero-size crop checks
fire; views read only their own byte window; rotateSelf without a centre
no longer yields NaN; decode keeps embedded NULs.
__test__/images.test.mjs covers the above (30 node tests pass).
* feat(windows): 2D on Direct3D 12, presented in a WinUI SwapChainPanel
- canvas-core gpu::dxgi: CompositionSwapChain, a flip-model BGRA
composition swapchain (2 buffers, frame-latency waitable,
premultiplied or ignored alpha) bound to a panel through WinUI 3's
ISwapChainPanelNative; SetMatrixTransform maps physical pixels back to
the panel's DIPs.
- canvas-2d surface_d3d (feature d3d): canvases draw into a persistent
Skia render target on the thread's shared D3D12 device and
DirectContext; presenting blits it into the current back buffer.
Resizing clears the canvas and resizes the swapchain.
- canvas-c: Engine::D3D, canvas_native_context_create_d3d (null without
a usable device), _attach_swap_chain_panel and
_set_swap_chain_transform; render() presents and resize() goes through
the D3D path.
- canvas-napi host::windows: CanvasModule.NSCCanvas, the native side of
the canvas view. It takes the panel's pointer key (offscreen without
one), owns the context like the iOS view, applies the composition
scale, and falls back to a CPU context when D3D12 is unavailable.
__test__/windows-host.test.mjs renders, reads back and resizes offscreen
on the hardware adapter and on WARP.
* feat(canvas-napi): frame scheduling and view layout for the Windows host
- frame: a default scheduler, installed at module init, so drawing
presents without the host calling __flushAll. The first context
dirtied in a JS turn queues one microtask that flushes every dirty
context, so a turn (a rAF batch, an event handler) presents once,
after all of its drawing. Works on any Node-API host.
- canvas_core::fit: the iOS/Android fit modes (none, fill, fitX, fitY,
scaleDown) as one neutral buffer-to-view transform, per-axis density.
- NSCCanvas: 300x150 until sized (as on the web), surfaceWidth/Height
setters, fit, and setViewSize (the panel's DIPs). With the
composition scale these feed the swapchain's matrix transform.
* feat(windows): the canvas view on NativeScript Windows
The TS side of 2D on Windows, verified in a NativeScript Windows app
(core, webpack and the classic runtime with Node-API addons): 2D draws
into a SwapChainPanel at the display's refresh rate at 100% and 125%.
- platform.ts: capability flags (NAPI_HOST, POINTER_CONTEXT_HOST) that
call sites branch on instead of platform lists. __WINDOWS__ is read
through typeof, so bundlers that do not define it get false.
- helpers.ts: Node-API hosts load system_lib://canvasnative.node, which
installs globalThis.CanvasModule.
- Canvas/napi-host.ts: the view shared by Node-API hosts, driving
CanvasModule.NSCCanvas like the iOS view drives its native view:
width/height coalescing, context creation, fit from the CSS size,
rAF pause on unload, toDataURL, parent DOM shims. Hosts with native
layout report the view's size and scale; a `%` size then sizes the
buffer to the view's pixels (NativeScript never measures the view).
- Canvas/index.windows.ts: the SwapChainPanel, its SizeChanged and
CompositionScaleChanged, and pointer/wheel events in the shape
_handleEvents takes (which now also accepts objects).
- WebGL/WebGL2/WebGPU context pointer lookups use POINTER_CONTEXT_HOST.
- Packaging: platforms/windows plugin.props/targets copy
<arch>/canvasnative.node next to the app's executable;
tools/scripts/build-napi.sh and `make windows` build it;
package.json declares the windows platform.
* feat(windows): canvas-polyfill on Windows; svg, media and audio import cleanly
- canvas-polyfill: file.windows (reads through CanvasModule.readFile,
writes through core's file access, deletes through WinRT promises) and
http.windows on Windows.Web.Http: one shared HttpClient, headers and
content headers, string/binary/JSON bodies, onHeaders/onProgress,
timeout and cancel through the WinRT operation, the body read into an
ArrayBuffer without a copy, and the same text/JSON/bytes result rules
as iOS.
- XMLHttpRequest takes ArrayBuffer response content first for every
responseType (its non-Apple branches assumed Android's NSData /
ByteBuffer and threw on Windows). Local files: readFile's result is
{ buffer, mime }, so decode and return `buffer` itself; text, json,
document and arraybuffer responses were handed the wrapper object on
every platform.
- canvas-svg, canvas-media, audio-context have no Windows backend yet.
canvas-svg imports cleanly (canvas-polyfill's DOM references it at
startup) and throws a clear error from Helpers.initialize() when an
SVG is used; media and audio throw on import, so canvas-polyfill's
probes leave AudioContext absent and fall back to its VideoFrame stub.
* feat(demo): run apps/demo on NativeScript Windows
- Windows-capable @nativescript/core and @nativescript/webpack from the
core PR (pkg.pr.new 11272) and @nativescript/windows. typescript is
pinned for ts-loader (unpinned, fork-ts-checker's peer resolution
installed TypeScript 7, which has no JS compiler API); the type checker
it would switch on stays off, as the demo has never type-checked.
- @nativescript/font-manager is linked into the demo like the other root
packages, so the CLI stages its Windows plugin (as a dependency of the
file:-linked canvas package it was never installed in the app).
- Asset names with parentheses or commas are not copied on Windows,
where MSIX packaging cannot handle them.
- Launch arguments on Windows: launch-args.json in the app's LocalState,
read (asynchronously) and removed at launch; the main page waits for
them before opening the requested demo.
- App_Resources/Windows (manifest, PerMonitorV2 app.manifest, assets).
* test(canvas-napi): contract check against the V8 bindings
tools/tests/check-napi-contract.mjs scrapes the classes, methods,
accessors, Fast API methods and module members the V8 bindings register
(packages/canvas/platforms/ios/src/cpp/**, following inherited
Set{Methods,Props,Constants}) and checks them on the Node-API module.
Constants are reported separately (packages/canvas defines its own);
--strict fails on anything else missing, --json for tooling.
* feat(windows): WebGL and WebGL 2 on ANGLE in a SwapChainPanel
- canvas-core: GLContext::create_texture_context renders the default
framebuffer into a BGRA D3D11 texture on ANGLE's device; present()
flushes, copies it into a D3D11 composition swapchain bound to the
panel and presents. A copy leaves ANGLE's cached D3D11 state alone
(a draw would not); GL's bottom-up rows are flipped by a negative Y
scale in the swapchain's matrix transform. ANGLE cannot wrap a
multisampled texture, so these contexts report antialias: false.
dxgi::CompositionSwapChain gains a D3D11 constructor (and builds with
the gl feature).
- canvas-webgl / canvas-c: Windows attach / present / transform / resize,
canvas_native_webgl_create_d3d, and canvas_native_webgl_present (every
platform: present where on screen, else swap).
- canvas-napi: NSCCanvas.initContext + nativeContext (the iOS view's
contract), one context kind per view; createWebGLContext /
createWebGL2Context wrap the view's state (non-owning) or create an
offscreen one from the options object. WebGL contexts are presented at
frame end through crate::frame and gain __toDataURL, __flipY,
__getSupportedExtensions, __resized, __startRaf/__stopRaf and
continuousRenderMode.
- Fixes (Node-API, all platforms): shader sources were rewritten to
desktop GLSL (#version 120 / 330 core), now macOS only; getUniform
freed a result the into_* accessor had already freed (heap
corruption); texImage2D gains ImageData/ImageBitmap sources and takes
pixels from any ArrayBuffer or view; texSubImage2D(ImageData) passed
RGBA as the type; bind* / isFramebuffer / useProgram / VAO calls
accept null, undefined and 0 (what packages/canvas passes) via
GLObject; WebGL 2 getParameter answers the WebGL 2 pnames (and stops
leaking its result); getFragDataLocation returns -1 for unknown names.
- Canvas.createCustomView() on Node-API hosts (offscreen canvases).
- Packaging: tools/scripts/download-angle.sh (pinned, checksummed ANGLE
build); build-napi.sh ships libEGL/libGLESv2 next to the module;
THIRD_PARTY_NOTICES.txt.
apps/demo canvas-spec on Windows: 2d 180/180, webgl 74/74.
* feat(windows): WebGPU on wgpu DX12 in a SwapChainPanel
- canvas-core: dxgi::PanelSurfaceTarget, a COM stand-in for the panel's
ISwapChainPanelNative that wgpu binds its swapchain through. It
forwards to the panel and remembers the swapchain, so the DPI / fit
matrix transform stays ours across wgpu's reconfigurations.
- canvas-c: wgpu-core/hal with dx12 on Windows;
canvas_native_webgpu_context_create_swap_chain_panel,
_set_swap_chain_transform and _resize_swap_chain_panel (reconfigures
in place, keeping the page's configuration, and resizes the toDataURL
read-back texture). onSubmittedWorkDone now polls the device, so it
fires without a frame loop.
- canvas-napi host: NSCCanvas.initWebGPUContext(gpuPointer); WebGPU
joins the one-context-per-view rule, transforms and resizes.
- canvas-napi gpu: rewritten against the packages/canvas WebGPU contract
(objects.rs's strict descriptors replaced by lenient parsing like the
V8 bindings'): callback-style requestAdapter / requestDevice / async
pipelines, GPUSupportedLimits, error scopes and uncaptured errors,
device.lost, mapAsync rejecting on every error type, destroy() /
__releaseHandle on the transient objects, bounds-checked writeBuffer
and setBindGroup offsets (canvas-c would panic / assert). The canvas
context presents at frame end unless presentSurface() already did.
Uncaptured-error callbacks are held weakly (the device was never
collected); raw threadsafe functions are not released twice at
teardown (segfault at exit in about half the runs).
- TS: bgra8unorm preferred and the capability checks on Node-API hosts;
GPUDevice's event target reads WeakRef via deref() or get();
getCurrentTexture releases the previous frame's wrappers when the host
presented at frame end (they accumulated forever);
GPURenderBundleEncoder.drawIndexed passes baseVertex (firstInstance
landed in its slot).
- Demo: WEBGPU_SAMPLES / runWebGPUSample, and canvas-perf's
`webgpu:<sample>` suite.
canvas-napi node:test 66/66 (webgpu 21/21 on D3D12). apps/demo on
Windows: canvas-spec webgpu 17/19 (the two webgpu.video tests need
canvas-media video, not ported yet); occlusionQuery animates through
auto-present, survives resizes, memory flat over 30s. Samples that load
their WGSL with File.readText() fail on Windows: @nativescript/core's
Windows PathIO calls get the app path joined with '/' and its sync
readers return before the async read completes.
* feat(windows): TS completeness, WebGL extensions on ANGLE, GPU device loss
TS through the platform/ helpers (Node-API hosts):
- ImageSource / XAML images in drawImage, createPattern, drawAtlas,
createImageBitmap, WebGL texImage2D / texImage3D / texSubImage3D and
ImageAsset.loadFromNative: the encoded bytes an ImageSource keeps, or
the file behind its UriSource, decoded once per source.
- Fonts registered from font-manager's loadingdone; Dom's container is a
XAML Grid; key events (key / code / repeat) from the panel; snapshot()
as a PNG ImageSource.
WebGL on ANGLE (canvas-webgl angle.rs): ANGLE's WebGL-compatibility mode
exposes a GL extension only once requested, so getExtension enabled
nothing. Extensions are now mapped to the GL extensions ANGLE needs,
requested on getExtension (glRequestExtensionANGLE), offered only on the
WebGL versions they exist in, and getSupportedExtensions lists WebGL
names (not GL_*). WebGL 1 contexts are ES 2, so the extension objects
call the suffixed entry points (GL_ANGLE_instanced_arrays added to
gl-bindings). canvas-napi extension objects gain ext_name (packages/canvas
switches on it: every getExtension returned null), the constants the TS
reads and the web method names (drawArraysInstancedANGLE,
createVertexArrayOES, COLOR_ATTACHMENT0_EXT, ...; napi-rs camel-cased
them), a WebGL 2 EXT_color_buffer_float; OES_texture_half_float was
unreachable (a typo), EXT_disjoint_timer_query's TIMESTAMP_EXT was wrong.
TS: EXT_disjoint_timer_query called *EXT methods the bindings name *Ext
(every platform), getQueryEXT's null check was inverted, and
getExtension('EXT_color_buffer_float') had no case.
Canvas as an image source (every GPU engine):
- Context::get_image copied GPU snapshots to the CPU for GL, Vulkan and
Metal only: D3D snapshots were unreadable, so createImageBitmap(canvas),
the bitmap renderer, texImage2D(2d canvas) and copyExternalImageToTexture
got nothing.
- A WebGL canvas as a source is read as an image: RGBA8, top row first,
from its default framebuffer, bindings restored (read with format RGBA as
the type before: GL error, white).
- copyExternalImageToTexture: the row pitch was width x the padded pitch
(write_texture overran), flipY was ignored, and a 2D canvas copied
smaller than itself mismatched its layout.
GPU device loss (Windows):
- 2D: a removed D3D12 device is detected at present; restoreContext()
moves the canvas to a new device (the thread's shared device and Skia
context are recreated), cleared, and back into its panel. TS fires
contextlost, then contextrestored unless prevented, retrying while the
driver comes back.
- WebGL: ANGLE contexts ask for reset notification; a reset makes
isContextLost() true and fires webglcontextlost (no restore yet).
- __setContextLostListener / NSCCanvas.isContextLost / restoreContext;
__simulateD3DDeviceRemoval and __createHeadlessPanel (a stand-in
SwapChainPanel) for tests.
- Native log records (canvas-c, wgpu) now reach the host's console (and
OutputDebugString); on Windows they went nowhere. CANVAS_LOG sets the
level.
Also: ImageAsset.saveSync / saveCb (PNG, JPG; the C function the bindings
declare had gone); canvas-polyfill's devicePixelRatio / innerWidth read the
window when asked (the Windows window does not exist at import: 1x,
1920x1080); canvas-chartjs survives Intl polyfill probes that throw.
canvas-napi node:test 78/78; contract check: only GPUDevice's Apple-only
__getMetalDevicePointer missing. apps/demo canvas-spec on Windows 319/321
(webgpu.video needs canvas-media video); suite "contextlost" passes run on
its own. Known: removing the device in a packaged app leaves the process
unable to make hardware D3D12 devices for other APIs (WebGPU) afterwards;
the NativeScript Windows V8 engine loads no ICU data (Intl without
locales).
* feat(windows): canvas-svg on Node-API; canvas-polyfill gaps on Windows
canvas-svg:
- crates/canvas-svg-napi (canvassvg.node): the global.SVGModule the V8
bindings install (SVGDocument, SVGNode, createSVGDocument, createElement,
createTextNode), over canvas-svg-c, so NativeNode.ts runs unchanged.
renderToBuffer takes an optional `bgra`.
- canvas-svg-c: canvas_native_svg_document_render_to_buffer_ordered renders
premultiplied RGBA or BGRA whatever the platform's N32 order (BGRA on
Windows: canvas-image loads the buffer as RGBA).
- Windows view: the document rasterized on the CPU straight into a
WriteableBitmap's pixels (NSWinRT.interop.arrayBufferFromBuffer), shown
by an Image sized to the content in DIPs; SvgData holds the markup and
its natural size. Helpers.initialize loads system_lib://canvassvg.node.
- Packaging: platforms/windows (plugin.targets copies canvassvg.node);
build-napi.sh takes the crate (`make windows-svg`).
canvas-polyfill on Windows: HTMLElement's WeakRef read deref() only on
Apple and get() only on Android (undefined elsewhere); lang from
Device.language; getBoundingClientRect from the view's window location and
size for non-canvas elements; data: images decode in memory where there is
no native base64-to-file helper; SVG lengths in in/cm/mm use the CSS inch
(96) instead of NaN.
canvas-svg-napi node:test 6/6. apps/demo on Windows: canvas-svg renders
DOM-built SVGs, SMIL animations and drawImage(svg) into a canvas (1.2 ms
for two per frame); canvas-spec 319/321. SVGs loaded from `~/` files fail
in @nativescript/core (PR build): knownFolders.currentApp() is the package
root, not <root>\app, and path.join leaves '/' inside the joined part,
which Windows PathIO rejects.
* fix(windows): canvases take their style size; WebGL buffer calls take any view
- The canvas overrode width/height setNative to size its drawing buffer
without chaining to the platform view, which on Windows is what sets the
XAML element's Width / Height: every canvas stretched to fill its cell
(the canvas-spec page's 64x64 canvas was a full-width band). The
overrides now chain. Core watches SizeChanged itself for % sizes and an
event takes one delegate, so the canvas also hears size changes through
_onSizeChanged and only wires SizeChanged when core has not.
- bufferData / bufferSubData / compressedTexImage2D /
compressedTexSubImage2D / texSubImage2D took Uint8Array (or a few fixed
view types) only: Float32Array vertex updates, Uint16Array indices,
Uint8ClampedArray pixels threw. They take any ArrayBuffer or view now,
read in place.
- Demo: canvas-perf `alpha` / `alpha-webgl` suites (a cleared canvas over a
magenta parent).
Transparency: a cleared canvas still shows black on Windows. WinUI 3's
SwapChainPanel is external content and cannot blend with XAML behind it
(documented; microsoft-ui-xaml #6893), whatever the swapchain's alpha mode.
* feat(windows): transparent canvases blend with the page
WinUI 3's SwapChainPanel is external content: nothing in XAML shows
through it, whatever the swapchain's alpha mode (microsoft-ui-xaml #6893),
so a cleared canvas showed the window's black/white fill. A canvas with
alpha (the default) now presents into a XAML SurfaceImageSource shown by an
Image inside its panel (a panel with no swapchain stays see-through);
alpha: false keeps the zero-copy swapchain.
- canvas-core: WinUI 3's ISurfaceImageSourceNative and XamlSurface
(BeginDraw / copy / EndDraw); D3D12Context::d3d11_on_12, a D3D11On12
device on the canvas queue. ANGLE now runs on a D3D11 device of ours made
with BGRA support (EGL_ANGLE_device_creation), which SurfaceImageSources
require; ANGLE's own device is the fallback.
- canvas-2d: a transparent D3D12 canvas draws its frame into a texture (as
into a swapchain buffer) that D3D11On12 copies into the image on the
same queue. Device loss re-attaches the same surface.
- WebGL: the ANGLE texture is copied into the image; the view flips it (GL
rows are bottom-up).
- canvas-c / canvas-napi: *_attach_xaml_surface,
NSCCanvas.attachSurfaceImageSource and surfaceTransform (where the
drawing buffer sits in the view, for placing the Image).
- TS: napi-host's _prepareSurface / _layoutSurface hooks; on Windows a
SurfaceImageSource of the drawing buffer's size (a new one per size),
placed with surfaceTransform, flipped for WebGL. The Image stays
hit-testable, so pointer events bubble to the panel. % sizes are passed
to core as `auto`: the panel stretches over its cell (core sizes % views
against the whole parent).
WebGPU canvases still present through the panel (wgpu owns the swapchain):
their transparent areas show the window background.
apps/demo on Windows: canvas-perf alpha / alpha-webgl show the page through
the canvas (WebGL upright); canvas-svg's canvases draw over the page;
canvas-spec 319/321 (webgpu.video: canvas-media).
* perf(canvas-napi): raw fast members for the hottest calls; benchmark harness
napi-rs 3's generated methods and accessors create a reference to `this`
and register a native borrow on every call: ~300 ns, against ~75 ns for a
plain module function and a few times the work of a call like
translate(). src/fast.rs installs raw Node-API callbacks over the napi-rs
members on the class prototypes; they unwrap `this` checking napi-rs's
per-class type tag (a foreign receiver still throws "Illegal invocation")
and never call back into JS, which is what makes skipping the borrow
bookkeeping sound. They mirror the members they replace (canvas-c calls,
dirty marking):
- 2D: save/restore, resetTransform, translate/rotate/scale, beginPath,
closePath, moveTo/lineTo/bezierCurveTo/quadraticCurveTo/arcTo/arc/rect,
fillRect/strokeRect/clearRect, lineWidth and globalAlpha, and
fillStyle/strokeStyle: colour strings read on the stack, the last parsed
colour per style reused (no CSS parse when set again), colours
serialized directly by the getter, gradients and patterns through the
napi-rs accessors.
- WebGL / WebGL 2: uniform{1,2,3,4}f, uniform1i, uniformMatrix{2,3,4}fv,
viewport, clear, clearColor, enable/disable, activeTexture,
enableVertexAttribArray, vertexAttribPointer, bindBuffer, bindTexture,
useProgram, drawArrays, drawElements. A null uniform location is a
no-op, as in WebGL (the napi-rs members threw); numbers are coerced as
WebIDL does (the napi-rs members threw on "5").
CANVAS_NAPI_FAST=0 turns them off. Release build, per call (median, this
laptop; unchanged calls vary +-30% run to run): translate 477 -> 180-230
ns, lineTo 349 -> 166, lineWidth 332 -> 171-191, fillStyle (same colour)
557 -> 301, uniformMatrix4fv 1171 -> 438-504, uniform4f 849 -> 417-453,
drawArrays 1552 -> 475-1186.
bench/run.mjs (the demo's `native` scenarios on the offscreen host:
median and p95, --save to bench/results/<commit>[-label].json, which is
ignored) and bench/compare.mjs (per-scenario change, exits 1 past a
threshold). PORTING.md documents both.
canvas-napi node:test 83/83; apps/demo canvas-spec on Windows 319/321.
* fix(windows): a lost 2D context is restored onto its GPU, not WARP
D3D12 devices are per-adapter singletons: while anything holds a removed
device, D3D12CreateDevice on that adapter fails. Restoring one lost canvas
while the others still held the device fell through to the next adapter,
usually WARP, for the rest of the process (and failed outright where WARP
was the only adapter, as on GPU-less CI runners).
- canvas-2d keeps a thread registry of D3D canvases (registered by
canvas-c once boxed). Before a new device is made after a loss, every
canvas on the removed one lets go of it: Skia context and surfaces,
swapchain (unbound from its panel), XAML surface. A lost canvas draws
into a raster stand-in until restored; resizing it while lost sizes the
restore.
- XAML SurfaceImageSources keep the device they were given, and what they
last drew with it, until they draw again (SetDevice(null) does not
release it): a lost one is given a small D3D11 WARP stand-in device and
drawn blank once.
- For 10 s after losing a GPU device, a new device has to be on a GPU
again (restores fail and packages/canvas retries) rather than settle for
WARP; after that WARP will do.
- CanvasModule.__d3dAdapterInfo() (tests): the device-lost suites assert
the restore comes back on the same adapter.
* ci(windows): build the Node-API modules for x64 and arm64
build-native.yml gains canvas-windows and canvas-svg-windows (cached on
its sources, like the other SVG jobs), each for x64 and arm64
(cross-compiled). The x64 jobs run the Node-API suites on the shipped
modules, on WARP. .github/actions/setup-windows-native uses the image's
LLVM and ninja (installing LLVM over it with choco fails). The npm job
places the Windows modules; make windows / windows-svg build arm64 too.
Internal notes (canvas-napi PORTING.md, napi-sys PATCHED.md) stay local
(gitignored).
* build(windows): commit the x64 and arm64 Node-API modules
Release builds (release-napi) of canvasnative.node (+ ANGLE libEGL/libGLESv2)
and canvassvg.node, so npm_release.yml ships them. Temporary: the
canvas-windows / canvas-svg-windows CI jobs will provide them instead.
* feat(windows): audio-context on Node-API (#155)
* fix(canvas-napi): device.lost settles on the JS thread; the flush test keeps its host
- destroy() (and reading `lost` on a destroyed device) settled the weak lost promise through
its threadsafe function. An unref'd function does not keep Node's loop alive, so an await on
`lost` right after could see the loop end first (Node 22, CI). It now settles in place, and
destroy() takes the pending promise before canvas-c's own lost callback can.
- windows-host: the flush test held only the pointer-wrapped context; the NSCCanvas that owns
it could be collected across the awaits, freeing the context under it (transparent reads or
an access violation on Node 22).
* feat(windows): audio-context on Node-API (web-audio-api, WASAPI)
- crates/audio-context-napi (audiocontext.node): AudioContext / OfflineAudioContext, every node
in index.d.ts, AudioParam automation, AudioBuffer, PeriodicWave and the listener over the
web-audio-api crate, played through WASAPI (cpal). web-audio-api panics on spec violations;
each call turns the panic into a JS error ("InvalidStateError: ...") instead of aborting the
host. Decoding (bytes, base64, files) and offline rendering run as async work; ended and
statechange arrive through weak threadsafe functions.
- index.windows.ts: the public classes over it, sharing common.ts with iOS and Android.
canvas-polyfill's probe now finds AudioContext on Windows. Not yet: MediaElementAudioSourceNode
(canvas-media has no Windows backend); a WaveShaperNode takes one curve.
- build-napi.sh audio-context-napi, `make windows-audio`, the MSBuild copy targets, a CI job
(x64 and arm64; the x64 job runs the Node-API suite offline and on the 'none' sink) and the
PR workflow's artifact placement. THIRD_PARTY_NOTICES.txt for web-audio-api, Symphonia
(MPL-2.0) and cpal.
- demo: an `audio` spec suite on Windows (offline graphs, decoding, realtime state and ended
events on the default device).
* build(windows): commit the x64 and arm64 audiocontext modules
* build(windows): rebuild the x64 and arm64 canvasnative modules with the device.lost fix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Windows support to
@nativescript/audio-contextand fixes the two Canvas Windows (x64) test failures from run 36277464282.Audio context on Windows
crates/audio-context-napi→audiocontext.node, a napi-rs 3 module over the web-audio-api crate (MIT). Graph, DSP and decoding (wav, mp3, ogg/vorbis, flac, aac via Symphonia) come from the crate; output is WASAPI through cpal.start()twice, badconnect, invalidfftSize, …). Every call goes through a guard that returns the panic as a JS error (InvalidStateError: …) rather than unwinding into Node-API, which aborts the host.decodeAudioDataandstartRenderingrun as async work;ended/statechangeare delivered through weak threadsafe functions.packages/audio-context/index.windows.tsimplements the public API over it, sharingcommon.tswith iOS and Android. canvas-polyfill's existing probe now installsAudioContextand the node globals on Windows.build-napi.sh … audio-context-napi,make windows-audio, MSBuild copy targets, anaudio-context-windowsjob (x64 + arm64; x64 runs the Node-API suite offline and on the'none'sink, since runners have no audio device), and the PR workflow's artifact placement.THIRD_PARTY_NOTICES.txtfor web-audio-api, Symphonia (MPL-2.0, unmodified) and cpal.Not supported on Windows yet (README updated):
createMediaElementSourcethrowsNotSupportedError;createSourceNodeFromPlayerreturnsnull(canvas-media has no Windows backend).WaveShaperNodeaccepts one curve (engine limitation); a second assignment throwsInvalidStateError.Canvas Windows (x64) test fixes
Both reproduce every time on Node 22 (what CI uses) and never on Node 24.
device.destroy() resolves lost with reason 1(68–70 were cancelled after it):destroy()settled the weaklostpromise through an unref'd threadsafe function, which does not keep Node's loop alive, so the loop could end while the test awaitedlost. It now settles on the JS thread directly, and takes the pending promise before canvas-c's own lost callback can.NSCCanvasthat owns it could be collected across the test's awaits, freeing the context under it. Test fix only — the view owning the context is the same contract as iOS/Android, and packages/canvas keeps the view alive.Testing
crates/audio-context-napi/__test__: 11 tests, debug and release builds (offline rendering, automation, buffers/loops/ended, spec errors as exceptions, param connections, filters/analyser, HRTF panning, periodic waves, decoding the demo's wav/mp3/ogg and base64, realtime state on the'none'sink).{"demo":"canvas-spec","suite":"audio"}: 7/7; full spec run 326/328 (the twowebgpu.videofailures need canvas-media video, which has no Windows backend).Runtime dependency
In the NS Windows runtime,
AsyncWork::completeread the work after calling the addon's complete callback, which napi-rs uses to free it (allowed by Node-API): a use-after-free for every napi-rsAsyncTask(heredecodeAudioDataandstartRendering, which crashed the demo app). Fixed in the runtime byfix/napi-async-work-after-free(merged); apps need a runtime release that includes it.The committed x64 and arm64
canvasnative.nodemodules are rebuilt with thedevice.lostfix (84/84 on Node 22 with the x64 one).