async_spi: SPI transfers for asyncio by lynt-smitka · Pull Request #11489 · adafruit/circuitpython

lynt-smitka · 2026-10-02T09:38:07Z

Adds the async_spi module, based on @kamocat's work in #11040. Part of #10856.

async_spi.SPI is configured and locked like busio.SPI, but write(), readinto() and write_readinto() are awaited, so other asyncio tasks run during a transfer. Cancelling a task stops its transfer. Tasks that share the bus can use an asyncio.Lock.

busio on raspberrypi gets async reads, full-duplex transfers and cancel, next to the async write from displayio: send one area buffer while composing the next #11468. A DMA interrupt marks the transfer as done, and asyncio then resumes the waiting task.
supervisor/shared/awaitable.c lets C functions return awaitables, so other async modules (I2C, UART) can use it later.
Only raspberrypi enables the module so far. It adds about 1.9 KB there; other ports are unchanged.

Tested on a Pico W with MOSI jumpered to MISO: data reads back correctly at up to 62.5 MHz, other tasks keep running during transfers, and cancelling transfers works while audio plays.

tannewt

Thanks for picking this up! I'd prefer it to be more like a new module than changing the implementation in busio.SPI().

lynt-smitka · 2026-10-03T17:40:38Z

I would like to clarify why the PR looks the way it does:

The blocking transfers, displayio's async write and async_spi share one DMA path. A separate module would either carry a second copy of the DMA code for the same SPI block, or busio would have to call into the new module.
Reusing busio's constructor and configure()/try_lock()/unlock() keeps the module at about 1.9 KB. With its own bindings it is about 0.5 KB more.
The lock is busio's. An earlier version had an awaitable lock() that polled - in a test with two tasks it let one of them run 790 transfers while the other got 1. Without a native lock, tasks use asyncio.Lock, which costs nothing in flash.

What I'm working on:

Moving the RP2 driver out of busio into a port file (peripherals/spi.c), with busio.SPI and async_spi.SPI as thin layers over it. busio's bindings stay unchanged.
async_spi.SPI as its own type: constructor, configure(), deinit(), frequency and the three awaited transfers, no lock methods. Tasks share the bus with asyncio.Lock.
A Zephyr implementation tested on native_sim. That needed zephyr-cp: fix asyncio and busio.SPI on native_sim, add an SPI loopback for tests #11504 first: asyncio tasks from create_task() did not run there, and busio.SPI crashed.
The current result is about +2 KB on RP2 boards, and other ports get a few bytes smaller. It passes on a Pico W and a Pico 2 W so far. I'll push once zephyr-cp: fix asyncio and busio.SPI on native_sim, add an SPI loopback for tests #11504 is in.

Questions before I push

Is it ok to keep the shared driver in peripherals/, so busio does not depend on the Python module?
Is it ok for async_spi to reuse busio's argument tables? Same arguments, 160 B less flash, but a link between the two modules.
Is it ok that configure() and deinit() wait for a running transfer instead of raising?

tannewt · 2026-10-05T16:39:48Z

busio would have to call into the new module.

This is what I was thinking. Build busio on top of async under the hood.

Reusing busio's constructor and configure()/try_lock()/unlock() keeps the module at about 1.9 KB. With its own bindings it is about 0.5 KB more.

The constructor can be shared by building on the async portion.

I don't think we want busio's locking. Instead, we likely want something async friendly. (Like an async lock)

Questions before I push

* Is it ok to keep the shared driver in peripherals/, so busio does not depend on the Python module?

Yup, that's fine. Code in peripherals shouldn't have any python specific code though.

* Is it ok for async_spi to reuse busio's argument tables? Same arguments, 160 B less flash, but a link between the two modules.

The reverse but yes. Have it in async and reuse it in busio.

* Is it ok that configure() and deinit() wait for a running transfer instead of raising?

Yes, but I image they too should be async then.

buffer_slice() turns a buffer object and element bounds into a byte slice, as busio.SPI's write(), readinto() and write_readinto() did inline. busio.SPI gets a little smaller, and async_spi will use it too.

busio.SPI's port code becomes rp2_spi_t and rp2_spi_*() in peripherals/spi.c, with common-hal/busio/SPI.c as a thin layer over it, so that another module can drive an SPI bus the same way. No change in behaviour.

rp2_spi_read_start() and rp2_spi_transfer_start() start the other two transfers the way rp2_spi_write_start() starts a write. rp2_spi_stop() stops a transfer, or finishes it if it is already done; it does nothing once the bus has started a later transfer. rp2_spi_configure_start() applies new settings once the running transfer has finished, and rp2_spi_deinit() stops a running transfer. With CIRCUITPY_ASYNC_SPI, a DMA_IRQ_0 handler, shared like the audio and rp2pio ones, sets the done flag when the RX channel finishes, so a waiter does not need to call rp2_spi_end() to find out.

CIRCUITPY_DEFINE_ASYNC_FUN_OBJ_KW(obj, n_args_min, fun) defines a method that returns an awaitable. As with a CPython coroutine, its arguments are stored and only parsed when it is awaited. While an operation runs, the task waits on asyncio's I/O queue: the awaitable is a stream that polls readable once the operation's done flag is set, possibly from an interrupt. Based on the awaitable in adafruit#11040.

async_spi.SPI is configured like busio.SPI, and configure(), write(), readinto() and write_readinto() are awaited: other tasks run during a transfer. Tasks that share the bus can use an asyncio.Lock. Enabled on raspberrypi, where it drives the bus through peripherals/spi.c. A manual test is in tests/circuitpython-manual/async_spi. Co-authored-by: Marshal Horn <175229+kamocat@users.noreply.github.com>

busio.SPI takes the same arguments as async_spi.SPI, so where that module exists its tables are used. Saves 160 bytes on RP2040.

Transfers use spi_transceive_cb() where the driver has an async path, and otherwise run on the system work queue. Zephyr cannot stop a transfer, so cancel waits for it. Enabled on boards with an SPI bus; tested on native_sim through the loopback device.

The driver's result reaches the await as an exception. cancel() and deinit() wait for the transfer without running background tasks, since they may run from a finalizer during GC. async_spi needs asyncio, not only async/await, so raspberrypi enables it with MICROPY_PY_ASYNCIO.

…O_SPI Unused, so a build error, on boards without busio.SPI.

busio.SPI is always built, async_spi only on some ports, so the shared tables belong to busio. This drops the CIRCUITPY_ASYNC_SPI conditionals from busio's bindings. Firmware size is unchanged.

tannewt

Thanks for working on this. I've got a couple suggestions. I'm being extra picky on this change because I expect to apply these patterns to more ports and APIs.

tannewt · 2026-10-12T18:38:13Z

-    self->target_frequency = baudrate;
-    self->real_frequency = spi_set_baudrate(self->peripheral, baudrate);
-
+    rp2_spi_configure(&self->spi, baudrate, polarity, phase, bits);


Instead of targeting a shared peripherals implementation, could this be in shared-modules and target async_spi? Targeting async_spi would allow it to be port-agnostic.

This will be pretty large refactor, working on it.

The driver takes a volatile bool * for a transfer's done flag instead of circuitpy_async_flag_t, so peripherals/ does not depend on the supervisor. common-hal/busio/SPI.h includes the flag header again, as before the driver moved.

common_hal_async_spi_spi_*_start() return the context that end() and cancel() get, so an implementation can keep per-operation state. Both ports return the SPI object, as the bindings did before.

tannewt

Getting closer thanks! Please resolve or reply to inline comments.

The context is for state of one operation; start() returns NULL on both ports, which keep that state in the bus object. end() and cancel() take self and the context as separate arguments, and the macro takes the end and cancel functions by name.

shared-module/busio/SPI.c implements busio.SPI over common_hal_async_spi: each blocking call starts a transfer and waits for its flag, the lock sits on top, and FourWire's write_start/end map onto the async start and end. A port turns it on with CIRCUITPY_BUSIO_SPI_OVER_ASYNC, which also builds its async_spi common-hal without the Python module. A driver error reaches busio as false and an await as an exception, through transfer_finish() and transfer_end(). raspberrypi and zephyr-cp use it, so their own busio.SPI goes away. The RP2 driver is always async now and loses its blocking entry points; Zephyr keeps a fixed-device constructor for board.SPI() and reuses the fill buffer of reads. Pico 898680 -> 898760 bytes, Pico 2 W 1375564 -> 1375500; other ports are unchanged.

bus->spi.spi.peripheral read as a typo.

Without a TX buffer the nRF SPIM driver sends its over-read character, 0xFF, so readinto() with the default write_value clocked out 0xFF instead of 0. Seen on a Feather nRF52840; native_sim's loopback sends zeros, which hid it.

lynt-smitka · 2026-10-14T19:57:17Z

Tested on a Feather nRF52840 under zephyr-cp too: busio and async_spi loopback pass from 250 kHz to 24 MHz. It showed one old bug: without a TX buffer the nRF SPIM sends 0xFF, so readinto() with the default write_value clocked out 0xFF instead of 0 (the previous zephyr busio did the same); fixed in 4546345 by always sending write_value from a buffer.

close() or throw() before the first await left the state NEW, so a later await started the transfer. It is now finished, as with a closed coroutine.

configure_start() checked for a running transfer and then stored the waiting configure(); a transfer_done() between the two left it waiting forever.

Errata RP2350-E5: an enabled channel can retrigger after the abort.

…ransfers "ticker ran" once also passed with a blocking transfer; now it has to run more than 20 times during 10 transfers of 4 KB at 1 MHz (about 330 ms).

tannewt requested changes Oct 2, 2026

View reviewed changes

Comment thread ports/raspberrypi/common-hal/busio/SPI.c Outdated

Comment thread shared-module/async_spi/SPI.h Outdated

Comment thread tests/circuitpython-manual/async_spi/async_spi_loopback.py

lynt-smitka mentioned this pull request Oct 3, 2026

zephyr-cp: fix asyncio and busio.SPI on native_sim, add an SPI loopback for tests #11504

Merged

lynt-smitka and others added 7 commits October 8, 2026 20:34

shared/runtime: add buffer_slice() and use it in busio.SPI

c1d61d2

buffer_slice() turns a buffer object and element bounds into a byte slice, as busio.SPI's write(), readinto() and write_readinto() did inline. busio.SPI gets a little smaller, and async_spi will use it too.

raspberrypi: move the SPI driver to peripherals/spi.c

4719ff9

busio.SPI's port code becomes rp2_spi_t and rp2_spi_*() in peripherals/spi.c, with common-hal/busio/SPI.c as a thin layer over it, so that another module can drive an SPI bus the same way. No change in behaviour.

busio: reuse async_spi's argument tables

4f18be7

busio.SPI takes the same arguments as async_spi.SPI, so where that module exists its tables are used. Saves 160 bytes on RP2040.

zephyr-cp: async_spi

8bbdf9b

Transfers use spi_transceive_cb() where the driver has an async path, and otherwise run on the system work queue. Zephyr cannot stop a transfer, so cancel waits for it. Enabled on boards with an SPI bus; tested on native_sim through the loopback device.

lynt-smitka force-pushed the async-spi branch from 5e39446 to 8bbdf9b Compare October 8, 2026 20:38

lynt-smitka added 3 commits October 8, 2026 21:10

busio: keep the SPI constructor's argument table under CIRCUITPY_BUSI…

9aaf7f5

…O_SPI Unused, so a build error, on boards without busio.SPI.

Merge remote-tracking branch 'upstream/main' into async-spi-v3

f2b40a4

tannewt reviewed Oct 12, 2026

View reviewed changes

Comment thread shared-bindings/busio/SPI.c Outdated

busio: own the SPI argument tables, async_spi uses them

7fe5f7a

busio.SPI is always built, async_spi only on some ports, so the shared tables belong to busio. This drops the CIRCUITPY_ASYNC_SPI conditionals from busio's bindings. Firmware size is unchanged.

tannewt requested changes Oct 12, 2026

View reviewed changes

lynt-smitka added 2 commits October 12, 2026 20:19

async_spi: start functions return the context

b1de0e7

common_hal_async_spi_spi_*_start() return the context that end() and cancel() get, so an implementation can keep per-operation state. Both ports return the SPI object, as the bindings did before.

tannewt requested changes Oct 13, 2026

View reviewed changes

Comment thread shared-bindings/async_spi/SPI.h Outdated

Comment thread ports/raspberrypi/common-hal/async_spi/SPI.c Outdated

tannewt mentioned this pull request Oct 14, 2026

Implemented asyncio SPI for rp2040 & rp2350 #11040

Closed

lynt-smitka added 3 commits October 14, 2026 17:34

busio: name the embedded async_spi object async_spi

cff9b57

bus->spi.spi.peripheral read as a typo.

lynt-smitka added 3 commits October 14, 2026 20:50

async_spi: close() on an awaitable that was never awaited finishes it

6f5d10a

close() or throw() before the first await left the state NEW, so a later await started the transfer. It is now finished, as with a closed coroutine.

zephyr-cp: register a waiting configure() under irq_lock()

5178374

configure_start() checked for a running transfer and then stored the waiting configure(); a transfer_done() between the two left it waiting forever.

raspberrypi: clear the DMA enable bits before an SPI abort on RP2350

c96badb

Errata RP2350-E5: an enabled channel can retrigger after the abort.

zephyr-cp: the async_spi test requires the ticker to run during the t…

629e8a4

…ransfers "ticker ran" once also passed with a blocking transfer; now it has to run more than 20 times during 10 transfers of 4 KB at 1 MHz (about 330 ms).

Navigation Menu

Repository navigation

async_spi: SPI transfers for asyncio - #11489

async_spi: SPI transfers for asyncio#11489
lynt-smitka wants to merge 21 commits into
adafruit:mainfrom
MakerClassCZ:async-spi

lynt-smitka commented Oct 2, 2026

Uh oh!

tannewt left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lynt-smitka commented Oct 3, 2026

Uh oh!

tannewt commented Oct 5, 2026

Uh oh!

Uh oh!

tannewt left a comment

Uh oh!

Uh oh!

tannewt Oct 12, 2026

Uh oh!

lynt-smitka Oct 14, 2026

Uh oh!

Uh oh!

tannewt left a comment

Uh oh!

Uh oh!

Uh oh!

lynt-smitka commented Oct 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

lynt-smitka commented Oct 2, 2026

Uh oh!

tannewt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lynt-smitka commented Oct 3, 2026

Uh oh!

tannewt commented Oct 5, 2026

Uh oh!

Uh oh!

tannewt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tannewt Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

lynt-smitka Oct 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tannewt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lynt-smitka commented Oct 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants