Skip to content

refactor: Include Bluesky plan stubs by default - #1694

Open
shree-iyengar-dls wants to merge 39 commits into
mainfrom
1691_blueapi_plan_stubs
Open

shree-iyengar-dls wants to merge 39 commits into
mainfrom
1691_blueapi_plan_stubs

Conversation

@shree-iyengar-dls

@shree-iyengar-dls shree-iyengar-dls commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1691

Summary of changes

  • Since the improvement of type checking of plan stubs in bluesky, dodal wrapped plan_stubs are no longer required and we can import plan stubs directly from bluesky - these are hardcoded in a list of DEFAULT_PLAN_STUBS
  • NB: bluesky plan stubs have the annotation Moveable[Any]. The is_compatible_args function in blueapi checks type arguments and so there might be a mismatch between for example Moveable[float] and the bluesky plan stub annotation? I had to change the type annotation of FakeDevice but not sure what effect this might have on other devices

@shree-iyengar-dls shree-iyengar-dls changed the title feat: add default plans list refactor: Include Bluesky plan stubs by default Oct 8, 2026
@codecov

codecov Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.32%. Comparing base (5424f13) to head (66ba601).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1694   +/-   ##
=======================================
  Coverage   96.31%   96.32%           
=======================================
  Files          50       50           
  Lines        3831     3836    +5     
=======================================
+ Hits         3690     3695    +5     
  Misses        141      141           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@shree-iyengar-dls

Copy link
Copy Markdown
Contributor Author

Blocked by #1662 and #1663

Comment thread pyproject.toml Resolved
@oliwenmandiamond

Copy link
Copy Markdown
Contributor

@shree-iyengar-dls I would put this in draft for now as this depends on a few different things which are yet to be merged e.g updated bluesky version, supporting *args and **kwargs

@abbiemery

Copy link
Copy Markdown
Contributor

Unblocked partially by the merging of #1715, might still be blocked by a bluesky release.

@oliwenmandiamond

Copy link
Copy Markdown
Contributor

Blocked by #1720

@oliwenmandiamond

oliwenmandiamond commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

I have tested this

>>> pl.mv(dev.stage.x, 90, dev.stage.theta, 5)
>>> pl.abs_set(dev.stage.x, 100, wait=True)
>>> pl.mvr(dev.stage.x, -2)
>>> pl.mvr
mvr(
    args: list[Movable | Any] | None = None,
    group: Any | None = None,
    timeout: float | None = None,
    kwargs: dict | None = None
)

Seems to work fine for plan stubs now

Comment thread pyproject.toml Resolved
Comment thread pyproject.toml Resolved

def with_config(self, config: EnvironmentConfig) -> None:
if config.metadata is not None:
self.run_engine.md |= config.metadata.model_dump()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change not related to this change

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the original |= I'm getting an error:

  Operator "|" not supported for types "RunEngineMetadata" and "Dict[str, Any]"

@shree-iyengar-dls
shree-iyengar-dls marked this pull request as ready for review November 13, 2026 11:14
@shree-iyengar-dls
shree-iyengar-dls requested a review from a team as a code owner November 13, 2026 11:14
Comment thread pyproject.toml Resolved

@oliwenmandiamond oliwenmandiamond left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good, thank you! I think a small amount of documentation is needed to go with this about blueapi providing default bluesky plan stubs and that they can be overwritten for whatever reason is also possible.

Then am happy to approve



class FakeDevice(Movable[float]):
class FakeDevice(Movable[Any]):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this changed to Any? If it does indeed take Any, then the type hint for set would need to be updated to Any as well

def set(self, value: Any) -> Status:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah not sure if this is right but the NB point in my PR description should explain

Comment thread docs/tutorials/4. config-add-plans.md Outdated
module: dodal.plan_stubs.wrapped
```
These plans that will be used in this tutorial can be found in the [dodal repository](https://github.com/DiamondLightSource/dodal/tree/b4b38f1e0cf480faeb29d67695440c45a148f169). Specifically, in `dodal/src/dodal/plans/wrapped.py` and `dodal/src/dodal/plan_stubs/wrapped.py`.
These plans that will be used in this tutorial can be found in the [dodal repository](https://github.com/DiamondLightSource/dodal/tree/b4b38f1e0cf480faeb29d67695440c45a148f169). Specifically, in `dodal/src/dodal/plans/wrapped.py` and [bluesky repository](https://github.com/bluesky/bluesky/blob/1fb2f4e13437edf4a72372dacd455cfaf905f09e/src/bluesky/plan_stubs.py#L321).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should make a distinction between the plans that were just added in the example and the plans available by default. At the moment it looks like you add one module and get two loads of plans to work with.

(and nit: Was this meant to point to a specific line? )

Comment thread tests/unit_tests/core/test_context.py Resolved


class FakeDevice(Movable[float]):
class FakeDevice(Movable[Any]):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needing to change is a sign that the type hints in the bluesky plan stubs are wrong. I'll open an issue there.

@shree-iyengar-dls shree-iyengar-dls Nov 14, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I leave it as Moveable[Any] for this PR or wait for the change in bluesky to go through?

PR in bluesky

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tpoliaw I don't think the plans stubs are wrong, I think this is another blueapi bug that has been discovered.

With the help with some AI, I've managed to find the issue

https://chatgpt.com/s/t_6afacb2423988191817fe58a4634baef

In summary, it is because the is_compatible_args function is too strict and doesn't work correctly with Any being the sub script of a class.

This is the function currently

def is_compatible_args(
    val: Device, target: type, args: tuple[type, ...] | None
):
    if not args:
        return True

    actual_args = generic_bounds(val, target)

    if not actual_args:
        return False

    return all(
        actual is Any
        or type(actual) is TypeVar
        or type(expected) is TypeVar
        or expected == actual
        or issubclass(actual, expected)
        for expected, actual in zip(args, actual_args, strict=True)
    )

This is the suggested fix. I've tested and it indeed does work.

def is_compatible_args(
    val: Device, target: type, args: tuple[type, ...] | None
):
    return (not args) or all(
        expected is Any
        or actual is Any
        or type(actual) is TypeVar
        or type(expected) is TypeVar
        or expected == actual
        or issubclass(actual, expected)
        for expected, actual in zip(args, generic_bounds(val, target), strict=False)
    )

So the Any is not being checked for expected or actual, only actual.

Your bluesky Movable change I've also tested and it doesn't fix the problem and unfortunately will cause more problems, especially ophyd-async because all signals take a value plus a timeout value as optional which i don't your change would work with any more.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the plans stubs are wrong

I do but I will open an issue/PR in bluesky and we can discuss it there.

I think this is another blueapi bug that has been discovered.

There is an issue in blueapi but not the one that you copied from the LLM. I'll open an issue for what is going on.

Your bluesky Movable change I've also tested and it doesn't fix the problem and unfortunately will cause more problems, especially ophyd-async because all signals take a value plus a timeout value as optional which i don't your change would work with any more

The PR I have open in bluesky at the moment doesn't fix anything because it doesn't change anything other than making the current behaviour more readable. If there are signals that break because of it, I think they're already broken. Do you have an example of one?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shree-iyengar-dls, I think it might be worth waiting for the changes in bluesky if it doesn't look like they'll take too long to go through. As it is we won't be able to use the plan stubs through blueapi anyway.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR in bluesky: bluesky/bluesky#2116

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tpoliaw that bluesky change makes more sense. I was confused on how the other one would fix this issue so thank you for clearing that up. I'm happy to approve this in bluesky.

I do have one question though, why is it you think the blueapi bug fix I applied from the AI shouldn't be applied? All the tests pass with it and I've also confirmed using it via the client all works well.

I am not sure why we need to wait again for another bluesky release for a couple of months? Surely we can do both, merge your bluesky PR and also still make progress here?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is it you think the blueapi bug fix I applied from the AI shouldn't be applied?

If it makes plans accept arguments that they shouldn't accept (as is the case here) it suggests the 'fix' is not correct.

There are more complications here that I mentioned earlier but I'm writing that up in a new issue.

@oliwenmandiamond oliwenmandiamond Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it makes plans accept arguments that they shouldn't accept (as is the case here) it suggests the 'fix' is not correct.

That doesn't make sense. If a plan is annotated as Movable[Any], I would expect a device implementing Movable[float] to be accepted. Any explicitly allows any value type, so I don't think accepting Movable[float] here is making the plan accept an invalid argument.

Simple test

def move(device: Movable[str]) -> MsgGenerator:
    yield from ()

m = SimMotor("")

move(m)

The above correctly has type checking error

Argument of type "Motor" cannot be assigned to parameter "device" of type "Movable[str]" in function "move"
  "Motor" is not assignable to "Movable[str]"
    Type parameter "T_co@Movable" is contravariant, but "float" is not a supertype of "str"
      "str" is not assignable to "float"PylancereportArgumentType
(variable) m: Motor

because SimMotor implements Movable[float]. If I change the plan to be Movable[Any]

def move(device: Movable[Any]) -> MsgGenerator:
    yield from ()

m = SimMotor("")

move(m)

There's no type checking error.

The blueapi check is too strict and wrongly rejects the task without the suggested change

test_does_not_allow_simultaneous_running_tasks

   File "/scratch/bluesky_development/blueapi/src/blueapi/worker/task.py", line 50, in prepare_params
    raise TaskParameterError.from_validation_error(ve) from None
blueapi.worker.task.TaskParameterError: 1 validation error for abs_set
obj
  Value error, Device fake_device (test_task_worker.FakeDevice) is not of type bluesky.protocols.Movable[typing.Any] [type=value_error, input_value='fake_device', input_type=str]
    For further information visit https://errors.pydantic.dev/2.14/v/value_error

If the expected value is Any, then the actual value should accept any python type for is_compatible_args args.

You do this already for TypeVar so that Movable[T] is accepted. We should do this for Any too.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Include Bluesky plan stubs by default

4 participants