Skip to content

Scenario stories green on caldav's servers; one kind per collection - #44

Merged
tobixen merged 8 commits into
mainfrom
fix/scenarios-all-servers
Oct 31, 2026
Merged

tobixen merged 8 commits into
mainfrom
fix/scenarios-all-servers

Conversation

@tobixen

@tobixen tobixen commented Oct 30, 2026

Copy link
Copy Markdown
Member

Work done on the scenario stories in tests/test_scenarios.py:

  • Human review and code comments
  • Work done to get it to pass on every server in caldav's test-server registry. Some stories are skipped where a server can't support them, and two servers only pass when run on their own or over several runs. docs/caldav-servers.md lists the result and the reason for each server. Note that the caldav code used to let this pass has not been released yet.

This has been fixed:

  • Time logging is unsupported where the server strips RELATED-TO (Zimbra, robur and a few others).
  • A new task.uncomplete capability: on GMX, uncomplete() now refuses instead of silently doing nothing.
  • collections() leaves out a listed calendar that answers 404
  • The configured search-cache delay is now handled same way in the calendaring tests as in the caldav tests, the caldav test helper has been exported in feat: caldav.testing.search_cache_delay python-caldav/caldav#843 (to be released in caldav 3.8.0)

A new collection.mixed capability says whether one collection can hold several component kinds - many systems treat calendars and tasklists as two different things.

The last commit narrates the stories in comments, makes the change-feed story check that an edit before the token is old news and an add after it is not, and adds a story moving a task to a task list on another backend. Moving to another backend should not depend on the source server's one-UID-per-calendar rules, but it does today: #43.

A time entry is tied to its task by RELATED-TO.  Zimbra and robur
store the entry without it, so time_spent() came back zero and
log_time() lost the link without a word.  caldav's
save-load.icalendar.related-to hint now makes TIME_LOG unsupported,
as it already did for the task relations; that also covers Yahoo,
which refuses the entry (400), and Ox and Bedework 3, whose hint is
"broken" (Bedework 3 keeps only the first RELATED-TO, which would do).

Prompt: [read the session handover notes] - we should make sure the scenario tests can be run towards all the caldav servers without failures/errors first, and after that the conformance tests

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Bedework makes no task lists, so the story got an event calendar and
add_task() raised.  It now checks that refusal, as the other task
stories do.

Prompt: [read the session handover notes] - we should make sure the scenario tests can be run towards all the caldav servers without failures/errors first, and after that the conformance tests

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Bedework 3 and Purelymail answer searches from an index that lags
behind writes; a story searches right after its write.  The stories
now wait for what is left of the delay, with the helper caldav's own
integration tests use (caldav.testing.search_cache_delay, caldav 3.8).
On Bedework 3 the etag listing of changes() waits too, as it lags as
well.  With an older caldav nothing waits.

Prompt: An arbitrary sleep after writing something to the calendar can be quite intrusive, particularly for purelymail which is defined with several minutes of search cache delay.  It's typically only in tests that the search cache delay causes problems.  There is a fix already in the caldav tests (~/caldav), please check that up before reinventing the wheel.  Skip memory as for now.  Retry the failed tests for ecloud and stalwart and robur. [the caldav source checkout] [answering a proposal to make caldav itself sleep after writes on servers with a search-cache delay]
Followup-Prompt: Ok, do the upstream testing export fix.  Since the CI doesn't do the full integration run (yet), we don't have to wait for caldav 3.8 before releaseing (sic) calendaring 0.1, it suffices to write it out clearly somewhere what servers the tests are passing at, and what servers they are failing at and the reason.  Plus, importantly: I still want the tests to pass locally towards the master branch (or a bleeding-edge branch) on caldav.

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
GMX answers the PUT of a reopened task with 204 and keeps it
completed, so uncomplete() seemed to work and did nothing.  A new
capability, task.uncomplete, follows caldav's
save-load.todo.uncomplete hint; where it is unsupported, uncomplete()
raises UnsupportedError and writes nothing.  The minimal memory profile
declares it unsupported, so the conformance suite runs that branch.

Prompt: [read the session handover notes] - we should make sure the scenario tests can be run towards all the caldav servers without failures/errors first, and after that the conformance tests

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Bedework lists a tasks/ collection in the calendar home and answers
404 when asked for its properties, so collections() failed on every
call.  A calendar that is listed but not found, as also one another
client deletes meanwhile, is now left out of the list.

Prompt: [read the session handover notes] - we should make sure the scenario tests can be run towards all the caldav servers without failures/errors first, and after that the conformance tests

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
CI runs the embedded servers only, so the hand-run results on caldav's
docker and external servers, with the reason for every skip and
error, now live in docs/caldav-servers.md, linked from CONTRIBUTING.

Prompt: Ok, do the upstream testing export fix.  Since the CI doesn't do the full integration run (yet), we don't have to wait for caldav 3.8 before releaseing (sic) calendaring 0.1, it suffices to write it out clearly somewhere what servers the tests are passing at, and what servers they are failing at and the reason.  Plus, importantly: I still want the tests to pass locally towards the master branch (or a bleeding-edge branch) on caldav.
Followup-Prompt: please try to rerun just the failed tests on ecloud, and let the purelymail tests run over the night
Followup-Prompt: try again with ecloud now.  What's the status of all the runs?

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Some servers keep events and tasks apart: CCS refuses a calendar
asked for both, and a task tracker holds tasks only.  A new backend
feature, collection.mixed, says so (on CalDAV from caldav's
...with-supported-component-types.multiple hint), and where it is
unsupported create_collection() makes a collection for one kind.  The
conformance suite then gives each test a collection for the kind it
writes, instead of failing in setup on CCS and Bedework 3.  That hint
is new in caldav 3.7, which the caldav extra now needs.

Prompt: "calendars for several components" is a capability.  Some systems will have separate task lists and calendars (be it caldav servers or other servers).  Some systems will not support tasks at all, and others will not support events at all.
Followup-Prompt: yes.  So the scenario tests have passed green on all servers except for eCloud? [approving the proposed design: a collection.mixed capability, one-kind create_collection(), and a conformance suite giving each test a collection for the kind it writes]

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
The stories say in comments what each step does.  The change-feed
story now also edits a task before the token is taken, which is old
news, and adds one after, which is not.  A new story moves a task to a
task list on another backend (local files), which is not atomic.

Prompt: Seems like we should add one more commit to the PR - please look into my changes and TODO-nodes (sic) in the scenario test file

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
@tobixen
tobixen merged commit 0536885 into main Oct 31, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant