ci: move system tests from Github Actions to Testing Farm by pbrezina · Pull Request #9304 · SSSD/sssd

pbrezina · 2026-09-21T11:16:36Z

The system tests can quickly deplete available Github runners (20)
and we need them for process purposes (Accepted label). So we keep
only quick checks in GHA now.

Testing Farm is supported infrastructure for Fedora Project, we should
be able to see infra issues resolved more quickly as we can actually
ping people when needed.

pbrezina · 2026-09-22T07:52:49Z

This is waiting for new feature in packit:

And SSSD/sssd-ci-containers#219

pbrezina · 2026-09-22T11:05:56Z

Note: Do not forget to switch sssd-ci-containers repo in the test plan after SSSD/sssd-ci-containers#219 is merged.

pbrezina · 2026-09-22T12:10:41Z

/packit build
/packit check

pbrezina · 2026-09-24T07:25:18Z

/packit build
/packit check

pbrezina · 2026-09-29T17:36:22Z

/packit check

packit-as-a-service · 2026-09-29T17:36:26Z

There appears to be a syntax error in the command provided. Please refer to the Packit documentation or use the /packit help command.

pbrezina · 2026-09-29T19:03:26Z

/packit test

pbrezina · 2026-09-30T07:01:44Z

/packit test

pbrezina · 2026-09-30T10:11:26Z

I get this failure on centos-10... it is a segfault in autofs, I can't reproduce this locally so I'm trying to catch it in reserved TF run.

FAILED tests/test_autofs.py::test_autofs__new_map_entries_added_to_provider_are_visible_after_reload (ldap) - pytest_mh.conn.ssh.SSHProcessTimeoutError: 
Command #19113 timed out after 300 seconds:
  Command:
    set -ex
    pushd "/var/export/export2"
    mount | grep "nfs.test:/dev/shm/exports/export2 on /var/export/export2"
    popd
    umount "/var/export/export2"
  CWD:
  Env:
  Output:
  Error output:
    + pushd /var/export/export2

Other two errors are in provisioning phase. Both looks like a network error.

On TF side, reported, they will implement a retry

[2026-09-30 07:04:47] [+] [worker_0] [guest-setup-order] Running guest setup for fedora-copr-build artifact
[2026-09-30 07:04:47] [+] [worker_0] [fedora-copr] Initialized with 11049366:fedora-rawhide-x86_64: package 'sssd' build '11049366' target 'fedora-rawhide-x86_64' (https://copr.fedorainfracloud.org/coprs/packit/SSSD-sssd-9304/build/11049366/)
[2026-09-30 07:04:47] [E] [Fedora-latest:x86_64:/tmt/plans/system-tests] guest setup failed: 'utf-8' codec can't decode byte 0xe2 in position 4642988: invalid continuation byte

Other one gets further but fails when downloading containers. I implemented a retry on our side.

unable to copy from source docker://quay.io/sssd/ci-keycloak:fedora-45: writing blob: storing blob to file "/var/tmp/container_images_storage484318116/2": happened during read: unexpected EOF (while reconnecting: Get "https://cdn01.quay.io/quayio-production-s3/sha256/5e/5e45f5cd39f75b56f8265cfad98cc04c5ca0c13ec20bc28b6b87dc757e650c0d?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260930%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260930T070847Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=f65e83168abafa3d93496c56b822de2a33b720be40388ce4189d8f19cb740a45&region=us-east-1&namespace=sssd&repo_name=ci-keycloak&akamai_signature=exp=1790753027~hmac=acd19ced820e2b607fc7b6017192e006e942755280af2463b297ac9bda1be5b3": EOF)
        Error: executing /usr/libexec/docker/cli-plugins/docker-compose up --no-recreate --detach: exit status 1

pbrezina · 2026-09-30T11:06:11Z

/packit retest-failed

pbrezina · 2026-09-30T14:51:24Z

/packit retest-failed

alexey-tikhonov · 2026-10-19T17:39:29Z

Looks like Check polarion metadata step is dropped -- is this intentional?

pbrezina · 2026-10-20T12:03:24Z

Looks like Check polarion metadata step is dropped -- is this intentional?

No, good catch. I wanted to add it to static analyzers but forgot. Done.

alexey-tikhonov · 2026-10-20T13:27:34Z

f-45: "Request was canceled on user request." -- why?

pbrezina · 2026-10-20T13:39:08Z

f-45: "Request was canceled on user request." -- why?

Dunno. I can see there is SIGTERM in setup (TF not ours). I asked, waiting for answer.

alexey-tikhonov · 2026-10-20T15:23:29Z

Perhaps not related to this PR, but looking at logs (even at Rawhide):

(1) why there are so many SKIPPED tests of different kinds? Is this expected?

(2) stdout or stderr leaks to pytest output?

tests/test_smartcard_pinpad.py::test_smartcard_pinpad__su_as_local_user (client) SKIPPEDr — rc=0, out=Available slots:
Slot 0 (0x0):
Virtual PCD 00 00
  (empty)
Slot 1 (0x4): Virtual PCD 00 01
  (empty),
err=)

alexey-tikhonov · 2026-10-20T17:18:54Z

@alexey-tikhonov alexey-tikhonov closed this 2 hours ago

Sorry, it was accidentally.

thrix · 2026-10-20T22:39:27Z

@alexey-tikhonov the packit issue is being discussed here:

https://redhat-internal.slack.com/archives/C04MU29TES1/p1792506940634379

Most probably some race, but did not yet get full answer

pbrezina · 2026-10-21T10:59:55Z

Perhaps not related to this PR, but looking at logs (even at Rawhide):

(1) why there are so many SKIPPED tests of different kinds? Is this expected?

It's not error in the test run, these tests are really skipped and they call pytest.skip intentionally. If they should or not, is different question and should not belong to this PR.

(2) stdout or stderr leaks to pytest output?

No. This is a skip reason from detect_pinpad_reader. But malformed. For some reason, other skip reasons are not printed. I did not look into it yet, but I do not think it should prevent us moving forward with this PR. It is not caused by it, it is present in GH as well.

tests/test_smartcard_pinpad.py::test_smartcard_pinpad__su_as_local_user (client) SKIPPEDr — rc=0, out=Available slots:
Slot 0 (0x0):
Virtual PCD 00 00
  (empty)
Slot 1 (0x4): Virtual PCD 00 01
  (empty),
err=)

alexey-tikhonov · 2026-10-21T11:06:26Z

these tests are really skipped and they call pytest.skip intentionally. If they should or not, is different question and should not belong to this PR.

I wanted to double check the reason isn't some difference in TF env.
But it's the same vs gh:

$ grep "SKIPPED" job-logs.txt | wc -l
36

$ grep "SKIPPED" output.txt | wc -l
36

Feel free to merge.

This enables automatic collection of coredumps, failing test if the coredumps occurs in any SSSD binary. Other coredumps will be ignored if test is passed. All coredumps are collected if test fails. Reviewed-by: Justin Stephenson <jstephen@redhat.com>

The system tests can quickly deplete available Github runners (20) and we need them for process purposes (Accepted label). So we keep only quick checks in GHA now. Testing Farm is supported infrastructure for Fedora Project, we should be able to see infra issues resolved more quickly as we can actually ping people when needed. Reviewed-by: Justin Stephenson <jstephen@redhat.com>

This is a quick job and we can run it immediately without waiting for the rpms to be built. Reviewed-by: Justin Stephenson <jstephen@redhat.com>

sssd-bot · 2026-10-21T13:48:39Z

The pull request was accepted by @pbrezina with the following PR CI status:

🟢 CodeQL (success)
🟢 osh-diff-scan:fedora-rawhide-x86_64:upstream (success)
🟢 rpm-build:centos-stream-10-x86_64:upstream (success)
🟢 rpm-build:fedora-43-x86_64:upstream (success)
🟢 rpm-build:fedora-44-x86_64:upstream (success)
🟢 rpm-build:fedora-45-x86_64:upstream (success)
🟢 rpm-build:fedora-rawhide-x86_64:upstream (success)
🟢 testing-farm:centos-stream-10-x86_64:system (success)
🟢 testing-farm:fedora-43-x86_64:system (success)
🟢 testing-farm:fedora-44-x86_64:system (success)
🟢 testing-farm:fedora-45-x86_64:system (success)
🟢 testing-farm:fedora-rawhide-x86_64:system (success)
🟢 Build / freebsd (success)
🟢 Build / make-distcheck (success)
➖ Coverity scan / coverity (skipped)
🟢 Static code analysis / codeql (success)
🟢 Static code analysis / files-changed (success)
🟢 Static code analysis / polarion (success)
🟢 Static code analysis / pre-commit (success)
🟢 Static code analysis / python-system-tests (success)

There are unsuccessful or unfinished checks. Make sure that the failures are not related to this pull request before merging.

pbrezina force-pushed the tft branch 3 times, most recently from ef7f7e8 to 400131b Compare September 21, 2026 12:08

This was referenced Sep 21, 2026

ci: add tmt scripts that can be reused in tmt plans SSSD/sssd-ci-containers#219

Merged

Add skip_install option to tests job packit/packit#2775

Closed

pbrezina force-pushed the tft branch from 400131b to 38c1ebb Compare September 21, 2026 14:17

pbrezina force-pushed the tft branch from 38c1ebb to 8c7620a Compare September 22, 2026 11:04

pbrezina force-pushed the tft branch 2 times, most recently from afee380 to 482ba1f Compare September 22, 2026 11:25

alexey-tikhonov requested a review from justin-stephenson September 24, 2026 12:56

alexey-tikhonov assigned justin-stephenson Sep 24, 2026

pbrezina force-pushed the tft branch 2 times, most recently from b8273c4 to fa6197b Compare September 25, 2026 09:59

justin-stephenson reviewed Sep 25, 2026

View reviewed changes

Comment thread tmt/plans/system-tests.fmf Outdated

justin-stephenson reviewed Sep 25, 2026

View reviewed changes

Comment thread tmt/plans/system-tests.fmf Outdated

justin-stephenson reviewed Sep 25, 2026

View reviewed changes

Comment thread tmt/plans/system-tests.fmf Outdated

justin-stephenson reviewed Sep 25, 2026

View reviewed changes

Comment thread tmt/plans/system-tests.fmf Outdated

pbrezina force-pushed the tft branch 2 times, most recently from 66e45be to 752e0b5 Compare September 29, 2026 09:27

pbrezina added backport-to-sssd-2-10 backport-to-sssd-2-9-4 Corresponds to C8S backport-to-sssd-2-12 backport-to-sssd-2-13 labels Oct 19, 2026

alexey-tikhonov reviewed Oct 19, 2026

View reviewed changes

Comment thread tmt/readme.md Outdated

pbrezina force-pushed the tft branch 2 times, most recently from 340b5b0 to c7d2537 Compare October 20, 2026 12:28

alexey-tikhonov closed this Oct 20, 2026

alexey-tikhonov reopened this Oct 20, 2026

pbrezina added the Accepted label Oct 21, 2026

pbrezina added 3 commits October 21, 2026 13:48

ci: add polarion metadata check to static analyzers

963f937

This is a quick job and we can run it immediately without waiting for the rpms to be built. Reviewed-by: Justin Stephenson <jstephen@redhat.com>

sssd-bot force-pushed the tft branch from c7d2537 to 963f937 Compare October 21, 2026 13:48

pbrezina merged commit d76af0f into SSSD:master Oct 21, 2026
9 checks passed

Conversation

pbrezina commented Sep 21, 2026

Uh oh!

pbrezina commented Sep 22, 2026

Uh oh!

pbrezina commented Sep 22, 2026

Uh oh!

pbrezina commented Sep 22, 2026

Uh oh!

pbrezina commented Sep 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pbrezina commented Sep 29, 2026

Uh oh!

packit-as-a-service Bot commented Sep 29, 2026

Uh oh!

pbrezina commented Sep 29, 2026

Uh oh!

pbrezina commented Sep 30, 2026

Uh oh!

pbrezina commented Sep 30, 2026

Uh oh!

pbrezina commented Sep 30, 2026

Uh oh!

pbrezina commented Sep 30, 2026

Uh oh!

Uh oh!

alexey-tikhonov commented Oct 19, 2026

Uh oh!

pbrezina commented Oct 20, 2026

Uh oh!

alexey-tikhonov commented Oct 20, 2026

Uh oh!

pbrezina commented Oct 20, 2026

Uh oh!

alexey-tikhonov commented Oct 20, 2026

Uh oh!

alexey-tikhonov commented Oct 20, 2026

Uh oh!

thrix commented Oct 20, 2026

Uh oh!

pbrezina commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexey-tikhonov commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sssd-bot commented Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pbrezina commented Oct 21, 2026 •

edited

Loading

alexey-tikhonov commented Oct 21, 2026 •

edited

Loading