Repository navigation
Conversation
- Prefetch only the ids of a risk acceptance's accepted findings. - Treat an empty file name as a missing file in the stream and the manifest. - Return a 400, not a 500, for a max_file_bytes value with too many digits. - Build the finding prefetch objects per queryset, not once at import. - Resolve the endpoint mode and status serializer once per serializer. - Let the database return distinct scan types. - Build the end counts from COUNT_KEYS. - Name the export file after the product id, not the raw URL value.
The inherited request_response field decoded the pairs with errors="ignore", dropped blank lines and stopped at DD_MAX_REQRESP_FROM_API. Finding lines now carry every stored pair as request_response_pairs, in id order, with the base64 text unchanged. File entries carry the stored file name, so the extension survives. A non-finite float renders as null instead of ending the stream, the product id route accepts ASCII digits only, and the stream response tells nginx not to buffer it.
The stream seed now puts a note and a file on the engagement and on the test, and stores a request/response pair with non-UTF-8 bytes and a blank line. The tests read those values back from the lines, and the end counts are checked against the seeded totals, so a dropped note, file or byte fails the suite. One seed helper runs once per class from setUpTestData. Tests that the stream and manifest tests already cover end to end are removed, and the performance tests no longer import from the export test module.
A DRF IntegerField now parses max_file_bytes and documents it in the schema. The two routes sit in api_v2_urls beside the other plain API views, so dojo/export/api/urls.py goes. The single-caller queryset and tally helpers are inlined, the reference tables come from one serializer map, both endpoint modes share one loop, and one file_size helper serves the stream and the manifest. The views rely on IsSuperUser alone, and the finding query no longer joins tables that no serializer reads.
One counts table now compares the manifest with end.counts, and the nesting rules, use cases and endpoint examples say each thing once. The page now lists what the export leaves out (endpoints without findings, JIRA keys, members, presets, configurations, contacts, import history and more), says when install_id changes, says that a risk acceptance line names one engagement, gives the line size and memory use a reader must plan for, and says that token.expires_at is null under session or basic authentication.
An empty max_file_bytes value uses the default cap. The docs now say so, and a test pins it. The manifest-to-stream comparison runs at the default cap and at a cap of 4 again, and checks files_over_cap and finding_groups. The docs also name the report type and CI/CD infrastructure ids that stay on engagement lines.
svader0
added
enhancement
New Migration
|
This pull request modifies sensitive codepaths in
Configured Sensitive Codepath Modified by Non-Allowed Author in
|
| Vulnerability | Configured Sensitive Codepath Modified by Non-Allowed Author |
|---|---|
| Description | File 'dojo/db_migrations/0301_system_settings_install_id.py' matches configured sensitive codepath pattern 'dojo/db_migrations/*.py' and was modified by 'svader0' (commit de75317) who is not in the allowed authors list. |
Configured Sensitive Codepath Modified by Non-Allowed Author in dojo/urls.py (drs_606c52df)
| Vulnerability | Configured Sensitive Codepath Modified by Non-Allowed Author |
|---|---|
| Description | File 'dojo/urls.py' matches configured sensitive codepath pattern 'dojo/urls.py' and was modified by 'svader0' (commit db063e1) who is not in the allowed authors list. |
Comment to provide feedback on these findings.
Report false positive: @dryrunsecurity fp [FINDING ID] [FEEDBACK]
Report low-impact: @dryrunsecurity nit [FINDING ID] [FEEDBACK]
Example: @dryrunsecurity fp drs_90eda195 This code is not user-facing
All finding details can be found in the DryRun Security Dashboard.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Two superuser-only endpoints export one product with everything under it, so a product can be backed up, moved to another DefectDojo instance, or mirrored.
GET /api/v2/export/manifest/: counts per product type and product, reference data (SLA configurations, development environments, regulations, test types, note types, product platforms, lifecycles and origins, users with contact info), the distinct scan types in use, the token's privilege, and the instance'sinstall_id.GET /api/v2/export/products/{id}/: anapplication/x-ndjsonstream. Oneheaderline, then every engagement, test, finding and risk acceptance of the product as its own line in id order, then anendline with counts.What the lines carry:
dataobject is the existing v2 serializer output for that object. Only the fields that were ids are swapped: a finding'sendpointsare inlined with their status (endpoint status, or location reference status when Locations is on),fileson engagements, tests and findings are inlined as base64 under amax_file_bytescap (default 10 MiB; a larger file keeps its name and size withdata_base64null), request/response pairs cross as their stored base64, risk acceptance ids replace the nestedaccepted_risks, and a risk acceptance line carries its proof file and its notes.System_Settings.install_idis a new read-only UUID (migration0301_system_settings_install_id). The header and the manifest carry it so a tool can tell two instances apart.How it stays flat: each collection is walked with
iterator(chunk_size=500)plus per-chunk prefetches, one serializer instance per line type, one JSON line at a time into aStreamingHttpResponse. The manifest runs a fixed number of grouped queries.Known limits:
install_idregenerates if the initializer re-runs first boot (the admin account missing). That path also resets every other system setting. A separate fix follows.Migrations: this PR and #16386 both add a
0301migration onSystem_Settings. Whichever merges second renumbers to0302and points its dependency at the other.Test results
unittests/test_apiv2_export.py(35 tests) andunittests/test_apiv2_export_performance.py(3 tests), run with Locations on and off. One test pins the query count per finding batch (17 with Locations on, 16 off), and a second fails on any query per row.test_rest_framework,test_apiv2_methods_and_endpoints,test_system_settingsandtest_apiv2_usergreen.makemigrations --checkclean,spectacular --fail-on-warnsilent, ruff clean.Documentation
docs/content/automation/api/product_export.mddocuments both endpoints, the line shapes, the counts, and what does not cross (secrets and instance settings, endpoints with no finding, JIRA links, members and groups; the page lists all of them).