Repository navigation
Add opt-in archive size and entry count limits to PaArchive - #822
Open
Crash Collison (tehcrashxor) wants to merge 1 commit into
Open
Crash Collison (tehcrashxor) wants to merge 1 commit into
Crash Collison (tehcrashxor) wants to merge 1 commit into
Conversation
Callers that need to bound the resources used when opening archives can now pass PaArchiveLimits to PaArchive, MsappArchive, MsappArchiveFactory or AddMsappArchiveFactory(limits). When set, opening an existing archive in Read or Update mode checks the sizes and entry count recorded in the zip central directory before any entry is read, and throws PaArchiveLimitExceededException (PersistenceErrorCode 5005) when a limit is exceeded. Create mode is not checked. - PaArchive.GetStats() and PaArchiveLimits.Evaluate() can be used without limits. - When the stream is seekable, the entry count declared in the end of central directory record (including Zip64) is checked before ZipArchive reads the central directory. - Update mode is checked with a separate read-only pass, so an archive that exceeds the limits is left unchanged. Behavior is unchanged for callers that don't pass limits. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issue: N/A
Problem
Some callers need to bound the resources used when they open
.msapparchives, for example to keep memory and processing within service limits. TodayPaArchivehas no option for this, so those callers wrapIMsappArchiveFactorywith their own checks of the zip central directory. That duplicates archive-format knowledge outside this library.Solution
Add opt-in limits to
PaArchive. When a caller passesPaArchiveLimits, opening an existing archive inReadorUpdatemode checks the sizes and entry count recorded in the zip central directory, before any entry is read. It throwsPaArchiveLimitExceededExceptionwhen a limit is exceeded.Createmode is never checked. Callers that don't pass limits see no change in behavior, includingMsappArchiveFactory.DefaultandAddMsappArchiveFactory().The raw stats and the evaluator are public and work without limits. A caller can open an archive normally, then call
GetStats()andEvaluate()to observe how archives compare to the limits before enforcing them.Changes
Microsoft.PowerPlatform.PowerApps.Persistence.Compression:PaArchiveLimits: a record withMaxExpandedSize,MaxCompressedSize,MaxCompressionRatio,RatioCheckMinExpandedSize,MaxEntrySize,MaxEntryCountandMaxTextEntrySize.Defaultholds recommended values: 4 GiB expanded, 1 GiB compressed, ratio 250 above 100 MiB expanded, 512 MiB per entry, 10,000 entries, and 256 MiB per text entry.Defaultis only used when a caller passes it..json,.yaml,.yml,.xmlor.txt.PaArchiveLimits.Evaluate(PaArchiveStats)returns the first limit exceeded. The order isEntryCount,CompressedSize,ExpandedSize,EntrySize,TextEntrySize,CompressionRatio. It returnsNonewhen no limit is exceeded. A value equal to a limit is allowed.PaArchiveStatsis a record withCompressedSize,ExpandedSize,EntryCount,LargestEntrySizeandLargestTextEntrySize, plusFromEntries(...). Totals saturate instead of overflowing.PaArchiveLimitenum.PaArchiveLimitExceededException, a subclass ofPersistenceLibraryException, exposesLimit,StatsandLimits.PersistenceErrorCode.PaArchiveLimitExceeded = 5005.PaArchiveLimits? limitsonPaArchiveandMsappArchive. The existing constructors are unchanged and chain to them. Both types get aLimitsproperty.MsappArchiveFactory: a new init-onlyLimitsproperty, used asnew MsappArchiveFactory(logger) { Limits = limits }. The factory keeps its single constructor, so dependency injection still builds a factory without limits, even whenPaArchiveLimitsis registered in the container.PaArchive.GetStats()computes stats from all raw zip entries. That includes the directory, duplicate and invalid-path entries thatEntriesskips. It doesn't loadEntries.ZipArchivereads the central directory. If the record is missing or can't be read,ZipArchivehandles the archive as it does today.ZipArchivepass. An archive that exceeds the limits is never opened inUpdatemode and is left unchanged. When a constructor with limits throws, the stream is disposed unlessleaveOpenis true.AddMsappArchiveFactory(PaArchiveLimits)overload. It replaces any existingIMsappArchiveFactoryregistration, so the limits apply even ifAddMsappPackingService()was called first.EntriesandGetStats()..github/copilot-instructions.md, including fixing the archive error-code range to 5xxx.Validation
dotnet build src/PASopa.sln: passed, 0 warnings, 0 errors.dotnet test --no-build --project src/Persistence.Tests/Persistence.Tests.csproj(net48, net8.0, net10.0): 1137 passed, 3 failed.MsappPackingServiceTests.PackFromMsappReferenceFile_*cases. They throwDirectoryNotFoundExceptionbecause my local worktree path pushes their test output paths past MAX_PATH (271 chars).dotnet test --no-build --project src/Persistence.Tests/Persistence.Tests.csproj -f net48 -p:ArtifactsPath=%TEMP%\pat\ --filter "FullyQualifiedName~PackFromMsappReferenceFile"reported 6 passed, 0 failed. They aren't related to this change.dotnet pack --no-build -c Debug src/Persistence/Microsoft.PowerPlatform.PowerApps.Persistence.csproj: passed.PaArchiveLimitsTests, plus additions toMsappArchiveTests, cover:EntriesReadandUpdatemodesMsappArchiveFactory.Update(path)leaveOpenhandlingCreatemode and emptyUpdatestreams not being checkedPaArchiveLimitsbut still resolves a factory without limits