Skip to content

DAPI responses are not compressed (rs-dapi defers to Envoy, Envoy has no compressor) #5576

Closed as not planned

Description

@PastaPastaPasta

Summary

DAPI responses, including proofs, are currently sent uncompressed. Neither rs-dapi nor the Envoy gateway compresses them, but the code says Envoy does:

  • packages/rs-dapi/src/clients/drive_client.rs:64: "Compression (gzip) is intentionally DISABLED at rs-dapi level; Envoy handles it."
  • packages/dashmate/templates/platform/gateway/envoy.yaml.dot: the HTTP filter chain is cors → grpc_web → ratelimit → router, with no envoy.filters.http.compressor.
  • tonic's gzip/zstd features and send_compressed / accept_compressed are not enabled in rs-dapi, dapi-grpc, or rs-dapi-client.

Defacto raised this in Slack. I measured how much it would actually save.

Measurement

I extracted the raw grovedb_proof bytes from the recorded SDK test vectors (packages/rs-sdk/tests/vectors, at 371a3b3): 162 proofs, 116 unique, all ≤ 4.4 KB. Each one was compressed on its own, the way gRPC compresses messages.

Slice Savings (zstd -3)
All unique proofs 27%
Data contract proofs 52% (brotli -q 11 reaches ~60%)
Everything else 18%
Proofs < 1 KB (identity, balance, nonce, keys, token status) 8–10%; a few grow slightly
Proofs 1–4 KB ~25%

gzip -6, zstd -3 and brotli -q 5 land within a couple of points of each other, so the codec choice matters little. The savings come from embedded payloads (contract schemas, document bodies, vote-poll keys). The 32-byte Merk hashes barely compress.

Caveats: these vectors come from small test networks. Mainnet trees are deeper, so hash-heavy proofs will compress somewhat less, while large document queries will likely compress more. Response metadata and the 96-byte signature were not included.

Proposal

  1. Add an Envoy compressor filter (gzip, and zstd/brotli if wanted) with a min_content_length threshold of roughly 1 KB, so small hash-only responses skip it. This also covers grpc-web/browser clients. Alternatively, enable tonic compression in rs-dapi.
  2. Make sure clients advertise grpc-accept-encoding / Accept-Encoding: rs-dapi-client (native and tonic-web-wasm-client) and js-dapi-client.
  3. Fix or remove the misleading comment in drive_client.rs.
  4. Check evonode CPU cost under load before turning it on by default.

@shumkov could you take a look and improve this?

Activity

  1. PastaPastaPasta commented on Oct 25, 2026

    @PastaPastaPasta
    MemberAuthor

    Superseded by #5577, which implements this directly (rs-dapi gzip responses, native client support, Envoy grpc-web compressor) with mainnet measurements in the description.


    🤖 Posted autonomously by Claude on behalf of pasta.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions