Skip to content

feat: support dataset and user-provided bindings - #885

Draft
daanjo3 wants to merge 1 commit into
ldelements:mainfrom
daanjo3:feat/pipeline-dataset-custom-bindings
Draft

daanjo3 wants to merge 1 commit into
ldelements:mainfrom
daanjo3:feat/pipeline-dataset-custom-bindings

Conversation

@daanjo3

@daanjo3 daanjo3 commented Oct 7, 2026

Copy link
Copy Markdown

One of the issues raised in development of the LDE-based LODA pipeline is the need to provide constant bindings from either the dataset description or just purely static.

Changes

This PR aims to resolve this by making two changes:

  1. adding an additional injectValuesOuter method that adds another VALUES expression to bind the following predefined dataset bindings: ?dataset, ?datasetPublisher, ?datasetPublisherName, ?datasetLicense, ?datasetLanguage.
  2. providing a DatasetBindingsProvider that can be passed as a StageOption to provide user-defined bindings based on the dataset and distribution. It is merged in the same VALUES expression as the dataset bindings.

Consideration / open questions

  1. Originally I combined the new "custom" bindings in the main value bindings produced by the selector. However these end up in the innermost subquery. This would not be ideal however as most of the user-defined bindings are likely to end up in construct statements, and would then need to be passed outside the subquery first, making for a very verbose query.
  2. The new dataset bindings logic is in pipeline/src/sparql/datasetBindings.ts, I'm not sure this is entirely the right place.
  3. Currently only a subset of the dataset properties are added as default bindings, perhaps for completeness sake it is best to just add all the properties.
  4. The current implementation results in cross-products for multiple dataset languages or language-specific names of the publisher. I'm assuming that the deduplicate logic should be helping in avoiding this from polluting the data, but I haven't been able to verify this yet.

Draft notice

I'm in the process of running tests with this version on the LODA pipeline to verify that it works properly.

fixes #884

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Substitute Dataset properties into SPARQL queries

1 participant