Skip to content

Add a "Create a Dataset from a CSV" tutorial and make the getting-started tutorial read-only - #131

Draft
AntoninKns wants to merge 3 commits into
OCHA-DAP:mainfrom
AntoninKns:docs/tutorial-create-dataset
Draft

AntoninKns wants to merge 3 commits into
OCHA-DAP:mainfrom
AntoninKns:docs/tutorial-create-dataset

Conversation

@AntoninKns

Copy link
Copy Markdown
Contributor

This PR corresponds to the suggested tutorial rewrite mentioned in #129. A follow-up to the documentation restructure in #126.

Following the discussion in #129, where it was confirmed that the getting-started tutorial doesn't explain authorisation for writing, this PR proposes one way to cover that: separate reading from writing, and add a tutorial whose main part runs offline. I'm happy to adjust the approach.

What changed (one commit each, so they can be reviewed separately)

  1. Make the getting-started tutorial read-only to make scope clearer. Retitled "Read a Dataset"; the path is unchanged, so existing links keep working. Removed the API key step, the stage write configuration and the steps that modify datasets, renumbered the steps and fixed the Docker "jump to step" reference. A closing paragraph points to the new tutorial.
  2. Add "Create a Dataset from a CSV". It goes from a made-up CSV to a complete Dataset, ending just before create_in_hdx(). Steps: prepare the CSV, connect read-only, build the dataset, generate the resource with generate_resource, validate with check_required_fields, look at what would be sent, then explain where a dry run stops. A final section covers organisation members: token setup, rehearsal on demo or stage, then prod. It follows the naming and format guidance in the HDX pipeline recommendations. The sample CSV is in documentation/tutorials/assets/.
  3. Test the tutorial's code. tests/hdx/data/test_tutorial_create_dataset.py runs steps 3 to 6 offline with the existing fixtures.

How it was tested

  • I ran the code of steps 2 to 6 as written, from a clean folder, against prod in read-only mode, using the sample CSV.
  • The new test passes, and ruff check and format are clean. The rest of tests/hdx/data passes except test_datastore_queries, which also errored before this change and which I haven't investigated.
  • mkdocs build runs without warnings (uv run --with mkdocs-material --with mkapi mkdocs build -f documentation/mkdocs.yaml). The page renders under Tutorials.
  • The "Before you start" commands were checked in a fresh virtualenv with Python 3.14.4, installing from this checkout.
  • The error in step 7 (Access denied: User not authorised to create packages) was observed on demo with a valid token and no organisation.

Not tested: needs a maintainer

  • Section 8 ("For organisation members") needs publishing rights, which I don't have. Could you run those steps once on demo? If they work, the sentence saying they haven't been run end to end can be removed.
  • Not tested: Windows activation of the virtualenv, and installing the released PyPI package (I installed this checkout).

Questions

  • The tutorials use user_agent="tutorial_create_dataset" as an example name. Would you prefer a different one?
  • The test repeats the tutorial's code by hand. Should I change it to make it run the code blocks from the page directly?

Generated with Claude, fully reviewed and edited manually.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant