From eaf8a87a33bb6afd54432958f2de787d9e0326ba Mon Sep 17 00:00:00 2001 From: "Michael J. Williams" Date: Wed, 14 Oct 2026 13:46:01 +0200 Subject: [PATCH 1/5] DOC(docs): remove maintenance notes Remove the separate maintenance document and its README link. Keep the local setup, build, and validation instructions in the README. --- MAINTENANCE.rst | 284 ------------------------------------------------ README.md | 4 +- 2 files changed, 1 insertion(+), 287 deletions(-) delete mode 100644 MAINTENANCE.rst diff --git a/MAINTENANCE.rst b/MAINTENANCE.rst deleted file mode 100644 index 7fb8818..0000000 --- a/MAINTENANCE.rst +++ /dev/null @@ -1,284 +0,0 @@ -Setup and maintenance -===================== - -Local execution ---------------- - -Install Pixi 0.81.0 or newer, then run from the repository root:: - - pixi install --locked -e default -e docs -e check - pixi run --locked example gaussian - pixi run --locked example rosenbrock - pixi run --locked example adaptive-mutation - pixi run --locked -e docs docs - pixi run --locked -e check test - pixi run --locked -e check lint - pixi run --locked -e check format-check - pixi run --locked -e check pre-commit run --all-files - -The Makefile in ``docs/`` wraps these Pixi tasks. From the repository root:: - - make -C docs setup - make -C docs check - make -C docs example EXAMPLE=rosenbrock - make -C docs examples - make -C docs docs - make -C docs docs-only - make -C docs validate - -``make -C docs docs`` checks existing output completeness and provenance, then builds -the full Sphinx site. It reruns examples only if outputs are missing or stale. -``make -C docs validate`` runs checks, explicitly reruns all examples, then builds the -site. Both preserve execution order with ``make -C docs -j``. -You can also change into ``docs/`` and run these targets without ``-C docs``. ``make -C docs docs-only`` never -executes examples and fails if outputs are missing or stale. Use this target -when editing documentation on a machine with limited memory. Use ``make -C docs example EXAMPLE= ENV=`` for -a single example requiring another backend. ``make -C docs hooks`` runs all pre-commit -hooks and may edit files. ``make -C docs help`` lists the available targets. The -``SEQUINCE_INVENTORY`` environment override described below also works with make. - -``pixi run --locked examples`` runs every manifest entry using its declared -environment. Open ``docs/_build/html/index.html`` after a successful build. -The maintained narratives remain beside the scripts under ``examples/``. -The pipeline copies them and script sources into ``docs/_generated`` so relative ``literalinclude`` -paths still resolve. Both that directory and ``build-docs`` are ignored. - -The runner imports each script and overrides only its output-directory global. -It preserves the original working directory, parameters, sampler, seed and RNG -calls. For CLI examples the runner passes the script defaults without the -pipeline's own arguments. The three additional scripts write concise JSON -statistics only when the runner sets ``documentation_output``; ordinary script -execution keeps its existing output behavior. Matplotlib uses Agg. Raw particles and HDF5 results live in a temporary -run directory and are removed after extracting plots and weighted summaries. -Logs remain in ``build-docs``. The covariance in the summary is the weighted -population covariance; it does not apply a finite-sample correction. No target, -weighting, evidence convention or mutation setting is changed for publication. - -Dependencies and new examples ------------------------------ - -During development, Sequince follows the latest ``main`` branch in -``pyproject.toml``. Normal example commands refresh that dependency before -execution. ``make -C docs setup`` and ``make -C docs docs`` refresh it too; -``make -C docs docs-only`` uses the existing snapshot and never runs sampling. -Use ``make -C docs refresh`` to resolve main without executing examples. -Direct standalone Python script execution uses whichever Sequince is installed; -use the runner commands for automatic refresh. - -Actions resolves main once in the manifest job, shares the resulting -``pixi.lock`` with every matrix job, and uses locked installs afterward. This -keeps one workflow internally consistent even if main advances during the run. -``SEQUINCE_LOCK_READY=1`` is reserved for such already-resolved batches or -intentional reproduction of an archived execution; it suppresses refresh. - -``pixi.lock`` remains versioned as a bootstrap snapshot. Metadata verifies and -records the installed version and exact resolved Git commit. Each output bundle -includes ``execution-pixi.lock`` with checksums. RTD restores that archived lock -before validating outputs; it does not fetch today's main. The publishing job -allows this intentional lock-file change but rejects other tracked edits. - -Separate executions of the same example source commit can now produce different -results as Sequince main changes. Existing immutable-by-policy bundles still -cannot be overwritten with different bytes: publish a new example source commit -for a new dependency snapshot. To reproduce an old run, use its archived lock -and ``SEQUINCE_LOCK_READY=1``. This development policy supersedes the original -requirement that dependency resolution be identical for every run of a source -commit. When Sequince releases are available, replace the branch dependency with -the latest compatible release and revise the refresh command at the same time. - -The docs environment contains no sampler or optional backend. The separate -``flow`` environment uses CPU-only PyTorch and pins ``sequince-density`` to a -full Git commit in its repository (currently named ``sequince-flows``). Both Git -dependency commits and installed versions are verified and recorded for flow -runs. The separate ``jax`` environment is a CPU template for future examples. - -The Sequince repository is currently private and there is no public PyPI -release. Existing SSH access was used to resolve the local lock. To use existing -SSH access locally without changing Git configuration permanently:: - - GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=url.git@github.com:.insteadOf \ - GIT_CONFIG_VALUE_0=https://github.com/ pixi install --locked - -Before enabling this workflow for public/fork PRs, make the dependency -publicly readable (or replace it with a compatible exact public release and -regenerate the lock). Never add a private dependency token to PR execution. -RTD only installs Sphinx and reads public output assets, so it needs no Sequince -repository access or sampling environment. The Sequince documentation inventory -at ``https://sequince.readthedocs.io/en/latest/objects.inv`` must also be publicly -readable; current probes returned HTTP 403/404 in this environment. For local -validation against an inventory generated by the main Sequince Sphinx build, -set ``SEQUINCE_INVENTORY=/absolute/path/to/objects.inv``. This overrides only the -inventory location; generated API links still point to the main Sequince docs. - -Place new example directories under ``examples/`` and add a record to -``docs/examples.json`` with a unique ID, script, working -directory, environment, timeout in seconds, narrative, parameter globals, -required raw outputs and captioned figures. Scripts must expose ``main()`` and -``outdir``, save weighted particles in ``posterior.npz`` with ``particles``, -``weights`` and scalar ``log_evidence``, and list every required output. -Seed must be a declared parameter. Examples can provide a raw ``summary.json`` -instead of ``posterior.npz``; the runner validates and packages it without -rendering it in the tutorial. CLI examples expose ``build_parser()`` so metadata -records their actual defaults. Optional Git dependencies list their feature -names under ``dependency_features`` in the execution manifest. Add backend features/environments to -``pyproject.toml`` and regenerate the lock as needed. The execution matrix comes from the manifest; no workflow edit is needed for -another entry. Put figure directives, captions, and summary includes directly -in the example README using ``/_generated/results//`` paths. The staging -step copies the narrative, script, and validated outputs without creating a -separate result page. Link its staged README from ``docs/index.rst`` in reading -order. Narrative-only tutorials keep their ``README.rst`` beside the script in -``examples/``. Add their directory and source filename to the staging list in -``prepare()`` and link the staged README from ``docs/index.rst``. This preserves -relative source includes without executing optional backends or adding an -execution-manifest entry. Keep contributor and hosting instructions -in this repository document rather than the tutorial navigation. - -Provenance and durable bundles ------------------------------- - -Every example records the checked-out source SHA, hashes of its script and -narrative, manifest, runner and Pixi inputs, installed Sequince version/resolved main commit, -Python version, environment, parameters and seed. Per-example checksums are -validated before Sphinx. Local edited inputs are fingerprinted, but publication -requires a clean tracked checkout except for the resolved execution lock. Actions checks out the same SHA in each job -(including the synthetic merge commit on PRs) and downloads only same-run artifacts. - -Trusted main runs create ``docs-outputs-.tar.gz`` and a companion -``.sha256`` asset on the dedicated prerelease ``docs-output-`` in -**sequince-examples**, never the library repository. The archive includes plots, -concise summaries, provenance, checksums, ``execution-pixi.lock`` and -``bundle-manifest.json``. It excludes -raw chains, checkpoints and HDF5 files. Tar and gzip headers are deterministic. -Existing assets are downloaded and compared before reuse; different bytes fail -rather than replacing archived results. Draft releases allow recovery from a -partial upload. Do not move these tags or edit release assets. - -Retain release assets for as long as any corresponding RTD version may be -rebuilt. Actions artifacts expire after 14 days (logs after 7) and serve only -intermediate/PR validation. To regenerate, check out the exact source revision, -restore the bundle execution lock and set ``SEQUINCE_LOCK_READY=1`` before -executing examples, building docs and creating the bundle. A rerun -with different results intentionally fails publication: investigate the mismatch, -then publish a new source commit instead of replacing history. Hardware/BLAS -variation can change numerical or figure bytes despite a fixed seed. - -Read the Docs Community setup ------------------------------ - -These are manual maintainer steps; no external settings or credentials are -created by this change. - -#. Make sequince-examples and its output release assets public. Import the repo - at ``app.readthedocs.org`` as a separate Community project, with main as the - default branch and ``.readthedocs.yaml`` as its configuration. -#. Keep ``latest`` active/default. Disable automatic push/tag builds and PR - builds for this project. For a GitHub App integration, use automation rules - with an enabled **Trigger build for version** rule that never matches real - refs (for example a reserved ``^disabled-auto-build$`` ref), and remove other - build-trigger rules. Such rules gate webhook builds; explicit API builds - remain available. For legacy webhook integrations, disable automatic builds - on the integration. Verify no ordinary push starts a build before enabling - publication. Do not enable rules that activate ``docs-output-*`` tags. -#. In the main Sequince project's **Subprojects** settings attach this imported - project with alias ``examples``. Select the examples default version as - latest. Its URL is ``https://sequince.readthedocs.io/projects/examples/en/latest/``. - The main Sequince build system remains unchanged. A parent navigation link to - this URL can be added when the subproject is live. -#. Set GitHub Actions variable ``RTD_PROJECT`` to the imported project's slug - and secret ``RTD_TOKEN`` to an RTD Community API token. Tokens inherit their - user's permissions: use a dedicated maintainer account with access only to - this project where possible. RTD needs no GitHub publishing token. The workflow - uses the automatic GitHub token with ``contents: read`` for validation and - ``contents: write`` only for the trusted publishing job. Allow Actions to - create releases in the examples repository. No secrets enter PR jobs. -#. Merge the files, then run the workflow on main once dependency access is - ready. Inspect the release assets and monitored RTD result. Initial import - builds can fail until the first matching bundle is uploaded; rerun afterward. - -RTD runs standard Sphinx on Community, downloading and verifying the exact bundle -for ``git rev-parse HEAD`` in ``pre_build``. It never falls back to an older -bundle. A trigger for ``latest`` does not pin a commit: the actual checkout is -verified and must have its own matching bundle. Main freshness is checked before -Sphinx and in ``post_build``, before RTD's upload. - -The publishing job holds a non-cancelling concurrency lock until RTD finishes, -waits for any existing RTD builds, and checks current main before triggering. -Stale workflow runs skip triggering. A newer push during an older RTD build -causes the freshness hook to fail; its workflow then releases the lock and the -newer run can build. This prevents older workflow builds from overwriting newer -successful docs. Do not run concurrent manual builds or enable automatic builds: -RTD has no customizable upload hook or API commit-pinning guarantee. After a -monitor timeout or manual cancellation, inspect/drain RTD builds before retrying; -the next publishing job also waits for active builds. - -Tagged versions and stable --------------------------- - -Initially publish only main/latest. Do not automatically activate artifact tags -or a stable alias. For a curated example release, choose an immutable ``vX.Y.Z`` -tag on a main commit whose output bundle already exists; sync and activate that -version in RTD, then manually build it. The hook downloads the bundle for the -tag's actual checkout. Configure stable to the newest curated release only -when adopting this policy. Never retarget a release tag; artifact prereleases -are not example releases. The current workflow intentionally does not trigger -tag/stable builds or create semantic version tags. - -PRs and troubleshooting ------------------------ - -PRs execute the full matrix and build Sphinx, with downloadable HTML artifacts. -They do not publish releases, receive RTD credentials or create RTD previews. -Use the HTML artifact for review. RTD PR previews would require a separate -trusted bundle/authorization policy and are disabled here. - -For failed examples, inspect the named job and its log artifact, reproduce with -``pixi run --locked example ``, and check required filenames and the timeout. -Do not reduce particle counts or change seeds just to pass CI. For a missing -bundle, verify the full RTD checkout SHA has a successful publishing workflow -and matching release assets; execute that revision instead of selecting another -commit's outputs. A checksum/provenance failure means corruption or mixed inputs: -re-download or reproduce the exact locked revision. Missing figures fail before -Sphinx. Warnings (including missing Intersphinx inventories) fail the build. - -For RTD API failures check the project slug, account permissions, token, rate -limits and build logs. A main-advanced error is expected during rapid pushes; -check that the newest workflow eventually succeeds. A private dependency error -in Actions requires public dependency access, not credentials in PR jobs. - -Official integration references -------------------------------- - -* `Pixi setup action and cache/locked inputs `_ -* `GitHub release and asset API `_ -* `RTD build hooks `_ -* `RTD API v3, builds and authentication `_ -* `RTD webhook automation rules `_ -* `RTD subprojects `_ - -Prepared plot integrations --------------------------- - -The flow preconditioning, persistent sampling, and waste-free examples are now -configured for the matrix and durable bundles, with inline figure directives. -Their new integrations have not been executed or numerically validated locally. -The standalone scripts retain their particle counts, dimensions, seeds, fitting -budgets, and mutation settings. Some constants have moved to module scope so -the runner can record them. Documentation-only summary writes do not draw RNG. -Flow preconditioning uses its existing four-dimensional, 1,000-particle setup; -persistent sampling uses the parser defaults (four dimensions, 256 active -particles); waste-free SMC keeps 100 chains of length 100 and seed 123. - -Do not treat pre-existing loose PNG files as verified documentation outputs. -A full build requires fresh, matching provenance for all seven execution entries. -In particular, ``make -C docs docs`` may launch sampling when outputs are stale -or missing; ``make -C docs docs-only`` never launches it. The runner has timeouts -but no enforced memory ceiling yet. Validate the new runs on suitable hardware -or after adding memory limits. Dependency resolution and mocked worker tests -do not validate numerical behavior or plot quality. - -Particle filtering is also configured for automated execution, with its existing -defaults of 1,000 particles, 20 observations, seed 2026, and resampling after -every observation. The runner isolates its plot and captures accuracy, ESS, -and evidence diagnostics in the bundle without rendering numerical summaries. -This integration has mocked worker and layout checks; no filtering run was -performed while preparing it. diff --git a/README.md b/README.md index 61ebadb..b2c5a35 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,5 @@ Run `make -C docs check` for tests, lint, and formatting checks, or `make -C doc individual targets. New example runs refresh Sequince main; all jobs in a batch use the same resolved lock. Later, the dependency will switch to releases. -See [setup and maintenance](MAINTENANCE.rst) for dependency access, adding -examples, publication and Read the Docs Community subproject setup. Generated -results are ignored. Environments, dependencies, and tasks are defined in +Generated results are ignored. Environments, dependencies, and tasks are defined in `pyproject.toml`; `pixi.lock` is intentionally tracked for reproducibility. From 5274db198a86005292ded8c8436fc25a1dfb3b1a Mon Sep 17 00:00:00 2001 From: "Michael J. Williams" Date: Wed, 14 Oct 2026 13:53:11 +0200 Subject: [PATCH 2/5] CI(docs): prepare builds for released Sequince dependencies Resolve Sequince and Sequince Density from PyPI once per execution batch and validate the installed versions against the shared execution lock. Reject stale Git locks instead of falling back to development sources. --- .github/workflows/documentation.yml | 4 +- docs/Makefile | 6 +-- pyproject.toml | 4 +- scripts/docs_pipeline.py | 73 ++++++++++++++++++++++++----- tests/test_docs_pipeline.py | 68 +++++++++++++++++++++++++-- 5 files changed, 132 insertions(+), 23 deletions(-) diff --git a/.github/workflows/documentation.yml b/.github/workflows/documentation.yml index 697ac93..d4245d9 100644 --- a/.github/workflows/documentation.yml +++ b/.github/workflows/documentation.yml @@ -27,8 +27,8 @@ jobs: manifest-path: pyproject.toml run-install: false cache: false - - name: Resolve current Sequince main once for this workflow - run: pixi update sequince + - name: Resolve released dependencies once for this workflow + run: pixi update sequince sequince-density - uses: actions/upload-artifact@v4 with: name: execution-lock diff --git a/docs/Makefile b/docs/Makefile index e13c724..49f98a1 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -11,8 +11,8 @@ ENV ?= default help: @printf '%s\n' \ - 'make setup Refresh Sequince main and install environments' \ - 'make refresh Resolve current Sequince main without sampling' \ + 'make setup Refresh released dependencies and install environments' \ + 'make refresh Resolve released dependencies without sampling' \ 'make check Run tests, lint and formatting checks' \ 'make test Run pipeline tests' \ 'make lint Run Ruff lint checks' \ @@ -26,7 +26,7 @@ help: 'make validate Run checks, all examples, then build docs' refresh: - cd "$(REPO_ROOT)" && $(PIXI) update sequince + cd "$(REPO_ROOT)" && $(PIXI) update sequince sequince-density setup: refresh cd "$(REPO_ROOT)" && $(PIXI) install --locked -e default -e docs -e check diff --git a/pyproject.toml b/pyproject.toml index 32ea428..17729a7 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -18,7 +18,7 @@ requires-pixi = ">=0.81.0" python = "3.13.*" [tool.pixi.feature.cpu.pypi-dependencies] -sequince = { git = "https://github.com/sequince-dev/sequince.git", branch = "main", extras = ["corner", "hdf5"] } +sequince = { version = "*", extras = ["corner", "hdf5"] } [tool.pixi.feature.jax.pypi-dependencies] jax = ">0.5.3" @@ -30,7 +30,7 @@ JAX_PLATFORMS = "cpu" pytorch-cpu = "*" [tool.pixi.feature.flow.pypi-dependencies] -sequince-density = { git = "https://github.com/sequince-dev/sequince-flows.git", rev = "cbe32939713f17be2c5deb8f813ba1ec4550ea7e", extras = ["zuko"] } +sequince-density = { version = "*", extras = ["zuko"] } [tool.pixi.feature.docs.dependencies] sphinx = "8.2.*" diff --git a/scripts/docs_pipeline.py b/scripts/docs_pipeline.py index 29b3a97..18ad5bf 100644 --- a/scripts/docs_pipeline.py +++ b/scripts/docs_pipeline.py @@ -58,9 +58,9 @@ def entry(name: str) -> dict: raise ValueError(f"Unknown example: {name}") -def dependency_commit() -> str: +def dependency_commit() -> str | None: """Read Sequince's resolved commit from the execution lock.""" - return git_dependencies({})["sequince"] + return git_dependencies({}).get("sequince") def git_dependencies(example: dict) -> dict[str, str]: @@ -97,9 +97,50 @@ def git_dependencies(example: dict) -> dict[str, str]: return revisions +def resolved_versions(example: dict) -> dict[str, str]: + """Read exact package versions, rejecting stale Git locks for releases.""" + config = tomllib.loads((ROOT / "pyproject.toml").read_text())["tool"][ + "pixi" + ] + lock = (ROOT / "pixi.lock").read_text() + records = re.findall( + r"^- pypi: ([^\n]+)\n name: ([^\n]+)\n version: ([^\n]+)", + lock, + re.MULTILINE, + ) + versions = {} + for feature in ["cpu", *example.get("dependency_features", [])]: + for package, specification in ( + config["feature"][feature].get("pypi-dependencies", {}).items() + ): + matches = [ + (source, version) + for source, name, version in records + if name == package + ] + found = {version for _, version in matches} + if len(found) != 1: + raise ValueError( + f"Missing or inconsistent locked version: {package}" + ) + if not ( + isinstance(specification, dict) and "git" in specification + ): + if any("git+" in source for source, _ in matches): + raise ValueError( + f"Stale Git lock for {package}; run pixi update sequince sequince-density after the releases are available" + ) + versions[package] = found.pop() + return versions + + def refresh() -> None: - """Resolve current Sequince main once before a new execution batch.""" - subprocess.run(["pixi", "update", "sequince"], cwd=ROOT, check=True) + """Resolve the latest released dependencies once per execution batch.""" + subprocess.run( + ["pixi", "update", "sequince", "sequince-density"], + cwd=ROOT, + check=True, + ) def source_hashes(example: dict) -> dict[str, str]: @@ -121,16 +162,24 @@ def worker(name: str, raw: Path, destination: Path) -> None: example = entry(name) revisions = git_dependencies(example) - versions = {} - for package, revision in revisions.items(): + versions = resolved_versions(example) + for package, version in versions.items(): distribution = importlib.metadata.distribution(package) + if distribution.version != version: + raise ValueError( + f"Installed {package} version differs from the execution lock" + ) direct = json.loads(distribution.read_text("direct_url.json") or "{}") resolved = direct.get("vcs_info", {}).get("commit_id") - if resolved != revision: + if package in revisions: + if resolved != revisions[package]: + raise ValueError( + f"Installed {package} commit differs from the execution lock" + ) + elif "vcs_info" in direct: raise ValueError( - f"Installed {package} commit {resolved} != pinned {revision}" + f"Expected a release of {package}, found a Git installation" ) - versions[package] = distribution.version spec = importlib.util.spec_from_file_location( "documentation_example", ROOT / example["script"] ) @@ -197,7 +246,7 @@ def worker(name: str, raw: Path, destination: Path) -> None: "source_commit": git("rev-parse", "HEAD"), "source_hashes": source_hashes(example), "sequince_version": importlib.metadata.version("sequince"), - "sequince_commit": revisions["sequince"], + "sequince_commit": revisions.get("sequince"), "git_dependencies": revisions, "dependency_versions": versions, "python_version": sys.version.split()[0], @@ -294,6 +343,8 @@ def validate(base: Path | None = None) -> None: "source_commit": git("rev-parse", "HEAD"), "source_hashes": source_hashes(example), "sequince_commit": dependency_commit(), + "sequince_version": resolved_versions(example)["sequince"], + "dependency_versions": resolved_versions(example), "git_dependencies": git_dependencies(example), "environment": example["environment"], } @@ -305,7 +356,7 @@ def validate(base: Path | None = None) -> None: if metadata["seed"] != metadata["parameters"]["seed"]: raise ValueError(f"Seed mismatch: {directory}") versions = metadata.get("dependency_versions", {}) - if set(versions) != set(git_dependencies(example)) or not all( + if set(versions) != set(resolved_versions(example)) or not all( versions.values() ): raise ValueError( diff --git a/tests/test_docs_pipeline.py b/tests/test_docs_pipeline.py index c283211..4c71d44 100644 --- a/tests/test_docs_pipeline.py +++ b/tests/test_docs_pipeline.py @@ -13,6 +13,28 @@ sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts")) import docs_pipeline as pipeline # noqa: E402 +read_resolved_versions = pipeline.resolved_versions + + +@pytest.fixture(autouse=True) +def release_versions(monkeypatch: pytest.MonkeyPatch) -> None: + """Use synthetic release versions; no unavailable packages are resolved.""" + monkeypatch.setattr( + pipeline, + "resolved_versions", + lambda example: { + name: "test-version" + for name in [ + "sequince", + *( + ["sequince-density"] + if example.get("dependency_features") + else [] + ), + ] + }, + ) + @pytest.fixture def outputs(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: @@ -32,13 +54,10 @@ def outputs(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: "example": example["id"], "source_commit": "a" * 40, "source_hashes": pipeline.source_hashes(example), - "sequince_version": "0.1.dev1", + "sequince_version": "test-version", "sequince_commit": pipeline.dependency_commit(), "git_dependencies": pipeline.git_dependencies(example), - "dependency_versions": { - name: "test-version" - for name in pipeline.git_dependencies(example) - }, + "dependency_versions": pipeline.resolved_versions(example), "environment": example["environment"], "seed": 42, "parameters": {key: 42 for key in example["parameters"]}, @@ -225,6 +244,8 @@ def main() -> None: version="test-version", read_text=lambda filename: json.dumps( {"vcs_info": {"commit_id": revisions[package]}} + if package in revisions + else {} ), ), ) @@ -290,3 +311,40 @@ def test_execution_batch_refreshes_once( assert calls[0] == "refresh" assert len(calls) == len(pipeline.examples()) + 1 assert all(call["env"]["SEQUINCE_LOCK_READY"] == "1" for call in calls[1:]) + + +@pytest.mark.parametrize( + "source, version, error", + [ + ("https://files.pythonhosted.org/sequince.whl", "0.1.0", None), + ( + "git+https://github.com/sequince-dev/sequince@abc", + "0.1.0", + "Stale Git lock", + ), + (None, None, "Missing or inconsistent locked version"), + ], +) +def test_release_lock_provenance( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + source: str | None, + version: str | None, + error: str | None, +) -> None: + """Accept a release lock and reject missing or obsolete Git records.""" + monkeypatch.setattr(pipeline, "ROOT", tmp_path) + (tmp_path / "pyproject.toml").write_text( + "[tool.pixi.feature.cpu.pypi-dependencies]\n" + 'sequince = { version = "*" }\n' + ) + (tmp_path / "pixi.lock").write_text( + f"- pypi: {source}\n name: sequince\n version: {version}\n" + if source + else "" + ) + if error: + with pytest.raises(ValueError, match=error): + read_resolved_versions({}) + else: + assert read_resolved_versions({}) == {"sequince": version} From 05e82f68802a3797ee59d59bc83d0ac311e1ccd6 Mon Sep 17 00:00:00 2001 From: "Michael J. Williams" Date: Wed, 14 Oct 2026 13:53:12 +0200 Subject: [PATCH 3/5] DOC: simplify the README and add contribution guidelines Match the Sequince repository layout and move contributor setup, checks, and documentation build instructions into CONTRIBUTING.md. --- CONTRIBUTING.md | 200 ++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 59 +++++++------- 2 files changed, 233 insertions(+), 26 deletions(-) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..f7cd14f --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,200 @@ +# Contributing to Sequince examples + +Contributions are welcome, including tutorials, bug reports, and improvements to +existing examples. These guidelines follow the conventions of +[Sequince](https://github.com/sequince-dev/sequince/blob/main/CONTRIBUTING.md) +and describe the documentation workflow used here. + +## Community and responsibility + +Treat other contributors with respect and follow the +[Python Community Code of Conduct](https://www.python.org/psf/codeofconduct/). +Contributions must comply with the project's [license](LICENSE). Preserve +required attribution when incorporating third-party material. + +AI-assisted contributions are welcome under Sequince's +[AI policy](https://github.com/sequince-dev/sequince/blob/main/AI_POLICY.md). +Contributors remain responsible for understanding, validating, and explaining +all submitted content, including its scientific correctness and performance. + +## Scope of contributions + +Keep pull requests focused on one problem or a closely related set of changes. +Discuss substantial new dependencies or workflows in an issue first. Small fixes +can go directly to a pull request. Reusable sampling components generally belong +in Sequince; this repository demonstrates their use in concrete applications. + +## Development setup + +Fork `sequince-dev/sequince-examples`, clone your fork, and configure upstream: + +```bash +git clone git@github.com:YOUR_USERNAME/sequince-examples.git +cd sequince-examples +git remote add upstream https://github.com/sequince-dev/sequince-examples.git +git fetch upstream +git switch main +git branch --set-upstream-to=upstream/main main +git pull --ff-only +git switch -c fix-descriptive-name +``` + +Keep local `main` for upstream changes and develop on separate branches. Install +[Pixi](https://pixi.prefix.dev/latest/installation/), then run: + +```bash +make -C docs setup +pixi run --locked -e check pre-commit install +``` + +Dependencies, environments, and tasks are defined in `pyproject.toml`. +Unlike Sequince itself, this repository tracks `pixi.lock` for reproducible +example execution. Include intentional dependency updates with the corresponding +manifest changes; do not commit environments, caches, or generated results. + +The configuration requests released `sequince` and `sequince-density` packages +from PyPI, with no fallback to Git main. Until both releases are available, +setup and CI dependency resolution are expected to fail. The current lock is an +older Git snapshot. After release, regenerate it using +`pixi update sequince sequince-density`. Existing local check environments can +still run lightweight checks with `pixi run --frozen -e check test`; this does +not validate a release installation. + +## Code style and scientific correctness + +Follow existing module conventions. Use Ruff for formatting and import ordering, +four-space indentation, and a 79-character line limit. Use descriptive names and +NumPy-style docstrings for public functions, including array shapes and numerical +assumptions where relevant. Explain non-obvious choices rather than restating +code in comments. + +Add parameter and return annotations to new functions and helpers. Prefer modern +Python syntax, precise callback signatures, and existing backend-neutral types. +Keep optional imports optional and avoid broad type suppressions. + +Preserve array backends, dtypes, RNG behavior, and intended target distributions. +Explain intentional changes to weighting, evidence estimates, sampler settings, +or diagnostics. Use fixed seeds and justified tolerances for numerical tests. +Avoid retaining unnecessary histories or adding expensive diagnostics by default. + +## Checks and tests + +Before requesting review, run: + +```bash +make -C docs check +make -C docs hooks +``` + +The first command runs pytest, Ruff linting, and formatting checks. Pre-commit +also checks spelling, whitespace, YAML, and large files, and may modify files. +Review any edits and rerun the checks. Do not bypass hooks to make commits pass. +The configurations in `pyproject.toml` and `.pre-commit-config.yaml` are +authoritative; this repository does not currently configure mypy. + +Tests live in `tests/`. Test observable behavior and numerical invariants, using +small fixed inputs or mocked execution for pipeline tests. Some tests require +optional flow or particle-filtering dependencies. Report checks actually run, +including skipped tests and unavailable environments. Lightweight pipeline tests +do not establish that a complete sampling run succeeds. + +## Writing and building the documentation + +Keep scripts and their reStructuredText tutorials together under `examples/`. +Introduce the problem and sampling method before showing plots. Include figures +directly in the tutorial, with explanations and the relevant source code. The +reading order in `docs/index.rst` starts with basic examples, then introduces +other methods, with flow preconditioning before persistent sampling. + +Documentation commands run from the repository root: + +```bash +make -C docs help +make -C docs example EXAMPLE=gaussian +make -C docs example EXAMPLE=flow-preconditioning ENV=flow +make -C docs examples +make -C docs docs +make -C docs docs-only +make -C docs validate +``` + +`example` runs one manifest entry; `examples` runs all entries sequentially in +their declared environments. Each execution batch resolves released dependencies +once and uses the same lock throughout. `docs` refreshes dependencies, reuses +valid outputs when possible, and executes examples if outputs are missing or +stale. `docs-only` validates existing outputs and builds Sphinx HTML without +sampling. `validate` runs checks, reruns all examples, then builds the docs. +HTML is written to `docs/_build/html/`. + +Full example execution can consume substantial memory and time. Choose a single +example when developing, and use `docs-only` when valid results are available. +Execution has timeouts, but does not currently enforce a memory limit. Do not +assume that parallel local execution is safe. + +### Adding an example + +Add the script and tutorial under `examples/`, then register automated examples +in `docs/examples.json`. Declare the environment, settings, timeout, expected +outputs, and figure captions. Add the tutorial to `docs/index.rst`. + +Follow existing scripts' documentation-output adapters: write outputs to the +provided directory and keep ordinary script behavior intact. Register additional +dependencies in `pyproject.toml`, using a dedicated environment when needed. +Add focused tests for new pipeline behavior and verify the tutorial with Sphinx. + +The runner writes plots and metadata under `build-docs/outputs//`. +Validation checks the exact output set, checksums, source hashes, settings, +source commit, and dependency provenance before staging pages under +`docs/_generated/`. Changes to source files or the execution lock invalidate old +results. Do not edit metadata to make stale outputs pass, or commit generated +plots and staging directories. + +### CI and hosting + +The documentation workflow resolves one execution lock and shares it across +parallel example jobs. It collects outputs from that same workflow run, runs +checks, builds Sphinx, and uploads HTML and a validated output bundle. + +Trusted runs on `main` can attach the bundle to a dedicated GitHub release keyed +by the full source commit and trigger a Read the Docs build. Read the Docs +installs `docs/requirements.txt`, fetches the exact matching bundle, and verifies +it before building; it does not run sampling. Freshness checks guard `latest` +builds, and there is no fallback to results from another commit. + +Maintainers must configure the Read the Docs project, the repository variable +`RTD_PROJECT`, and the secret `RTD_TOKEN` before enabling this hosting workflow. +Keep credentials out of source files and pull request jobs. The configuration +uses standard Read the Docs hosting; private access requires a hosting service +and account configuration that supports it. + +## Commit messages + +Use Sequince's uppercase SciPy-style prefixes and optional lowercase scopes: + +```text +BUG(docs): reject stale example outputs +ENH(examples): add a particle filtering tutorial +DOC: clarify local documentation builds +CI(docs): collect example artifacts +BLD: update documentation dependencies +``` + +Other supported prefixes include `PERF`, `API`, `DEP`, `MAINT`, `TST`, `REL`, and +`STY`. Use `BUG` rather than `FIX`. Start the summary with a lowercase imperative +verb, omit a trailing period, and aim for at most 72 characters. Use one prefix +by default, or two separated by `/` when both are needed. Make coherent commits; +pull request titles should follow the same convention for squash merging. + +## Pull requests and review + +Push your contribution branch to your fork and open a pull request targeting +`sequince-dev/sequince-examples:main`. Use a draft for work in progress. + +Explain the problem and resulting behavior, link related issues, and describe +scientific assumptions or compatibility changes. Summarize validation with +commands and results, including sampling runs that were not performed and any +skips. For documentation changes, identify which pages and examples were built. + +Respond to reviewer feedback and keep the description aligned with the final +implementation. Maintainers should wait for applicable checks and review +conversations to be resolved, and check the squash commit message before merging. diff --git a/README.md b/README.md index b2c5a35..e3ace05 100644 --- a/README.md +++ b/README.md @@ -1,30 +1,37 @@ # Sequince examples -Example scripts and their narrative tutorials live together under `examples/`. +Tutorials demonstrating Bayesian inference and particle filtering with Sequince. +Example scripts and their explanations live together under `examples/`. + Start with the [basic examples](examples/basic_examples/README.rst), then explore [adaptive mutation](examples/adaptive_mutation_steps/README.rst), -[flow preconditioning](examples/preconditioning/README.rst), and -[persistent sampling](examples/persistent_sampling/README.rst). The documentation -also covers waste-free SMC and particle filtering. - -The documentation pipeline is configured to execute Gaussian, Rosenbrock, -adaptive mutation, flow preconditioning, persistent sampling, and waste-free -SMC and particle-filtering examples using the latest Sequince main, resolved -once per execution batch. Their plots appear directly in the tutorial pages. - -```bash -pixi install --locked -e default -e docs -pixi run --locked examples -pixi run --locked -e docs docs -``` - -Use `make -C docs setup` followed by `make -C docs docs` to build the full documentation. -Valid example outputs are reused; missing or stale outputs trigger example -execution. `make -C docs docs-only` never executes examples. Use `make -C docs validate` to run -checks, rerun all examples, and build the documentation. -Run `make -C docs check` for tests, lint, and formatting checks, or `make -C docs help` for -individual targets. New example runs refresh Sequince main; all jobs in a batch -use the same resolved lock. Later, the dependency will switch to releases. - -Generated results are ignored. Environments, dependencies, and tasks are defined in -`pyproject.toml`; `pixi.lock` is intentionally tracked for reproducibility. +[flow preconditioning](examples/preconditioning/README.rst), +[persistent sampling](examples/persistent_sampling/README.rst), +[waste-free SMC](examples/waste_free_smc/README.rst), and +[particle filtering](examples/particle_filtering/README.rst). + +## Documentation + +The tutorials are collected in the [documentation](docs/index.rst), with plots +from the example scripts. See the [contributing guidelines](CONTRIBUTING.md) +for instructions on building the documentation locally. + +## Contributing + +Bug reports, feature requests, and pull requests are welcome via the GitHub +repository. Please see the [contributing guidelines](CONTRIBUTING.md) +and [AI policy](https://github.com/sequince-dev/sequince/blob/main/AI_POLICY.md). + +## Related repositories + +The following repositories are part of the `sequince` ecosystem: + +- [`sequince`](https://github.com/sequince-dev/sequince): the main sampling framework +- [`sequince-density`](https://github.com/sequince-dev/sequince-density): density estimation and preconditioning +- [`sequince-bilby`](https://github.com/sequince-dev/sequince-bilby): interfaces for using Sequince samplers with Bilby +- [`orng`](https://github.com/sequince-dev/orng): cross-backend random number generation +- [`acai`](https://github.com/sequince-dev/acai): array-API compatible control flow and iteration + +## License + +Licensed under the [MIT License](LICENSE). From a5506c6388381232a7a26d1c87a2e481e5b8cc03 Mon Sep 17 00:00:00 2001 From: "Michael J. Williams" Date: Wed, 14 Oct 2026 13:54:03 +0200 Subject: [PATCH 4/5] DOC: link to documentation instead of listing tutorials --- README.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/README.md b/README.md index e3ace05..1c88729 100644 --- a/README.md +++ b/README.md @@ -3,13 +3,6 @@ Tutorials demonstrating Bayesian inference and particle filtering with Sequince. Example scripts and their explanations live together under `examples/`. -Start with the [basic examples](examples/basic_examples/README.rst), then explore -[adaptive mutation](examples/adaptive_mutation_steps/README.rst), -[flow preconditioning](examples/preconditioning/README.rst), -[persistent sampling](examples/persistent_sampling/README.rst), -[waste-free SMC](examples/waste_free_smc/README.rst), and -[particle filtering](examples/particle_filtering/README.rst). - ## Documentation The tutorials are collected in the [documentation](docs/index.rst), with plots From 8a2027670d023fa9523c7cd60296c5cf2e10e4a7 Mon Sep 17 00:00:00 2001 From: "Michael J. Williams" Date: Wed, 14 Oct 2026 15:41:59 +0200 Subject: [PATCH 5/5] DOC: add repository instructions for coding agents --- AGENTS.md | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..9a8c3b5 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,40 @@ +# Agent instructions + +Before making changes, read and follow [CONTRIBUTING.md](CONTRIBUTING.md) and +Sequince's [AI policy](https://github.com/sequince-dev/sequince/blob/main/AI_POLICY.md). +CONTRIBUTING.md defines development setup, code style, typing, validation, +commit-message conventions, and the documentation workflow. The AI policy +applies to AI-assisted contributions to this repository. + +## Working in the repository + +- Inspect the working tree before editing. Preserve existing work and do not + overwrite or revert unrelated changes. Stage only files relevant to the task. +- Keep changes focused and follow existing interfaces and helpers. Avoid + unrelated refactors and unnecessary dependencies or abstractions. +- Keep scripts and their reStructuredText tutorials together under `examples/`. + Register automated examples in `docs/examples.json` and update the reading + order in `docs/index.rst` when adding pages. +- Dependencies and environments belong in `pyproject.toml`. This repository + intentionally tracks `pixi.lock`; change it only for intentional dependency + updates. Preserve local environments, caches, and generated results, and + keep them out of commits. + +## Scientific correctness and validation + +- Identify changes that affect target distributions, weighting, evidence + estimates, RNG behavior, sampler settings, or backend compatibility. Explain + intentional changes and support them with appropriate tests. +- Prefer lightweight pipeline tests and small fixed numerical inputs during + development. Full examples can consume substantial memory. Respect execution + constraints from the user; commands such as `make -C docs docs` can trigger + sampling when outputs are missing or stale. `docs-only` builds from existing + validated outputs without sampling. +- Preserve output provenance checks. Do not edit metadata or weaken validation + to reuse stale results. Keep generated plots directly in tutorial pages using + the existing staging workflow. +- Follow CONTRIBUTING.md for checks. Report commands actually run and their + results, including failures, skips, sampling runs not performed, and checks + that could not be completed. Do not describe skipped checks as passing. +- Do not bypass hooks or disable checks to make a commit succeed. Address the + failure, or report the blocker when it cannot be resolved within the task.