Repository navigation
Conversation
Add rocrate-validator to the dev dependencies and CI, with tests that build a crate and check it against the ro-crate-1.2 and process-run-crate-0.5 profiles (offline, REQUIRED level), plus a check that a missing licence is caught. The real-app e2e workflow now runs the same validation, in its own virtual environment because rocrate-validator requires rich<14 while current Flower requires rich>=14. Pin the RO-Crate version written by build_crate to 1.2 and require rocrate>=0.15: ro-crate-py 0.16 changed its default to RO-Crate 1.3, so the version flwrCrate wrote depended on which ro-crate-py was installed.
elichad
left a comment
elichad
left a comment
There was a problem hiding this comment.
Looks good, minor suggestions only!
| **settings, | ||
| )) | ||
| issues = [f"{i.check.identifier}: {i.message}" for i in result.get_issues()] | ||
| issues += [f"{e.identifier} (extra): {e.message}" for e in result.extra_checks] |
There was a problem hiding this comment.
Noting that e.message isn't always as helpful as we'd like it to be - came across this recently in eScienceLab/RO-Crate-Profile-Portal#64 (comment) which also suggests an alternative.
There was a problem hiding this comment.
extra thought - yesterday I was thinking that while this is on the "testing only" side, the exact format is less important, but if we put the validation output in front of users later, it will be more important to make sure all the information is definitely there using the TextOutputFormatter thing.
But actually for ease of development it's probably beneficial to have the full error output here as well.
There was a problem hiding this comment.
Good point, thanks. Both the tests and this script now use the validator's own text report (TextOutputFormatter, wrapped in a report() helper), so a failure shows the entity, property and value behind each issue for a missing licence, and a table for extra-check findings. The script still prints the PASS/FAIL summary first, then the full report for any profile that fails. #14 can reuse the same helper for user-facing output.
| def _validate(crate_dir, profile, **settings): | ||
| return services.validate(services.ValidationSettings( | ||
| rocrate_uri=str(crate_dir), | ||
| profile_identifier=profile, | ||
| requirement_severity=models.Severity.REQUIRED, | ||
| offline=True, | ||
| extra_checks=True, | ||
| **settings, | ||
| )) | ||
|
|
||
|
|
||
| def _issues(result): | ||
| """Readable list of failures and extra-check findings, shown if an assertion fails.""" | ||
| lines = [f"{i.check.identifier}: {i.message}" for i in result.get_issues()] | ||
| lines += [f"{e.identifier} (extra): {e.message}" for e in result.extra_checks] | ||
| return "\n".join(lines) |
There was a problem hiding this comment.
this looks very similar to https://github.com/eScienceLab/flwrCrate/pull/25/changes#diff-a4f60deb7907c632852386785aa779892c62b34ac017421d00c4079711b22a6aR16-R25 - could make a shared utils.py module with validation helper function(s)?
Will also be useful if we want to use it in crate_builder.py later (when the compatibility issues are solved)
There was a problem hiding this comment.
Done: flwrcrate/validation.py now has validate() and report(), used by both tests/test_validation.py and tests/e2e/validate_crate.py. I named it validation.py rather than utils.py, since it only holds validation helpers and is where #14's runtime validation will go.
Move validate() into flwrcrate/validation.py, used by both the tests and the e2e script, and add report(), which uses rocrate-validator's TextOutputFormatter so failures show the entity, property and value behind each issue.
Adds rocrate-validator to the test setup and CI, and validates generated crates against RO-Crate 1.2 and Process Run Crate.
Changes
roc-validator>=0.13added to thedevextra and installed intests.yml.tests/test_validation.py: builds a crate and checks it againstro-crate-1.2andprocess-run-crate-0.5at REQUIRED level, plus a test that a missing licence is reported.realapps.yml/tests/e2e/validate_crate.py) now runs the same validation on a crate from an actual Flower run.rocrateversion, the Validation section and the development setup updated; a duplicated comment intest_integration.pyremoved.Validation runs with
offline=True, which since 0.13 reads only the validator's HTTP cache, so CI fills the cache first withrocrate-validator cache warm. The profile is named explicitly, because recent versions default to RO-Crate 1.3. Process Run Crate is checked against the 0.5 profile withdisable_inherited_profiles_issue_reporting=True, as suggested, since the validator has no 0.6 profile yet.The opt-in checks added in 0.13 are enabled as well (
extra_checks=True). Their findings don't affectpassed(), so the tests assert separately that there are none. Our crates currently have none.RO-Crate version is now pinned
ro-crate-py 0.16.0 (released 6 Oct) changed its default output to RO-Crate 1.3. flwrCrate never set the version, so what it wrote depended on the installed ro-crate-py: 1.1 with 0.13/0.14, 1.2 with 0.15, 1.3 with 0.16. The e2e crate from 8 October was already RO-Crate 1.3, and the new 1.2 test failed under ro-crate-py 0.16.
build_cratenow usesROCrate(version="1.2"), and the minimum is raised torocrate>=0.15(older versions don't acceptversion=). This also affects users of the released 0.4.0.Dependency conflict to be aware of
rocrate-validator requires
rich<14, while Flower 1.32+ (including the latest, 1.41) requiresrich>=14, so they can't be installed in the same environment:rich14. I'm working on a PR to add that support upstream.All 54 tests pass locally and in CI (Python 3.11–3.14, rocrate 0.16, roc-validator 0.13.0), and so does the real-app e2e.
Closes #15