Skip to content

FAIR Signposting and detached RO-Crate - #1390

Draft
stain wants to merge 9 commits into
ElixirTeSS:masterfrom
stain:fair-signposting
Draft

stain wants to merge 9 commits into
ElixirTeSS:masterfrom
stain:fair-signposting

Conversation

@stain

@stain stain commented Oct 22, 2026 •

Copy link
Copy Markdown

Summary of changes

  • Adds FAIR Signposting to Material and Event pages, both in the HTTP Link header and as <link> elements in the HTML <head>:
    • describedby → Bioschemas JSON-LD (.jsonld, with the Bioschemas profile), Detached RO-Crate (with profile="https://w3id.org/ro/crate") and JSON-API
    • cite-as → the DOI, normalised to https://doi.org/… (also from doi:, bare 10.…, http://doi.org/, http://dx.doi.org/)
    • author → authors' ORCIDs, license → SPDX licence URL, type → schema.org type + AboutPage
  • Adds a Detached RO-Crate 1.3 for each Material and Event, generated from the existing Bioschemas generators (lib/ro_crate_export/):
    • GET /materials/:id/ro-crate-metadata.json and GET /events/:id/ro-crate-metadata.json
    • or content negotiation on the page URL: Accept: application/ld+json; profile="https://w3id.org/ro/crate" (plain application/ld+json still returns Bioschemas as before)
    • Root Data Entity is the TeSS page URL, typed Dataset + the Bioschemas type; nested objects are flattened; the Bioschemas profile is declared via conformsTo; properties not defined by the RO-Crate context (contact, eligibility, event_types, hostInstitution) are left out of the crate.
  • Fixes an interoperability problem with Rails' own asset preload Link entries, which have unquoted values (integrity=sha256-…=) and a valueless nopush. Strict parsers (e.g. httplink, used by the Python signposting library) reject the whole header because of these, including any signposting links. On pages with signposting the values are now quoted and nopush dropped; the preload hints are kept.
  • Tests:
    • Rails: test/integration/fair_signposting_test.rb, test/unit/ro_crate_export_identifier_test.rb (new code has 100% line coverage)
    • Python integration tests in test/signposting/ using the signposting library as an independent client, plus rocrate-validator (no REQUIRED issues). These run against a live TeSS server, so they are opt-in and not part of CI. Happy to discuss adding them to test.yml if wanted.
  • Docs: new section in docs/api.md.

The design follows SEEK's FairSignposting concern.

Motivation and context

TeSS already publishes Bioschemas markup, but machine clients have no standard way to discover it or other representations from a record's URL. FAIR Signposting provides that discovery through typed links, and it is what tools such as the RO-Crate retrieval algorithm, F-UJI and other FAIR assessment tools look for. A Detached RO-Crate makes each TeSS record consumable by RO-Crate tooling (WorkflowHub, rocrate-validator, ro-crate-py/js/ruby). It is also a first step towards packaging learning paths and collections as RO-Crates.

Points for review:

  • cite-as uses the material's DOI. That DOI usually resolves to the material itself (e.g. Zenodo), not to the TeSS record, so it is used for Signposting but deliberately not as cite-as on the crate's Root Data Entity. RO-Crate 1.3 requires a root cite-as to resolve to the crate itself.
  • Events have no licence in TeSS, so their crates carry a textual licence statement saying none has been specified, which RO-Crate allows.
  • The crate route uses its own ro_crate action. This avoids it being picked when Rails generates URLs for action: :show.
  • The public pages on tess.elixir-europe.org are currently behind a Cloudflare bot challenge, which also blocks non-browser clients from the Link headers and .jsonld. Signposting will only be useful to machines if the challenge allows these requests, which is a deployment question.

Screenshots

    curl -I http://localhost:3000/materials/training-material-example
    curl http://localhost:3000/materials/training-material-example/ro-crate-metadata.json
    curl -H 'Accept: application/ld+json; profile="https://w3id.org/ro/crate"' http://localhost:3000/materials/training-material-example

Example Link header (test fixtures, preload links omitted):

    <http://localhost:3000/materials/training-material-example.jsonld> ; rel="describedby" ; type="application/ld+json" ; profile="https://bioschemas.org/profiles/TrainingMaterial/1.0-RELEASE",
    <http://localhost:3000/materials/training-material-example/ro-crate-metadata.json> ; rel="describedby" ; type="application/ld+json" ; profile="https://w3id.org/ro/crate",
    <http://localhost:3000/materials/training-material-example.json_api> ; rel="describedby" ; type="application/vnd.api+json",
    <http://dx.doi.ac.uk/2q3093032> ; rel="cite-as",
    <https://spdx.org/licenses/CC-BY-4.0.html> ; rel="license",
    <https://schema.org/LearningResource> ; rel="type",
    <https://schema.org/AboutPage> ; rel="type"

Example RO-Crate

image image

Checklist

  • I have read and followed the CONTRIBUTING guide.
  • I confirm that I have the authority necessary to make this contribution on behalf of its copyright owner and agree to license it to the TeSS codebase under the BSD license.

Notes from Claude

What it does

  • Signposting on material and event pages. Links go in the HTTP Link header and as <link> elements in the HTML head:
  • A Detached RO-Crate 1.3 at /materials/:id/ro-crate-metadata.json and /events/:id/ro-crate-metadata.json. It's also served on the page URL to requests whose Accept header has application/ld+json; profile="https://w3id.org/ro/crate". The crate is built from the existing Bioschemas output.
  • Commits:
    a. the crate generator and DOI helper
    b. the Signposting concern and routes
    c. the Python tests and a CI workflow
    d. the docs in docs/api.md

Tests

  • New Rails tests: 13 pass, and the new code has 100% line coverage.
  • Python signposting tests: 11 pass against a running TeSS loaded with the test fixtures. They cover the Link header, the HTML links (which must match the header), following describedby to the crate, content negotiation, DOI rewriting, and rocrate-validator. The validator finds no REQUIRED issues.
  • Full Rails suite: 4 failures on the branch, none caused by this change. Upstream master, run in the same environment, fails 2 of the same tests.
    • The 2 failures that also happen on master come from this database's C collation, which sorts upper case first.
    • The other 2 are the order-dependent MaterialTest tests from the issue text above. They pass when run alone.
  • Signposting workflow in CI: I added it, but it hasn't run yet. Pushing to your fork may start it there if Actions is enabled.

Things you should know

  • The Python tests found a real bug. Rails' own preload links in the Link header have unquoted integrity=…= values and a bare nopush. Strict parsers (httplink, which signposting uses) then reject the whole header, Signposting links included. On these pages the values are now quoted and nopush is dropped; the preload hints themselves stay.
  • cite-as points at the material's DOI, which isn't a TeSS record. The DOI usually resolves to Zenodo or similar, not to the TeSS page. I kept it in the header as you asked, but left it out of the crate root: RO-Crate requires a root cite-as to resolve to the crate itself.
  • Four Bioschemas properties are left out of the crate: contact, eligibility, event_types and hostInstitution. The RO-Crate context doesn't define them; they stay in the Bioschemas JSON-LD.
  • Events have no licence in TeSS. Their crates carry a text licence statement saying none is specified, which RO-Crate allows.
  • I hit the routing bug I described and fixed it. The first version pointed the crate route at show, which broke 40 existing tests. The crate now has its own ro_crate action.

Claude Sandbox and others added 6 commits October 22, 2026 21:33
RoCrateExport::DetachedCrate builds an RO-Crate 1.3 metadata document
for a single resource from its Bioschemas representation: the TeSS
page URL is the Root Data Entity (typed Dataset plus the Bioschemas
type), nested objects are flattened into the @graph, the Bioschemas
profile is declared with conformsTo, and properties not defined by the
RO-Crate context are left out.

RoCrateExport::Identifier normalises DOIs (bare, doi:, http(s)://doi.org/,
dx.doi.org/) to https://doi.org/ URLs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Material and event pages now advertise FAIR Signposting links, in the
HTTP Link header and as <link> elements in the HTML head:
describedby for the Bioschemas JSON-LD, the Detached RO-Crate and the
JSON-API; cite-as for the DOI; author for ORCIDs; license; and type.

The Detached RO-Crate is served at /materials/:id/ro-crate-metadata.json
and /events/:id/ro-crate-metadata.json, and on the page URL by content
negotiation for application/ld+json with the RO-Crate profile.

Rails' own asset preload links in the Link header have unquoted
attribute values (base64 integrity hashes) and a valueless nopush,
which make strict Link header parsers reject the whole header. These
values are now quoted and nopush is dropped where signposting is added.

Modelled on SEEK's FairSignposting concern.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Python tests in test/signposting use the signposting library as an
independent client against a running TeSS: they check the Link header
and HTML <link> elements of material and event pages, follow
describedby to the Detached RO-Crate, check content negotiation and
DOI rewriting, and validate the crates with rocrate-validator.

A GitHub Actions workflow loads the test fixtures, starts TeSS and runs
them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep the Python signposting tests as an opt-in suite run against a
running TeSS, rather than adding a second CI job that duplicates the
setup of the main test workflow. CI integration can be discussed with
the maintainers separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Signposting Sniffing browser extension only looks at the first rel
of a Link header and ignored the whole header when it started with
Rails' asset preload links. Emit the signposting links first and keep
the preload links after them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@stain
stain marked this pull request as draft October 22, 2026 22:24
Claude Sandbox and others added 3 commits October 22, 2026 22:30
The Root Data Entity is now a plain Dataset for the TeSS record, with
TeSS as publisher, instead of the resource's Bioschemas type mixed into
the Dataset. The Material or Event is the root's mainEntity, with its
Bioschemas properties, profile and sdPublisher, identified by its
external URL (or a local identifier if it has no valid URL) rather than
a plain url string. A Material is also a data entity of the crate
(hasPart); an Event is not.

Give SPDX licences as SPDX license URIs (http://spdx.org/licenses/<id>),
which match SPDX's RDF, in the crate and the Signposting license link,
instead of the HTML page from the licence dictionary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…page

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved metadata exposure and unsafe Link serialization block approval, alongside licence handling, negotiation and test assertion defects.

6 open issues
What changed in this PR

Adds machine-readable discovery and Detached RO-Crate exports for TeSS materials and events, building on the existing Bioschemas generators.

Changes:

  • Publishes FAIR Signposting links in HTTP headers and HTML, while preserving asset preload hints.
  • Adds RO-Crate endpoints, profile-based content negotiation, and identifier normalization.
  • Documents the interfaces and adds Rails tests plus opt-in Python interoperability checks.
File Description
test/​unit/​ro_crate_export_identifier_test.rb Tests DOI and licence URL normalization.
test/​signposting/​test_signposting.py Checks discovery and crate interoperability against a live server.
test/​signposting/​requirements.txt Defines Python test dependencies.
test/​integration/​fair_signposting_test.rb Tests signposting, exports, negotiation and access control.
test/​controllers/​about_controller_test.rb Checks developer documentation content.
lib/​ro_crate_export/​identifier.rb Normalizes DOI and licence identifiers.
lib/​ro_crate_export/​detached_crate.rb Builds flattened Detached RO-Crate graphs.
docs/​api.md Documents discovery and export interfaces.
config/​routes.rb Adds material and event crate routes.
app/​views/​layouts/​_head.html.erb Renders HTML signposting links.
app/​views/​about/​developers.erb Adds discovery and export examples.
app/​controllers/​materials_controller.rb Integrates material signposting and crate negotiation.
app/​controllers/​events_controller.rb Integrates event signposting and crate negotiation.
app/​controllers/​concerns/​fair_signposting.rb Shares link generation, header handling and crate rendering.
app/​controllers/​application_controller.rb Allows anonymous crate requests, retaining action-level authorization.
.gitignore Ignores Python bytecode artifacts.

Review effort: Balanced

return if links.empty?

signposting = links.map do |url, attributes|
(["<#{url}>"] + attributes.map { |key, value| %(#{key}="#{value}") }).join(' ; ')
# "Accept: application/ld+json; profile="https://w3id.org/ro/crate"".
def ro_crate_requested?
profile = Regexp.escape(RoCrateExport::DetachedCrate::PROFILE)
request.headers['Accept'].to_s.match?(%r{application/ld\+json\s*;[^,]*profile="?#{profile}(?=["\s,;]|\z)}i)
# An array of [url, { attribute => value }] pairs.
def fair_signposting_links
resource = fair_signposting_resource
return [] unless resource&.persisted?
'name' => main['name'],
'description' => main['description'],
'datePublished' => date_published,
'license' => licence || NO_LICENCE,
def self.licence_url(licence)
return nil if licence.blank?

reference = LicenceDictionary.instance.lookup_value(licence, 'reference').presence
crate['@graph'].each do |entity|
assert entity['@type'].present?, "#{entity['@id']} has no @type"
entity.except('@id', '@type').each_value do |value|
Array(value).grep(Hash).each do |ref|
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants