Skip to content

Latest commit

 

History

6,960 Commits

Folders and files

Repository files navigation

The low-code Knowledge Graph application platform

LinkedDataHub (LDH) is open source software you can use to manage data, create visualizations and build apps on RDF Knowledge Graphs.

LinkedDataHub rendering a SKOS concept with its multilingual labels

Version 6 redraws the interface on a design system that ships with the platform: the app shell, content blocks, action bar, property lists, tables, modals and forms all render one class vocabulary, and ldh.css is the single app layer a dataspace stylesheet overrides. Bootstrap 2 is gone with the bs2: template modes — the CHANGELOG has what that means for an application stylesheet written against it.

LinkedDataHub 6 overview

We started the project with the intention to use it for Linked Data publishing, but gradually realized that we've built a multi-purpose data-driven platform.

We are building LinkedDataHub primarily for:

  • researchers who need an RDF-native FAIR data environment that can consume and collect Linked Data and SPARQL documents and follows the FAIR principles
  • developers who are looking for a declarative full stack framework for Knowledge Graph application development, with out-of-the-box UI and API

What makes LinkedDataHub unique is its completely data-driven architecture: applications and documents are defined as data, managed using a single generic HTTP API and presented using declarative technologies. The default application structure and user interface are provided, but they can be completely overridden and customized. Unless a custom server-side processing is required, no imperative code such as Java or JavaScript needs to be involved at all.

XHTML documents can be edited in-place using a built-in RDFa-aware rich text editor — annotations link selected text directly to Knowledge Graph terms, embedding machine-readable RDF statements in the markup without leaving the page.

Follow the Get started guide to LinkedDataHub. The setup and basic configuration sections are provided below and should get you running.

LinkedDataHub is also available as a free AWS Marketplace product! AWS Marketplace
It takes a few clicks and filling out a form to install the product into your own AWS account. No manual setup or configuration necessary!

Setup

Click to expand

Prerequisites

CLI

The ldh command line interface is attached to every release and needs only a Java 21 runtime; building it from source additionally requires Maven. The certificate and WebID scripts that remain in the bin/ directory require openssl and keytool (part of the JDK).

Steps

  1. Fork this repository and clone the fork into a folder

  2. In the folder, create an .env file and fill out the missing values (you can use .env_sample as a template). For example:

    COMPOSE_CONVERT_WINDOWS_PATHS=1
    COMPOSE_PROJECT_NAME=linkeddatahub
    
    PROTOCOL=https
    HTTP_PORT=81
    HTTPS_PORT=4443
    HOST=localhost
    
    OWNER_MBOX=john@doe.com
    OWNER_GIVEN_NAME=John
    OWNER_FAMILY_NAME=Doe
    OWNER_ORG_UNIT=My unit
    OWNER_ORGANIZATION=My org
    OWNER_LOCALITY=Copenhagen
    OWNER_STATE_OR_PROVINCE=Denmark
    OWNER_COUNTRY_NAME=DK
    
  3. Setup server's SSL certificates by running this from command line:

    ./bin/server-cert-gen.sh .env nginx ssl

    The script will create an ssl sub-folder where the SSL certificates and/or public keys will be placed.

  4. Create the following secrets with certificate/truststore passwords:

    • secrets/client_truststore_password.txt
    • secrets/owner_cert_password.txt
    • secrets/secretary_cert_password.txt
    • secrets/openai_api_key.txt — optional, only if you enable the assistant: an OpenAI API key, declared in docker-compose.override.yml like the OAuth secrets; without it the assistant runs plans but cannot write them The one you will need to remember in order to authenticate with LinkedDataHub using WebID client certificate is owner_cert_password.
  5. Launch the application services by running this from command line:

    make up -- --build

    make up passes its arguments on to docker-compose up. The -- is required before any argument starting with -, otherwise make treats it as one of its own options.

    It will build LinkedDataHub's Docker image, start its container and mount the following sub-folders:

    • ssl
      • owner stores root owner's WebID certificate, keystore, and public key
      • secretary stores root application's WebID certificate, keystore, and public key
      • server stores the server's certificate (also used by nginx)
    • data where the triplestore(s) will persist RDF data
    • datasets where LDH persists agent metadata files
    • uploads where LDH stores content-hashed file uploads It should take up to half a minute as datasets are being loaded into triplestores. After a successful startup you should see periodic healtcheck requests being made to the https://localhost:4443/ns URL.
  6. Install ssl/owner/keystore.p12 into a web browser of your choice (password is the owner_cert_password secret value)

    • Google Chrome: Settings > Advanced > Manage Certificates > Import...
    • Mozilla Firefox: Options > Privacy > Security > View Certificates... > Import...
    • Apple Safari: The file is installed directly into the operating system. Open the file and import it using the Keychain Access tool (drag it to the local section).
    • Microsoft Edge: Does not support certificate management, you need to install the file into Windows. Read more here.
  7. For authenticated API access use the ssl/owner/cert.pem HTTPS client certificate with curl, or the ssl/owner/keystore.p12 keystore beside it with the ldh CLI. If you are running Linux with user other than root, you might need to fix the certificate permissions because Docker bind mounts are owned by root by default. For example:

    sudo setfacl -m u:$(whoami):r ./ssl/owner/*
  8. Open https://localhost:4443/ in the web browser or use the API, for example:

    curl -k -E ./ssl/owner/cert.pem:<your cert password> -H "Accept: text/turtle" 'https://localhost:4443/'
    ldh get -c ./ssl/owner/keystore.p12 -p <your cert password> --accept text/turtle 'https://localhost:4443/'

Notes

  • There might go up to a minute before the web server is available because the nginx server depends on healthy LinkedDataHub and the healthcheck is done every 20s

  • You will likely get a browser warning such as Your connection is not private in Chrome or Warning: Potential Security Risk Ahead in Firefox due to the self-signed server certificate. Ignore it: click Advanced and Proceed or Accept the risk to proceed.

    • If this option does not appear in Chrome (as observed on some MacOS), you can open chrome://flags/#allow-insecure-localhost, switch Allow invalid certificates for resources loaded from localhost to Enabled and restart Chrome
  • MacOS: Chrome subdomain support: Chrome on macOS requires the server certificate to be installed to the System keychain to properly load resources from dataspace subdomains (e.g., admin.localhost:4443). Firefox is more lenient and will work without this step.

    1. Open Keychain Access (Applications > Utilities > Keychain Access)
    2. Select System keychain in the left sidebar
    3. File → Import Items → select ssl/server/server.crt
    4. Enter your admin password when prompted
    5. Double-click the "localhost" certificate
    6. Expand the Trust section
    7. Set "When using this certificate:" to Always Trust
    8. Close the window (enter password again)
    9. Completely quit Chrome (Cmd+Q) and restart

    Alternatively, use the command line:

    sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ssl/server/server.crt
  • .env_sample and .env files might be invisible in MacOS Finder which hides filenames starting with a dot. You should be able to create it using Terminal however.

  • On Linux your user may need to be a member of the docker group. Add it using

sudo usermod -aG docker ${USER}

and re-login with your user. An alternative, but not recommended, is to run

sudo make up

Configuration

Click to expand

Base URI

A common case is changing the base URI from the default https://localhost:4443/ to your own.

Lets use https://ec2-54-235-229-141.compute-1.amazonaws.com/ as an example. We need to split the URI into components and set them in the .env file using the following parameters:

PROTOCOL=https
HTTP_PORT=80
HTTPS_PORT=443
HOST=ec2-54-235-229-141.compute-1.amazonaws.com

A dataspace serves its documents from the root of its origin, so the base URI is always the origin followed by / — there is no sub-path component to configure. Deploying several dataspaces on one instance is a matter of giving each its own subdomain, as described in Dataspaces below.

Dataspaces

Since version 5.1.0, a single LinkedDataHub instance supports multiple dataspaces, each identified by a distinct subdomain (origin). Each dataspace consists of a pair of applications: an end-user app (e.g. https://northwind-traders.demo.localhost:4443) and an admin app on the admin. subdomain (e.g. https://admin.northwind-traders.demo.localhost:4443).

Dataspace configuration is split across two files:

  • config/dataspaces.trig — public metadata: origins (lds:origin), ontologies, stylesheets
  • config/system.trig — internal wiring: SPARQL service bindings and dataspace types (lds:AdminDataspace/lds:EndUserDataspace)

To add a new dataspace, add corresponding entries to both files. Relative URIs will be resolved against the base URI configured in the .env file.

⚠️ Do not use blank nodes to identify applications or services. We recommend using the urn: URI scheme, since LinkedDataHub application resources are not accessible under their own dataspace.

Graph versioning

Since version 5.9.0, LinkedDataHub can mirror every document of a dataspace into a GitHub repository. Each document (named graph) becomes an N-Triples file, and every write (POST/PUT/PATCH/DELETE) becomes a commit authored with the agent's WebID — giving you a full history, audit trail, and undo capability using plain git tooling. Commits happen asynchronously in the background and are best-effort: if GitHub is unavailable or the token is invalid, writes succeed as usual and the failures are only logged.

Historical versions can be retrieved with the version query parameter using a commit SHA:

curl -k -E ./ssl/owner/cert.pem:<your cert password> -H "Accept: text/turtle" 'https://localhost:4443/my-doc/?version=<commit-sha>'

Version responses are immutable and carry a Memento-Datetime header with the commit's datetime. They are subject to the same access control as the live document.

Versioning is disabled by default. To enable it for a dataspace:

  1. Create a GitHub repository (a private one is recommended) that will store the graph files.
  2. Obtain an access token: on GitHub, go to Settings > Developer settings > Personal access tokens > Fine-grained tokens > Generate new token. Under Repository access select Only select repositories and pick the repository from step 1; under Permissions > Repository permissions set Contents to Read and write. Generate the token and copy it (it starts with github_pat_). A classic token with the repo scope works as well, but grants far broader access.
  3. Describe the repository in config/system.trig and link the application to it (an example is included in the file):
    <urn:linkeddatahub:apps/end-user>
    {
        <urn:linkeddatahub:apps/end-user> lds:versioningRepository <urn:linkeddatahub:versioning/end-user> .
    }
    
    <urn:linkeddatahub:versioning/end-user>
    {
        <urn:linkeddatahub:versioning/end-user> a doap:GitRepository ;
            doap:location <https://github.com/OWNER/REPO> ;
            github:branch "main" ;
            github:pathPrefix "graphs" .
    }
  4. Put the token into secrets/credentials.trig (create the file if it does not exist) as an a:authToken of the repository resource:
    @prefix a: <https://w3id.org/atomgraph/core#> .
    
    <urn:linkeddatahub:versioning/end-user>
    {
        <urn:linkeddatahub:versioning/end-user> a:authToken "github_pat_..." .
    }
  5. Enable the credentials secret in docker-compose.yml by uncommenting it in the top-level secrets: block and in the linkeddatahub service's secrets: list.
  6. Restart with make up. The startup log will confirm: Graph versioning enabled for application <...>.

Multiple dataspaces can be versioned into different repositories with different tokens. The token never appears in the environment or the process table — it is merged into the internal context dataset from the Docker secret, the same mechanism used for SPARQL service credentials.

Secrets

Secrets used in docker-compose.yml:

owner_cert_password
Password of the owner's WebID certificate
secretary_cert_password
Password of the secretary's WebID certificate
client_truststore_password
Password of the client truststore
openai_api_key
OpenAI API key the web-algebra service writes the assistant's plans with. Optional, declared in the override together with OPENAI_API_KEY_FILE; the service executes plans without it, and the drawer then reports that a plan could not be written
google_client_id
Google's OAuth client ID
Login with Google authentication is enabled when this value is provided
google_client_secret
Google's OAuth client secret
credentials
TriG dataset (secrets/credentials.trig) with service credentials such as SPARQL auth and graph versioning tokens, merged into the system dataset at startup

Environment

LinkedDataHub supports a range of configuration options that can be passed as environment parameters in docker-compose.yml. The most common ones are:

CATALINA_OPTS
Tomcat's command line options
SELF_SIGNED_CERT
true if the server certificate is self-signed
SIGN_UP_CERT_VALIDITY
Validity of the WebID certificates of signed up users (not the owner's)
IMPORT_KEEPALIVE
The period for which the data import can keep an open HTTP connection before it times out, in ms. The larger files are being imported, the longer it has to be in order for the import to complete.
MAX_CONTENT_LENGTH
Maximum allowed size of the request body, in bytes
COMPOSE_PROFILES
assistant to run and offer the assistant. Disabled by default
MAIL_SMTP_HOST
Hostname of the mail server
MAIL_SMTP_PORT
Port number of the mail server

The options are described in more detail in the configuration documentation.

Reset

If you need to start fresh and wipe the existing setup (e.g. after configuring a new base URI), you can do that using

make drop

It asks for confirmation, then stops the services and removes their volumes before deleting the datasets, fuseki, ssl, and uploads folders. Stopping first matters: deleting those folders while the containers are running leaves Fuseki writing into directories that no longer exist.

⚠️ This will remove the persisted data and files as well as Docker volumes.

ldh wraps the HTTP API into a single executable with convenient parameters. It can be used for testing, automation, scheduled execution and such. It is usually much quicker to perform actions using the CLI rather than the user interface, as well as easier to reproduce.

Every release attaches an ldh-<version>.tar.gz archive, which needs only a Java 21 runtime — no build tools and no source checkout:

tar -xzf ldh-<version>.tar.gz
export PATH="$PWD/ldh-<version>:$PATH"

ldh --help

To build it from source instead — the CLI lives in the cli subfolder and needs Java 21 and Maven:

make cli

which prints the export PATH=... line to run afterwards. If you will be using LinkedDataHub's CLI regularly, add that export to your shell profile.

Commands authenticate with a WebID client certificate, read from a PKCS12 keystore or a PEM file holding the certificate and its private key — ssl/owner/keystore.p12 and ssl/owner/cert.pem are the owner's, either works. Options that repeat across commands can be set once as environment variables:

export LDH_CERT_FILE=./ssl/owner/keystore.p12
export LDH_CERT_PASSWORD=$(cat secrets/owner_cert_password.txt)
export LDH_BASE=https://localhost:4443/

ldh create container --parent "$LDH_BASE" --title "Concepts" --slug concepts
ldh create item --container "${LDH_BASE}concepts/" --title "Example" --slug example

Commands that create or append to a document print its URL as the only line on stdout, so they compose in shell pipelines: item=$(ldh create item ...). Commands group by verb — ldh create container, ldh add xhtml-block, ldh import csv, ldh admin create group — with import holding the workflows that compose the atomic commands. See cli/README.md for the full command table and the differences from the scripts.

⚠️ The bin/ HTTP API scripts that ldh replaces are deprecated. The certificate and WebID tooling (webid-keygen.sh, webid-keygen-pem.sh, webid-uri.sh, webid-modulus.sh, server-cert-gen.sh) talks to no API and stays in bin/.

Sample applications

These demo applications can be installed into a LinkedDataHub instance using make install. You will need to provide the path to your WebID certificate as well as its password.

AI-Powered Automation

AI agents work with LinkedDataHub through the same HTTP API and access control as people do: an agent acts as a WebID, and can read and write only what that WebID may.

Assistant

Users signed in with a WebID certificate who have append access to a document can open the assistant from its creation bar. A question becomes a Web-Algebra plan, and the plan is shown before anything runs. A plan that only reads can run straight away, and one that writes waits for the user to press Execute. The plan runs as the user who asked. Each conversation is stored in the document as a block, so it is still there when the user comes back.

⚠️ The assistant is opt-in. It is served by the web-algebra service, whose image (ghcr.io/atomgraph/webalgebra-server-ldh) is proprietary: AtomGraph publishes the image but not its source, and LinkedDataHub's license does not cover it.

To enable it, add to .env:

COMPOSE_PROFILES=assistant

The profile starts the service, and the platform reads the same variable to show the Assistant button. Set it in .env rather than with docker compose --profile, which the platform does not see. Writing plans needs an OpenAI API key (secrets/openai_api_key.txt, see Setup). Without one, the assistant can still run plans but cannot write them. See the assistant reference for details.

The ldh plugin gives Claude Code skills for building on LinkedDataHub with the ldh CLI: provisioning a dataspace, modelling and importing a dataset, building an app, and writing a reusable package. Install it in Claude Code with:

/plugin marketplace add AtomGraph/LinkedDataHub
/plugin install ldh@linkeddatahub

Claude loads a skill by itself when a request matches it, for example:

Here are products.csv and categories.csv. Model them with schema.org, import them into the inventory dataspace, and make each category's page list its products.

The skills need a LinkedDataHub instance, the owner's WebID certificate and the ldh CLI. See plugins/ldh/README.md for the skills and more example requests.

Web-Algebra and MCP

Web-Algebra expresses LinkedDataHub operations, SPARQL queries and Linked Data reads as plans that run as one unit. The assistant writes its plans in it, and its Model Context Protocol server exposes LinkedDataHub operations as tools to any MCP client, such as Claude Code or Cursor. Connected with a WebID certificate, an agent can create containers and items, add views, queries and charts, and run SPARQL. It can build a working application in a dataspace this way. The application is documents like any other, and you can keep editing them by hand. See Connect an agent for the setup, and AGENTS.md for the conventions an agent should follow when it writes to LinkedDataHub.

How to get involved

Test suite

LinkedDataHub includes an HTTP test suite, run with make tests. It builds its fixtures with ldh, which make tests builds and puts on the $PATH for the run. The server implementation is also covered by the Processor test suite.

HTTP-tests

Dependencies

Browser

Java

Docker

Support

Please report issues if you've encountered a bug or have a feature request.

Commercial consulting, development, and support are available from AtomGraph.

Community