Skip to content

feat: Add the file-based override source - #224

Draft
kinyoklion wants to merge 8 commits into
rlamb/overrides-java-override-eventsfrom
rlamb/overrides-java-file-override-source
Draft

kinyoklion wants to merge 8 commits into
rlamb/overrides-java-override-eventsfrom
rlamb/overrides-java-file-override-source

Conversation

@kinyoklion

@kinyoklion kinyoklion commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Summary

This adds the file-based override source that the OVERRIDE specification defines, as FileOverrides.source() in the integrations package, configured through DataSystemBuilder.overrides(...):

LDConfig config = new LDConfig.Builder()
    .dataSystem(Components.dataSystem().defaultMode()
        .overrides(FileOverrides.source().filePaths("/etc/launchdarkly/overrides.json")))
    .build();

The source reads one or more JSON or YAML files in the file data source document format (flags, flagValues, segments) in the configured order. A flagValues entry becomes a flag that is on and serves the value as its single variation by fallthrough, so it evaluates with the FALLTHROUGH reason kind. Entries keep the versions that the documents specify.

  • duplicateKeysHandling: FAIL (the default) fails the reload when the same key appears in more than one file; IGNORE keeps the entry from the first configured file.
  • changeDetection: POLLING (the default) examines each file's modification time and size on an interval; WATCHING uses file system notifications. The two are alternatives. When the watch service cannot be created at all (for example, the limit on inotify instances is reached), the source logs an error and polls the files instead, at no less than the minimum poll interval.
  • pollInterval: one second by default and at minimum; a smaller value is raised to the minimum with a warning.
  • A configured file that does not exist contributes no overrides and is not an error. Creating it later takes effect, and deleting it removes its overrides.
  • A file that exists but cannot be read or parsed fails that whole reload: the last good overrides stay in effect, the failure is logged, and the source retries after a bounded delay as well as on the next change.
  • The initial load completes during client construction. Every applied change is logged at Info with the overrides in effect and what each file supplied.
  • No file paths, a null change detection mode, a null poll interval, or a file path that is not on the default file system (a zip file system, for example) with WATCHING selected is a configuration error that fails client construction. Polling reads such a path through its own provider. A path listed more than once is logged at Warn and the repeats are ignored; the first occurrence keeps its place in the order.

The source is built on OverrideFileLoader, FileDataReloader, FileDataPoller, and FileDataWatcher from the earlier change. The existing file data source keeps its current behavior; this change does not touch its code paths. Flag overrides are currently experimental and subject to change.

This PR depends on the override layer and the reload infrastructure changes and is based on the events branch (rlamb/overrides-java-override-events); retarget it to feat/overrides once that branch merges.

SDK-3246


Note

Overview
Adds FileOverrides.source() as the public way to load experimental flag/segment overrides from local JSON or YAML files via DataSystemBuilder.overrides(...), so operators can pin flags during outages without changing the main file data source.

The new builder supports multiple files (ordered merge, duplicate-key FAIL/IGNORE), polling vs file watching (with watch failures falling back to polling), poll interval clamping, and validation (paths required, WATCHING limited to the default file system). FileOverrideSourceImpl performs a synchronous initial load, pushes full snapshots to the override sink, keeps the last good set on parse/read errors with retries, and logs what each file contributed.

Docs on OverrideSource and DataSystemBuilder.overrides now point at FileOverrides. Tests cover reload behavior, edge cases, and client CLIENT_SIDE_ONLY behavior for flagValues vs full flags definitions.

Reviewed by Cursor Bugbot for commit 8ccc74b. Bugbot is set up for automated code reviews on this repo. Configure here.

Opened from termag

@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-override-events branch from bd3633b to e1dd774 Compare September 28, 2026 21:04
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-file-override-source branch from bc2e4a9 to 657a598 Compare September 28, 2026 21:04
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-override-events branch from e1dd774 to 752e614 Compare October 1, 2026 23:44
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-file-override-source branch from 657a598 to 5be8948 Compare October 1, 2026 23:44
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-override-events branch from 752e614 to f76f228 Compare October 3, 2026 00:58
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-file-override-source branch from 5be8948 to 9992774 Compare October 3, 2026 00:58
@kinyoklion

Copy link
Copy Markdown
Member Author

bugbot review

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-override-events branch from f76f228 to b03ecdf Compare October 3, 2026 01:43
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-file-override-source branch from 9992774 to 1728c29 Compare October 3, 2026 01:43
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-override-events branch from b03ecdf to ed4075d Compare October 13, 2026 21:02
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-file-override-source branch from 1728c29 to c134bcc Compare October 13, 2026 21:02
@kinyoklion

Copy link
Copy Markdown
Member Author

bugbot review

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-override-events branch from ed4075d to c5cfe53 Compare October 16, 2026 21:18
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-file-override-source branch from a0f4c55 to 21f7364 Compare October 16, 2026 21:18
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-override-events branch from c5cfe53 to c583704 Compare October 28, 2026 22:19
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-file-override-source branch from 21f7364 to 68fe0d8 Compare October 28, 2026 22:19
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-override-events branch from c583704 to dd7ee83 Compare October 28, 2026 22:39
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-file-override-source branch from 68fe0d8 to 7f0a3eb Compare October 28, 2026 22:39
@kinyoklion

Copy link
Copy Markdown
Member Author

bugbot review

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@kinyoklion

Copy link
Copy Markdown
Member Author

bugbot review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 8ccc74b. Configure here.

@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-override-events branch from dd7ee83 to 43080b1 Compare October 29, 2026 16:49
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-file-override-source branch from 8ccc74b to af78d3a Compare October 29, 2026 16:49
The OVERRIDE specification defines a file-based override source that
reads override entries from one or more local files and reloads them
when the files change. FileOverrides.source() provides it, configured
through DataSystemBuilder.overrides(...).

The source accepts the file data source document format (flags,
flagValues, segments) in JSON or YAML, merges the files in the
configured order with the configured duplicate keys handling (fail by
default, or ignore all but the first), and keeps the versions that the
documents specify. A flagValues entry becomes a flag that is off and
serves the value as its single variation. Change detection is polling
(the default, one second by default and at minimum) or watching. A
configured file that does not exist contributes no overrides. A file
that exists but cannot be read or parsed fails that reload, keeps the
last good overrides, logs the failure, and is retried. The initial load
completes during client construction. Every applied change is logged at
Info with the overrides in effect and what each file supplied. Missing
file paths, a null change detection mode, or a null poll interval fail
client construction.

Flag overrides are currently experimental and subject to change.
The poller logs a change callback that throws, so it needs the source's logger. This compiles once the branch includes the poller constructor that takes a logger.
…ntries as duplicate keys

A path listed more than once would make every entry of that file a duplicate of itself: under the
default fail handling the load failed, and under ignore the first listing won by accident. The
builder now drops the repeats, keeps the first occurrence in its place in the order, and logs a
warning once at build time. This is the behavior the Node SDK has.
The watcher uses the default file system's watch service, which cannot
watch a path of another provider such as a zip file system. Registering
one threw an unchecked exception from start, so a configuration error
surfaced as a failed start. The builder now rejects such a path when
watching is selected, so the error is reported when the client is
created. Polling reads such a path through its own provider and still
accepts it.
When watching was selected and the watch service could not be created,
for example because the limit on inotify instances was reached, the
source logged the failure and ran with no change detection, so a later
edit never took effect. It now polls the files instead, at no less than
the minimum poll interval, and the log says so.
… SDKs

A flagValues entry becomes a flag without client-side availability, so
allFlagsState with CLIENT_SIDE_ONLY leaves it out even when the flag it
overrides is client-side. A full definition with clientSide is the way
to override such a flag. A test pins both.
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-override-events branch from 43080b1 to 0a91418 Compare October 29, 2026 17:01
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-java-file-override-source branch from af78d3a to 2ab2fe7 Compare October 29, 2026 17:01

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant