Skip to content

FileMonitor: fix macOS shared-memory name exceeding shm_open limit - #802

Open
phildapunk wants to merge 4 commits into
stashapp:mainfrom
phildapunk:fix/filemonitor-macos-shm-name-too-long
Open

phildapunk wants to merge 4 commits into
stashapp:mainfrom
phildapunk:fix/filemonitor-macos-shm-name-too-long

Conversation

@phildapunk

Copy link
Copy Markdown

Fixes #801.

The problem

Start Library Monitor Service has never actually started the background service on macOS. The service's running/singleton lock is a POSIX named shared-memory segment:

SHAREDMEMORY_NAME = "DavidMaisonaveAxter_FileMonitor" # 31 chars

macOS's shm_open() rejects names longer than PSHMNAMLEN (31 chars, including the leading / that multiprocessing.shared_memory prepends on POSIX). This name is exactly 31 characters, so with the slash it's 32 — one over the limit:

>>> from multiprocessing import shared_memory
>>> shared_memory.SharedMemory(name="DavidMaisonaveAxter_FileMonitor", create=True, size=10)
OSError: [Errno 63] File name too long: '/DavidMaisonaveAxter_FileMonitor'

The except: around that call swallows the exception and logs a generic "must already be running" message, so the real cause never showed up in the log — just the same misleading line on every attempt, for well over a year of this user's filemonitor.log history.

What this does

  • Shorten SHAREDMEMORY_NAME to 26 characters. Verified empirically on current macOS that 30 chars succeeds and 31 fails, so this has headroom.
  • Log the real exception in the existing error message, so if this ever regresses (e.g. a future rename back over the limit) it's visible instead of misdiagnosable.
  • Harden executeProcess()'s detach path, shared byte-for-byte across FileMonitor, DupFileManager, and RenameFile's StashPluginHelper.py: pass start_new_session=True when detaching, so the spawned child survives the launching task process exiting. The code already carried a comment flagging this was needed on macOS (# On detach, may need the following for MAC OS ... start_new_session=True) but never applied it.

Testing

Against a live Stash instance running natively on macOS (not Docker):

  • Before: clicking "Start Library Monitor Service" logged a spawned pid and "transitioning to service mode", but Settings → Tools → FileMonitor immediately reported FileMonitor is NOT running!!! — reproduced on every attempt.
  • After: reproduced the exact OSError directly against multiprocessing.shared_memory with the old name, confirmed the new name succeeds up to 30 chars, applied the fix, reloaded the plugin in Stash, started the service, and confirmed Settings → Tools → FileMonitor now reports running.

Scope

Only touches filemonitor.py's shared-memory name/error message, and the identical executeProcess() block in all three vendored StashPluginHelper.py copies. Bumped each plugin's version (FileMonitor 1.0.7→1.0.8, DupFileManager 1.0.2→1.0.3, RenameFile 1.0.4→1.0.5) since StashPluginHelper.py changed in all three, and added a FileMonitor version_history entry.

LLM-assisted contribution disclosure

Per the repository's contribution policy: this change was prepared with LLM assistance (Claude). The diff has been reviewed by me, the testing described above was carried out and its results are reproduced verbatim, and I take full responsibility for the change and its license compliance.

start_library_monitor_service() has never actually started the background
service on macOS. SHAREDMEMORY_NAME ("DavidMaisonaveAxter_FileMonitor",
31 chars) exceeds macOS's POSIX shm_open name length limit (PSHMNAMLEN,
31 chars including the leading "/"), so SharedMemory(create=True) raises
OSError "File name too long" every time. The bare except clause in
start_library_monitor() swallowed that exception and logged the generic
"FileMonitor must already be running" message instead, making the real
cause invisible in the log for over a year of start attempts.

- Shorten SHAREDMEMORY_NAME to 26 chars (verified empirically: 30 chars
  succeeds, 31 fails, on current macOS).
- Log the real exception alongside the existing message.
- Harden executeProcess()'s detach path (shared byte-for-byte across
  FileMonitor, DupFileManager, and RenameFile's StashPluginHelper.py) to
  pass start_new_session=True, so a detached child survives the launching
  task process exiting; the code already carried a comment noting this
  was needed on macOS but never applied it.

Tested against a live Stash instance on macOS: before this change,
"Start Library Monitor Service" logged a spawned pid and then immediately
reported "FileMonitor is NOT running" on every status check. After, the
service starts and reports "running" as expected.

LLM-assisted contribution disclosure: this change was prepared with LLM
assistance (Claude). I reviewed the diff, the testing described above was
performed by me against my own Stash installation, and I take full
responsibility for the change and its license compliance.
getFileMonitorRunningStatus() and start_library_monitor_service()'s own
pre-flight "already running" guard both open the shared-memory lock with
create=False just to confirm it exists, then call shm_a.unlink() on
success. unlink() deletes the named POSIX shared memory object from the
OS immediately - it doesn't wait for the running service to close its
own mapping. The running loop is unaffected by this (it keeps its
already-open handle and keeps working), but the name is now gone, so:

- Every later getFileMonitorRunningStatus() call fails to reopen it by
  name and reports "NOT running", even while the service is genuinely
  still running.
- stop_library_monitor() (same open-by-name) can no longer signal that
  instance to stop via the UI/CLI.
- start_library_monitor_service()'s "already running" guard destroys the
  real instance's lock as a side effect of merely detecting it, before
  refusing to start a duplicate.

Found this while verifying the previous commit's fix end-to-end: the
service started fine and kept running (confirmed via `ps`), but the very
first status check flipped every later check to "NOT running" by
unlinking the lock out from under it. Both functions now just
attach-and-close without unlinking; only stop_library_monitor() (which
is actually supposed to tear the lock down) still unlinks.

Tested against the same live macOS Stash instance: restarted the service
after killing the orphaned process whose lock had already been
destroyed by the old code, and confirmed repeated status checks now
consistently report "running".
@phildapunk

Copy link
Copy Markdown
Author

Follow-up found while verifying the first commit end-to-end: once the shared-memory rename took effect, the service genuinely started and kept running (confirmed via a live `ps` check), but the very first "Tools -> FileMonitor" status check flipped every status check after it to "NOT running" — even though the process was still alive and logging its watch loop.

Cause: both `getFileMonitorRunningStatus()` and `start_library_monitor_service()`'s own "already running" pre-flight guard open the shared-memory lock with `create=False` just to confirm it exists, then call `shm_a.unlink()` on success. `unlink()` deletes the named segment from the OS immediately; the running service keeps its own already-open mapping and is unaffected, but the name is gone, so every later status check (and `stop_library_monitor()`, and the "already running" duplicate-start guard) can no longer find it by name.

Added a second commit that removes the unlink from both of those read-only checks — only `stop_library_monitor()` (which is actually supposed to tear the lock down) still unlinks. Verified by killing the now-orphaned process (whose lock had already been destroyed by the old code before this fix landed), restarting the service, and confirming repeated status checks now consistently report "running".

Removing the explicit unlink() calls (previous commit) wasn't enough.
Reproduced directly: any process that attaches to a
multiprocessing.shared_memory segment - even read-only, create=False -
gets it auto-registered with *that process's own* resource_tracker. When
that short-lived process exits normally, its tracker treats the
unregistered-but-tracked segment as "leaked" and unlinks it anyway,
regardless of whether another process (the real owner) is still using
it:

    creator process: SharedMemory(create=True) -> keeps running
    attacher process: SharedMemory(create=False); close(); exits
    -> attacher's own resource_tracker now deletes the segment
    -> next status check: "No such file or directory"

This is a known multiprocessing.shared_memory gotcha; the fix is to
call resource_tracker.unregister(shm._name, "shared_memory") right
after attaching read-only, telling *this* process's tracker it isn't
responsible for cleanup. Verified the failure and the fix in isolation
with a 3-process repro (creator/attacher/checker) before applying it to
getFileMonitorRunningStatus() and start_library_monitor_service()'s
"already running" guard - the same two read-only attach sites touched
in the previous commit.

stop_library_monitor() needs no change: SharedMemory.unlink() already
calls resource_tracker.unregister() internally as part of what it does.

Tested against the same live macOS Stash instance: killed the second
orphaned instance (whose lock the resource_tracker had destroyed before
this fix landed), restarted the service, and confirmed status checks
several seconds apart both report "running".
@phildapunk

Copy link
Copy Markdown
Author

Second follow-up: removing the explicit `unlink()` calls in the previous commit wasn't actually enough — it still went back to "NOT running" ~8-15s after a successful check. Reproduced the real mechanism in isolation (3-process repro: a creator that holds the segment, a separate attacher that opens it `create=False` and exits cleanly, then a checker): the attacher's own `resource_tracker` treats the segment as "leaked" on its own exit and unlinks it — regardless of `unlink()` never being called explicitly, and regardless of the real creator still being alive. This is a known `multiprocessing.shared_memory` gotcha; the fix is calling `resource_tracker.unregister(shm._name, "shared_memory")` right after a read-only attach, in both `getFileMonitorRunningStatus()` and `start_library_monitor_service()`'s "already running" guard. Verified the failure mode and the fix in isolation before applying, then re-tested end-to-end against the same live instance.

@DogmaDragon

Copy link
Copy Markdown
Contributor

Please limit the PR to a single plugin.

Per maintainer request, drop the DupFileManager and RenameFile changes
from this PR. Those two plugins share a vendored copy of
StashPluginHelper.py with FileMonitor, which is why the
start_new_session=True detach fix had touched all three — but only
FileMonitor is tied to an actual filed issue (stashapp#801). The same fix for
DupFileManager and RenameFile will follow as separate single-plugin
PRs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@phildapunk

Copy link
Copy Markdown
Author

Done — pushed a commit that drops the DupFileManager and RenameFile changes. This PR is now scoped to FileMonitor only. I'll open the same start_new_session=True detach fix for DupFileManager and RenameFile as separate single-plugin PRs.

@discourse-stashapp

Copy link
Copy Markdown

This pull request has been mentioned on Stash Forum. There might be relevant details there:

https://discourse.stashapp.cc/t/filemonitor/1333/13

@DogmaDragon DogmaDragon added the type:plugin Plugins label Oct 27, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FileMonitor: Start Library Monitor Service never actually starts on macOS

3 participants