Repository navigation
FileMonitor: fix macOS shared-memory name exceeding shm_open limit - #802
phildapunk wants to merge 4 commits into
Conversation
start_library_monitor_service() has never actually started the background
service on macOS. SHAREDMEMORY_NAME ("DavidMaisonaveAxter_FileMonitor",
31 chars) exceeds macOS's POSIX shm_open name length limit (PSHMNAMLEN,
31 chars including the leading "/"), so SharedMemory(create=True) raises
OSError "File name too long" every time. The bare except clause in
start_library_monitor() swallowed that exception and logged the generic
"FileMonitor must already be running" message instead, making the real
cause invisible in the log for over a year of start attempts.
- Shorten SHAREDMEMORY_NAME to 26 chars (verified empirically: 30 chars
succeeds, 31 fails, on current macOS).
- Log the real exception alongside the existing message.
- Harden executeProcess()'s detach path (shared byte-for-byte across
FileMonitor, DupFileManager, and RenameFile's StashPluginHelper.py) to
pass start_new_session=True, so a detached child survives the launching
task process exiting; the code already carried a comment noting this
was needed on macOS but never applied it.
Tested against a live Stash instance on macOS: before this change,
"Start Library Monitor Service" logged a spawned pid and then immediately
reported "FileMonitor is NOT running" on every status check. After, the
service starts and reports "running" as expected.
LLM-assisted contribution disclosure: this change was prepared with LLM
assistance (Claude). I reviewed the diff, the testing described above was
performed by me against my own Stash installation, and I take full
responsibility for the change and its license compliance.
getFileMonitorRunningStatus() and start_library_monitor_service()'s own pre-flight "already running" guard both open the shared-memory lock with create=False just to confirm it exists, then call shm_a.unlink() on success. unlink() deletes the named POSIX shared memory object from the OS immediately - it doesn't wait for the running service to close its own mapping. The running loop is unaffected by this (it keeps its already-open handle and keeps working), but the name is now gone, so: - Every later getFileMonitorRunningStatus() call fails to reopen it by name and reports "NOT running", even while the service is genuinely still running. - stop_library_monitor() (same open-by-name) can no longer signal that instance to stop via the UI/CLI. - start_library_monitor_service()'s "already running" guard destroys the real instance's lock as a side effect of merely detecting it, before refusing to start a duplicate. Found this while verifying the previous commit's fix end-to-end: the service started fine and kept running (confirmed via `ps`), but the very first status check flipped every later check to "NOT running" by unlinking the lock out from under it. Both functions now just attach-and-close without unlinking; only stop_library_monitor() (which is actually supposed to tear the lock down) still unlinks. Tested against the same live macOS Stash instance: restarted the service after killing the orphaned process whose lock had already been destroyed by the old code, and confirmed repeated status checks now consistently report "running".
|
Follow-up found while verifying the first commit end-to-end: once the shared-memory rename took effect, the service genuinely started and kept running (confirmed via a live `ps` check), but the very first "Tools -> FileMonitor" status check flipped every status check after it to "NOT running" — even though the process was still alive and logging its watch loop. Cause: both `getFileMonitorRunningStatus()` and `start_library_monitor_service()`'s own "already running" pre-flight guard open the shared-memory lock with `create=False` just to confirm it exists, then call `shm_a.unlink()` on success. `unlink()` deletes the named segment from the OS immediately; the running service keeps its own already-open mapping and is unaffected, but the name is gone, so every later status check (and `stop_library_monitor()`, and the "already running" duplicate-start guard) can no longer find it by name. Added a second commit that removes the unlink from both of those read-only checks — only `stop_library_monitor()` (which is actually supposed to tear the lock down) still unlinks. Verified by killing the now-orphaned process (whose lock had already been destroyed by the old code before this fix landed), restarting the service, and confirming repeated status checks now consistently report "running". |
Removing the explicit unlink() calls (previous commit) wasn't enough.
Reproduced directly: any process that attaches to a
multiprocessing.shared_memory segment - even read-only, create=False -
gets it auto-registered with *that process's own* resource_tracker. When
that short-lived process exits normally, its tracker treats the
unregistered-but-tracked segment as "leaked" and unlinks it anyway,
regardless of whether another process (the real owner) is still using
it:
creator process: SharedMemory(create=True) -> keeps running
attacher process: SharedMemory(create=False); close(); exits
-> attacher's own resource_tracker now deletes the segment
-> next status check: "No such file or directory"
This is a known multiprocessing.shared_memory gotcha; the fix is to
call resource_tracker.unregister(shm._name, "shared_memory") right
after attaching read-only, telling *this* process's tracker it isn't
responsible for cleanup. Verified the failure and the fix in isolation
with a 3-process repro (creator/attacher/checker) before applying it to
getFileMonitorRunningStatus() and start_library_monitor_service()'s
"already running" guard - the same two read-only attach sites touched
in the previous commit.
stop_library_monitor() needs no change: SharedMemory.unlink() already
calls resource_tracker.unregister() internally as part of what it does.
Tested against the same live macOS Stash instance: killed the second
orphaned instance (whose lock the resource_tracker had destroyed before
this fix landed), restarted the service, and confirmed status checks
several seconds apart both report "running".
|
Second follow-up: removing the explicit `unlink()` calls in the previous commit wasn't actually enough — it still went back to "NOT running" ~8-15s after a successful check. Reproduced the real mechanism in isolation (3-process repro: a creator that holds the segment, a separate attacher that opens it `create=False` and exits cleanly, then a checker): the attacher's own `resource_tracker` treats the segment as "leaked" on its own exit and unlinks it — regardless of `unlink()` never being called explicitly, and regardless of the real creator still being alive. This is a known `multiprocessing.shared_memory` gotcha; the fix is calling `resource_tracker.unregister(shm._name, "shared_memory")` right after a read-only attach, in both `getFileMonitorRunningStatus()` and `start_library_monitor_service()`'s "already running" guard. Verified the failure mode and the fix in isolation before applying, then re-tested end-to-end against the same live instance. |
|
Please limit the PR to a single plugin. |
Per maintainer request, drop the DupFileManager and RenameFile changes from this PR. Those two plugins share a vendored copy of StashPluginHelper.py with FileMonitor, which is why the start_new_session=True detach fix had touched all three — but only FileMonitor is tied to an actual filed issue (stashapp#801). The same fix for DupFileManager and RenameFile will follow as separate single-plugin PRs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Done — pushed a commit that drops the DupFileManager and RenameFile changes. This PR is now scoped to FileMonitor only. I'll open the same |
|
This pull request has been mentioned on Stash Forum. There might be relevant details there: |
Fixes #801.
The problem
Start Library Monitor Servicehas never actually started the background service on macOS. The service's running/singleton lock is a POSIX named shared-memory segment:macOS's
shm_open()rejects names longer thanPSHMNAMLEN(31 chars, including the leading/thatmultiprocessing.shared_memoryprepends on POSIX). This name is exactly 31 characters, so with the slash it's 32 — one over the limit:The
except:around that call swallows the exception and logs a generic "must already be running" message, so the real cause never showed up in the log — just the same misleading line on every attempt, for well over a year of this user'sfilemonitor.loghistory.What this does
SHAREDMEMORY_NAMEto 26 characters. Verified empirically on current macOS that 30 chars succeeds and 31 fails, so this has headroom.executeProcess()'s detach path, shared byte-for-byte acrossFileMonitor,DupFileManager, andRenameFile'sStashPluginHelper.py: passstart_new_session=Truewhen detaching, so the spawned child survives the launching task process exiting. The code already carried a comment flagging this was needed on macOS (# On detach, may need the following for MAC OS ... start_new_session=True) but never applied it.Testing
Against a live Stash instance running natively on macOS (not Docker):
Settings → Tools → FileMonitorimmediately reportedFileMonitor is NOT running!!!— reproduced on every attempt.OSErrordirectly againstmultiprocessing.shared_memorywith the old name, confirmed the new name succeeds up to 30 chars, applied the fix, reloaded the plugin in Stash, started the service, and confirmedSettings → Tools → FileMonitornow reports running.Scope
Only touches
filemonitor.py's shared-memory name/error message, and the identicalexecuteProcess()block in all three vendoredStashPluginHelper.pycopies. Bumped each plugin's version (FileMonitor1.0.7→1.0.8,DupFileManager1.0.2→1.0.3,RenameFile1.0.4→1.0.5) sinceStashPluginHelper.pychanged in all three, and added a FileMonitor version_history entry.LLM-assisted contribution disclosure
Per the repository's contribution policy: this change was prepared with LLM assistance (Claude). The diff has been reviewed by me, the testing described above was carried out and its results are reproduced verbatim, and I take full responsibility for the change and its license compliance.