Repository navigation
fix: make start.sh --stop skip nodes in other directories - #7075
SeriousCoding789 wants to merge 3 commits into
Conversation
A directory with a start.log but no pid file was used by an earlier version of the script, and --stop found its node by the jar name alone, so it also stopped nodes with the same jar name in other directories. Pick the node whose working directory is the current directory, read from /proc on Linux or with lsof on macOS. When neither is available, exit with an error that asks for the process id in the pid file.
The pid file path matched only the jar name, so a recorded id reused by a node that runs the same jar from another directory would have stopped that node. A recorded process whose working directory reads as another directory is now taken as a stopped node and the pid file is emptied; one whose directory cannot be read keeps the pid file and is stopped as before. Without a pid file, a process whose working directory could not be read was skipped silently, so --stop could report the node stopped while it was still running. If this user can signal such a process, --stop and restart now end with an error on stderr; a process this user cannot signal is left running with a warning.
There was a problem hiding this comment.
[NIT] Propagate stop failures and prevent unsafe restarts.
If a managed process is still running but cannot be signalled, stopService() may repeatedly print kill errors and eventually return success because its final command is sleep 5.
Meanwhile, restart() ignores the stop result and proceeds with rebuilding and starting the node, potentially overwriting the PID file of the still-running process.
As a separate follow-up, please ensure that stopService() returns a failure when a managed process remains running but cannot be stopped, and that restart() aborts subsequent steps on such failures.
Please distinguish this from the case where no node was previously started, so first-time --run remains supported. Also, recheck process liveness after a failed kill, since the process may have exited normally between inspection and signalling.
This is pre-existing behavior and need not block the current PR.
There was a problem hiding this comment.
Thanks, addressed in b2a4b67 rather than as a follow-up, since the change is small. stopService now signals each process separately, and a failed kill counts only if the process still exists afterwards, so one that exited in between is treated as stopped. If a process cannot be signalled, or the node still runs after SIGKILL, it reports an error on stderr and returns 2: --stop exits with that status, and restart stops there before starting anything. A return of 1 still means no node was ever started from the directory, so a first --run is unchanged.
| ``` | ||
|
|
||
| `--run` records the process id in `<jar name>.pid` (`FullNode.jar.pid` by default) next to `start.log`, and `--stop` reads it. Run `--stop` in the directory the node was started from, with the same `-j` name if one was given. After `--release` or `-cb` the node runs in `FullNode/`; `--stop` finds it there from the parent directory as well. A node started by an earlier version of the script (a `start.log` but no pid file) is still found by its jar name. | ||
| `--run` records the process id in `<jar name>.pid` (`FullNode.jar.pid` by default) next to `start.log`, and `--stop` reads it. Run `--stop` in the directory the node was started from, with the same `-j` name if one was given. After `--release` or `-cb` the node runs in `FullNode/`; `--stop` finds it there from the parent directory as well. `--stop` only stops a process that runs the jar from that directory, read from `/proc` on Linux or with `lsof` on macOS, so nodes with the same jar name in other directories keep running; a pid file whose process now runs from another directory counts as a stopped node. A node started by an earlier version of the script (a `start.log` but no pid file) is found by its jar name; if the working directory of such a process cannot be read, `--stop` exits with an error: write the node's process id to the pid file and run `--stop` again. |
There was a problem hiding this comment.
[NIT, documentation] Clarify the limits of directory isolation in --stop.
Could we qualify the statement that --stop only stops processes running from the target directory?
As discussed, the PID-file path intentionally trusts the recorded PID when its CWD cannot be inspected. In this fallback case, a stale PID reused by another process running the same JAR name could still be selected and stopped.
Please clarify in the documentation that directory isolation is guaranteed only when CWD inspection succeeds, and that the PID-file fallback requires the recorded PID to be independently verified.
This would accurately describe the intended compatibility tradeoff without requiring implementation changes.
There was a problem hiding this comment.
Thanks, clarified in b2a4b67: shell.md now says the directory check needs a readable working directory, and that when the process in the pid file cannot be inspected, the recorded id is trusted as it is, so it has to be known to belong to this node (for example when it is written into the pid file by hand).
stopService ended with sleep 5 and returned success even when the node was still running: a process this user cannot signal made every kill fail until the timeout, and a restart then started another node and wrote its id over the pid file of the running one. A failed signal now counts while the process still exists, and a node still running after SIGKILL fails too; both report an error on stderr and return 2, which ends --stop with that status and stops a restart before it starts a node. A return of 1 still means no node was ever started from the directory, so --run starts the node as before. shell.md now states that the directory check needs a readable working directory, and that a pid file whose process cannot be inspected is trusted as recorded.
What does this PR do?
start.sh --stopnow stops only the node started from the current directory when that directory has astart.logbut no<jar name>.pid, that is, a node started by an earlier version of the script or one whose pid file was removed. Such a node is matched by its jar name and its working directory, instead of by its jar name alone. The process recorded in a pid file is checked the same way, so an id reused by a node of another directory no longer gets that node stopped.start.shprocessDir, which reads a process's working directory from/proc/<pid>/cwdon Linux and withlsofon macOS.matchDirPid, which keeps the jar-name matches whose working directory is the current directory (pwd -P);checkPiduses it when there is no pid file.--stopand the stop step of a restart with an error on stderr, which asks for the node's process id in the pid file; no signal is sent. A process this user cannot signal (another user's, or one already gone) is left running with a warning.shell.md: describe the directory check, and what to do when a working directory cannot be read.Why are these changes required?
#7003 keeps the node's process id in
<jar name>.pidand still finds a node of the earlier script by its jar name. That lookup takes every process on the machine whose command line has-jar <jar name>, and all of them are passed tokill -15, then tokill -9afterMAX_STOP_TIME. On a host that runs several nodes with the same jar name in different directories,--stopor a restart in one directory also stops the nodes of the other directories, and the restart starts only its own node again. This happens even when the node of the current directory is not running.The earlier script stopped nothing when more than one process matched (its
[ $pid ]test fails on several ids), and otherwise stopped the one match, whatever directory it ran from.The pid-file path checked only the jar name as well: when a node stopped without its pid file being emptied and its id was later reused by a node that runs the same jar from another directory,
--stopstopped that node.Both the earlier and the current script start the node from its directory, and the JVM keeps that working directory, so the working directory tells the node started from a directory apart from the nodes of other directories.
This PR has been tested by:
lsof). Dummy processes with-jar <test jar>in their command line ran from separate directories, and--stopran with the script before and after this change:--stopin A--stopin A--stopin Ash start.sh--stopin BPATHthat has nolsof--stopin AmatchDirPidbash -n start.sh;shellcheck start.shreports nothing new compared with the base./procbranch on Linux, and a restart of a real node.Follow up
None.
Extra details
matchNodePid, the stop loop and the way a node is started are unchanged, and no Java code is changed./procorlsof: Linux has/proc, and macOS shipslsofin/usr/sbin.