Repository navigation
Conversation
9822261 to
b95b96a
Compare
A guest RLIMIT_CPU lives in the guest rlimit table and nothing acted on
it, so a guest that spins past its limit runs until the vCPU watchdog
fires after 10 s. Linux sends SIGXCPU at the soft limit and raises the
soft limit by one second, so a process that survives gets another
SIGXCPU each CPU second, and sends SIGKILL at the hard limit
(check_process_timers in kernel/time/posix-cpu-timers.c). Both are
sent as SEND_SIG_PRIV, so a handler sees SI_KERNEL and no sender.
A guest spinning at EL0 takes no exit where elfuse could check the
limit, so rlimit_cpu_main samples the host process CPU time from a
thread of its own. It starts on the first finite soft limit, from the
boot seed, prlimit64, or in a fork child after signal_set_state, since
that call replaces the shared pending set and would drop a signal the
sampler queued earlier. It sleeps for the remaining CPU time divided
by the online CPU count, from 10 ms up to an hour. A limit change wakes
it, so the cap only keeps a soft limit near saturation out of the host
timed wait. After a SIGXCPU it waits the 10 ms before checking again,
so a soft limit set far below the CPU time already used rises one step
per check, as it does per tick on Linux, not in one burst whose
signals merge into one.
At the hard limit the sampler sends SIGKILL again every 10 ms rather
than once. A CLONE_THREAD worker that dequeues a fatal signal ends
alone instead of taking the thread group with it as do_group_exit
does, so a single send could leave the other threads running past the
limit with nothing left to enforce it.
When the sampler thread cannot be created the limit would go
unenforced, so prlimit64 restores the old slot and fails with EAGAIN,
and boot and the fork child fail rather than run the guest without its
limit.
A signal the sampler queues raises attention in the guest shim
globals, so launch_destroy_guest stops the sampler first and waits out
a send already in flight, before guest_destroy unmaps that memory.
tests/test-rlimit-cpu covers SIGXCPU at its default action, the
repeated SIGXCPU with the raised soft limit and its siginfo, a soft
limit lowered below the CPU time used catching up, SIGKILL at the hard
limit and at {0, 0}, a sleeping process that is not charged, a fork
child that dies of an inherited soft limit, and a sleeping leader whose
spinning worker crosses the hard limit. That fork child's parent blocks
SIGXCPU, so its own SIGXCPU cannot end it before the fork. One step per
check is checked as a lower bound, soft limit 4 not visible within 1 ms
of the lowering, which a busy host can only make longer; a burst
reaches 4 in about 0.05 ms, and Linux at HZ 1000 takes 2 ms. The
parent kills a child still alive after 10 s, so a limit that is never
enforced fails its case rather than the run; alarm in the child would
not, because elfuse checks a guest ITIMER_REAL only when the vCPU
exits. The test skips under an inherited hard limit below 4 s, since
one child needs 3.5 s of CPU.
sys_rlimit_init copies a finite host RLIMIT_CPU into the guest rlimit
table, where rlimit_cpu_main enforces it, but the host soft limit stays
armed on elfuse itself. Past it the host kernel sends elfuse SIGXCPU,
whose default action terminates elfuse even when the guest caught
SIGXCPU or raised its own soft limit.
Ignore SIGXCPU next to SIGXFSZ, and lift the host soft limit to the
hard one, as sys_rlimit_init already does for RLIMIT_FSIZE, so the
guest table decides. XNU sends nothing at the hard limit: a host
process that ignores SIGXCPU spins to 4 CPU seconds under {1, 2} and
{2, 2} and exits 0. With the soft limit lifted, nothing on the host
stops elfuse, and a guest with euid 0 can raise its own hard limit
through prlimit64. rlimit_cpu_main therefore sends SIGKILL at the
lower of the guest hard limit and the host hard limit, and runs while
either the guest soft limit or the host hard limit is finite. The host
hard limit is read at boot and in a fork child, which inherits it.
Under ulimit -St 1 and under ulimit -St 1000 with SIGXCPU ignored in
the shell, test-hello stalled at boot until a 20 s timeout in 3 of 3
runs before this change and exits 0 in 3 of 3 after. A finite hard
limit (ulimit -Ht 100000) still stalls in 3 of 3 runs, see sysprog21#474, so
the host hard limit cannot be tested end to end until it is fixed. A
build that reads a host hard limit of 2 from the environment kills a
fakeroot guest that raised its limits to unlimited after 2 to 3 s
with status 137.
tests/test-rlimit-seed.sh spins a guest until times reports 1 s of
user time, after the guest raises its soft limit above a 1 s host soft
limit.
There was a problem hiding this comment.
3 issues found across 11 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="tests/test-rlimit-cpu.c">
<violation number="1" location="tests/test-rlimit-cpu.c:86">
P3: Use `volatile sig_atomic_t` for the state shared with `on_xcpu_late`; ordinary volatile `int`/`unsigned char` objects are not the signal-safe communication type and can make this timing test flaky or undefined.</violation>
</file>
<file name="tests/test-rlimit-seed.sh">
<violation number="1" location="tests/test-rlimit-seed.sh:107">
P2: This test can race the guest's initial one-second limit during elfuse startup. Ignore guest `SIGXCPU` before raising the limit, or otherwise prevent startup CPU from being charged, so a queued startup signal cannot terminate the test after `ulimit -S -t unlimited` runs.</violation>
</file>
<file name="src/main.c">
<violation number="1" location="src/main.c:563">
P2: This installs the host `SIGXCPU` ignore only after substantial startup work. A finite inherited `RLIMIT_CPU` can therefore kill elfuse during argument or sysroot setup before this line runs; move the registration to the start of `main()`.</violation>
</file>
Tip: Get a Slack DM when cubic finishes reviewing your PRs. Connect Slack
View guided diff | Re-trigger cubic
| # writes to a file, since in a command substitution it would report the | ||
| # subshell; the file case above is done with that file. | ||
| # shellcheck disable=SC2016 # Expanded by the guest shell. | ||
| SPIN_PAST_1S='ulimit -S -t unlimited || exit 1 |
There was a problem hiding this comment.
P2: This test can race the guest's initial one-second limit during elfuse startup. Ignore guest SIGXCPU before raising the limit, or otherwise prevent startup CPU from being charged, so a queued startup signal cannot terminate the test after ulimit -S -t unlimited runs.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At tests/test-rlimit-seed.sh, line 107:
<comment>This test can race the guest's initial one-second limit during elfuse startup. Ignore guest `SIGXCPU` before raising the limit, or otherwise prevent startup CPU from being charged, so a queued startup signal cannot terminate the test after `ulimit -S -t unlimited` runs.</comment>
<file context>
@@ -97,6 +97,32 @@ else
+# writes to a file, since in a command substitution it would report the
+# subshell; the file case above is done with that file.
+# shellcheck disable=SC2016 # Expanded by the guest shell.
+SPIN_PAST_1S='ulimit -S -t unlimited || exit 1
+n=0
+while [ $n -lt 1000 ]; do
</file context>
| SPIN_PAST_1S='ulimit -S -t unlimited || exit 1 | |
| SPIN_PAST_1S='trap : XCPU | |
| ulimit -S -t unlimited || exit 1 |
| * hard limit, so the RLIMIT_CPU sampler (sys.c) enforces both the guest | ||
| * limits and the host hard limit. | ||
| */ | ||
| signal(SIGXCPU, SIG_IGN); |
There was a problem hiding this comment.
P2: This installs the host SIGXCPU ignore only after substantial startup work. A finite inherited RLIMIT_CPU can therefore kill elfuse during argument or sysroot setup before this line runs; move the registration to the start of main().
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/main.c, line 563:
<comment>This installs the host `SIGXCPU` ignore only after substantial startup work. A finite inherited `RLIMIT_CPU` can therefore kill elfuse during argument or sysroot setup before this line runs; move the registration to the start of `main()`.</comment>
<file context>
@@ -553,6 +553,15 @@ int main(int argc, char **argv)
+ * hard limit, so the RLIMIT_CPU sampler (sys.c) enforces both the guest
+ * limits and the host hard limit.
+ */
+ signal(SIGXCPU, SIG_IGN);
+
/* Block the vCPU-preemption signals and start the sigwait thread before any
</file context>
| static volatile unsigned char late_seen[8]; | ||
| static volatile int late_n; |
There was a problem hiding this comment.
P3: Use volatile sig_atomic_t for the state shared with on_xcpu_late; ordinary volatile int/unsigned char objects are not the signal-safe communication type and can make this timing test flaky or undefined.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At tests/test-rlimit-cpu.c, line 86:
<comment>Use `volatile sig_atomic_t` for the state shared with `on_xcpu_late`; ordinary volatile `int`/`unsigned char` objects are not the signal-safe communication type and can make this timing test flaky or undefined.</comment>
<file context>
@@ -0,0 +1,330 @@
+ spin();
+}
+
+static volatile unsigned char late_seen[8];
+static volatile int late_n;
+
</file context>
| static volatile unsigned char late_seen[8]; | |
| static volatile int late_n; | |
| static volatile sig_atomic_t late_seen[8]; | |
| static volatile sig_atomic_t late_n; |
| sigaddset(&xcpu, SIGXCPU); | ||
| sigprocmask(SIG_BLOCK, &xcpu, NULL); | ||
| set_cpu_soft(0); | ||
| pid_t pid = fork(); |
There was a problem hiding this comment.
This case passes without exercising the fork-child start. signal_get_state copies the shared pending set into the snapshot and signal_set_state restores it, so the SIGXCPU the parent earned while blocked arrives already pending in the child. The child dies the moment it unblocks, and deleting the sys_rlimit_cpu_start call in fork_child_main leaves the test green.
Drain the pending signal in the parent before forking (sigtimedwait with a zero timeout), so the child inherits only the limit and has to be killed by its own sampler.
| * are properly restored to the thread entry. | ||
| */ | ||
| signal_set_state(&sig); | ||
| if (!sys_rlimit_cpu_start()) { |
There was a problem hiding this comment.
This failure exit comes after the fork was committed: the parent has already returned the child pid, and proc_lifecycle_sync_self has run. If pthread_create fails here, the guest sees a successful fork whose child exits 1 with no proc_process_exit record, where Linux would have failed the fork with EAGAIN.
Consider creating the sampler thread disarmed before the commit (next to sys_rlimit_restore) and having sys_rlimit_cpu_start only arm it. Then a thread-creation failure takes one of the existing pre-commit exits, and this call can no longer fail.
| static void rlimit_cpu_send_locked(int sig) | ||
| { | ||
| rlimit_cpu_sending = true; | ||
| pthread_mutex_unlock(&rlimit_lock); |
There was a problem hiding this comment.
The send decision is made under rlimit_lock, but the signal is queued after the lock is dropped. A guest thread can complete prlimit64 in that gap (for example, raise the hard limit from 1 s to 10 s), and the sampler then queues a SIGKILL that the new limit would not have produced. Linux decides and queues under one lock.
The window is tiny, so this may be acceptable. If so, note it in the comment above rlimit_cpu_send_locked. Otherwise, re-check the limit after taking sig_lock and drop the signal if it no longer applies.
Intent: a guest
RLIMIT_CPUlimits the guest's CPU time as it does on Linux. This is follow-up 2 of #448, which stores the limit in the guest table and leaves it unenforced.Failure fixed: on main, a guest that sets
RLIMIT_CPUto 1 s and spins gets no signal and runs until the vCPU watchdog ends elfuse after 10 s with status 124. On Linux the guest getsSIGXCPUafter 1 s of CPU time.This PR follows
check_process_timers(kernel/time/posix-cpu-timers.c). At the soft limit the process getsSIGXCPUand the soft limit rises by one second, so a process that catches or ignoresSIGXCPUgets another one each CPU second. At the hard limit it getsSIGKILL; when soft equals hard,SIGKILLalone. A guest spinning at EL0 takes no exit where elfuse could check the limit, sorlimit_cpu_mainsamples the host process CPU time, the clock guestCLOCK_PROCESS_CPUTIME_IDreads, from a thread of its own. The thread starts on the first finite soft limit (boot seed, fork-child restore, orprlimit64) and sleeps for the remaining CPU time divided by the online CPU count, at least 10 ms. A limit change wakes it.Review focus:
rlimit_lockis a leaf, so the sampler drops it beforesignal_queue_info, which takessig_lock.SIGXCPUis queued, so a handler that callsgetrlimitreads the new value, as on Linux.SI_KERNELwithsi_pid0, matchingSEND_SIG_PRIVincheck_rlimit.sys_rlimit_cpu_start, called right aftersignal_set_state, since that call replaces the shared pending set and would drop a signal queued earlier.SIGXCPUand lifts its ownRLIMIT_CPUsoft limit to the hard one, aseeada5bdoes forRLIMIT_FSIZE. XNU sends nothing at the hard limit, so the sampler sendsSIGKILLat the lower of the guest and host hard limits, which keeps a guest with euid 0 from raising its way past the host limit.Known gaps: a fork child is a new host process, so its CPU time starts with what its elfuse process spent before the guest resumes (a fresh launch measured 6 to 17 ms), where a Linux child starts at 0. The signal can land up to 10 ms after the limit, the sampler's shortest sleep. A fatal default signal taken by a worker thread ends only that thread (#476), so the sampler sends
SIGKILLagain every 10 ms, and a defaultSIGXCPUtaken by a worker is not sent again. Under a finite hostRLIMIT_CPUhard limit, elfuse stalls before the guest runs, on main as well (#474), so the host hard limit is checked only with a build that reads it from the environment. ACLONE_VMchild withoutCLONE_THREADruns in its parent's host process, so the two share one CPU clock and the parent can be signalled for the child's CPU time (#478).Verified on
8134b88, rebased onto3347188:make check-formatandmake checkpass;make test-matrix-elfuse-aarch64gives 317 passed, 0 failed, 10 skipped.