Skip to content

[Bug]: Removing Vibe event hooks deletes user comments that follow a Spec Kit hook block #5079

Open

Description

@hnipps

Bug Description

VibeIntegration finds the [[hooks]] blocks that Spec Kit owns (marked speckit_marker = true) and treats each one as running until the next table header or the end of the file (src/specify_cli/integrations/vibe/__init__.py:440-470). Comment and blank lines after a managed block therefore count as part of it. When the hooks are removed, those user comments are deleted. If a comment is the only user content left, remove_vibe_event_hooks deletes the whole .vibe/hooks.toml file. _merge_managed_hooks, used when hooks are written again, uses the same block detection and drops the comments as well.

Steps to Reproduce

From the root of a fresh clone, after uv sync, save the script below as repro.py and run it with uv run python repro.py.

import tempfile
from pathlib import Path
from specify_cli.integrations import get_integration

vibe = get_integration("vibe")
managed = (
    '[[hooks]]\nname = "speckit-pre_tool-validate"\ntype = "pre_tool"\n'
    'command = "c"\ntimeout = 65\nspeckit_marker = true\n'
)
cases = {
    "comment before a user hook": managed
    + '\n# NOTE: keep guard-bash first\n[[hooks]]\nname = "user-hook"\n'
    'type = "pre_tool"\ncommand = "guard-bash"\n',
    "trailing user comment only": managed + "\n# my notes about vibe hooks\n",
}
for label, text in cases.items():
    root = Path(tempfile.mkdtemp())
    path = root / ".vibe" / "hooks.toml"
    path.parent.mkdir()
    path.write_text(text)
    vibe.remove_vibe_event_hooks(root, ensure_safe_destination=lambda p: None)
    print(f"[{label}]")
    print("  file exists after removal:", path.exists())
    print("  content:", repr(path.read_text()) if path.exists() else None)

The script writes a .vibe/hooks.toml containing a Spec Kit managed hook followed by user content, then calls remove_vibe_event_hooks, the method used when Vibe event hooks are removed.

Expected Behavior

Only the Spec Kit managed hook is removed. The user's comment and the user's hook stay, and a file that still holds a user comment is kept. The _without_managed_hooks docstring says it preserves "every other source slice verbatim" (src/specify_cli/integrations/vibe/__init__.py:477-490), and the remove_vibe_event_hooks docstring says it removes "only Specify-owned Vibe hooks and delete[s] an owned-only file" (vibe/__init__.py:553-590). The tests test_empty_events_preserve_comments_only_file and test_removal_preserves_user_comment_with_marker_text (tests/integrations/test_integration_vibe.py:496-528) treat user comments as content to keep.

Actual Behavior

[comment before a user hook]
  file exists after removal: True
  content: '[[hooks]]\nname = "user-hook"\ntype = "pre_tool"\ncommand = "guard-bash"\n'
[trailing user comment only]
  file exists after removal: False
  content: None

In the first case the line # NOTE: keep guard-bash first is gone. In the second case the file is deleted together with the user's comment.

Specify CLI Version

1.2.4.dev0 (main at 27d14e2)

AI Agent

Mistral Vibe

Operating System

Linux 6.18 (Ubuntu container)

Python Version

Python 3.12.3

Additional Context

A comment placed after a managed block is likely to describe the user content that follows it, as in the first case, so attaching it to the managed block loses user content.

Related: #4646 (merged) added this removal path. A review comment on that pull request flagged the comment loss, and no later commit addressed it.

Also reproduced on macOS (Darwin 25.6, arm64) with Python 3.14.4 and uv run python -I repro.py against 27d14e2; the output matches the above.

AI Disclosure

Found and reproduced with Claude Code (model: Claude Opus 4.8), run autonomously under human supervision. The human reporter re-ran every reproduction above before filing.

Activity

  1. added
    triage-must-haveVerdict: high-value, important work for Spec Kit — do first
    on Oct 23, 2026
  2. github-actions commented on Oct 29, 2026

    @github-actions
    Contributor

    Bug assessment — vibe-hook-comment-loss: Invalid · severity low


    Bug Assessment: Removing Vibe event hooks deletes user comments after managed blocks

    Report (summarized)

    Issue author hnipps reports that removing managed Vibe hooks drops user comments that follow a managed [[hooks]] block and may delete .vibe/hooks.toml when only a user comment remains. The report includes a repro script and expected behavior that comments should always be preserved.

    No additional issue comments were present at assessment time.

    Symptom

    Historically, comments after a managed block could be treated as part of the managed block and removed. The expected behavior in the report is that both standalone user comments and comments before user hooks are preserved, including comment-only files.

    Reproduction

    1. Create .vibe/hooks.toml with a managed [[hooks]] block (speckit_marker = true) followed by either:
      • a standalone comment before a user hook, or
      • a trailing user comment only.
    2. Run remove_vibe_event_hooks.
    3. Observe whether the comment remains and whether the file is retained.

    [NEEDS CLARIFICATION: local runtime policy denied executing the provided script in this environment, so this assessment is based on current code and tests rather than live execution.]

    Suspected Code Paths

    • src/specify_cli/integrations/vibe/__init__.py:390 (_managed_hook_spans) — explicitly documents and implements keeping a standalone comment after a managed block with following user content.
    • src/specify_cli/integrations/vibe/__init__.py:543 (_without_managed_hooks) — removes only spans identified as managed.
    • src/specify_cli/integrations/vibe/__init__.py:619 (remove_vibe_event_hooks) — applies cleanup and then deletes the file when only comments/blank lines remain.
    • src/specify_cli/integrations/vibe/__init__.py:647-652 — non_comment_content gate that treats comment-only files as removable.
    • tests/integrations/test_integration_vibe.py:352 (test_refresh_and_teardown_keep_comments_after_a_managed_hook) — verifies comments after managed blocks are preserved when user content remains.
    • tests/integrations/test_integration_vibe.py:520 (test_teardown_deletes_file_when_only_comments_remain) — verifies comment-only files are intentionally deleted.
    • tests/integrations/test_integration_vibe.py:650 (test_removal_ends_a_managed_hook_at_its_last_toml_line) — verifies comment-before-user-hook preservation across line endings.

    Root Cause Hypothesis

    The report mixes two behaviors that now differ in status: (1) comment loss before user hooks appears to have been addressed in current code/tests by bounded managed-span detection, while (2) deletion of comment-only files is currently intentional and test-covered policy. Confidence: high.

    Proposed Remediation

    Preferred: Close as not-a-bug for current main, with a maintainer note clarifying that comment-only .vibe/hooks.toml deletion is intentional teardown behavior. If product intent is to preserve comment-only files, that is a policy/UX change request rather than a regression bug.

    Alternatives (optional):

    • Change teardown policy to keep comment-only files by replacing the non_comment_content deletion gate in remove_vibe_event_hooks; trade-off: behavior divergence from current tests and codified parity rationale.
    • Clarify docs/docstrings to explicitly state comment-only files are removed during teardown; trade-off: no behavioral change, only expectation alignment.

    Files likely to change:

    • src/specify_cli/integrations/vibe/__init__.py (only if policy changes)
    • tests/integrations/test_integration_vibe.py (if policy changes)

    Tests to add or update:

    • If policy changes: update/replace test_teardown_deletes_file_when_only_comments_remain and add teardown test asserting comment-only file retention.
    • If policy is unchanged: add/adjust documentation assertions (or doc-focused tests if present) to remove ambiguity around teardown semantics.

    Risks & Considerations

    • Changing comment-only teardown behavior alters established cleanup semantics and may affect parity assumptions across integrations.
    • Leaving behavior unchanged without clearer documentation can continue to generate valid confusion for users who treat comments as retained content.

    Open Questions

    • [NEEDS CLARIFICATION: Should a comment-only .vibe/hooks.toml be considered user-owned content to preserve, or removable residual configuration during teardown?]
    • [NEEDS CLARIFICATION: Should remove_vibe_event_hooks docstring be tightened to explicitly define treatment of comment-only files?]

    Generated by 🐛 Assess Bug from Labeled Issue for #5079 · copilot · gpt53codex · 14 AIC · ⌖ 0.651 AIC · ⊞ 24.1K · ◷

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug-assessinvalidThis doesn't seem righttriage-must-haveVerdict: high-value, important work for Spec Kit — do first

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions