Give each Tomcat JSP engine its own Dependabot scan

.github/DEPENDENCY_DECISIONS.md

@@ -134,7 +134,7 @@ should confirm that staging still authenticates before the release relies on it.
 | Plexus Utils 3.6.2 → 4.1.0 in GPG/Central plugin dependencies | Keep the reviewed 3.6.2 mitigation. The [upstream 4.x migration](https://github.com/codehaus-plexus/plexus-utils) moves XML utilities to a separate artifact; 4.1 also changes DirectoryScanner default exclusions. A newer major is not a drop-in plugin-realm security fix. Reconsider with actual plugin linkage, isolated signing/bundle/rehearsal evidence, transitive-advisory review and repeatable payloads. |
 | javax Servlet 3.0.1 → 4.0.1; EL 2.2.5 → 3.0.0 | Keep the test-only minimum API fixtures. These are not bundled production container implementations. Modern engine coverage is separate; replacing the minimum tests would remove evidence for existing consumers. |
 | Jakarta Pages 3.0.0 → 4.0.0 | Keep the published provided Pages 3 API and existing `[3.0,4)` package ranges. Reconsider only with a reviewed minimum-runtime/API migration, public POM implications and compatibility evidence. |
-| Jasper/annotations 9.0.122 or 10.1.60 → 11.0.26 in the isolated tag fixtures | Keep coherent Tomcat 9 (`javax`) and 10.1 (`jakarta`) engines. PR #188 fails the javax engine with missing `javax.servlet.jsp.tagext.SimpleTagSupport` after the Tomcat 11 switch. The optional Boot/browser WAR already exercises Tomcat 11. Patch updates within each intended engine line remain candidates for manual review; Dependabot's lowest-version classification cannot automate both lines independently. Cross-line migration needs a separate coverage decision. |
+| Jasper/annotations 9.0.122 or 10.1.60 → 11.0.26 in the isolated tag fixtures | Keep coherent Tomcat 9 (`javax`) and 10.1 (`jakarta`) engines. PR #188 fails the javax engine with missing `javax.servlet.jsp.tagext.SimpleTagSupport` after the Tomcat 11 switch. The optional Boot/browser WAR already exercises Tomcat 11. Patch updates within each engine line come from the separate `/jsp` and `/jakarta` Dependabot entries described below. Cross-line migration needs a separate coverage decision. |
 
 A dependency's test/build scope does not dismiss an advisory. Check each finding's
 actual affected versions, executed path and proposed remedy; use a supported fix
@@ -157,20 +157,22 @@ record; a grouped PR closure is not proof that every proposed upgrade was applie
 
 ## Future Dependabot proposals
 
-The [configuration](dependabot.yml) excludes fourteen coordinates from the broad
-Maven **version-update group**: the eleven baseline-sensitive coordinates above,
+The [configuration](dependabot.yml) excludes twelve coordinates from the broad
+Maven **version-update group**: the nine baseline-sensitive coordinates above,
 the two Boot plugin-realm Jackson coordinates and the Maven distribution used by
-the wrapper. For the ten coordinates with a rejected proposal in #218–#227 and
-the two Jackson coordinates from #232, it also ignores only the SemVer minor or
-major version-update classes covered by the decisions above. The Maven wrapper
-has no ignore rule.
+the wrapper. For the eight of those coordinates with a rejected proposal in
+#218–#227 and the two Jackson coordinates from #232, it also ignores only the
+SemVer minor or major version-update classes covered by the decisions above. The
+Maven wrapper has no ignore rule. The two Tomcat engine coordinates, the other
+rejected proposals in that range, have their own entries described below.
 This prevents the weekly job from recreating proposals that merely replace
 historical comparators, minimum-consumer fixtures, coherent servlet-engine
 lines, or reviewed tool majors. The accepted Felix Maven Bundle Plugin remains
 individually updateable.
 
-Every new rule uses `update-types`, not a version range or an unqualified
-coordinate ignore. GitHub documents that `update-types` affects version updates,
+Apart from the Felix framework fixture and the root entry's two Tomcat rules,
+every rule uses `update-types`, not a version range or an unqualified coordinate
+ignore. GitHub documents that `update-types` affects version updates,
 not security updates, so the Maven security-update group and security alerts
 remain eligible:
 <https://docs.github.com/en/code-security/reference/supply-chain-security/dependabot-options-reference#update-types-ignore>.
@@ -190,13 +192,31 @@ Dependabot's [`DependencySet`](https://github.com/dependabot/dependabot-core/blo
 and [`IgnoreCondition`](https://github.com/dependabot/dependabot-core/blob/a6e095f540c6542facc06ba47f66418da50d669c/common/lib/dependabot/config/ignore_condition.rb#L50-L65)
 implementations.
 
+For the Tomcat JSP engine fixtures, that combination writes a wrong version
+instead of suppressing one. `jsp/` pins Jasper and the annotations API on Tomcat
+9.0 (`javax`) and `jakarta/` pins the same coordinates on 10.1 (`jakarta`). The
+reactor scan combined them from 9.0 and wrote 9.0.123 into both modules in #238
+and #239, which broke the Jakarta engine; #241 applied each line's patch instead.
+The root entry therefore ignores both coordinates outright, so its combined scan
+cannot propose a cross-line version or security update. Separate `/jsp` and
+`/jakarta` entries allow only those two coordinates. A module scan fetches the
+module and its parent POM but not sibling modules (Dependabot's Maven
+[`FileFetcher`](https://github.com/dependabot/dependabot-core/blob/fed09ef6f7252c3d527cf592f871a6b83fdeea17/maven/lib/dependabot/maven/file_fetcher.rb#L42-L48)),
+so each sees one engine line and proposes its patch release as one grouped pull
+request. These entries ignore only major version updates, which would replace
+the engine, so security updates remain eligible there. A policy test requires
+every reactor module that pins these coordinates to have its own entry on its
+expected line.
+
 The root directory monitors the complete Maven reactor once; listing each reactor
-module again produced duplicate pull requests. The standalone
-`compatibility/dependencies` Maven project remains a separate monitored directory.
+module again produced duplicate pull requests. The Tomcat-only `/jsp` and
+`/jakarta` entries do not duplicate it because the root entry ignores everything
+they allow. The standalone `compatibility/dependencies` Maven project remains a
+separate monitored directory.
 No security-alert dismissals or automatic merges are introduced. The pre-existing
 unqualified Felix framework fixture exception remains scoped and documented in
-[CI/security operations](CI_SECURITY.md); it is the only ignore rule without an
-explicit version-update class.
+[CI/security operations](CI_SECURITY.md); it and the root entry's two Tomcat
+rules are the only ignore rules without an explicit version-update class.
 
 When a new security proposal or manually raised compatibility change revisits a
 deferred baseline, compare it with this dated record and current upstream evidence.

.github/dependabot.yml

@@ -30,8 +30,6 @@ updates:
           - jakarta.servlet.jsp:jakarta.servlet.jsp-api
           - jakarta.servlet:jakarta.servlet-api
           - jakarta.el:jakarta.el-api
-          - org.apache.tomcat.embed:tomcat-embed-jasper
-          - org.apache.tomcat:tomcat-annotations-api
           # Exact Boot plugin-realm pins checked by CI policy tests.
           - tools.jackson.core:jackson-databind
           - tools.jackson.core:jackson-core
@@ -82,12 +80,12 @@ updates:
         update-types:
           - version-update:semver-minor
           - version-update:semver-major
+      # The JSP engine fixtures pin Tomcat 9.0 in jsp/ and 10.1 in jakarta/
+      # under the same coordinates. This scan would combine them from 9.0 and
+      # write that line's release into both modules, so the /jsp and /jakarta
+      # entries below own these coordinates instead.
       - dependency-name: org.apache.tomcat.embed:tomcat-embed-jasper
-        update-types:
-          - version-update:semver-major
       - dependency-name: org.apache.tomcat:tomcat-annotations-api
-        update-types:
-          - version-update:semver-major
       # Keep Boot 4.1.1's plugin realm on its own 3.1 line; patches and
       # security updates remain eligible.
       - dependency-name: tools.jackson.core:jackson-databind
@@ -98,6 +96,52 @@ updates:
         update-types:
           - version-update:semver-minor
           - version-update:semver-major
+  # Each JSP engine fixture gets its own scan, limited to Tomcat. A module scan
+  # reads the module and its parent POM but not sibling modules, so it sees one
+  # engine line and proposes that line's patch release. Major updates would
+  # replace the engine; see DEPENDENCY_DECISIONS.md.
+  - package-ecosystem: maven
+    directory: /jsp
+    schedule:
+      interval: weekly
+      day: monday
+      time: '09:00'
+      timezone: Etc/UTC
+    open-pull-requests-limit: 1
+    allow:
+      - dependency-name: org.apache.tomcat.embed:tomcat-embed-jasper
+      - dependency-name: org.apache.tomcat:tomcat-annotations-api
+    groups:
+      tomcat-9-jsp-engine:
+        patterns: ['*']
+    ignore:
+      - dependency-name: org.apache.tomcat.embed:tomcat-embed-jasper
+        update-types:
+          - version-update:semver-major
+      - dependency-name: org.apache.tomcat:tomcat-annotations-api
+        update-types:
+          - version-update:semver-major
+  - package-ecosystem: maven
+    directory: /jakarta
+    schedule:
+      interval: weekly
+      day: monday
+      time: '09:00'
+      timezone: Etc/UTC
+    open-pull-requests-limit: 1
+    allow:
+      - dependency-name: org.apache.tomcat.embed:tomcat-embed-jasper
+      - dependency-name: org.apache.tomcat:tomcat-annotations-api
+    groups:
+      tomcat-10-jakarta-engine:
+        patterns: ['*']
+    ignore:
+      - dependency-name: org.apache.tomcat.embed:tomcat-embed-jasper
+        update-types:
+          - version-update:semver-major
+      - dependency-name: org.apache.tomcat:tomcat-annotations-api
+        update-types:
+          - version-update:semver-major
   - package-ecosystem: github-actions
     directory: /
     schedule:

scripts/tests/test_ci_policy.py

@@ -209,16 +209,16 @@ def test_dependabot_scopes_reviewed_ignores_to_version_updates(self):
             'jakarta.servlet.jsp:jakarta.servlet.jsp-api': minor_and_major,
             'jakarta.servlet:jakarta.servlet-api': minor_and_major,
             'jakarta.el:jakarta.el-api': minor_and_major,
-            'org.apache.tomcat.embed:tomcat-embed-jasper': [
-                'version-update:semver-major'],
-            'org.apache.tomcat:tomcat-annotations-api': [
-                'version-update:semver-major'],
+            'org.apache.tomcat.embed:tomcat-embed-jasper': [],
+            'org.apache.tomcat:tomcat-annotations-api': [],
             'tools.jackson.core:jackson-databind': minor_and_major,
             'tools.jackson.core:jackson-core': minor_and_major,
         }
         self.assertEqual(expected, rules)
         self.assertEqual(
-            {'org.apache.felix:org.apache.felix.framework'},
+            {'org.apache.felix:org.apache.felix.framework',
+             'org.apache.tomcat.embed:tomcat-embed-jasper',
+             'org.apache.tomcat:tomcat-annotations-api'},
             {name for name, update_types in rules.items() if not update_types})
 
         mutations = {
@@ -264,6 +264,62 @@ def test_dependabot_routes_reviewed_coordinates_out_of_broad_group(self):
         self.assertEqual(set(), versioned.difference(excluded))
         self.assertNotIn('org.apache.maven:apache-maven', ignore)
 
+    def test_dependabot_scans_each_tomcat_engine_line_separately(self):
+        # jsp/ and jakarta/ pin different Tomcat lines under the same
+        # coordinates. The reactor scan combines them from the lowest version
+        # and writes that line into both modules (#238, #239), so each module
+        # needs its own Tomcat-only entry.
+        tomcat = {'org.apache.tomcat.embed:tomcat-embed-jasper',
+                  'org.apache.tomcat:tomcat-annotations-api'}
+        dependabot = (ROOT / '.github/dependabot.yml').read_text()
+        entries = dependabot.split('\n  - package-ecosystem: ')[1:]
+        maven = [entry for entry in entries if entry.startswith('maven\n')]
+
+        root_ignore = parse_dependabot_ignore(maven[0].split('    ignore:\n', 1)[1])
+        self.assertEqual({name: [] for name in tomcat},
+                         {name: root_ignore.get(name) for name in tomcat})
+
+        engines = {}
+        for entry in maven[1:]:
+            directory = re.search(r'^    directory: (\S+)$', entry, re.M)
+            self.assertIsNotNone(directory, entry)
+            engines[directory.group(1)] = entry
+        self.assertEqual(len(maven) - 1, len(engines))
+
+        for directory, entry in engines.items():
+            with self.subTest(directory=directory):
+                allow = entry.split('    allow:\n', 1)[1].split('    groups:\n', 1)[0]
+                allowed = re.findall(r'^      - dependency-name: (\S+)$', allow, re.M)
+                self.assertEqual(sorted(tomcat), sorted(allowed))
+                self.assertEqual(1, entry.count("        patterns: ['*']"))
+                self.assertEqual(
+                    {name: ['version-update:semver-major'] for name in tomcat},
+                    parse_dependabot_ignore(entry.split('    ignore:\n', 1)[1]))
+
+        expected_lines = {'/jsp': '9.0.', '/jakarta': '10.1.'}
+        root = ET.parse(ROOT / 'pom.xml').getroot()
+        pinned = {}
+        for node in root.findall('.//p:modules/p:module', version.NS):
+            module = node.text.strip()
+            pom = ET.parse(ROOT / module / 'pom.xml').getroot()
+            for dependency in pom.findall('.//p:dependency', version.NS):
+                name = '%s:%s' % (
+                    dependency.findtext('p:groupId', namespaces=version.NS),
+                    dependency.findtext('p:artifactId', namespaces=version.NS))
+                declared = dependency.findtext('p:version', namespaces=version.NS)
+                if name in tomcat and declared:
+                    pinned.setdefault('/' + module, set()).add(declared.strip())
+
+        # A module that pins Tomcat without its own entry would get no updates.
+        self.assertEqual(set(expected_lines), set(engines))
+        self.assertEqual(set(expected_lines), set(pinned))
+        for directory, versions in pinned.items():
+            with self.subTest(directory=directory):
+                self.assertEqual(1, len(versions), versions)
+                self.assertTrue(
+                    next(iter(versions)).startswith(expected_lines[directory]),
+                    versions)
+
     def test_only_executed_plugins_are_submitted(self):
         workflow = (ROOT / '.github/workflows/dependency-submission.yaml').read_text()
         self.assertIn('-DincludeArtifactIds=', workflow)