diff --git a/.agents/docs/2026-10-19-xpkgindex-update-design.md b/.agents/docs/2026-10-19-xpkgindex-update-design.md index 7febaefb..8fc2a13f 100644 --- a/.agents/docs/2026-10-19-xpkgindex-update-design.md +++ b/.agents/docs/2026-10-19-xpkgindex-update-design.md @@ -1,7 +1,7 @@ # xpkgindex-update:两个索引共用的声明式版本更新工具 日期:2026-10-19 -状态:实现联调中。工具开发 PR:openxlings/xpkgindex-update#1;mcpp 为首个 draft CI 接入仓库。 +状态:工具和索引接入已合并;2026-10-20 补齐每日 bot PR 与自动 CI,验收以第 16 节为准。 ## 1. 目标与结论 @@ -546,3 +546,80 @@ package = { 先等待索引候选与现有消费者 CI 全部成功,再 squash 合并 #543。将 #544 同步到 main 基线,以工具重新生成共享格式并验证后合入。随后手动触发 main 上的 Action,使用可选 package 输入限定一个真实更新包;从 artifact 取回同一份固定计划,由已合并的 CLI 创建针对 main 的更新 PR,等待实际构建、运行、锁文件及旧版本保留检查成功后 squash 合入。 写 PR 使用本地已有授权,不新建长期 token secret。现有 libmysqlclient 源故障豁免继续明确保留,不改变其历史版本。此节是实施时的计划及已核实证据,最终合并与 CI 结果另见汇报和 PR 记录。 + +## 15. 共享资源与真实更新验收记录 + +本节为用户授权 squash 合并后的真实结果;不创建 docs-only 提交以重复触发整套 CI。 + +| PR | 合并提交 | 结果 | +|---|---|---| +| [openxlings/xpkgindex-update #1](https://github.com/openxlings/xpkgindex-update/pull/1) | `23c5e04948c55e606bc56ee2c649ddd4849b7ec6` | squash 合并;59 测试、六种 OS/Python 组合及 Action/打包检查成功;工具 main CI 成功 | +| [mcpp-index #543](https://github.com/mcpplibs/mcpp-index/pull/543) | `1a514dbd1d71de29f0c8e6a9c854bab92b928df9` | squash 合并前 74 个执行 checks 成功,52/52 共享资源候选构建/运行/锁文件检查和 13 个消费者回归分片成功 | +| [mcpp-index #544](https://github.com/mcpplibs/mcpp-index/pull/544) | `54cad76b3058a46ceec4c63b7bfa2122d5fdb194` | 以 main 为基线,20 个执行 checks 成功后 squash 合并;hiredis 1.5.0 单资源定义,三个平台引用;旧 1.2.0 不变 | +| [mcpp-index #545](https://github.com/mcpplibs/mcpp-index/pull/545) | `192c8653516a3057e6103ffacd9bc0e6e613b921` | 真实 Action 计划生成的 yyjson 0.13.0 更新,20 个执行 checks 成功后 squash 合并;旧 0.12.0 不变 | + +### 15.1 真正的 main 更新流程 + +1. 在 main 的 `54cad76` 手动触发 [Action 39308549012](https://github.com/mcpplibs/mcpp-index/actions/runs/39308549012),输入 `package=compat:yyjson`。 +2. Action 扫描确认 `0.12.0 -> 0.13.0`,资源完整下载并计算 SHA256;四种平台/工具链的实际构建/运行/锁文件检查全部成功。 +3. 原样取回 artifact `12215250910` 的固定计划,不重新构造计划;用合并提交 `23c5e04` 的 CLI 消费,自动创建 #545。使用已有本地授权,不配置新的长期 token secret。 +4. 再次执行同一计划的 propose 返回 `pending`,未创建重复 PR。 +5. #545 的实际 diff 为一个文件新增 10 行,无删除;URL/SHA 只定义一次,三个平台引用同一资源。全部 PR CI 成功后按已验证 head squash 合并。 +6. 合并后重新 check 返回 `compat:yyjson 0.13.0 up-to-date`;再 propose 返回 `{ "packages": [] }`。 +7. 独立 guard 继续确认原始基线 `981a4f2` 的 254 个描述符历史版本条目全部保留。 + +### 15.2 发布索引的普通客户端验证 + +- 最新 main 的 [发布](https://github.com/mcpplibs/mcpp-index/actions/runs/39309716725)、[站点部署](https://github.com/mcpplibs/mcpp-index/actions/runs/39309716804)、[消费者验证](https://github.com/mcpplibs/mcpp-index/actions/runs/39309716862) 均成功。 +- 固定客户端 `mcpp 2026.10.1.2` 刷新正式发布索引,报告 mcpplibs revision `192c865`。 +- 新建独立消费者仅依赖 `compat:yyjson = 0.13.0`,没有 `[indices]` 或本地索引覆盖。 +- 静态断言版本宏为 0.13.0;运行时 `yyjson_version()` 与版本宏一致;真实解析布尔 JSON 成功,输出 `PUBLISHED_YYJSON_0_13_0=PASS`。 +- 实际构建/运行结果为 1 passed、0 failed;锁文件确认为 `compat.yyjson`、`version = 0.13.0`、`source = index+compat@0.13.0`。 + +### 15.3 保留的边界 + +旧版本不做批量去重迁移,原镜像、哈希、注释和顺序保持原样。共享优化用于新追加的版本,后续追加复用同一个表。URL 不同的平台不因哈希相同而混用来源。工具不执行 Lua、不自动改安装配方、不自动 mirror、不自动 merge;本轮 squash 合并由用户明确授权。 + +原有两个条件性通知/计时发布任务按原规则跳过,不是构建测试跳过。wellwei/libmysqlclient 的源故障豁免继续明确显示 error 和 NOT TESTED,不计作成功构建;其余失败均未屏蔽。xim-pkgindex 的 latest.ref 变更仍未启用。README 为 34 行,详细使用文档独立。 + +较早的 integration main 提交 `1a514db` 的 [自动完整回归 39307639699](https://github.com/mcpplibs/mcpp-index/actions/runs/39307639699) 也已全部成功,19 个执行 jobs 全部通过,无剩余失败或待完成构建。至此,工具 main、索引接入 PR/main、hiredis PR/main、真实 yyjson 更新 PR/main 的构建、测试、发布及部署均完成。 + +## 16. 每日无人值守更新与贡献规范(2026-10-20) + +### 16.1 补齐原验收缺口 + +#545 是 Action 扫描、候选 CI 后,由本地 CLI 使用 Sunrisepeak 的已有授权创建的 PR。 +它证明了更新内容及发布有效,但不证明定时触发、Action 内提 PR 或 bot PR 无人值守 CI。 +此前“完成”的判断范围错误;本节单独验收这三个环节。 + +### 16.2 正式流程 + +- 使用原设计的 `17 2 * * *`,每日 02:17 UTC / 11:17 JST;手动默认分支 dispatch 走同一流程。 +- 扫描只读;固定 artifact 计划交给独立 propose job,不依赖本地 CLI 或本地用户 token。 +- 使用内置 `GITHUB_TOKEN`,PR 作者为 `github-actions[bot]`,不新增长期 secret。 +- 组织此前阻止 Actions 创建 PR(API 409);用户已开启组织设置,仓库级也已开启。 + 仓库默认权限仍为 read,仅 propose job 获得 contents/pull-requests/actions write。 +- 按中央 `max_prs: 3` 每轮最多三个新 PR;每包一个打开 PR,关闭候选不重开。 + 已有人工作业 PR 修改同一描述符时也不重复创建(例如已有 httplib #542);人工 PR 继续走普通 PR CI。 +- propose 后显式 dispatch validate、候选、site、openkal 四个 workflow,避免依赖受限的普通 PR 事件。 +- validate/openkal 以 PR number、bot 身份、同仓 head/base、默认 base、head SHA 和单描述符 diff 校验请求, + 然后选择受影响成员;普通不带 update_pr 的手动 validate 仍保持完整回归。 +- 新版本在 PR CI 内验证,不要求全量候选全部成功才能先开 PR;失败升级保留红色结果,由维护者适配。 +- 已有 PR 可补发缺失 dispatch;已有运行不自动重跑,失败仍可见。默认分支移动后拒绝旧扫描计划,须重跑。 +- 写入仅默认分支 schedule/dispatch 可执行;普通 PR 和更新分支都不会提 PR,无递归,也不自动审批或合并。 + +### 16.3 agent 与文档约定 + +新增或维护符合条件的公共 GitHub release/数字 tag 包时,skill 要求主动添加并验证 +`extensions.autoupdate`。快照、独立源码版本、非 GitHub 来源和需要修改配方的升级须说明例外。 +新版本仍只追加,旧版本/mirror 不改;不要求未来新版本自动 mirror。 + +两份根 README 精简为用途、使用、贡献与链接;删除逐包 Additional example、重复示例表与 agent 长提示词。 +包形态和示例保留在专门文档;自动更新新增中英文独立文档并登记站点导航。 +skill 明确普通包 PR 不改根 README 或站点配置;用户明确要求的文档整理除外。 + +### 16.4 验收 + +工具 #2 已 squash 合并为 `6c0bee2941e36132bc2ac3d1f7edbef8b5790320`,61 项回归和全部 7 个 CI checks 成功;索引固定使用此提交。 +本地 8 项编排/基线回归覆盖 bot/head/仓库/文件边界、默认分支移动、局部 dispatch 恢复、重复 PR、人工 PR 和局部提案失败。 +Actionlint、真实 PR CI 与 bot 生产验证结果在本节补充;未完成前不再报告无人值守链路已通过。 diff --git a/.agents/skills/add-mcpp-index-package/SKILL.md b/.agents/skills/add-mcpp-index-package/SKILL.md index 7037206f..907c4bd8 100644 --- a/.agents/skills/add-mcpp-index-package/SKILL.md +++ b/.agents/skills/add-mcpp-index-package/SKILL.md @@ -6,7 +6,7 @@ description: Use when adding a new third-party library/package to the mcpp-index # 向 mcpp-index 新增一个库(标准作业流程) 本文件定义将上游库收录进 [`mcpp-index`](https://github.com/mcpplibs/mcpp-index) 的标准流程。其产出包括:一个 -`pkgs//.lua` 描述符、一个 GitCode CN 镜像、一个位于 `tests/examples/` 的最小工程、一条 README 记录, +`pkgs//.lua` 描述符、可用时的 GitCode CN 镜像、一个位于 `tests/examples/` 的最小工程、适用时的自动更新声明, 以及一份设计文档。完整流程为:本地验证通过,提交 PR,CI 全部通过,由维护者合并。合并后,`publish-artifact.yml` 将自动重新发布 index artifact。 @@ -21,6 +21,7 @@ description: Use when adding a new third-party library/package to the mcpp-index - [docs/zh/package-types.md](../../../docs/zh/package-types.md) —— 四种库形态的描述符模板与样例路径。 - [docs/zh/cn-mirror.md](../../../docs/zh/cn-mirror.md) —— CN 镜像闭环,含无 `mcpp-res` 权限时的回退方案。 +- [docs/zh/automatic-updates.md](../../../docs/zh/automatic-updates.md) —— 自动更新适用条件、声明、每日 bot PR 和候选 CI。 - [docs/zh/repository-and-schema.md](../../../docs/zh/repository-and-schema.md) —— 仓库结构、schema、CI 行为、关键文件与注意事项。 ## 适用范围 @@ -71,7 +72,7 @@ description: Use when adding a new third-party library/package to the mcpp-index 编译为 no-op `main`(`#ifdef __linux__ … #else int main(){return 0;} #endif` 模式)。 - 如需测试 feature,依赖采用长式声明 `name = { version = "…", features = ["…"] }`。 7. **本地验证**(使用与 CI 相同版本的 mcpp,详见下文“本地验证”)。必须实际执行 `mcpp test -p ` 并通过。 -8. **更新 README**:在对应分类表中新增一条记录 —— `README.md`(英文,默认)与 `README.zh-CN.md`(中文)两份都要更新。 +8. **接入自动更新**:按下节检查适用条件;符合条件的包必须添加 `extensions.autoupdate` 并验证。普通包 PR 不修改两份根 README,也不为新包修改站点配置;站点会自动发现描述符。 9. **撰写设计文档** `.agents/docs/-add--plan.md`,记录形态判定、镜像、feature 评估、验证结论 与注意事项。 10. **本地 lint**:在本地复现 `validate.yml` 的 lint 检查(语法、必填字段、无前导 v、镜像表校验)。详见 @@ -82,6 +83,23 @@ description: Use when adding a new third-party library/package to the mcpp-index (日志中 `selected members: `),`lint`(含 `mcpp xpkg parse`)与 `mirror-cn-reachable` 覆盖新增描述符与 CN url。合并由维护者执行。 +## 自动更新与 PR 范围 + +- 对公共 GitHub release/数字 tag、稳定下载模板、无需改变构建配方的包,主动添加 + `package.extensions.autoupdate = { tag = "v${version}", url = "https://github.com/acme/foo/archive/refs/tags/${tag}.tar.gz" }`。 + 实际 tag 不带 `v` 时使用 `${version}`;按真实上游规则填写,不照抄示例。 +- 共享 URL 或按平台映射均可。新版本可以直接使用 GitHub,不要求自动 mirror;不要声明尚不存在的 CN 模板。 +- commit 快照、独立源码/包版本映射、非 GitHub 来源、需要改配方的升级不强行接入;在 PR 中说明原因。 +- 使用 workflow 固定的工具提交运行 `inspect --package ` 和 + `check --package --output plan.json`;确实存在候选时,在干净 worktree 中 `apply` 并检查补丁。 + 新增版本必须有真实构建、运行、锁文件验证;不能只用仍依赖旧版本的测试证明升级成功。 +- 现有版本、URL、镜像、SHA256 和注释均保留;不迁移旧内联条目。相同新资源可以前置定义一次并由平台引用。 +- 每日默认分支 Action 由 `github-actions[bot]` 开 PR,并显式 dispatch 候选、消费者、站点和 openkal CI。 + 它不审批、不自动合并;普通 PR 和更新分支不会创建更多 PR。 +- **一般包 PR 不改 README**:不要追加 “Additional example / 新增示例”、包列表、更新记录或 agent 提示词。 + 用户明确要求整理文档或说明确有变化时才改入口文档。新增包通常也不改 `.xpkgindex.json`。 +- 详细字段、例外和认证机制见 [自动更新文档](../../../docs/zh/automatic-updates.md)。 + ## feature 机制 mcpp **0.0.68** 的包描述符 `features` 表**仅能门控 `sources`**;其余子字段会被解析器忽略(经 `manifest.cppm` 的 diff --git a/.github/workflows/openkal-compat.yml b/.github/workflows/openkal-compat.yml index 302cd33d..606683c1 100644 --- a/.github/workflows/openkal-compat.yml +++ b/.github/workflows/openkal-compat.yml @@ -13,6 +13,10 @@ on: - cron: "0 7 * * 0" workflow_dispatch: inputs: + update_pr: + description: "Bot update PR number; measure only members affected by its exact head" + required: false + default: "" members: description: "Space-separated members to measure (empty = every listed member)" required: false @@ -62,6 +66,9 @@ jobs: name: measure (linux, windows through wine) runs-on: ubuntu-24.04 timeout-minutes: 240 + permissions: + contents: read + pull-requests: read outputs: members: ${{ steps.select.outputs.members }} steps: @@ -75,11 +82,20 @@ jobs: - name: Select the members id: select shell: bash + env: + UPDATE_PR: ${{ inputs.update_pr }} + GH_TOKEN: ${{ github.token }} + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} run: | if [ "${{ github.event_name }}" = "pull_request" ]; then base=$(git merge-base "origin/${{ github.base_ref }}" HEAD) files=$(git diff --name-only "$base" HEAD) members=$(python3 tests/openkal/compat.py select --base "$base" $files) + elif [ -n "$UPDATE_PR" ]; then + requested_base=$(python3 tests/autoupdate/automation.py verify --pr "$UPDATE_PR") + base=$(git merge-base "$requested_base" HEAD) + files=$(git diff --name-only "$base" HEAD) + members=$(python3 tests/openkal/compat.py select --base "$base" $files) else members="${{ github.event.inputs.members }}" [ -n "$members" ] || members=$(python3 tests/openkal/compat.py select tests/openkal/) diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index 3614cce5..2b9592c0 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -23,6 +23,10 @@ on: - cron: "0 6 * * 0" workflow_dispatch: inputs: + update_pr: + description: "Bot update PR number; dispatch its exact head with selective regression" + type: string + required: false cache: description: "Package build cache — 'local' rebuilds every dependency per member, which is what the timing table should be read against when comparing" type: choice @@ -680,6 +684,9 @@ jobs: # a selective one does not. select: runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: read outputs: members: ${{ steps.fanout.outputs.members }} matrix: ${{ steps.fanout.outputs.matrix }} @@ -716,6 +723,10 @@ jobs: - name: Select affected workspace members id: plan shell: bash + env: + UPDATE_PR: ${{ inputs.update_pr }} + GH_TOKEN: ${{ github.token }} + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} run: | # A FULL RUN IS NO LONGER SOMETHING A PR CAN TRIGGER BY ACCIDENT. # @@ -799,6 +810,13 @@ jobs: || { widen "push predecessor $base not in history"; usable=0; } fi if [ "$usable" = 1 ]; then range="$base..HEAD"; else range=""; fi ;; + workflow_dispatch) + if [ -n "$UPDATE_PR" ]; then + base=$(python3 tests/autoupdate/automation.py verify --pr "$UPDATE_PR") + range="$base...HEAD" + else + full "event=workflow_dispatch" + fi ;; *) full "event=${{ github.event_name }}" ;; esac @@ -1743,7 +1761,7 @@ jobs: if: >- always() && (needs.workspace.result == 'failure' || needs.mirror-cn-reachable.result == 'failure') - && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') + && (github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && inputs.update_pr == '')) runs-on: ubuntu-latest permissions: issues: write diff --git a/.github/workflows/xpkgindex-update.yml b/.github/workflows/xpkgindex-update.yml index 524fc007..448a196b 100644 --- a/.github/workflows/xpkgindex-update.yml +++ b/.github/workflows/xpkgindex-update.yml @@ -1,6 +1,8 @@ name: xpkgindex-update on: + schedule: + - cron: "17 2 * * *" # Daily 02:17 UTC (11:17 JST). pull_request: paths: - 'pkgs/**/*.lua' @@ -19,7 +21,7 @@ permissions: concurrency: group: xpkgindex-update-${{ github.ref }} - cancel-in-progress: true + cancel-in-progress: ${{ github.event_name == 'pull_request' }} env: PYTHONUTF8: '1' @@ -43,7 +45,7 @@ jobs: - uses: actions/checkout@v4 with: repository: openxlings/xpkgindex-update - ref: 23c5e04948c55e606bc56ee2c649ddd4849b7ec6 + ref: 6c0bee2941e36132bc2ac3d1f7edbef8b5790320 path: .updater persist-credentials: false - name: Use the pinned updater source @@ -60,7 +62,7 @@ jobs: - name: Discover and download real upstream candidates id: scan continue-on-error: true # The mandatory gate below accepts only the approved source outage. - uses: openxlings/xpkgindex-update@23c5e04948c55e606bc56ee2c649ddd4849b7ec6 + uses: openxlings/xpkgindex-update@6c0bee2941e36132bc2ac3d1f7edbef8b5790320 with: mode: check token: ${{ github.token }} @@ -97,7 +99,9 @@ jobs: name: candidate (${{ matrix.id }} ${{ matrix.candidate }}, ${{ matrix.platform }}, ${{ matrix.toolchain }}) needs: scan if: >- - always() && !cancelled() && needs.scan.outputs.matrix != '' && needs.scan.outputs.matrix != '{"include": []}' + always() && !cancelled() && + (github.event_name == 'pull_request' || github.ref_name != github.event.repository.default_branch) && + needs.scan.outputs.matrix != '' && needs.scan.outputs.matrix != '{"include": []}' strategy: fail-fast: false max-parallel: 8 @@ -115,7 +119,7 @@ jobs: - uses: actions/checkout@v4 with: repository: openxlings/xpkgindex-update - ref: 23c5e04948c55e606bc56ee2c649ddd4849b7ec6 + ref: 6c0bee2941e36132bc2ac3d1f7edbef8b5790320 path: .updater persist-credentials: false - uses: actions/download-artifact@v4 @@ -212,3 +216,51 @@ jobs: name: candidate-diagnostics-${{ strategy.job-index }} path: candidate-diagnostics if-no-files-found: ignore + + propose: + # PR validation and dispatches on update branches can never create more PRs. + if: >- + (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') && + github.ref_name == github.event.repository.default_branch + needs: scan + runs-on: ubuntu-latest + timeout-minutes: 45 + permissions: + contents: write + pull-requests: write + actions: write + steps: + - uses: actions/checkout@v4 + with: + ref: ${{ github.sha }} + fetch-depth: 0 + persist-credentials: false + - uses: actions/setup-python@v5 + with: + python-version: '3.12' + - uses: actions/checkout@v4 + with: + repository: openxlings/xpkgindex-update + ref: 6c0bee2941e36132bc2ac3d1f7edbef8b5790320 + path: .updater + persist-credentials: false + - uses: actions/download-artifact@v4 + with: + name: xpkgindex-update-plan + path: .update-plan + - name: Create bot PRs from the fixed scan plan and dispatch their CI + env: + GH_TOKEN: ${{ github.token }} + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + run: | + export PYTHONPATH="$PWD/.updater/src" + git switch -C "$DEFAULT_BRANCH" "$GITHUB_SHA" + python tests/autoupdate/automation.py propose \ + --plan .update-plan/xpkgindex-update-plan.json --output automation-report.json + - uses: actions/upload-artifact@v4 + if: always() + with: + name: xpkgindex-update-proposals + path: | + automation-report.json + proposals.json diff --git a/.xpkgindex.json b/.xpkgindex.json index fda8f365..50b6cbbf 100644 --- a/.xpkgindex.json +++ b/.xpkgindex.json @@ -181,6 +181,18 @@ "zh": "docs/zh/descriptor-examples.md" } }, + { + "slug": "automatic-updates", + "title": { + "en": "Automatic updates", + "zh": "自动更新", + "zh-Hant": "自動更新" + }, + "path": "docs/automatic-updates.md", + "translations": { + "zh": "docs/zh/automatic-updates.md" + } + }, { "slug": "cn-mirror", "title": { diff --git a/README.md b/README.md index 294339a9..da76f448 100644 --- a/README.md +++ b/README.md @@ -1,101 +1,37 @@ # mcpp-index -**English** | [简体中文](README.zh-CN.md) +[English](README.md) · [简体中文](README.zh-CN.md) -> The default package index for the [`mcpp`](https://github.com/mcpp-community/mcpp) build tool. -> Browse every package online: **https://mcpplibs.github.io/mcpp-index/** +The default package index for [mcpp](https://github.com/mcpp-community/mcpp): +C++ modules and third-party C/C++ libraries built from source. -The C++23 packages `mcpp` can `add` directly: modular libraries ready to `import`, and third-party C/C++ libraries -built from upstream sources in `compat` form. Each package is one `pkgs//.lua` descriptor. - -> **Requires mcpp 2026.9.18.3 or later** (`min_mcpp` in [`index.toml`](index.toml)). Older engines silently misbuild -> `[c-abi]` packages such as `openkal-musl`. Upgrade: `xlings install mcpp --force`. +[Browse packages](https://mcpplibs.github.io/mcpp-index/) · [Quick start](docs/quick-start.md) ## Usage -```bash -mcpp add ftxui@6.1.9 # add the dependency to mcpp.toml -mcpp build # fetch sources and build; dependencies propagate along the chain - -mcpp search # search and refresh the index -mcpp self config --mirror CN # switch to the CN mirror; GLOBAL upstream is the default -``` - -## Package kinds - -- **Native mcpp module libraries** (Form A): upstream carries its own `mcpp.toml`; the descriptor only declares - metadata and a download address. `mcpplibs.*`, `nlohmann.json`, `imgui`, `opencv`, `tensorvia-cpu`, … -- **Third-party C/C++ libraries** (`compat`, Form B): upstream has no mcpp support, so the descriptor inlines the build. - Header-only, C/C++ sources, or a C++23 module wrapper; optional parts sit behind `features`; a GitCode CN mirror - serves the same bytes. - -### Reference examples - -One descriptor per common shape: - -| Shape | Example | -|------|------| -| Native module library (Form A) | [`mcpplibs.tinyhttps`](pkgs/t/tinyhttps.lua) · [`gzj-creator.galay`](pkgs/g/gzj-creator.galay.lua) | -| C sources + `features` | [`compat.cjson`](pkgs/c/compat.cjson.lua) | -| C++ sources, several versions | [`compat.yaml-cpp`](pkgs/c/compat.yaml-cpp.lua) | -| Header-only | [`compat.gtl`](pkgs/c/compat.gtl.lua) · [`compat.safeint`](pkgs/c/compat.safeint.lua) | -| Generated config header | [`compat.c-ares`](pkgs/c/compat.c-ares.lua) | -| C++23 module wrapper | [`nlohmann.json`](pkgs/n/nlohmann.json.lua) | -| C++23 module shipped by upstream | [`khronos.vulkan-hpp`](pkgs/k/khronos.vulkan-hpp.lua) | -| External build system (`install()`) | [`compat.openssl`](pkgs/c/compat.openssl.lua) · [`compat.openmp`](pkgs/c/compat.openmp.lua) | - -Every other shape, and why each descriptor is written the way it is, is in -**[Descriptor examples by shape](docs/descriptor-examples.md)**. - -### Adding a package - -The procedure is the agent skill [`add-mcpp-index-package`](.agents/skills/add-mcpp-index-package/SKILL.md). Hand an -agent (Claude Code, for example) this instruction: - -```text -Following this repo's skill `.agents/skills/add-mcpp-index-package`, add @ to -mcpp-index: determine the shape; configure the CN mirror (use a plain-string upstream url when you have no mcpp-res -access); write pkgs//.lua; add a tests/examples// test project and register it as a workspace -member; verify locally with the same mcpp version CI pins by running `mcpp test -p `; update the README and -the online index; open a PR and confirm CI is green. +```sh +mcpp search +mcpp add ftxui@6.1.9 +mcpp build ``` -A PR runs lint and tests only the workspace members that depend on the changed descriptors; after the merge, -`deploy-site` publishes the site. - -Additional example: [`compat.rapidcsv`](pkgs/c/compat.rapidcsv.lua) — Header-only CSV parsing; [`consumer test`](tests/examples/rapidcsv/mcpp.toml). -Additional example: [`compat.pybind11`](pkgs/c/compat.pybind11.lua) — Header-only Python bindings; [`consumer test`](tests/examples/pybind11/mcpp.toml). -Additional example: [`compat.etl`](pkgs/c/compat.etl.lua) — Header-only; [`consumer test`](tests/examples/etl/mcpp.toml). -Additional example: [`compat.richdem`](pkgs/c/compat.richdem.lua) — C++ sources, serial terrain analysis; [`consumer test`](tests/examples/richdem/mcpp.toml). -Additional example: [`compat.fmt`](pkgs/c/compat.fmt.lua) — {fmt} in header form (`#include `; the non-module sibling of `fmtlib.fmt`), its C API behind a `c-api` feature; [`consumer test`](tests/examples/fmt/mcpp.toml), [`feature test`](tests/examples/fmt-c/mcpp.toml). -Additional example: [`compat.soci`](pkgs/c/compat.soci.lua) — C++ database API, with the SQLite3 backend compiled in and the engine taken from `compat.sqlite3`; [`consumer test`](tests/examples/soci/mcpp.toml). -Additional example: [`compat.imgui`](pkgs/c/compat.imgui.lua) — Dear ImGui 1.93.0 core sources, including the docking variant; [`headless test`](tests/examples/imgui/mcpp.toml), [`GLFW/OpenGL3 backend test`](tests/examples/imgui-window/mcpp.toml). -Additional example: [`yizhinailong.mcr`](pkgs/y/yizhinailong.mcr.lua) — Form A module library: libcurl-based C++23 HTTP client with sync, async, coroutine and batch requests, built from the upstream's own `mcpp.toml`; [`consumer test`](tests/examples/mcr/mcpp.toml). - -## Documentation - -- [Library shapes and descriptor templates](docs/package-types.md) -- [Descriptor examples by shape](docs/descriptor-examples.md) -- [The CN mirror loop](docs/cn-mirror.md) -- [openkal compatibility](docs/openkal-compat.md): the `openkal-ecosystem` / `openkal-compat` labels on the site -- [Repository layout, schema and CI](docs/repository-and-schema.md) -- The authoritative check of a field is `mcpp xpkg parse`, which CI runs; semantics are in mcpp's - [`docs/spec/`](https://github.com/mcpp-community/mcpp/tree/main/docs/spec). - -## Related links +Use `mcpp self config --mirror CN` to select the CN mirror. +The minimum supported mcpp version is recorded in [index.toml](index.toml). -| Project | Description | -|------|------| -| [mcpp](https://github.com/mcpp-community/mcpp) | Modern C++23 build and package management tool | -| [xlings](https://github.com/d2learn/xlings) | The package installation engine and sandbox environment underneath mcpp | -| [xpkg V1 spec](https://github.com/d2learn/xim-pkgindex/blob/main/docs/V1/xpackage-spec.md) | Package descriptor specification | -| [mcpplibs](https://github.com/mcpplibs) | The collection of modular C++23 libraries in the mcpp ecosystem | -| [mcpp-res](https://gitcode.com/mcpp-res) | The CN mirror organization for package resources (gitcode) | +## Contributing -## Community +Follow the [package contribution guide](docs/README.md) or the +[agent skill](.agents/skills/add-mcpp-index-package/SKILL.md). +Each package has a Lua descriptor and a consumer test. Eligible GitHub packages +use [automatic updates](docs/automatic-updates.md): daily checks open version PRs +for review while preserving existing versions. -[mcpp issues](https://github.com/mcpp-community/mcpp/issues) · [d2learn forum](https://forum.d2learn.org) +- [Package types and templates](docs/package-types.md) +- [Descriptor examples](docs/descriptor-examples.md) +- [CN mirrors](docs/cn-mirror.md) +- [Repository, schema and CI](docs/repository-and-schema.md) +- [openkal compatibility](docs/openkal-compat.md) ## License -The package descriptors are CC0; each upstream library keeps its own license. +Package descriptors are CC0. Upstream libraries retain their own licenses. diff --git a/README.zh-CN.md b/README.zh-CN.md index 358ab893..e1543e72 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -1,97 +1,37 @@ # mcpp-index -[English](README.md) | **简体中文** +[English](README.md) · [简体中文](README.zh-CN.md) -> [`mcpp`](https://github.com/mcpp-community/mcpp) 构建工具的默认包索引。 -> 在线浏览所有包:**https://mcpplibs.github.io/mcpp-index/** +[mcpp](https://github.com/mcpp-community/mcpp) 的默认包索引, +收录 C++ 模块库和从源码构建的第三方 C/C++ 库。 -收录 `mcpp` 可直接 `add` 的 C++23 包:`import` 即用的模块化库,以及以 `compat` 形态从上游源码构建的第三方 C/C++ 库。 -每个包对应一个 `pkgs/<首字母>/<包名>.lua` 描述文件。 - -> **需要 mcpp 2026.9.18.3 或更高版本**([`index.toml`](index.toml) 的 `min_mcpp`)。更旧的引擎会静默错误构建 -> `openkal-musl` 等 `[c-abi]` 包。升级:`xlings install mcpp --force`。 +[浏览包](https://mcpplibs.github.io/mcpp-index/) · [快速开始](docs/zh/quick-start.md) ## 使用 -```bash -mcpp add ftxui@6.1.9 # 添加依赖到 mcpp.toml -mcpp build # 自动拉取源码并构建,依赖沿链路自动传递 - -mcpp search # 搜索并刷新索引 -mcpp self config --mirror CN # 切换至国内镜像,默认使用 GLOBAL 上游源 -``` - -## 包的种类 - -- **原生 mcpp 模块库**(Form A):上游自带 `mcpp.toml`,描述文件只声明元数据与下载地址。如 `mcpplibs.*`、 - `nlohmann.json`、`imgui`、`opencv`、`tensorvia-cpu` 等。 -- **第三方 C/C++ 库**(`compat`,Form B):上游不支持 mcpp,描述文件内联构建信息。形态有 header-only、C/C++ 源码、 - C++23 module wrapper;可选组件经 `features` 门控;GitCode CN 镜像提供相同字节。 - -### 参考示例 - -每种常见形态一个描述符: - -| 形态 | 示例 | -|------|------| -| 原生模块库(Form A) | [`mcpplibs.tinyhttps`](pkgs/t/tinyhttps.lua) · [`gzj-creator.galay`](pkgs/g/gzj-creator.galay.lua) | -| C 源码 + `features` | [`compat.cjson`](pkgs/c/compat.cjson.lua) | -| C++ 源码,多版本 | [`compat.yaml-cpp`](pkgs/c/compat.yaml-cpp.lua) | -| header-only | [`compat.gtl`](pkgs/c/compat.gtl.lua) · [`compat.safeint`](pkgs/c/compat.safeint.lua) | -| 生成 config 头 | [`compat.c-ares`](pkgs/c/compat.c-ares.lua) | -| C++23 module wrapper | [`nlohmann.json`](pkgs/n/nlohmann.json.lua) | -| 上游自带 C++23 module | [`khronos.vulkan-hpp`](pkgs/k/khronos.vulkan-hpp.lua) | -| 外部构建系统(`install()`) | [`compat.openssl`](pkgs/c/compat.openssl.lua) · [`compat.openmp`](pkgs/c/compat.openmp.lua) | - -其余形态,以及每个描述符为何这样写,见 **[描述符示例总览(按形态)](docs/zh/descriptor-examples.md)**。 - -### 新增一个包 - -流程定义于 agent skill [`add-mcpp-index-package`](.agents/skills/add-mcpp-index-package/SKILL.md)。把下面的指令交给 -agent(如 Claude Code)即可: - -```text -参考本仓 skill `.agents/skills/add-mcpp-index-package`,将 <库名 / 仓库URL> @<版本> 收录进 mcpp-index: -判定形态;配置 CN 镜像(无 mcpp-res 权限时使用 plain-string 上游 url);编写 pkgs/<首字母>/<包名>.lua; -添加 tests/examples/<库>/ 测试工程并登记为 workspace 成员;使用与 CI 同版本的 mcpp 本地执行 -`mcpp test -p <成员>` 进行验证;更新 README 与在线索引;提交 PR 并确认 CI 通过。 +```sh +mcpp search +mcpp add ftxui@6.1.9 +mcpp build ``` -提交 PR 后,CI 执行 lint,并只测试依赖了被修改描述符的 workspace 成员;合并后,`deploy-site` 发布站点。 - -新增示例: [`compat.rapidcsv`](pkgs/c/compat.rapidcsv.lua) — Header-only CSV 解析; [`消费测试`](tests/examples/rapidcsv/mcpp.toml)。 -新增示例: [`compat.pybind11`](pkgs/c/compat.pybind11.lua) — Header-only Python bindings; [`消费测试`](tests/examples/pybind11/mcpp.toml)。 -新增示例: [`compat.etl`](pkgs/c/compat.etl.lua) — Header-only; [`消费测试`](tests/examples/etl/mcpp.toml)。 -新增示例: [`compat.richdem`](pkgs/c/compat.richdem.lua) — C++ sources, serial terrain analysis; [`消费测试`](tests/examples/richdem/mcpp.toml)。 -新增示例: [`compat.fmt`](pkgs/c/compat.fmt.lua) — {fmt} 头文件形态(`#include `,即 `fmtlib.fmt` 的非模块兄弟),C API 由 `c-api` feature 门控; [`消费测试`](tests/examples/fmt/mcpp.toml)、[`feature 测试`](tests/examples/fmt-c/mcpp.toml)。 -新增示例: [`compat.soci`](pkgs/c/compat.soci.lua) — C++ 数据库访问层,SQLite3 后端编入同一库,引擎来自 `compat.sqlite3`; [`消费测试`](tests/examples/soci/mcpp.toml)。 -示例: [`compat.imgui`](pkgs/c/compat.imgui.lua) — Dear ImGui 1.93.0 核心源码,含 docking 版本; [`无窗口测试`](tests/examples/imgui/mcpp.toml)、[`GLFW/OpenGL3 后端测试`](tests/examples/imgui-window/mcpp.toml)。 -新增示例: [`yizhinailong.mcr`](pkgs/y/yizhinailong.mcr.lua) — Form A 模块库:基于 libcurl 的 C++23 HTTP 客户端,支持同步、异步、协程与批量请求,构建由上游自带 `mcpp.toml` 负责; [`消费测试`](tests/examples/mcr/mcpp.toml)。 - -## 文档 - -- [库形态与描述符模板](docs/zh/package-types.md) -- [描述符示例总览(按形态)](docs/zh/descriptor-examples.md) -- [CN 镜像闭环](docs/zh/cn-mirror.md) -- [openkal 兼容性](docs/zh/openkal-compat.md):站点上的 `openkal-ecosystem` / `openkal-compat` 标签 -- [仓库结构、schema 与 CI](docs/zh/repository-and-schema.md) -- 字段的权威判定是 `mcpp xpkg parse`(CI 用的就是它);语义见 mcpp 仓的 - [`docs/spec/`](https://github.com/mcpp-community/mcpp/tree/main/docs/spec)。 - -## 相关链接 +使用 `mcpp self config --mirror CN` 切换国内镜像。 +最低 mcpp 版本见 [index.toml](index.toml)。 -| 项目 | 说明 | -|------|------| -| [mcpp](https://github.com/mcpp-community/mcpp) | 现代 C++23 构建与包管理工具 | -| [xlings](https://github.com/d2learn/xlings) | mcpp 底层的包安装引擎与沙箱环境 | -| [xpkg V1 spec](https://github.com/d2learn/xim-pkgindex/blob/main/docs/V1/xpackage-spec.md) | 包描述文件规范 | -| [mcpplibs](https://github.com/mcpplibs) | mcpp 生态的模块化 C++23 库集合 | -| [mcpp-res](https://gitcode.com/mcpp-res) | 包资源的 CN 镜像组织(gitcode) | +## 贡献 -## 社区 +参考[包贡献指南](docs/zh/README.md)或 +[agent skill](.agents/skills/add-mcpp-index-package/SKILL.md)。 +每个包包含 Lua 描述符和消费测试。符合条件的 GitHub 包接入 +[自动更新](docs/zh/automatic-updates.md):每日检查新版本并创建 PR, +保留已有版本,由维护者 review 后合并。 -[mcpp issues](https://github.com/mcpp-community/mcpp/issues) · [d2learn 论坛](https://forum.d2learn.org) +- [包形态与模板](docs/zh/package-types.md) +- [描述符示例](docs/zh/descriptor-examples.md) +- [国内镜像](docs/zh/cn-mirror.md) +- [仓库、schema 与 CI](docs/zh/repository-and-schema.md) +- [openkal 兼容性](docs/zh/openkal-compat.md) ## License -包描述文件采用 CC0;各上游库保留其自身许可证。 +包描述符采用 CC0;上游库保留各自的许可证。 diff --git a/docs/README.md b/docs/README.md index 4fe2aa78..c19d6dde 100644 --- a/docs/README.md +++ b/docs/README.md @@ -11,6 +11,7 @@ behind each step. |------|------| | [package-types.md](package-types.md) | Descriptor templates and samples for the four library shapes (C-source compat, header-only, C++23 module wrapper, external Form-A module repo) | | [descriptor-examples.md](descriptor-examples.md) | The complete catalog of descriptors in this index, grouped by shape — what each package compiles, and what it deliberately leaves out | +| [automatic-updates.md](automatic-updates.md) | Eligibility, declarations, daily bot PRs and candidate CI | | [cn-mirror.md](cn-mirror.md) | The GitCode `mcpp-res` CN mirror loop (`gtc` tooling, closed-loop verification, caveats), including the fallback when you lack write access to `mcpp-res` (plain-string upstream url) | | [repository-and-schema.md](repository-and-schema.md) | Repository layout, descriptor schema cheat-sheet, `validate.yml` CI behavior, reproducing lint locally, case index | | [openkal-compat.md](openkal-compat.md) | What the `openkal` label means, how it is measured, and how a package that fails in an openkal graph is adapted | diff --git a/docs/automatic-updates.md b/docs/automatic-updates.md new file mode 100644 index 00000000..0a4e1b61 --- /dev/null +++ b/docs/automatic-updates.md @@ -0,0 +1,101 @@ +# Automatic updates + +[English](automatic-updates.md) · [简体中文](zh/automatic-updates.md) + +[openxlings/xpkgindex-update](https://github.com/openxlings/xpkgindex-update) +checks declared packages daily at 02:17 UTC (11:17 JST). GitHub may delay scheduled +runs. A manual run of [the workflow](../.github/workflows/xpkgindex-update.yml) +on the default branch performs the same update cycle; the optional `package` +input limits it to one `namespace:name`. + +The workflow downloads new release resources, calculates SHA256 and opens at +most three new PRs per run as `github-actions[bot]`. It then explicitly dispatches +candidate, consumer, site and openkal checks on each PR branch. Candidate checks +build and run the new version and verify its lockfile; ordinary consumer tests +also cover the versions their manifests already select. Failed upgrades stay +visible in PR CI for maintainers to resolve. PRs are never automatically merged. + +## Declare a package + +When adding or updating a package, include `package.extensions.autoupdate` if: + +- A public GitHub release or numeric tag identifies the package version. +- The download URLs follow a stable template and resources exist on all declared platforms. +- Updating the version does not require changing its build recipe, hooks or dependency policy. + +```lua +extensions = { + autoupdate = { + tag = "v${version}", + url = "https://github.com/acme/foo/archive/refs/tags/${tag}.tar.gz", + }, +}, +``` + +The tag template contains exactly one `${version}`. URLs can use `${tag}`, +`${version}` or `${version:dash}` and may be shared or mapped by platform. +A `GLOBAL`/`CN` template table is valid only if both resources already exist and +have identical bytes; the updater does not create mirrors. A GitHub-only URL +for future versions is allowed even when historical entries have mirrors. +Use `package.repo` as a fallback when the URL does not identify a GitHub repo. + +Commit snapshots, independent source/package version mappings, prereleases, +non-GitHub providers and upgrades requiring recipe changes need manual handling. +Do not invent a rule to opt them in; explain the exception in the PR instead. +The current xim adapter does not advance `latest.ref`. + +## Verify a declaration + +Use the reviewed updater commit pinned in the workflow (Python 3.11+): + +```sh +python -m xpkgindex_update inspect --package compat:foo +python -m xpkgindex_update check --package compat:foo --output plan.json +# If a new version is ready, inspect its generated patch in a clean worktree: +python -m xpkgindex_update apply --plan plan.json --package compat:foo +``` + +`check` must parse the declaration and report the actual upstream result. +If already current, keep that result rather than inventing a candidate. An added +version must pass the candidate build/run/lockfile checks. Future upgrades may +still need manual adaptation when upstream changes its API or source layout. + +Only new versions are appended. Historical version definitions, URLs, mirrors, +hashes and comments remain unchanged. When every platform uses identical +resources, new URL/SHA256 definitions are stored once in a leading local table +and referenced by platform entries. Different resources remain separate. + +## Workflow and credentials + +The scan and build jobs have read access. Only the default-branch `propose` job +receives `contents: write`, `pull-requests: write` and `actions: write`. +It consumes the exact scan artifact using the pinned CLI; no local user token +or long-lived secret is involved. Organization and repository settings must +allow GitHub Actions to create PRs. The combined setting also permits approvals, +but this workflow never approves or merges a PR. + +Under [GitHub’s trigger rules](https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow), +`GITHUB_TOKEN` does not provide unattended ordinary PR-triggered CI, so the job +explicitly dispatches each workflow. The consumer and openkal dispatches verify +an open, same-repository bot PR, its exact head SHA, default base and single +changed descriptor before selecting affected members. A regular manual +consumer dispatch without `update_pr` still runs the full regression. + +One open update PR per package is retained. An existing manual PR editing the +same descriptor also blocks duplicates and keeps its ordinary PR CI. Closed +candidate PRs are not reopened. +A later daily run can resume missing CI dispatches on an existing bot PR without +creating duplicates; existing runs, including failures, are not silently rerun. +`max_prs` in [.github/xpkgindex-update.yml](../.github/xpkgindex-update.yml) limits +new PRs per run. No update PR can recursively create more PRs. + +The temporary `wellwei/libmysqlclient` outage exception retains its scan error; +its affected consumers are reported as **NOT TESTED**, not successful builds. + +## Contribution scope + +A normal package PR changes its descriptor, consumer test and workspace entry +when needed. Include the automatic-update declaration when eligible and describe +any exception in the PR. Package additions are discovered by the site from the +descriptors: do not append package announcements or examples to either root +README, and do not edit site configuration for a routine package addition. diff --git a/docs/descriptor-examples.md b/docs/descriptor-examples.md index 31082371..53f1fe0a 100644 --- a/docs/descriptor-examples.md +++ b/docs/descriptor-examples.md @@ -7,8 +7,7 @@ meant to be read the way you would read a case book: find the row whose situatio and reuse its reasoning. Every entry records not just what the package compiles but **what it deliberately does not**, which is usually the part worth copying. -For the templates behind each shape see [package-types.md](package-types.md); the short list of starting points lives -in the [root README](../README.md#reference-examples). +For descriptor templates see [package-types.md](package-types.md). | Shape | Examples | |------|------| diff --git a/docs/repository-and-schema.md b/docs/repository-and-schema.md index aabef946..5128da34 100644 --- a/docs/repository-and-schema.md +++ b/docs/repository-and-schema.md @@ -278,3 +278,9 @@ on `main` — no new mcpp release required. Browse online at: https://mcpplibs.g | External build system (`install()`-driven) | `pkgs/c/compat.openblas.lua` (Make), `compat.openssl.lua` (Perl Configure + Make) | `tests/examples/openblas/`, `openssl/` | `docs/superpowers/specs/2026-07-26-openssl-asio-tls-design.md` / #124 | | A feature pulling in a dependency (cross-package) | the `ssl` feature of `pkgs/c/chriskohlhoff.asio.lua` → `compat.openssl` | `tests/examples/asio-ssl/` | same as above | | Multi-component upstream flattened into one lib | `pkgs/c/compat.recastnavigation.lua` (five upstream CMake targets → a base build plus `crowd` / `tilecache` / `debug-utils`; `debug-utils` `implies` `tilecache`) | `tests/examples/recastnavigation/`, `recastnavigation-features/` | `.agents/docs/2026-08-29-add-recastnavigation-plan.md` | + +## Automatic version updates + +See [Automatic updates](automatic-updates.md) for declarations, the daily bot workflow, +permissions and exact-candidate CI. Normal package PRs include the declaration when eligible +and leave the root READMEs and site configuration unchanged. diff --git a/docs/zh/README.md b/docs/zh/README.md index da06fdd3..2f2556b4 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -9,6 +9,7 @@ |------|------| | [package-types.md](package-types.md) | 四种库形态(C 源码 compat、header-only、C++23 module wrapper、外部 Form-A 模块仓)的描述符模板与样例 | | [descriptor-examples.md](descriptor-examples.md) | 本索引全部描述符的完整目录,按形态分组 —— 每个包编了什么,以及刻意没编什么 | +| [automatic-updates.md](automatic-updates.md) | 适用条件、声明、每日 bot PR 与候选 CI | | [cn-mirror.md](cn-mirror.md) | GitCode `mcpp-res` CN 镜像闭环(`gtc` 工具、闭环校验、注意事项),含无 `mcpp-res` 写权限时的回退方案(plain-string 上游 url) | | [repository-and-schema.md](repository-and-schema.md) | 仓库布局、描述符 schema 速查、`validate.yml` CI 行为、本地 lint 复现、案例索引 | diff --git a/docs/zh/automatic-updates.md b/docs/zh/automatic-updates.md new file mode 100644 index 00000000..8a7ffa4f --- /dev/null +++ b/docs/zh/automatic-updates.md @@ -0,0 +1,85 @@ +# 自动更新 + +[English](../automatic-updates.md) · [简体中文](automatic-updates.md) + +[openxlings/xpkgindex-update](https://github.com/openxlings/xpkgindex-update) +每天 02:17 UTC(日本时间 11:17)检查已声明的包;GitHub 的定时任务可能延迟。 +在默认分支手动运行[同一 workflow](../../.github/workflows/xpkgindex-update.yml) +也会执行完整更新流程,可用 `package` 输入限定一个 `namespace:name`。 + +workflow 下载新版本资源、计算 SHA256,每轮最多由 `github-actions[bot]` 创建三个新 PR, +随后显式触发各 PR 分支上的候选、消费者、站点和 openkal 检查。 +候选测试必须真实构建、运行新版本并核对锁文件;普通消费测试同时覆盖其原来选择的版本。 +升级失败会保留为 PR CI 的失败结果,交由维护者处理;不会自动合并。 + +## 为包声明自动更新 + +新增或维护包时,满足以下条件就应添加 `package.extensions.autoupdate`: + +- 公共 GitHub release 或数字 tag 能确定包版本。 +- 下载 URL 有稳定模板,已声明平台的资源均存在。 +- 更新版本不需要修改构建配方、hook 或依赖策略。 + +```lua +extensions = { + autoupdate = { + tag = "v${version}", + url = "https://github.com/acme/foo/archive/refs/tags/${tag}.tar.gz", + }, +}, +``` + +`tag` 必须且只能包含一个 `${version}`。URL 可使用 `${tag}`、`${version}`、 +`${version:dash}`,可全平台共用,也可按平台声明。 +仅当 GLOBAL/CN 资源均已存在且字节一致时,才能声明镜像模板;工具不创建镜像。 +即使历史版本有镜像,新版本也可以只声明 GitHub 来源。 +URL 不能识别 GitHub 仓库时,回退使用 `package.repo`。 + +commit 快照、独立的源码/包版本映射、预发布版本、非 GitHub 来源,以及需要修改配方的升级, +应保留手工维护,在 PR 中说明原因,不要猜测更新规则。当前 xim adapter 不推进 `latest.ref`。 + +## 验证声明 + +使用 workflow 固定的已 review 工具提交(Python 3.11+): + +```sh +python -m xpkgindex_update inspect --package compat:foo +python -m xpkgindex_update check --package compat:foo --output plan.json +# 如发现新版本,在干净 worktree 中查看补丁: +python -m xpkgindex_update apply --plan plan.json --package compat:foo +``` + +`check` 必须能解析声明并给出真实上游结果;已经最新时保留该结果,不虚构候选版本。 +新增版本须通过候选构建、运行和锁文件检查。上游未来改变 API 或源码布局时,仍可能需要人工适配。 + +只追加新版本;历史定义、URL、mirror、哈希与注释保留原样。 +全平台资源相同时,新 URL/SHA256 前置定义一次,由各平台引用;不同资源分别保留。 + +## workflow 与认证 + +扫描和构建 job 只有读取权限。只有默认分支的 `propose` job 获取 +`contents: write`、`pull-requests: write`、`actions: write`, +使用固定 CLI 消费原始扫描 artifact,不依赖本地用户 token 或长期 secret。 +组织和仓库须允许 GitHub Actions 创建 PR。GitHub 的对应开关也允许审批, +但此 workflow 不会审批或合并 PR。 + +根据 [GitHub 的触发规则](https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow), +`GITHUB_TOKEN` 无法提供无人值守的普通 PR 事件 CI,因此提 PR 后显式 dispatch 各 workflow。 +消费者和 openkal 的 dispatch 会校验 PR 已打开、属于本仓 bot、head SHA 完全一致、 +目标为默认分支且只修改一个描述符,再选择受影响成员。 +不带 `update_pr` 的普通手动消费者验证仍为全量回归。 + +每包最多保留一个打开的更新 PR;已有人工 PR 修改同一描述符时,也不重复提案,继续使用普通 PR CI。 +已关闭的候选不会重新打开。 +下一轮能为已有 bot PR 补齐未成功发出的 CI dispatch,不重复提 PR;已有运行(包括失败)不会被静默重跑。 +[中央配置](../../.github/xpkgindex-update.yml) 的 `max_prs` 限制每轮新 PR 数量。 +更新分支上的 CI 不会递归创建 PR。 + +暂时保留 `wellwei/libmysqlclient` 源故障豁免:扫描错误仍记录,受影响消费者标为 +**NOT TESTED**,不计为成功构建。 + +## 普通包 PR 的范围 + +一般只改描述符、消费测试和必要的 workspace 登记,符合条件就补上自动更新声明; +不适用时在 PR 中说明。站点会从描述符发现包,普通包 PR 不向两份根 README 追加包介绍、 +示例或更新记录,也不修改站点配置。只有文档整理或用户明确要求时才调整 README。 diff --git a/docs/zh/descriptor-examples.md b/docs/zh/descriptor-examples.md index cfe870f4..89590854 100644 --- a/docs/zh/descriptor-examples.md +++ b/docs/zh/descriptor-examples.md @@ -6,7 +6,7 @@ 行,再打开对应描述符复用其推理过程。每条记录的不只是这个包编了什么,更包括它**刻意没有编什么** —— 后者通常才是 值得照搬的部分。 -各形态的模板见 [package-types.md](package-types.md);精简版的入门示例见[根 README](../../README.zh-CN.md#参考示例)。 +各形态的模板见 [package-types.md](package-types.md)。 | 形态 | 示例 | |------|------| diff --git a/docs/zh/repository-and-schema.md b/docs/zh/repository-and-schema.md index 86d52fa4..58e2ff08 100644 --- a/docs/zh/repository-and-schema.md +++ b/docs/zh/repository-and-schema.md @@ -227,3 +227,8 @@ bash tests/run_members.sh --all --cache local # 绕过包构建缓 | 外部构建系统(`install()` 驱动) | `pkgs/c/compat.openblas.lua`(Make)、`compat.openssl.lua`(Perl Configure + Make) | `tests/examples/openblas/`、`openssl/` | `docs/superpowers/specs/2026-07-26-openssl-asio-tls-design.md` / #124 | | feature 拉起依赖(跨包) | `pkgs/c/chriskohlhoff.asio.lua` 的 `ssl` feature → `compat.openssl` | `tests/examples/asio-ssl/` | 同上 | | 多组件上游拍平进单个库 | `pkgs/c/compat.recastnavigation.lua`(上游五个 CMake target → 核心构建 + `crowd`/`tilecache`/`debug-utils`;`debug-utils` `implies` `tilecache`) | `tests/examples/recastnavigation/`、`recastnavigation-features/` | `.agents/docs/2026-08-29-add-recastnavigation-plan.md` | + +## 自动版本更新 + +[自动更新](automatic-updates.md)说明声明格式、每日 bot 流程、权限与真实候选 CI。 +普通包 PR 应为符合条件的包补齐声明,不改根 README 和站点配置。 diff --git a/tests/autoupdate/automation.py b/tests/autoupdate/automation.py new file mode 100755 index 00000000..ce2e6fc7 --- /dev/null +++ b/tests/autoupdate/automation.py @@ -0,0 +1,124 @@ +#!/usr/bin/env python3 +"""Create update PRs in Actions and explicitly dispatch their CI with GITHUB_TOKEN.""" +from __future__ import annotations + +import argparse +import json +import os +import re +import subprocess +import sys +from pathlib import Path + + +def gh(*args): + return subprocess.check_output(["gh", *args], text=True).strip() + + +def verify_pr(number, repo, sha=None): + if not str(number).isdigit(): + raise ValueError("update_pr must be a PR number") + pr = json.loads(gh("api", f"repos/{repo}/pulls/{number}")) + if not ( + pr["state"] == "open" + and pr["user"]["login"] == "github-actions[bot]" + and pr["head"]["repo"]["full_name"] == repo + and pr["base"]["repo"]["full_name"] == repo + and pr["base"]["ref"] == os.environ["DEFAULT_BRANCH"] + and re.fullmatch(r"auto/bump-[a-zA-Z0-9._-]+", pr["head"]["ref"]) + and (sha is None or pr["head"]["sha"] == sha) + ): + raise ValueError("expected an open bot update PR in this repository at the requested head") + files = json.loads(gh("api", f"repos/{repo}/pulls/{number}/files?per_page=100")) + if len(files) != 1 or not re.fullmatch(r"pkgs/[^/]+/[^/]+\.lua", files[0]["filename"]): + raise ValueError("an update PR must change exactly one package descriptor") + return pr + + +def dispatch(pr, repo, identity): + """Resume missing dispatches after partial API failure without duplicating runs.""" + workflows = { + "validate.yml": {"update_pr": str(pr["number"])}, + "xpkgindex-update.yml": {"package": identity}, + "site-check.yml": {}, + "openkal-compat.yml": {"update_pr": str(pr["number"])}, + } + results = [] + for workflow, inputs in workflows.items(): + previous = json.loads(gh( + "run", "list", "--repo", repo, "--workflow", workflow, + "--branch", pr["head"]["ref"], "--commit", pr["head"]["sha"], + "--event", "workflow_dispatch", "--limit", "1", "--json", "databaseId")) + if previous: + results.append({"workflow": workflow, "status": "existing", "run": previous[0]["databaseId"]}) + continue + args = ["workflow", "run", workflow, "--repo", repo, "--ref", pr["head"]["ref"]] + for key, value in inputs.items(): + args.extend(["-f", f"{key}={value}"]) + gh(*args) + results.append({"workflow": workflow, "status": "dispatched", "head": pr["head"]["sha"]}) + return results + + +def propose(plan, output): + repo = os.environ["GITHUB_REPOSITORY"] + base = os.environ["DEFAULT_BRANCH"] + report = output.with_name("proposals.json") + fixed = json.loads(plan.read_text(encoding="utf-8")) + remote = gh("api", f"repos/{repo}/git/ref/heads/{base}", "--jq", ".object.sha") + if remote != fixed["base_commit"] or remote != os.environ["GITHUB_SHA"]: + raise ValueError("default branch moved since scanning; rerun with a fresh plan") + # The pinned CLI owns append-only editing, PR limits and duplicate detection. + result = subprocess.run([ + sys.executable, "-m", "xpkgindex_update", "propose", "--plan", str(plan), + "--base", base, "--output", str(report), + ], check=False) + proposals = json.loads(report.read_text(encoding="utf-8")) + outcomes = [] + for record in proposals["packages"]: + if record["status"] not in {"created", "pending"}: + outcomes.append(record) + continue + try: + if record.get("url"): + number = record["url"].rsplit("/", 1)[-1] + existing = json.loads(gh("api", f"repos/{repo}/pulls/{number}")) + if record["status"] == "pending" and existing["user"]["login"] != "github-actions[bot]": + # Human PRs run ordinary pull_request CI; do not duplicate or dispatch their code. + outcomes.append({**record, "ci": [{"status": "ordinary-pr-ci"}]}) + continue + else: + slug = re.sub(r"[^a-zA-Z0-9._-]", "-", record["id"]) + opened = json.loads(gh("pr", "list", "--repo", repo, "--state", "open", + "--limit", "1000", "--json", "number,headRefName")) + matches = [p for p in opened if p["headRefName"].startswith(f"auto/bump-{slug}-")] + if len(matches) != 1: + raise ValueError("expected exactly one pending update PR") + number = matches[0]["number"] + pr = verify_pr(number, repo) + outcomes.append({**record, "url": pr["html_url"], "ci": dispatch(pr, repo, record["id"])}) + except (ValueError, subprocess.CalledProcessError) as error: + outcomes.append({**record, "status": "error", "reason": str(error)}) + output.write_text(json.dumps({"packages": outcomes}, indent=2) + "\n", encoding="utf-8") + print(output.read_text(encoding="utf-8")) + return result.returncode or int(any(r["status"] == "error" for r in outcomes)) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + commands = parser.add_subparsers(dest="command", required=True) + create = commands.add_parser("propose") + create.add_argument("--plan", type=Path, required=True) + create.add_argument("--output", type=Path, required=True) + verify = commands.add_parser("verify") + verify.add_argument("--pr", required=True) + args = parser.parse_args() + if args.command == "verify": + pr = verify_pr(args.pr, os.environ["GITHUB_REPOSITORY"], os.environ["GITHUB_SHA"]) + print(pr["base"]["sha"]) + return 0 + return propose(args.plan.resolve(), args.output.resolve()) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/autoupdate/test_automation.py b/tests/autoupdate/test_automation.py new file mode 100644 index 00000000..69ad8824 --- /dev/null +++ b/tests/autoupdate/test_automation.py @@ -0,0 +1,148 @@ +"""Exercise bot identity, exact-head validation and resumable CI dispatch.""" +import copy +import json +import os +import subprocess +import tempfile +import unittest +from pathlib import Path +from unittest.mock import patch + +import automation + +REPO = "acme/index" +PR = { + "number": 7, "state": "open", "user": {"login": "github-actions[bot]"}, + "html_url": f"https://github.com/{REPO}/pull/7", + "base": {"repo": {"full_name": REPO}, "ref": "main", "sha": "base"}, + "head": {"repo": {"full_name": REPO}, "ref": "auto/bump-compat-foo-2.0.0", "sha": "head"}, +} + + +class AutomationTests(unittest.TestCase): + def setUp(self): + self.environment = patch.dict(os.environ, { + "DEFAULT_BRANCH": "main", "GITHUB_REPOSITORY": REPO, "GITHUB_SHA": "head", + }) + self.environment.start() + self.addCleanup(self.environment.stop) + + def test_verify_exact_head_and_single_descriptor(self): + with patch.object(automation, "gh", side_effect=[ + json.dumps(PR), json.dumps([{"filename": "pkgs/c/compat.foo.lua"}]), + ]): + self.assertEqual(automation.verify_pr(7, REPO, "head"), PR) + + def test_reject_human_fork_closed_wrong_base_and_changed_head(self): + variants = [] + for keys, value in [ + (("user", "login"), "Sunrisepeak"), + (("head", "repo", "full_name"), "attacker/index"), + (("base", "ref"), "release"), + (("head", "sha"), "replacement"), + (("state",), "closed"), + (("head", "ref"), "feature/manual"), + ]: + pr = copy.deepcopy(PR) + target = pr + for key in keys[:-1]: + target = target[key] + target[keys[-1]] = value + variants.append(pr) + for pr in variants: + with (self.subTest(pr=pr), + patch.object(automation, "gh", return_value=json.dumps(pr)), + self.assertRaises(ValueError)): + automation.verify_pr(7, REPO, "head") + + def test_reject_workflow_changes(self): + with patch.object(automation, "gh", side_effect=[ + json.dumps(PR), json.dumps([{"filename": ".github/workflows/validate.yml"}]), + ]), self.assertRaisesRegex(ValueError, "one package descriptor"): + automation.verify_pr(7, REPO, "head") + + def test_partial_dispatch_is_resumable_and_targets_pr_head(self): + calls = [] + completed = {"validate.yml"} + + def gh(*args): + calls.append(args) + if args[:2] == ("run", "list"): + workflow = args[args.index("--workflow") + 1] + self.assertEqual(args[args.index("--commit") + 1], "head") + return '[{"databaseId": 12}]' if workflow in completed else '[]' + self.assertEqual(args[args.index("--ref") + 1], PR["head"]["ref"]) + return "" + + with patch.object(automation, "gh", side_effect=gh): + results = automation.dispatch(PR, REPO, "compat:foo") + self.assertEqual([r["status"] for r in results], ["existing"] + ["dispatched"] * 3) + dispatched = [c for c in calls if c[:2] == ("workflow", "run")] + self.assertEqual(len(dispatched), 3) + self.assertIn("package=compat:foo", dispatched[0]) + self.assertIn("update_pr=7", dispatched[2]) + + def test_moved_main_refuses_propose(self): + with tempfile.TemporaryDirectory() as directory: + plan = Path(directory) / "plan.json" + plan.write_text(json.dumps({"base_commit": "old"})) + with patch.object(automation, "gh", return_value="head"), patch.object(automation.subprocess, "run") as run: + with self.assertRaisesRegex(ValueError, "moved"): + automation.propose(plan, Path(directory) / "report.json") + run.assert_not_called() + + def test_created_pr_and_pending_retry_both_receive_ci(self): + for state in ("created", "pending"): + with self.subTest(state=state), tempfile.TemporaryDirectory() as directory: + plan = Path(directory) / "plan.json" + plan.write_text(json.dumps({"base_commit": "head"})) + output = Path(directory) / "automation-report.json" + + def run(args, check, state=state): + self.assertEqual(args[2:4], ["xpkgindex_update", "propose"]) + Path(args[-1]).write_text(json.dumps({"packages": [ + {"id": "compat:foo", "status": state, "url": PR["html_url"]}, + {"id": "compat:bar", "status": "error", "reason": "resource changed"}, + ]})) + return subprocess.CompletedProcess(args, 1) + + def gh(*args): + if args[:2] == ("pr", "list"): + return json.dumps([{"number": 7, "headRefName": PR["head"]["ref"]}]) + if args[0] == "api" and args[1].endswith("/pulls/7"): + return json.dumps(PR) + return "head" + + with patch.object(automation, "gh", side_effect=gh), \ + patch.object(automation.subprocess, "run", side_effect=run), \ + patch.object(automation, "verify_pr", return_value=PR), \ + patch.object(automation, "dispatch", return_value=[{"status": "dispatched"}]) as dispatch: + self.assertEqual(automation.propose(plan, output), 1) + dispatch.assert_called_once_with(PR, REPO, "compat:foo") + result = json.loads(output.read_text())["packages"] + self.assertEqual(result[0]["ci"], [{"status": "dispatched"}]) + self.assertEqual(result[1]["status"], "error") + + def test_pending_human_pr_uses_ordinary_ci_without_dispatch(self): + with tempfile.TemporaryDirectory() as directory: + plan = Path(directory) / "plan.json" + plan.write_text(json.dumps({"base_commit": "head"})) + output = Path(directory) / "automation-report.json" + + def run(args, check): + Path(args[-1]).write_text(json.dumps({"packages": [{ + "id": "compat:foo", "status": "pending", "url": PR["html_url"], + }]})) + return subprocess.CompletedProcess(args, 0) + + def gh(*args): + if args[1].endswith("/pulls/7"): + return json.dumps({**PR, "user": {"login": "maintainer"}}) + return "head" + + with patch.object(automation, "gh", side_effect=gh), \ + patch.object(automation.subprocess, "run", side_effect=run), \ + patch.object(automation, "dispatch") as dispatch: + self.assertEqual(automation.propose(plan, output), 0) + dispatch.assert_not_called() + self.assertEqual(json.loads(output.read_text())["packages"][0]["ci"], [{"status": "ordinary-pr-ci"}])