feat(webapp): open a bot's Mini App with real signed launch data

CLAUDE.md

@@ -40,7 +40,13 @@ gofmt -l .                           # list misformatted files; gofmt -w to fix
 go mod tidy                          # CI fails if this produces a diff
 ```
 
-Go 1.25. Releases are cut by pushing a `v*` tag, which triggers GoReleaser
+Go 1.25. On the maintainer's workstation Go is not on PATH: use
+`export PATH="$HOME/.local/opt/go-1.27.2/bin:$PATH"` and
+`~/.local/opt/golangci-lint-2.12.1/golangci-lint`. `scripts/install-local.sh` builds and installs
+`~/.local/bin/tg` with embedded app credentials (see its header). The first build of `gotd/td`
+takes a long time on this laptop; later builds are cached.
+
+Releases are cut by pushing a `v*` tag, which triggers GoReleaser
 (`.github/workflows/release.yml`); the release build embeds app credentials via `-ldflags` and
 fails if the `APP_ID`/`APP_HASH` secrets are missing.
 
@@ -52,7 +58,7 @@ Commits must follow Conventional Commits (`feat(...)`, `ci:`, `fix:`, …) — e
 The whole CLI lives in `cmd/tg/` (package `main`). Supporting packages are under `internal/`.
 
 **Command pattern.** Each command is a `func (a *app) newXxxCmd() *cobra.Command` method
-(spf13/cobra) registered in `root.go`'s `newRootCmd`. Commands carry a `GroupID`
+(spf13/cobra) registered in `root.go`'s `newRoot`. Commands carry a `GroupID`
 (`groupAuth`/`groupMessaging`/`groupChats`/`groupProfile`). Parent-with-subcommands is done via
 `cmd.AddCommand(...)` (see `devices`, `chats`, `invite-link`).
 
@@ -89,6 +95,38 @@ A `<peer>` argument accepts `me`/`self`, `@username`, a phone number, a `t.me/
 only after the peer has been cached). For sending, `builderFor(sender, peer)` turns a peer
 string into a `message.RequestBuilder` (the sender's resolver also understands `id:`).
 
+**Errors and exit codes** (`exitcode.go`). `main` calls `execute`, which maps the returned
+error to an exit code and, with `-o json`, also writes `{"schema":1,"error":{...}}` to stdout.
+Classification is by type, not text: return `usageError{err}` for bad invocations,
+`notFoundf(...)` for a missing peer/message/button, wrap `errWaitTimeout` when giving up waiting,
+and let Telegram RPC errors (`tgerr`) and context errors propagate wrapped. Use
+`parseMessageID` for `<message-id>` arguments. Exit codes are part of the contract.
+
+**Bot-testing primitives.** `conversation.go`: a `replyCollector` registered via
+`runParams.register` (handlers must be attached before the client connects — the dispatcher map
+is not safe for concurrent registration) collects a peer's replies and edits; commands opt in with
+`waitOptions` (`--wait/--settle/--replies`), `targetFor` (arms the collector before sending) and
+`awaitResponse`. `markup.go` models reply markup and button selection; `click.go` presses buttons;
+`bot.go` has `bot-info` and `start-bot`.
+
+**JSON compatibility.** Downstream scripts parse `history`/`send` output. Only add fields, always
+with `omitempty` (see `TestMessageItemJSONCompat`); never rename or remove one.
+
+**Binary size.** Never call cobra's `Set*Template` or otherwise link `text/template` (or
+`reflect.Value.Method`/`MethodByName`): reflective method lookup makes the linker keep every
+exported method of `gotd/td`'s generated API, growing the binary from ~19 MB to ~45 MB. Check with
+`go build -ldflags=-dumpdep ./cmd/tg 2>&1 | grep ReflectMethod` (should print nothing).
+
+**Files shared between processes** (sessions, peer cache, config) are written through
+`internal/atomicfile`, so concurrent `tg` invocations never read a torn file, and
+read-modify-write cycles hold an `internal/filelock` lock on a sidecar `<file>.lock` so they
+cannot lose each other's updates. The file session store also refuses to write a stale auth key
+over a session another process re-keyed or removed (see `fileSessionStore`).
+
+**Reply attribution.** `--wait` results distinguish `replies` (observed after the action),
+`linked` (tied to it by Telegram: `reply_to` / edit of the pressed message) and `earlier`
+(ordered before it). Call `collector.setAction(...)` once the action's RPC returns.
+
 **Testing convention.** Keep network code thin and factor the logic into pure functions that
 are unit-tested directly (e.g. `inviteLinkFromFull`, `newDevicesResult`). For code that must
 issue RPCs, use the mock invoker helpers in tests: `newTestAPI(t)` (tgmock, `Expect()`-based)

README.md

@@ -7,10 +7,30 @@ groups and channels, and stream new messages in real time.
 
 Built on [`gotd/td`](https://github.com/gotd/td).
 
+> **This is the ApexAiOfficial fork of [`gotd/cli`](https://github.com/gotd/cli).**
+> It stays command- and output-compatible with upstream and adds what an agent
+> needs to test Telegram bots end to end: pressing inline and keyboard buttons,
+> collecting a bot's replies and edits in one call, deep-link starts, bot
+> introspection, richer message JSON (buttons, entities, file details), stable
+> exit codes with JSON errors, a global `--timeout`, and session and cache files
+> that concurrent `tg` processes share safely. See [Testing bots](#testing-bots).
+
 ## Installation
 
 Download a prebuilt binary or package (`.deb`/`.rpm`/`.apk`) for your platform from
-the [latest release](https://github.com/gotd/cli/releases/latest).
+the [latest upstream release](https://github.com/gotd/cli/releases/latest).
+
+To build this fork from source and install it as `~/.local/bin/tg`:
+
+```console
+$ scripts/install-local.sh                       # reuses the app credentials of the installed tg
+$ APP_ID=… APP_HASH=… scripts/install-local.sh   # or embed your own
+$ scripts/install-local.sh --creds-from /path/to/old/tg
+```
+
+Sessions are tied to the app credentials they were created with, so replacing a
+release binary with a source build should embed the same credentials:
+`--creds-from` reads them from an existing binary's build info.
 
 ## Quick start
 
@@ -52,6 +72,9 @@ Bot login still works too: provide a token via `tg init --token <bot-token>` and
 - **Consistent peers:** every `<peer>` accepts `me`/`self`, `@username`, a phone number,
   or a `t.me/…` link. Resolved access-hashes are cached locally.
 - **Safety:** destructive actions (`delete`, `delete-history`, `unpin-all`) require `--yes`.
+- **Stable exit codes and JSON errors** (see [Exit codes](#exit-codes)), and a global
+  `--timeout` so no call can hang an agent.
+- **Secrets off the command line:** `send`, `reply` and `edit` take `--stdin`.
 - **Shell completion:** `tg completion bash|zsh|fish|powershell`, with dynamic completion
   for peers, accounts, output format and enum flags.
 
@@ -64,13 +87,14 @@ $ tg history @durov --limit 20 -o json
 
 This repo ships an installable [Claude Code](https://claude.com/claude-code) skill
 (`skills/tg`) that teaches an agent how to drive Telegram with `tg` — setup checks,
-the JSON/peers conventions, the `--yes` safety gate, and common recipes. Install it
+the JSON/peers conventions, the `--yes` safety gate, exit codes, common recipes,
+and a bot-testing playbook. Install it
 with [`skills`](https://www.npmjs.com/package/skills):
 
 ```console
-$ npx skills add https://github.com/gotd/cli --all
+$ npx skills add https://github.com/ApexAiOfficial/tg-cli --all
 # or a single skill:
-$ npx skills add https://github.com/gotd/cli --skill tg
+$ npx skills add https://github.com/ApexAiOfficial/tg-cli --skill tg
 ```
 
 ## What you can do
@@ -80,6 +104,11 @@ Run `tg --help` for the full, grouped command list, or `tg <command> --help` for
 - **Messaging:** `send`, `reply`, `edit`, `delete`, `delete-history`, `forward`,
   `history`, `search` (`--global`), `read`, `pin`/`unpin`/`unpin-all`/`pinned`,
   `react`/`unreact`/`reactions`, `draft`/`drafts`, `schedule`, `poll`, `link`, `context`.
+- **Diagnostics:** `doctor` checks config, credentials, session, peer cache, connection
+  and login, with a fix hint for each failure (`--offline` skips the network).
+- **Bots:** `click` (press inline/keyboard buttons), `start-bot` (deep-link start),
+  `bot-info`, `webapp` (open a Mini App and get its signed launch data), and `--wait` reply collection on
+  `send`/`reply`/`click`/`start-bot`/`upload`.
 - **Media:** `upload` (with `--type` video/audio/voice/gif/sticker), `album`, `download`,
   `stickers`.
 - **Chats & contacts:** `chats list`, `chat get`/`full`, `mute`/`unmute`,
@@ -94,14 +123,82 @@ Run `tg --help` for the full, grouped command list, or `tg <command> --help` for
 - **Realtime:** `watch [peer]` streams new messages as JSON lines; `wait [peer]` blocks
   until the next message (with `--timeout`).
 
+## Testing bots
+
+`tg` acts as a real user, so an agent (or a test script) can drive any bot the way
+a person would and check what actually happens:
+
+```console
+$ tg bot-info @mybot                                  # descriptions, command menu, menu button
+$ tg webapp @mybot --out session.json                 # open its Mini App: real signed launch data, owner-only file
+$ tg send --peer @mybot /start --wait 30s -o json     # send, then collect replies until the bot settles
+$ tg start-bot @mybot ref_42 --wait 20s               # deep link: t.me/mybot?start=ref_42
+$ tg click @mybot --text "Settings" --wait 15s -o json  # press an inline button, get the answer + edits
+$ tg click @mybot 4242 --data "page:2"                # by callback data, on a given message
+$ tg reply @mybot 4250 "Berlin" --wait 20s            # answer a prompt
+$ tg upload --peer @mybot ./sample.pdf --wait 30s     # media handling
+$ tg history @mybot --after 4242 -o json              # everything newer than a message
+```
+
+`--wait <dur>` (on `send`, `reply`, `click`, `start-bot`, `upload`) subscribes to
+updates *before* acting, so a fast reply is never missed. It stops when the bot has
+been quiet for `--settle` (default 2s, extended while the bot shows "typing…"),
+when `--replies N` messages arrived, or when `--wait` elapses (`timed_out: true`).
+
+Telegram does not record which message a bot is answering, so the result separates
+what is proven from what is observed:
+
+- `replies`: the bot's messages ordered after your action, in their latest version
+  (bots that stream by editing are handled). Very likely the response, but a bot
+  that also sends unprompted messages, or a second conversation with the same bot
+  from this account, can add messages here.
+- `edits`: other messages the bot edited, e.g. the one whose button was pressed.
+- `linked`: ids Telegram itself ties to your action (a `reply_to` your message, an
+  edit of the pressed message). Deterministic.
+- `earlier`: messages the bot sent before your action (ids are ordered per chat),
+  such as a late answer to a previous command. Never a response to this action.
+
+`replies` is always present (possibly `[]`); `edits`, `linked`, `earlier` and `timed_out`
+are omitted when empty or false, like every optional field in tg's JSON, so read them
+with a default (`jq '.data.linked // []'`).
+
+Message JSON includes `reply_markup` (inline buttons or reply keyboard, with each
+button's `type`, `text`, callback `data`, `url`), `entities` (formatting and links,
+with the covered `text`), `file` (`kind`, `name`, `mime`, `size`, `duration`,
+dimensions), `edit_date` and `via_bot_id`. New fields are omitted when empty, so
+existing consumers see the same shape.
+
+## Exit codes
+
+| Code | Meaning |
+| --- | --- |
+| 0 | success |
+| 1 | other failure |
+| 2 | usage: unknown command, bad flags or arguments |
+| 3 | auth: not logged in or session revoked (`tg login`) |
+| 4 | not found: peer, message or button |
+| 5 | flood wait: rate limited beyond the retry budget |
+| 6 | timeout: `--timeout`, or `wait --timeout` |
+| 7 | other Telegram API error |
+| 130, 143 | interrupted by SIGINT / SIGTERM (128 + signal) |
+
+Errors are printed to stderr as `tg: <message>`. With `--output json` a failure
+also writes an envelope to stdout, so a parser always gets JSON:
+
+```json
+{"schema": 1, "error": {"code": "flood_wait", "exit": 5, "message": "…", "retry_after": 42,
+  "rpc": {"code": 420, "type": "FLOOD_WAIT", "argument": 42}}}
+```
+
 ## Global flags
 
 | Flag | Description |
 | --- | --- |
 | `-o, --output text\|json` | output format (default `text`) |
 | `-a, --account <label>` | select an account, or `all` to fan out across accounts |
-| `-c, --config <path>` | config file to use |
+| `-c, --config <path>` | config file to use (env `TG_CONFIG`) |
 | `--proxy <url>` | `socks5://…` or `tg://proxy?…` (MTProxy) |
+| `--timeout <dur>` | abort the command after this long, exit code 6 (env `TG_TIMEOUT`) |
 | `--test` | connect to the Telegram test server (persisted by `tg init --test`) |
 
 ## Multiple accounts

cmd/tg/app.go

@@ -2,6 +2,7 @@ package main
 
 import (
 	"context"
+	"time"
 
 	"github.com/go-faster/errors"
 	"github.com/spf13/cobra"
@@ -58,6 +59,10 @@ type app struct {
 	outputFormat string
 	proxyURL     string
 	accountFlag  string
+	timeout      time.Duration
+
+	// cancelTimeout releases the --timeout context; see before and close.
+	cancelTimeout context.CancelFunc
 
 	cfg     Config
 	log     *zap.Logger
@@ -79,13 +84,26 @@ func newApp() *app {
 		panic(err)
 	}
 
+	// Surface rate limiting: without this a flood wait looks like a hang.
+	waiter := floodwait.NewWaiter().WithCallback(func(_ context.Context, w floodwait.FloodWait) {
+		defaultLog.Warn("Rate limited by Telegram (FLOOD_WAIT); retrying after the wait",
+			zap.Duration("wait", w.Duration))
+	})
+
 	return &app{
-		waiter:  floodwait.NewWaiter(),
+		waiter:  waiter,
 		log:     defaultLog,
 		printer: output.New(output.Text, nil),
 	}
 }
 
+// close releases resources held across the command run.
+func (a *app) close() {
+	if a.cancelTimeout != nil {
+		a.cancelTimeout()
+	}
+}
+
 // before is wired as the root command's PersistentPreRunE; it runs before every
 // subcommand. It sets up the output printer and (for commands that need it)
 // loads the config.
@@ -96,6 +114,12 @@ func (a *app) before(cmd *cobra.Command) error {
 	}
 	a.printer = output.New(format, cmd.OutOrStdout())
 
+	if a.timeout > 0 {
+		ctx, cancel := context.WithTimeout(cmd.Context(), a.timeout)
+		cmd.SetContext(ctx)
+		a.cancelTimeout = cancel
+	}
+
 	if skipConfig(cmd) {
 		return nil
 	}
@@ -157,11 +181,14 @@ func (a *app) ensureActive() error {
 	return a.activate(labels[0], false)
 }
 
+// helpCmdName is cobra's built-in help command.
+const helpCmdName = "help"
+
 // skipConfig reports whether the command runs without a loaded config/session.
 func skipConfig(cmd *cobra.Command) bool {
 	for c := cmd; c != nil; c = c.Parent() {
 		switch c.Name() {
-		case "init", "docs", "completion", "help",
+		case "init", docsCmdName, "completion", helpCmdName, versionCmdName, doctorCmdName,
 			cobra.ShellCompRequestCmd, cobra.ShellCompNoDescRequestCmd:
 			return true
 		}
@@ -173,6 +200,10 @@ func skipConfig(cmd *cobra.Command) bool {
 type runParams struct {
 	auth    authKind
 	updates bool
+	// register attaches update handlers to the dispatcher (updates only). It
+	// runs before the client connects: the dispatcher's handler map is not
+	// safe to modify while updates are being dispatched.
+	register func(d tg.UpdateDispatcher)
 	// authorize performs interactive authentication when the session is not yet
 	// authorized. If nil, user sessions error with errNotAuthorized and bot
 	// sessions authenticate with the configured token.
@@ -217,6 +248,9 @@ func (a *app) connectWith(
 	var d tg.UpdateDispatcher
 	if rp.updates {
 		d = tg.NewUpdateDispatcher()
+		if rp.register != nil {
+			rp.register(d)
+		}
 	}
 
 	appID, appHash, err := effectiveCreds(st.acc)