Skip to content

[fix](be) Redact HTTP query details from request diagnostics - #69506

Merged
mrhhsg merged 1 commit into
apache:masterfrom
mrhhsg:fix/redact-be-http-request-debug-output
Oct 26, 2026
Merged

mrhhsg merged 1 commit into
apache:masterfrom
mrhhsg:fix/redact-be-http-request-debug-output

Conversation

@mrhhsg

@mrhhsg mrhhsg commented Oct 23, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: HttpRequest::debug_string() rendered the full request URI and every parsed parameter key and value, so query credentials could appear in BE diagnostics. The query-parse failure warning also included the raw query. Render the parsed path and parameter count instead, and omit query text from that warning. Request parsing and URI/parameter accessors retain their existing behavior. This change is limited to these logging paths; action-specific HTTP logs are unchanged.

Release note

BE HttpRequest::debug_string() no longer emits query strings or parameter keys and values. Query-parse failure warnings no longer emit the raw query.

Check List (For Author)

  • Test:
    • Unit Test: HttpRequestTest.* passed (9/9). The unmodified full BE UT build was blocked by a pre-existing compile error in an unrelated test translation unit; the focused suite was run with that unit temporarily excluded from the UT build.
    • Build: BUILD_TYPE=ASAN ./build.sh --be passed.
    • Static and format checks: BE build hygiene, clang-format, check-format, and changed-line clang-tidy passed.
    • Regression test: Not run; this changes BE diagnostic log text, not SQL or HTTP responses.
  • Behavior changed: Yes (diagnostic log text only).
  • Does this need documentation: No.

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: BE HTTP request diagnostics rendered the full URI and each parsed parameter, so query credentials could appear in request logs. Render only the parsed path and parameter count, and omit raw query text from the parse-failure warning. Request accessors retain their original values.

### Release note

BE HTTP request diagnostic logs no longer include query strings or parameter details.

### Check List (For Author)

- Test:
    - Unit Test: HttpRequestTest.* (9 cases, using a temporary exclusion for an unrelated pre-existing test compile error)
    - Regression test: Not applicable; only diagnostic log rendering changes
- Behavior changed: Yes (diagnostic log text only)
- Does this need documentation: No
@mrhhsg

mrhhsg commented Oct 23, 2026

Copy link
Copy Markdown
Member Author

/review

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the complete patch at 9a9ffcbfe4cd36a7b29b7216b05a332f43da7b4b. No actionable new findings or existing blocking inline comments were identified.

Critical checkpoint conclusions:

  • Goal and proof: diagnostics render the parsed path and parameter count; the parse-failure warning omits the query. The added tests exercise real query parsing and preserved accessors.
  • Scope: the three production rendering edits and three focused unit tests are small and consistent with the stated logging-only goal.
  • Concurrency: no new shared mutation, thread, lock, lock-held work, or lock-order dependency. Initialization and route insertion precede handler logging.
  • Lifecycle: no new production ownership or static initialization. Fixtures detach the borrowed URI and freed components before libevent request destruction; the later wrapper destructor has no handler context (pinned libevent implementation).
  • Configuration: no new configuration or dynamic-update behavior.
  • Compatibility: diagnostic text intentionally changes; public interfaces, HTTP parsing/responses, wire/storage formats, and rolling-upgrade behavior are unchanged. No new libc API is introduced.
  • Thrift/protobuf: no definitions or field IDs change.
  • Parallel paths: auth, action, web-page, and async-timeout callers share the updated renderer. Header-only rendering needs no query change. Separate action-specific logs remain outside the declared scope.
  • Conditional/error handling: no condition changes; query-parse failures still return the original error before handler dispatch.
  • Test coverage: plain credentials, encoded/case-varied query parameters, an inserted route-map entry, no-query requests, path/count output, and unchanged URI/parameter getters are covered by the added assertions. Existing header-masking tests remain.
  • Test results: no golden files change; expected decoding, counts, and absence assertions match the code. Tests were inspected, not executed.
  • Observability: method, path, masked headers, and parameter count remain available. Reduced parameter detail is intentional; no new execution path requires metrics.
  • Transactions/persistence/failover: unaffected; no visibility, version, bitmap, or EditLog changes.
  • Data writes/atomicity: no data-write or crash-recovery behavior changes, and no new retained allocation.
  • FE/BE variables: no transmitted variables or sending paths change.
  • Performance: parameter iteration and full-query output are removed; map size is constant-time. No new expensive operation was found.
  • Further issues: URI component separation and fixture cleanup were independently rechecked; no unresolved candidate remains.

User focus: none supplied. Validation limits: this was a static review with no builds, tests, or live HTTP requests. The parse-failure warning has no dedicated log-capture test; its constant rendering was inspected directly. Path contents, existing header behavior, and separate action-specific logs are outside the query-diagnostic change.

Review converged after 2 round(s).

@mrhhsg

mrhhsg commented Oct 23, 2026

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 66.57% (31982/48045)
Line Coverage 51.37% (337338/656731)
Region Coverage 46.72% (272866/584041)
Branch Coverage 48.46% (128637/265456)

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 28802 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9a9ffcbfe4cd36a7b29b7216b05a332f43da7b4b, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17591	5154	5031	5031
q2	2104	311	292	292
q3	10229	1462	817	817
q4	4678	466	333	333
q5	7432	810	531	531
q6	181	187	153	153
q7	734	792	598	598
q8	9259	1335	1142	1142
q9	5449	4181	4164	4164
q10	6818	1286	1021	1021
q11	434	248	228	228
q12	641	398	288	288
q13	18092	3680	3063	3063
q14	271	262	247	247
q15	q16	724	731	661	661
q17	1194	746	584	584
q18	6937	6202	6073	6073
q19	1102	956	615	615
q20	389	338	232	232
q21	5150	2440	2525	2440
q22	401	324	289	289
Total cold run time: 99810 ms
Total hot run time: 28802 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5975	5945	6053	5945
q2	757	566	550	550
q3	5501	5666	5320	5320
q4	2004	2131	1544	1544
q5	5902	5738	5563	5563
q6	251	192	143	143
q7	2204	1983	1873	1873
q8	2964	2603	2591	2591
q9	7809	7917	7753	7753
q10	3635	3589	3149	3149
q11	578	403	379	379
q12	672	672	498	498
q13	3308	3540	3066	3066
q14	295	288	262	262
q15	q16	658	680	635	635
q17	7718	7010	6807	6807
q18	13015	12160	12863	12160
q19	964	806	806	806
q20	2175	2139	1927	1927
q21	5547	4644	4684	4644
q22	506	448	422	422
Total cold run time: 72438 ms
Total hot run time: 66037 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 150550 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9a9ffcbfe4cd36a7b29b7216b05a332f43da7b4b, data reload: false

query5	4338	586	432	432
query6	424	204	190	190
query7	4812	434	224	224
query8	318	162	147	147
query9	8765	3070	3084	3070
query10	489	331	249	249
query11	6059	3830	3557	3557
query12	152	92	88	88
query13	1263	436	328	328
query14	6099	3881	3619	3619
query14_1	3427	3365	3424	3365
query15	211	206	180	180
query16	933	436	409	409
query17	861	642	522	522
query18	2417	418	310	310
query19	190	161	135	135
query20	87	83	86	83
query21	205	124	109	109
query22	12865	13097	12893	12893
query23	13356	12903	12570	12570
query23_1	12556	12558	12474	12474
query24	7389	826	458	458
query24_1	469	486	467	467
query25	537	416	364	364
query26	1253	258	139	139
query27	2758	436	274	274
query28	4638	1925	1939	1925
query29	1563	612	450	450
query30	299	221	183	183
query31	1017	935	811	811
query32	154	98	92	92
query33	519	298	238	238
query34	910	845	510	510
query35	735	740	636	636
query36	801	847	713	713
query37	137	97	82	82
query38	1794	1751	1706	1706
query39	798	792	753	753
query39_1	732	724	743	724
query40	222	115	98	98
query41	73	69	66	66
query42	87	87	83	83
query43	345	356	315	315
query44	1313	684	687	684
query45	180	176	171	171
query46	826	987	605	605
query47	3056	2780	2731	2731
query48	292	308	219	219
query49	573	398	289	289
query50	663	269	202	202
query51	10284	10308	10588	10308
query52	77	75	68	68
query53	190	201	147	147
query54	237	183	190	183
query55	71	65	63	63
query56	229	236	219	219
query57	1648	1603	1549	1549
query58	281	263	256	256
query59	2110	2152	1923	1923
query60	289	240	225	225
query61	144	150	151	150
query62	378	330	285	285
query63	192	162	158	158
query64	2823	927	736	736
query65	3328	3288	3237	3237
query66	1772	419	293	293
query67	20195	19961	20036	19961
query68	3214	993	611	611
query69	408	293	246	246
query70	824	796	770	770
query71	306	237	207	207
query72	2843	2615	2351	2351
query73	513	520	293	293
query74	4652	4504	4326	4326
query75	2310	2291	1932	1932
query76	2408	997	592	592
query77	349	378	263	263
query78	9667	9741	9258	9258
query79	958	875	515	515
query80	672	430	344	344
query81	545	328	269	269
query82	677	131	104	104
query83	256	192	177	177
query84	310	122	111	111
query85	893	421	362	362
query86	334	238	229	229
query87	2005	1933	1795	1795
query88	3607	2656	2641	2641
query89	333	279	239	239
query90	1780	170	162	162
query91	160	152	122	122
query92	99	85	90	85
query93	988	989	565	565
query94	470	310	271	271
query95	572	385	303	303
query96	670	479	232	232
query97	2443	2423	2300	2300
query98	153	154	151	151
query99	765	769	643	643
Total cold run time: 234169 ms
Total hot run time: 150550 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 24.17 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9a9ffcbfe4cd36a7b29b7216b05a332f43da7b4b, data reload: false

query1	0.00	0.00	0.01
query2	0.10	0.05	0.05
query3	0.26	0.10	0.09
query4	1.61	0.10	0.10
query5	0.23	0.24	0.23
query6	1.17	0.68	0.67
query7	0.04	0.01	0.00
query8	0.05	0.04	0.04
query9	0.39	0.34	0.35
query10	0.58	0.59	0.61
query11	0.19	0.15	0.14
query12	0.19	0.14	0.15
query13	0.48	0.49	0.48
query14	1.04	1.01	1.01
query15	0.65	0.60	0.61
query16	0.35	0.34	0.33
query17	1.16	1.07	1.14
query18	0.20	0.19	0.19
query19	2.10	1.93	1.99
query20	0.02	0.01	0.02
query21	15.49	0.18	0.13
query22	4.97	0.06	0.05
query23	16.25	0.32	0.13
query24	3.04	0.53	0.43
query25	0.08	0.05	0.06
query26	1.37	0.22	0.16
query27	0.04	0.05	0.03
query28	2.91	0.73	0.35
query29	12.55	4.07	3.23
query30	0.29	0.13	0.11
query31	2.78	0.55	0.32
query32	3.22	0.59	0.50
query33	3.25	3.27	3.18
query34	15.61	3.98	3.42
query35	3.30	3.31	3.33
query36	0.54	0.44	0.41
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 97.01 s
Total hot run time: 24.17 s

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 77.19% (35931/46549)
Line Coverage 62.50% (406033/649693)
Region Coverage 58.97% (343167/581930)
Branch Coverage 59.89% (158591/264807)

@mrhhsg
mrhhsg marked this pull request as ready for review October 24, 2026 03:12
@mrhhsg
mrhhsg merged commit aed8ecd into apache:master Oct 26, 2026
44 checks passed
github-actions Bot pushed a commit that referenced this pull request Oct 26, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: `HttpRequest::debug_string()` rendered the full request
URI and every parsed parameter key and value, so query credentials could
appear in BE diagnostics. The query-parse failure warning also included
the raw query. Render the parsed path and parameter count instead, and
omit query text from that warning. Request parsing and URI/parameter
accessors retain their existing behavior. This change is limited to
these logging paths; action-specific HTTP logs are unchanged.

### Release note

BE `HttpRequest::debug_string()` no longer emits query strings or
parameter keys and values. Query-parse failure warnings no longer emit
the raw query.

### Check List (For Author)

- Test:
- Unit Test: `HttpRequestTest.*` passed (9/9). The unmodified full BE UT
build was blocked by a pre-existing compile error in an unrelated test
translation unit; the focused suite was run with that unit temporarily
excluded from the UT build.
    - Build: `BUILD_TYPE=ASAN ./build.sh --be` passed.
- Static and format checks: BE build hygiene, clang-format,
check-format, and changed-line clang-tidy passed.
- Regression test: Not run; this changes BE diagnostic log text, not SQL
or HTTP responses.
- Behavior changed: Yes (diagnostic log text only).
- Does this need documentation: No.
yiguolei pushed a commit that referenced this pull request Oct 27, 2026
…ics #69506 (#69640)

Cherry-picked from #69506

Co-authored-by: Jerry Hu <hushenggang@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants