Repository navigation
Back up foreign data wrapper objects on Cloudberry - #129
talmacschen-arch wants to merge 2 commits into
Conversation
retrieveFDWObjects gates foreign data wrapper, foreign server and user
mapping backup on "GPDB 6+ or Cloudberry". A misplaced negation --
!(IsGPDB && AtLeast6) || IsCBDB() instead of !((IsGPDB && AtLeast6) ||
IsCBDB()) -- makes the guard return early on Cloudberry, so none of these
objects are backed up there. Foreign tables are backed up through a separate
path, so a Cloudberry backup produces CREATE FOREIGN TABLE statements
referencing servers it never dumped, and the restore fails with:
ERROR: server "..." does not exist
Fix the negation so Cloudberry backs up foreign data wrappers, servers and
user mappings, matching the "GPDB 6+ or Cloudberry" pattern used in every
other hunk of the commit that introduced this guard.
Add an end-to-end test that backs up and restores a foreign data wrapper,
server, user mapping and foreign table and asserts they exist after restore.
Real cross-cluster validationVerified end-to-end with a real Setup: an Apache Cloudberry / SynxDB4 cluster with the patched binary, holding a
Without this fix, step 2 omits the wrapper/server/user mapping on Cloudberry, so the restored foreign table fails with |
liang8283
left a comment
There was a problem hiding this comment.
Thanks for the fix — the precedence analysis is right, and with this change FDW objects are backed up on Cloudberry again. I verified it end to end on a Cloudberry cluster with a full gpbackup → gprestore round trip, building both main and this branch. The test database had a postgres_fdw server pointing at another local database (user mappings for a named role and PUBLIC, plus a foreign table on it), a handler-less fdw with a server that has options, a comment, a GRANT USAGE and an owner change, and a regular external table:
| Check | main |
this PR |
|---|---|---|
CREATE FOREIGN DATA WRAPPER / CREATE SERVER / CREATE USER MAPPING in metadata.sql |
❌ none emitted | ✅ emitted, server before the foreign table |
| Foreign tables restore | ❌ server "pg_remote" does not exist |
✅ restored, and postgres_fdw table returns the remote rows |
| Server/wrapper options, owners and ACLs | — | ✅ identical to the source |
Extension-owned gp_exttable_fdw / gp_exttable_server |
✅ not emitted | ✅ still not emitted, no duplicate creation |
| External table | ✅ | ✅ |
COMMENT ON SERVER |
— | ❌ fails on restore (see below) |
go test ./backup/... passes.
COMMENT on a foreign server produces invalid SQL
Now that servers are backed up on Cloudberry, a server with a comment is emitted as:
COMMENT ON FOREIGN SERVER sc IS 'plain server';
ERROR: syntax error at or near "SERVER" (SQLSTATE 42601)
The correct syntax is COMMENT ON SERVER. GetOwnerStatement in backup/predata_acl.go already maps toc.OBJ_FOREIGN_SERVER to toc.OBJ_SERVER for ALTER SERVER ... OWNER, but GetCommentStatement uses the object type as is. The bug isn't new (it affects GPDB 6 too), but on Cloudberry it was unreachable until this PR, so the PR now exposes it: the comment is lost, and without --on-error-continue the restore aborts.
Could you apply the same mapping in GetCommentStatement, and add a COMMENT ON SERVER fdw_server IS '...' to the new end-to-end test so it covers this path?
Now that foreign servers are backed up on Cloudberry, a server carrying a
comment is emitted by GetCommentStatement as
COMMENT ON FOREIGN SERVER <name> IS '...';
which is a syntax error on restore (ERROR: syntax error at or near "SERVER"):
the correct form is COMMENT ON SERVER. GetOwnerStatement already maps
OBJ_FOREIGN_SERVER to OBJ_SERVER for ALTER ... OWNER; apply the same mapping in
GetCommentStatement. The bug predates this series (it affects GPDB 6 too) but
was unreachable on Cloudberry until foreign servers started being backed up.
Verified end to end on a live Cloudberry 2.1.0 cluster: a foreign server with
a comment now backs up as COMMENT ON SERVER and restores cleanly, with the
comment preserved. Extend the FDW end-to-end test with a COMMENT ON SERVER and
assert it survives restore.
|
Thanks — good catch on the comment syntax. Fixed: Verified end to end on a live Cloudberry 2.1.0 cluster with the rebuilt (confirmed in |
Problem
On Cloudberry, restoring a database that contains foreign tables fails to create them:
The backup's
metadata.sqlcontains theCREATE FOREIGN TABLEstatements but noCREATE FOREIGN DATA WRAPPER,CREATE SERVERorCREATE USER MAPPINGfor the objects they depend on.Root cause
retrieveFDWObjects(backup/wrappers.go) gates foreign data wrapper / server / user mapping backup:By operator precedence this is
(!(IsGPDB && >=6)) || IsCBDB(), which returns (skips) on Cloudberry. The intent was to run for "GPDB 6+ OR Cloudberry", i.e.!((IsGPDB && >=6) || IsCBDB())— the same pattern every other hunk of the commit that added this guard (#14, "Implement database-specific logic for Cloudberry") uses. Because foreign tables are dumped through a separate path, a Cloudberry backup emits foreign tables whose wrappers/servers/mappings were never dumped, producing an unrestorable backup.Confirmed against a real backup: its
metadata.sqlhad 0CREATE FOREIGN DATA WRAPPER, 0CREATE SERVER, 0CREATE USER MAPPING, but 6CREATE FOREIGN TABLE— consistent withretrieveFDWObjectsbeing skipped entirely.Fix
Correct the negation so Cloudberry backs up foreign data wrappers, servers and user mappings. Verified end-to-end on SynxDB4.5 / Apache Cloudberry 2.1.0: a backup now emits
CREATE FOREIGN DATA WRAPPER/CREATE SERVER/CREATE USER MAPPINGin correct dependency order (server before the foreign table), and the foreign table restores. Thebackupunit tests still pass and no test asserted the old skip.Testing
Added an end-to-end test that creates a foreign data wrapper, server, user mapping and foreign table, backs up and restores them, and asserts the server, user mapping and foreign table exist after restore.