Skip to content

Default deployment history to enforced file-backed recording - #7169

Draft
shreyas-goenka wants to merge 1 commit into
shreyas-goenka_data/shreyas/statedb-modesfrom
shreyas-goenka_data/shreyas/history-best-effort-default
Draft

Default deployment history to enforced file-backed recording#7169
shreyas-goenka wants to merge 1 commit into
shreyas-goenka_data/shreyas/statedb-modesfrom
shreyas-goenka_data/shreyas/history-best-effort-default

Conversation

@shreyas-goenka

@shreyas-goenka shreyas-goenka commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Both experimental.deployment_history: true and DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=true now default to WritingFileWithHistoryEnforced. Resources remain in resources.json; history errors warn, and the WAL opens only after approval.

Preserve WritingDMS, its existing error policy, and ordinary file deployments. Existing DMS-marked state keeps that backend. Version allocation uses max(file serial, latest DMS version) + 1; saved plans also validate the file serial independently so history failures cannot hide intervening file-state changes.

Stack: #7151 → #7169 (this PR) → #7171.

Validation: complete local bundle acceptance matrix, affected Go/race tests, and scoped Go/Python lint pass. The unrelated SSH acceptance timeouts also reproduce on the unchanged #7151 base.

@github-actions

github-actions Bot commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Commit: d2f84b526 | Workflow run

Regression test report

Selected 20 changed tests (limit=20, 2 not selected)

21 tests: regression=18 unreleased=0 coverage=3 failing=0

test branch main (bef6321) latest (v1.22.1)
TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS= ✅ ✅ ✅
TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/bind-unbind-unsupported/DMS=true ✅ ✅ ✅
TestAccept/bundle/dms/declined-deploy/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/depends-on/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/deployment-metadata-change/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/empty-grants/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/existing-state/DMS= ✅ ❌ ❌
TestAccept/bundle/dms/failed-delete/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/failed-recreate/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/failed-update-permissions/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/multiple-resources/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/no-drift/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/no-resources/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/operation-upload-fails-once/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/plan-without-dms/DMS= ✅ ✅ ✅
TestAccept/bundle/dms/provenance/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/record-failure/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/stale-local-state/DMS= ✅ ❌ ❌
TestAccept/bundle/dms/state-features/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/version-never-created/DMS=true ✅ ❌ ❌
TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1307
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1077
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:655
        	Error:      	Not equal: 
        	            	expected: "\n>>> musterr [CLI] bundle deploy --plan plan.json\nWarn: Plan was created with CLI version 0.0.0-test but current version is [CLI_VERSION]\nError: plan lineage \"different-lineage\" does not match state lineage \"\"; the state may have been modified by another process\n\n"
        	            	actual  : "\n>>> musterr [CLI] bundle deploy --plan plan.json\nWarn: Plan was created with CLI version 0.0.0-test but current version is [CLI_VERSION]\nError: this plan was created for a different set of state features than the target now has; run 'bundle plan' again\n\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,3 +3,3 @@
        	            	 Warn: Plan was created with CLI version 0.0.0-test but current version is [CLI_VERSION]
        	            	-Error: plan lineage "different-lineage" does not match state lineage ""; the state may have been modified by another process
        	            	+Error: this plan was created for a different set of state features than the target now has; run 'bundle plan' again
        	            	 
        	Test:       	TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS=true
        	Messages:   	bundle/deploy/readplan/lineage-mismatch/output.txt vs /tmp/TestAcceptbundledeployreadplanlineage-mismatchDMS=true804996204/002/output.txt

latest (v1.22.1): same output as main (bef6321)

TestAccept/bundle/dms/declined-deploy/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/dms/declined-deploy/output.txt
        +++ /tmp/TestAcceptbundledmsdeclined-deployDMS=true3292913932/002/output.txt
        @@ -66,9 +66,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -82,8 +80,16 @@
         }
         {
           "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
        +  "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         >>> print_state.py --no-dms
         {
        @@ -92,17 +96,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_declined_deploy_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "name": "dms_declined_deploy_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A destructive change without --auto-approve is declined: this console cannot prompt
        @@ -125,6 +120,10 @@
           "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         === The recorded resources are unchanged by the decline: the schema is still at version 1
         >>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-resources deployments/[DEPLOYMENT_ID] -o json
        @@ -161,6 +160,10 @@
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
         {
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
           "method": "POST",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions",
           "q": {
        
    acceptance_test.go:1114: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1792993390330,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_dhzzkzzwcje63fm67pflntnvdu",
        LOG.catalog:   "metastore_id": "7a06fce6-b6fe-91ca-7a06-fce6b6fe92a6",
        LOG.catalog:   "name": "dms_other_dhzzkzzwcje63fm67pflntnvdu",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1792993390330,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }

latest (v1.22.1):

acceptance_test.go:1307: Diff:
        --- bundle/dms/declined-deploy/output.txt
        +++ /tmp/TestAcceptbundledmsdeclined-deployDMS=true744756129/002/output.txt
        @@ -66,9 +66,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -82,8 +80,16 @@
         }
         {
           "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
        +  "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         >>> print_state.py --no-dms
         {
        @@ -92,17 +96,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_declined_deploy_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "name": "dms_declined_deploy_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A destructive change without --auto-approve is declined: this console cannot prompt
        @@ -125,6 +120,10 @@
           "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         === The recorded resources are unchanged by the decline: the schema is still at version 1
         >>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-resources deployments/[DEPLOYMENT_ID] -o json
        @@ -161,6 +160,10 @@
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
         {
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
           "method": "POST",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions",
           "q": {
        
    acceptance_test.go:1114: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1792993401845,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_pwcxr5xd7vhdto4uuxibhuclze",
        LOG.catalog:   "metastore_id": "7a06fce9-655f-ef83-7a06-fce9655ff088",
        LOG.catalog:   "name": "dms_other_pwcxr5xd7vhdto4uuxibhuclze",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1792993401845,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }
TestAccept/bundle/dms/depends-on/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/dms/depends-on/output.txt
        +++ /tmp/TestAcceptbundledmsdepends-onDMS=true652313027/002/output.txt
        @@ -5,7 +5,7 @@
           "plan_version": [PLAN_VERSION],
           "cli_version": "[CLI_VERSION]",
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -60,9 +60,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -92,53 +90,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.child": {
        -   "__id__": "[CHILD_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-depends-on-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "description": "depends on [PARENT_ID]",
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "child",
        -    "queue": {
        -     "enabled": true
        -    }
        -   },
        -   "depends_on": [
        -    {
        -     "node": "resources.jobs.parent",
        -     "label": "${resources.jobs.parent.id}"
        -    }
        -   ]
        -  },
        -  "resources.jobs.parent": {
        -   "__id__": "[PARENT_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-depends-on-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "parent",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         >>> [CLI] bundle destroy --auto-approve
        
FAIL

latest (v1.22.1):

acceptance_test.go:1307: Diff:
        --- bundle/dms/depends-on/output.txt
        +++ /tmp/TestAcceptbundledmsdepends-onDMS=true2709076431/002/output.txt
        @@ -5,7 +5,7 @@
           "plan_version": [PLAN_VERSION],
           "cli_version": "[CLI_VERSION]",
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -60,9 +60,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -92,53 +90,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.child": {
        -   "__id__": "[CHILD_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-depends-on-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "description": "depends on [PARENT_ID]",
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "child",
        -    "queue": {
        -     "enabled": true
        -    }
        -   },
        -   "depends_on": [
        -    {
        -     "node": "resources.jobs.parent",
        -     "label": "${resources.jobs.parent.id}"
        -    }
        -   ]
        -  },
        -  "resources.jobs.parent": {
        -   "__id__": "[PARENT_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-depends-on-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "parent",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         >>> [CLI] bundle destroy --auto-approve
TestAccept/bundle/dms/deployment-metadata-change/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/dms/deployment-metadata-change/output.txt
        +++ /tmp/TestAcceptbundledmsdeployment-metadata-changeDMS=true1964223723/002/output.txt
        @@ -66,9 +66,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -128,9 +126,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }

latest (v1.22.1): same output as main (bef6321)

TestAccept/bundle/dms/empty-grants/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/dms/empty-grants/output.txt
        +++ /tmp/TestAcceptbundledmsempty-grantsDMS=true4283091396/002/output.txt
        @@ -137,9 +137,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 2,
        -  "file_serial": 2,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -164,17 +162,9 @@
          "lineage": "[UUID]",
          "serial": 2,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_empty_grants_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "name": "dms_empty_grants_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         >>> [CLI] bundle destroy --auto-approve

latest (v1.22.1): same output as main (bef6321)

TestAccept/bundle/dms/existing-state/DMS=: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/dms/existing-state/output.txt
        +++ /tmp/TestAcceptbundledmsexisting-stateDMS=3432691005/002/output.txt
        @@ -63,11 +63,6 @@
         
         >>> print_requests.py --dms //api/2.0/bundle --oneline
         {"method": "POST", "path": "/api/2.0/bundle/deployments", "body": {"display_name": "dms-existing-state-[UNIQUE_NAME]", "initial_parent_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/state", "target_name": "default", "workspace_info": {"file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/files", "root_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default"}}}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "2"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_DELETE", "resource_key": "jobs.one"}], "version_type": "VERSION_TYPE_DESTROY"}}
        -{"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/operations/jobs.one", "q": {"update_mask": "state,error_message,status"}, "body": {"error_message": "", "sequence_id": 0, "status": "OPERATION_STATUS_SUCCEEDED"}}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/complete", "body": {"completion_reason": "VERSION_COMPLETE_SUCCESS"}}
        -{"method": "DELETE", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments", "body": {"display_name": "dms-existing-state-[UNIQUE_NAME]", "initial_parent_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/state", "target_name": "default", "workspace_info": {"file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/files", "root_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default"}}}
         {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "1"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_CREATE", "resource_key": "jobs.one"}], "version_type": "VERSION_TYPE_DEPLOY"}}
         {"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/1/operations/jobs.one", "q": {"update_mask": "state,error_message,resource_id,status"}, "body": {"error_message": "", "resource_id": "[ONE_ID]", "sequence_id": 0, "state": "{\"state\":{\"deployment\":{\"deployment_id\":\"[DEPLOYMENT_ID]\",\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/state/metadata.json\",\"version_id\":\"1\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"one\",\"queue\":{\"enabled\":true}}}", "status": "OPERATION_STATUS_SUCCEEDED"}}
         {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/1/complete", "body": {"completion_reason": "VERSION_COMPLETE_SUCCESS"}}
        @@ -78,6 +73,6 @@
         {
           "state_version": 3,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }

latest (v1.22.1): same output as main (bef6321)

TestAccept/bundle/dms/failed-delete/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/dms/failed-delete/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-deleteDMS=true2894469520/002/output.txt
        @@ -66,9 +66,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -93,28 +91,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.stuck": {
        -   "__id__": "[STUCK_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-delete-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "stuck",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A resource that fails to delete is recorded as a failed operation carrying the error, so the history says why rather than showing the resource gone
        @@ -202,30 +180,11 @@
          "state_version": 3,
          "cli_version": "[CLI_VERSION]",
          "lineage": "[UUID]",
        - "serial": 1,
        + "serial": 2,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.stuck": {
        -   "__id__": "[STUCK_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-delete-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "stuck",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === The remote checkpoint stays at serial 1 while DMS advances to version 2

latest (v1.22.1): same output as main (bef6321)

TestAccept/bundle/dms/failed-recreate/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/dms/failed-recreate/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-recreateDMS=true1364926506/002/output.txt
        @@ -66,9 +66,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -93,18 +91,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_failed_recreate_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "comment": "v1",
        -    "name": "dms_failed_recreate_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A recreate deletes and then fails to create. The delete went through, so the resource really is gone: the failed operation clears its state (self-contained, not assuming the delete's own write reached the service), rather than describing the resource as it was before the deploy
        @@ -189,7 +177,7 @@
          "lineage": "[UUID]",
          "serial": 2,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
          "state": {}
         }
        
    acceptance_test.go:1114: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1792993387736,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_jfd2ry2lefam7hzlcbhkpkbtwu",
        LOG.catalog:   "metastore_id": "7a06fce6-1c66-06d0-7a06-fce61c6607b7",
        LOG.catalog:   "name": "dms_other_jfd2ry2lefam7hzlcbhkpkbtwu",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1792993387736,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }
    acceptance_test.go:1114: 
        LOG.plan_final: create schemas.foo
        LOG.plan_final: 
        LOG.plan_final: Plan: 1 to add, 0 to change, 0 to delete, 0 unchanged

latest (v1.22.1):

acceptance_test.go:1307: Diff:
        --- bundle/dms/failed-recreate/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-recreateDMS=true3687123484/002/output.txt
        @@ -66,9 +66,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -93,18 +91,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_failed_recreate_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "comment": "v1",
        -    "name": "dms_failed_recreate_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A recreate deletes and then fails to create. The delete went through, so the resource really is gone: the failed operation clears its state (self-contained, not assuming the delete's own write reached the service), rather than describing the resource as it was before the deploy
        @@ -189,7 +177,7 @@
          "lineage": "[UUID]",
          "serial": 2,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
          "state": {}
         }
        
    acceptance_test.go:1114: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1792993397879,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_km62w463lrholpdak74wy23xtu",
        LOG.catalog:   "metastore_id": "7a06fce8-78fa-67eb-7a06-fce878fa68dc",
        LOG.catalog:   "name": "dms_other_km62w463lrholpdak74wy23xtu",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1792993397879,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }
    acceptance_test.go:1114: 
        LOG.plan_final: create schemas.foo
        LOG.plan_final: 
        LOG.plan_final: Plan: 1 to add, 0 to change, 0 to delete, 0 unchanged
TestAccept/bundle/dms/failed-update-permissions/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/dms/failed-update-permissions/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-update-permissionsDMS=true4088213005/002/output.txt
        @@ -14,9 +14,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -46,50 +44,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.foo": {
        -   "__id__": "[FOO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-update-permissions-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "foo",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.foo.permissions": {
        -   "__id__": "/jobs/[FOO_ID]",
        -   "state": {
        -    "object_id": "/jobs/[FOO_ID]",
        -    "__embed__": [
        -     {
        -      "level": "CAN_VIEW",
        -      "user_name": "viewer@example.com"
        -     },
        -     {
        -      "level": "IS_OWNER",
        -      "user_name": "[USERNAME]"
        -     }
        -    ]
        -   },
        -   "depends_on": [
        -    {
        -     "node": "resources.jobs.foo",
        -     "label": "${resources.jobs.foo.id}"
        -    }
        -   ]
        -  }
        - }
        + "state": {}
         }
         
         === A permissions update that fails records the failure with the permissions it still has. The resource ID here is a path rather than a number, and the state is the embedded permission list, so this covers a shape the resource tests do not
        @@ -140,52 +96,11 @@
          "state_version": 3,
          "cli_version": "[CLI_VERSION]",
          "lineage": "[UUID]",
        - "serial": 1,
        + "serial": 2,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.foo": {
        -   "__id__": "[FOO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-update-permissions-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "foo",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.foo.permissions": {
        -   "__id__": "/jobs/[FOO_ID]",
        -   "state": {
        -    "object_id": "/jobs/[FOO_ID]",
        -    "__embed__": [
        -     {
        -      "level": "CAN_VIEW",
        -      "user_name": "viewer@example.com"
        -     },
        -     {
        -      "level": "IS_OWNER",
        -      "user_name": "[USERNAME]"
        -     }
        -    ]
        -   },
        -   "depends_on": [
        -    {
        -     "node": "resources.jobs.foo",
        -     "label": "${resources.jobs.foo.id}"
        -    }
        -   ]
        -  }
        - }
        + "state": {}
         }
         
         >>> print_requests.py --dms //api/2.0/bundle --oneline

latest (v1.22.1): same output as main (bef6321)

TestAccept/bundle/dms/multiple-resources/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/dms/multiple-resources/output.txt
        +++ /tmp/TestAcceptbundledmsmultiple-resourcesDMS=true2513130267/002/output.txt
        @@ -24,9 +24,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -71,100 +69,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.five": {
        -   "__id__": "[FIVE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "five",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.four": {
        -   "__id__": "[FOUR_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "four",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.one": {
        -   "__id__": "[ONE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "one",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.three": {
        -   "__id__": "[THREE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "three",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.two": {
        -   "__id__": "[TWO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "two",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === Redeploy with no changes: nothing is applied, so no version is created at all - the deployment's version tracks state changes, like serial
        @@ -182,9 +88,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -229,98 +133,7 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.five": {
        -   "__id__": "[FIVE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "five",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.four": {
        -   "__id__": "[FOUR_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "four",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.one": {
        -   "__id__": "[ONE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "one",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.jobs.three": {
        -   "__id__": "[THREE_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-multiple-resources-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
... (29 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)

latest (v1.22.1): same output as main (bef6321)

TestAccept/bundle/dms/no-drift/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/dms/no-drift/output.txt
        +++ /tmp/TestAcceptbundledmsno-driftDMS=true1724822341/002/output.txt
        @@ -34,9 +34,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -51,52 +49,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.foo": {
        -   "__id__": "[FOO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "foo",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.pipelines.bar": {
        -   "__id__": "[BAR_ID]",
        -   "state": {
        -    "catalog": "main",
        -    "channel": "CURRENT",
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edition": "ADVANCED",
        -    "libraries": [
        -     {
        -      "file": {
        -       "path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/files/transform.py"
        -      }
        -     }
        -    ],
        -    "name": "bar-[UNIQUE_NAME]",
        -    "schema": "default",
        -    "serverless": true
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === Plan without touching anything: the deployment stamp must not show as drift
        @@ -224,9 +178,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 2,
        -  "file_serial": 2,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -245,53 +197,9 @@
          "lineage": "[UUID]",
          "serial": 2,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.foo": {
        -   "__id__": "[FOO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "foo",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.pipelines.bar": {
        -   "__id__": "[BAR_ID]",
        -   "state": {
        -    "catalog": "main",
        -    "channel": "CURRENT",
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "2"
        -    },
        -    "development": true,
        -    "edition": "ADVANCED",
        -    "libraries": [
        -     {
        -      "file": {
        -       "path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/files/transform.py"
        -      }
        -     }
        -    ],
        -    "name": "bar-[UNIQUE_NAME]",
        -    "schema": "default",
        -    "serverless": true
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === Plan again: no residual drift, and the version stamps are ignored

latest (v1.22.1):

acceptance_test.go:1307: Diff:
        --- bundle/dms/no-drift/output.txt
        +++ /tmp/TestAcceptbundledmsno-driftDMS=true255838219/002/output.txt
        @@ -34,9 +34,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -51,52 +49,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.foo": {
        -   "__id__": "[FOO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "foo",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.pipelines.bar": {
        -   "__id__": "[BAR_ID]",
        -   "state": {
        -    "catalog": "main",
        -    "channel": "CURRENT",
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edition": "ADVANCED",
        -    "libraries": [
        -     {
        -      "file": {
        -       "path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/files/transform.py"
        -      }
        -     }
        -    ],
        -    "name": "bar-[UNIQUE_NAME]",
        -    "schema": "default",
        -    "serverless": true
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === Plan without touching anything: the deployment stamp must not show as drift
        @@ -224,9 +178,8 @@
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
           "serial": 2,
        -  "file_serial": 2,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -245,53 +197,9 @@
          "lineage": "[UUID]",
          "serial": 2,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.foo": {
        -   "__id__": "[FOO_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "foo",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  },
        -  "resources.pipelines.bar": {
        -   "__id__": "[BAR_ID]",
        -   "state": {
        -    "catalog": "main",
        -    "channel": "CURRENT",
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "2"
        -    },
        -    "development": true,
        -    "edition": "ADVANCED",
        -    "libraries": [
        -     {
        -      "file": {
        -       "path": "/Workspace/Users/[USERNAME]/.bundle/dms-no-drift-[UNIQUE_NAME]/default/files/transform.py"
        -      }
        -     }
        -    ],
        -    "name": "bar-[UNIQUE_NAME]",
        -    "schema": "default",
        -    "serverless": true
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === Plan again: no residual drift, and the version stamps are ignored
        
FAIL
TestAccept/bundle/dms/no-resources/DMS=true: branch ✅ | main (bef6321) ❌ | latest (v1.22.1) ❌

main (bef6321):

acceptance_test.go:1307: Diff:
        --- bundle/dms/no-resources/output.txt
        +++ /tmp/TestAcceptbundledmsno-resourcesDMS=true3296731253/002/output.txt
        @@ -35,10 +35,8 @@
           "plan_version": [PLAN_VERSION],
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
        -  "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -55,9 +52,8 @@
          "state_version": 3,
          "cli_version": "[CLI_VERSION]",
          "lineage": "[UUID]",
        - "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
          "state": {}
         }
        @@ -75,6 +70,10 @@
         }
         {
           "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
        +  "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
         {
        @@ -83,8 +82,16 @@
         }
         {
           "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
        +  "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         === The plan records the deployment_history feature
         >>> [CLI] bundle plan -o json
        @@ -92,10 +97,8 @@
           "plan_version": [PLAN_VERSION],
           "cli_version": "[CLI_VERSION]",
           "lineage": "[UUID]",
        -  "serial": 1,
        -  "file_serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -112,9 +114,8 @@
          "state_version": 3,
          "cli_version": "[CLI_VERSION]",
          "lineage": "[UUID]",
        - "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
          "state": {}
         }

latest (v1.22.1): same output as main (bef6321)

6 more failure details omitted, see the full report in the run.

@shreyas-goenka shreyas-goenka mentioned this pull request Oct 23, 2026
Draft
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 4835105

Run: 39973645165

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 306 23 4:31
✅​ aws-windows-latest-4core-16gb 308 21 8:46
✅​ azure linux-2core-8gb 305 23 4:29
✅​ azure-windows-latest-4core-16gb 307 21 10:10
✅​ gcp linux-2core-8gb 306 23 4:40
✅​ gcp-windows-latest-4core-16gb 308 21 8:49
Top 3 slowest tests (at least 2 minutes):
duration env testname
8:42 azure-windows-latest-4core-16gb TestAccept
7:08 aws-windows-latest-4core-16gb TestAccept
7:02 gcp-windows-latest-4core-16gb TestAccept

@shreyas-goenka shreyas-goenka changed the title Use enforced best-effort history for the YAML opt-in Use enforced best-effort deployment history for all opt-ins Oct 26, 2026
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 4017609 to 0b70621 Compare October 26, 2026 05:25
@shreyas-goenka shreyas-goenka changed the title Use enforced best-effort deployment history for all opt-ins Default deployment history to enforced file-backed recording Oct 26, 2026
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 0b70621 to d2f84b5 Compare October 26, 2026 05:35

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@shreyas-goenka @eng-dev-ecosystem-bot