ONTAP: cache templates on primary storage and clone volumes from them by rajiv-jain-netapp · Pull Request #14243 · apache/cloudstack

rajiv-jain-netapp · 2026-09-24T15:15:24Z

Enable the managed-storage template cache flow for the ONTAP primary storage plugin so that VM root volumes are cloned from a template copy kept on the pool's FlexVolume instead of being copied from secondary storage for every deployment.

Advertise CAN_CREATE_VOLUME_FROM_VOLUME from the datastore driver.
Create the template cache on primary: an iSCSI LUN (identity tracked on template_spool_ref) or a qcow2 file for NFS, populated by the KVM host.
Clone new volumes from the cached template: LUN clone for iSCSI and ONTAP file clone (FileCloneRequest) for NFS, growing the clone when the disk offering is larger than the template.
Clean up the cached LUN/file when the template is removed from the pool; a missing file is treated as already deleted.
Move protocol-specific create/clone/resize logic into UnifiedSANStrategy and UnifiedNASStrategy, and add a StorageStrategy helper to poll an optional async job.
Add unit tests for the driver and the NAS/SAN strategies.

Description

This PR...

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Junit tests are incorporated.
Additional we tested by running the feature for
1. create VM for the first time for a given template and storage-pool, workflow created a template and VM subsequently
2. On create of another VM in sequence with the same template and storage-pool, we are finding VM instance created is pretty fast since it used local template cache.
3. We tested the workflows for both iSCSI and NFS3 protocol.

Enable the managed-storage template cache flow for the ONTAP primary storage plugin so that VM root volumes are cloned from a template copy kept on the pool's FlexVolume instead of being copied from secondary storage for every deployment. - Advertise CAN_CREATE_VOLUME_FROM_VOLUME from the datastore driver. - Create the template cache on primary: an iSCSI LUN (identity tracked on template_spool_ref) or a qcow2 file for NFS, populated by the KVM host. - Clone new volumes from the cached template: LUN clone for iSCSI and ONTAP file clone (FileCloneRequest) for NFS, growing the clone when the disk offering is larger than the template. - Clean up the cached LUN/file when the template is removed from the pool; a missing file is treated as already deleted. - Move protocol-specific create/clone/resize logic into UnifiedSANStrategy and UnifiedNASStrategy, and add a StorageStrategy helper to poll an optional async job. - Add unit tests for the driver and the NAS/SAN strategies.

rajiv-jain-netapp · 2026-09-25T06:15:32Z

@blueorangutan package

rajiv-jain-netapp · 2026-09-25T06:15:58Z

@DaanHoogland @nvazquez @weizhouapache FYI

rajiv-jain-netapp · 2026-10-01T10:14:57Z

@DaanHoogland @nvazquez this PR needs to be tagged for 4.24, can you do the needful?

codecov · 2026-10-01T15:45:45Z

Codecov Report

❌ Patch coverage is 79.74068% with 125 lines in your changes missing coverage. Please review.
✅ Project coverage is 20.53%. Comparing base (0a5bf30) to head (db56563).
⚠️ Report is 49 commits behind head on main.

Files with missing lines	Patch %	Lines
...ck/storage/driver/OntapPrimaryDatastoreDriver.java	83.07%	27 Missing and 28 partials ⚠️
...oudstack/storage/feign/model/FileCloneRequest.java	45.28%	28 Missing and 1 partial ⚠️
...cloudstack/storage/service/UnifiedNASStrategy.java	78.78%	13 Missing and 8 partials ⚠️
...cloudstack/storage/service/UnifiedSANStrategy.java	85.93%	8 Missing and 10 partials ⚠️
...he/cloudstack/storage/service/StorageStrategy.java	83.33%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #14243      +/-   ##
============================================
+ Coverage     20.05%   20.53%   +0.48%     
- Complexity    20656    21535     +879     
============================================
  Files          6427     6488      +61     
  Lines        584905   589871    +4966     
  Branches      71658    72296     +638     
============================================
+ Hits         117278   121159    +3881     
- Misses       454802   455537     +735     
- Partials      12825    13175     +350

Flag	Coverage Δ
uitests	`3.70% <ø> (-0.02%)`	⬇️
unittests	`21.84% <79.74%> (+0.52%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

4 open findings

pollJobIfPresent ignores the boolean return value of jobPollForSuccess, so async job failures… · New
templatePoolRef.getTemplateSize() can be null (or 0) depending on how the spool ref row was… · New
This hard cast to UnifiedNASStrategy can trigger a ClassCastException if… · New
updateCloudStackVolumeMetadata(...) is called without verifying… · New

What changed in this PR

Enables ONTAP primary-storage template caching so VM root disks are created via array-side clones (LUN clone for iSCSI, file clone for NFS) instead of re-copying from secondary storage each deployment.

Changes:

Adds template-cache create/clone/resize flows to the ONTAP primary datastore driver and SAN/NAS strategies.
Introduces ONTAP NFS file-clone request model and new Feign API(s), plus SAN LUN clone/resize behavior.
Expands unit tests to cover template caching, cloning, and resize error handling.

File	Description
plugins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/driver/OntapPrimaryDatastoreDriver.java	Implements template-cache create/delete and volume clone-from-template logic; adds pool capability.
plugins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/service/UnifiedSANStrategy.java	Adds SAN template-cache LUN creation, LUN clone, and LUN resize implementation.
plugins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/service/UnifiedNASStrategy.java	Adds NAS file clone, host-side qcow2 resize, and “delete file by path” helper.
plugins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/service/StorageStrategy.java	Extends strategy API with template-cache create, clone, resize, and job polling helper.
plugins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/feign/client/NASFeignClient.java	Adds ONTAP file clone endpoint.
plugins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/feign/client/SANFeignClient.java	Updates LUN PATCH endpoint and headers for resizing.
plugins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/feign/model/FileCloneRequest.java	New request model for ONTAP file clone API.
plugins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/utils/OntapStorageUtils.java	Removes protocol-specific volume request builder (now handled in driver).
plugins/storage/volume/ontap/src/main/java/org/apache/cloudstack/storage/utils/OntapStorageConstants.java	Adds template LUN prefix constant and clone-of-template volume detail key.
plugins/storage/volume/ontap/src/test/java/org/apache/cloudstack/storage/service/UnifiedSANStrategyTest.java	Adds tests for template cache, clone, resize, and incomplete LUN responses.
plugins/storage/volume/ontap/src/test/java/org/apache/cloudstack/storage/service/UnifiedNASStrategyTest.java	Adds tests for file clone, host-side resize, and 404-on-delete behavior.
plugins/storage/volume/ontap/src/test/java/org/apache/cloudstack/storage/driver/OntapPrimaryDatastoreDriverTest.java	Adds tests covering template cache lifecycle and clone/resize integration paths.
plugins/storage/volume/ontap/src/test/java/org/apache/cloudstack/storage/service/StorageStrategyTest.java	Updates test stub for new strategy interface methods.

🧠 Review effort: Lite

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

DaanHoogland

clgtm; nothing strange to se on a quick review. The code is contained in the ontap plugin completely so I think we must trust the maintainers on this one.

Copilot

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

5 open issues

Long.valueOf(detail.getValue()) may throw NumberFormatException if the persisted value is… · New
The Javadoc states “NAS returns an empty volume” / “return … an empty CloudStackVolume”, but… · New
Long.parseLong(cloudstackVolume.getDatastoreId()) can throw NumberFormatException, which… · New
Map.of(...) throws a NullPointerException if svmName or lunName is null, which can prevent… · New
updateCloudStackVolumeMetadata(...) is called without verifying…

3 resolved since last review

_{Review effort: Lite}

DaanHoogland · 2026-10-21T15:19:14Z

@nvazquez @RosiKyu , please pass judgement ;)

@rajiv-jain-netapp can you address the co-pilot remarks so far?

nvazquez · 2026-10-21T16:37:48Z

Thanks @DaanHoogland - @rajiv-jain-netapp can confirm but in my understanding this PR needs #13897 and #14192 first. We can proceed with this one after these 2 are merged

rajiv-jain-netapp · 2026-10-22T03:50:12Z

Thanks @DaanHoogland - @rajiv-jain-netapp can confirm but in my understanding this PR needs #13897 and #14192 first. We can proceed with this one after these 2 are merged

@nvazquez yes, Nicolas, I would recommend following the sequence on the PRs as I sent in mail, would help in staying away from integration issues

rajiv-jain-netapp · 2026-10-22T06:55:08Z

@nvazquez @RosiKyu , please pass judgement ;)

@rajiv-jain-netapp can you address the co-pilot remarks so far?

Hi @DaanHoogland thanks for the approval. @nvazquez taken care of the co-pilot comments

boring-cyborg Bot added component:primary-storage component:storage labels Sep 24, 2026

rajiv-jain-netapp force-pushed the ontap-template-cache branch from c4080ea to 9ab7ec5 Compare September 24, 2026 15:16

DaanHoogland added this to the 24.0 milestone Oct 1, 2026

nvazquez requested a balanced review from Copilot October 8, 2026 13:22

Copilot AI reviewed Oct 8, 2026

View reviewed changes

Copilot stopped reviewing on behalf of nvazquez due to an error October 8, 2026 13:53
An unexpected error occurred. For more details, see the detailed logs in GitHub Actions.

Resolving the co-pilot comments

9b7353d

DaanHoogland approved these changes Oct 20, 2026

View reviewed changes

DaanHoogland requested a balanced review from Copilot October 20, 2026 13:37

Copilot AI reviewed Oct 20, 2026

View reviewed changes

Copilot stopped reviewing on behalf of DaanHoogland due to an error October 20, 2026 17:57
The job was not acquired by a runner even after multiple attempts.

Resolving another co-pilot comments

db56563

Conversation

rajiv-jain-netapp commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

rajiv-jain-netapp commented Sep 25, 2026

Uh oh!

rajiv-jain-netapp commented Sep 25, 2026

Uh oh!

rajiv-jain-netapp commented Oct 1, 2026

Uh oh!

codecov Bot commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DaanHoogland commented Oct 21, 2026

Uh oh!

nvazquez commented Oct 21, 2026

Uh oh!

rajiv-jain-netapp commented Oct 22, 2026

Uh oh!

rajiv-jain-netapp commented Oct 22, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

rajiv-jain-netapp commented Sep 24, 2026 •

edited

Loading

codecov Bot commented Oct 1, 2026 •

edited

Loading