Repository navigation
feat(storage): add DirectPath over Cloud Interconnect channel telemetry and diagnostics - #14903
nidhiii-27 wants to merge 1 commit into
Conversation
…ry and diagnostics - Add GrpcChannelTelemetry with URI and host classification (CloudPath, DirectPath, DirectPathInterconnect), two-stage diagnostic logging (logChannelConfiguration and logChannelReady), non-owning async channel[0] readiness watcher via ChannelPrimer, and internal/client/channel_creation_latency OpenTelemetry histogram callback. - Register storage.googleapis.com/internal/client/channel_creation_latency histogram view and support DirectPath over Interconnect endpoints in OpenTelemetryBootstrappingUtils. - Fix GrpcStorageOptions builder precedence so setAttemptDirectPath(false) overrides DirectPath over Cloud Interconnect as intended, and wire channel telemetry and diagnostics. - Add unit tests in GrpcChannelTelemetryTest and OpenTelemetryBootstrappingUtilsTest. [Generated-by: AI]
There was a problem hiding this comment.
Code Review
This pull request introduces gRPC channel telemetry and diagnostics for Google Cloud Storage, adding the GrpcChannelTelemetry class to support and monitor CloudPath, DirectPath, and DirectPath over Cloud Interconnect. It integrates this telemetry into GrpcStorageOptions and enhances OpenTelemetryBootstrappingUtils to bootstrap gRPC metrics and resolve Cloud Monitoring endpoints. Feedback on the changes highlights critical robustness improvements, including catching Throwable during static class-probing to prevent initialization crashes, calling setAccessible(true) and catching Exception to safely reflect on package-private channel implementations, and adding a defensive null check for credentials to avoid potential null pointer exceptions.
| private static boolean detectGrpcForceXdsSupport() { | ||
| try { | ||
| Class<?> clazz = Class.forName("io.grpc.googleapis.GoogleCloudToProdNameResolver"); | ||
| clazz.getDeclaredField("forceXds"); | ||
| return true; | ||
| } catch (ReflectiveOperationException | SecurityException e) { | ||
| return false; | ||
| } | ||
| } |
There was a problem hiding this comment.
In environments where GoogleCloudToProdNameResolver is present but its dependencies or superclasses are missing, Class.forName can throw a LinkageError (such as NoClassDefFoundError). Since this method is called during static initialization of GrpcChannelTelemetry, any uncaught Error will result in an ExceptionInInitializerError, preventing the class from loading and crashing the application. Catching Throwable instead of ReflectiveOperationException | SecurityException ensures robust class-probing without risking initialization failures.
| private static boolean detectGrpcForceXdsSupport() { | |
| try { | |
| Class<?> clazz = Class.forName("io.grpc.googleapis.GoogleCloudToProdNameResolver"); | |
| clazz.getDeclaredField("forceXds"); | |
| return true; | |
| } catch (ReflectiveOperationException | SecurityException e) { | |
| return false; | |
| } | |
| } | |
| private static boolean detectGrpcForceXdsSupport() { | |
| try { | |
| Class<?> clazz = Class.forName("io.grpc.googleapis.GoogleCloudToProdNameResolver"); | |
| clazz.getDeclaredField("forceXds"); | |
| return true; | |
| } catch (Throwable e) { | |
| return false; | |
| } | |
| } |
| private static String defaultExtractLbPolicyName(@Nullable ManagedChannel channel) { | ||
| if (channel == null) { | ||
| return null; | ||
| } | ||
| try { | ||
| Method m = channel.getClass().getMethod("getLoadBalancingPolicyName"); | ||
| Object result = m.invoke(channel); | ||
| if (result instanceof String) { | ||
| return (String) result; | ||
| } | ||
| } catch (ReflectiveOperationException | SecurityException ignored) { | ||
| // Standard io.grpc.ManagedChannel does not expose getLoadBalancingPolicyName publicly. | ||
| } | ||
| return null; | ||
| } |
There was a problem hiding this comment.
If the runtime class of the ManagedChannel is package-private (such as io.grpc.internal.ManagedChannelImpl), calling Method.invoke on a Method retrieved via getClass().getMethod(...) will throw an IllegalAccessException because the class itself is not accessible, even if the method is public. To bypass this, m.setAccessible(true) should be called. Additionally, on JDK 9+, calling setAccessible(true) on a module-protected class can throw an InaccessibleObjectException (which is a RuntimeException). Since the catch block only catches ReflectiveOperationException and SecurityException, this runtime exception would propagate and crash the application. Catching Exception and calling setAccessible(true) resolves both issues safely.
| private static String defaultExtractLbPolicyName(@Nullable ManagedChannel channel) { | |
| if (channel == null) { | |
| return null; | |
| } | |
| try { | |
| Method m = channel.getClass().getMethod("getLoadBalancingPolicyName"); | |
| Object result = m.invoke(channel); | |
| if (result instanceof String) { | |
| return (String) result; | |
| } | |
| } catch (ReflectiveOperationException | SecurityException ignored) { | |
| // Standard io.grpc.ManagedChannel does not expose getLoadBalancingPolicyName publicly. | |
| } | |
| return null; | |
| } | |
| private static String defaultExtractLbPolicyName(@Nullable ManagedChannel channel) { | |
| if (channel == null) { | |
| return null; | |
| } | |
| try { | |
| Method m = channel.getClass().getMethod("getLoadBalancingPolicyName"); | |
| m.setAccessible(true); | |
| Object result = m.invoke(channel); | |
| if (result instanceof String) { | |
| return (String) result; | |
| } | |
| } catch (Exception ignored) { | |
| // Standard io.grpc.ManagedChannel does not expose getLoadBalancingPolicyName publicly. | |
| } | |
| return null; | |
| } |
References
- Catching Throwable instead of Exception is acceptable at Future boundaries or during stream cleanup in asynchronous operations to ensure all failures are captured and propagated. For general reflection or runtime exception handling, prefer catching Exception to avoid catching critical JVM Errors.
| InstantiatingGrpcChannelProvider telemetryProvider = | ||
| credentials instanceof NoCredentials | ||
| ? baseProvider | ||
| : (InstantiatingGrpcChannelProvider) baseProvider.withCredentials(credentials); |
There was a problem hiding this comment.
If credentials is null, credentials instanceof NoCredentials evaluates to false, which will lead to calling baseProvider.withCredentials(null). Depending on the GAX provider implementation, passing null to withCredentials might throw a NullPointerException or cause downstream initialization issues. Adding an explicit null check for credentials ensures defensive safety.
| InstantiatingGrpcChannelProvider telemetryProvider = | |
| credentials instanceof NoCredentials | |
| ? baseProvider | |
| : (InstantiatingGrpcChannelProvider) baseProvider.withCredentials(credentials); | |
| InstantiatingGrpcChannelProvider telemetryProvider = | |
| (credentials == null || credentials instanceof NoCredentials) | |
| ? baseProvider | |
| : (InstantiatingGrpcChannelProvider) baseProvider.withCredentials(credentials); |
Summary
Implements Phase 2 (Telemetry, Diagnostics, and Parity Fixes) of DirectPath over Cloud Interconnect (DP over GCI) for
google-cloud-storage, achieving cross-SDK parity with C++ PRs googleapis/google-cloud-cpp#16478, googleapis/google-cloud-cpp#16538, googleapis/google-cloud-cpp#16556, googleapis/google-cloud-cpp#16568, and googleapis/google-cloud-cpp#16605.Key Changes
GrpcChannelTelemetry:hasQueryParameter,detectTransportType,c2pTargetHost,authorityHost,isDirectPathInterconnectHost,selectedTransportType,grpcForceXdsIsSupported,isDirectPathInterconnectEnabled,resolveEffectiveTarget) forCloudPath,DirectPath, andDirectPathInterconnect.logChannelConfigurationandlogChannelReady/channelReadyNote) reporting actionableWARNINGandINFOmessages for DirectPath over Cloud Interconnect configuration, runtime gRPCforce-xdssupport, xDS vs DNS LB fallback, and channel readiness latency.channel[0]readiness watcher (startChannelTelemetry/createChannelPrimer) usingWeakReference<ManagedChannel>andnotifyWhenStateChanged.makeChannelMetricsCallback) recordinginternal/client/channel_creation_latency(in seconds) withtransport_typeattribute (CloudPath,DirectPath,DirectPathInterconnect).OpenTelemetryBootstrappingUtils:storage.googleapis.com/internal/client/channel_creation_latencyhistogram view increateMeterProvider.bootstrapGrpcMetricsreturning both theChannelConfiguratorandMeterProvider.getCloudMonitoringEndpointto handlegoogle-c2p:////google-c2p-experimental:///targets,storage-direct.googleapis.com,storage-preprod-direct.sandbox.googleapis.com, andstorage-direct.<universeDomain>.GrpcStorageOptions:setAttemptDirectPath(false)disables DirectPath even whensetAttemptDirectPathXdsOverInterconnect(true)orGOOGLE_CLOUD_ENABLE_DIRECT_PATH_XDS_OVER_INTERCONNECT=trueis set.GrpcChannelTelemetryconfiguration logging,ChannelPrimer, and OpenTelemetry channel metrics callback intoresolveSettingsAndOpts.GrpcChannelTelemetryTestandOpenTelemetryBootstrappingUtilsTestcovering URI/host helpers, diagnostic logs and warnings, async state transitions, OpenTelemetry histogram recording, and builder precedence.(Scoped to
java-storage/google-cloud-storage/to avoid any merge conflict with #14864.)