GRPC and Http/2 errors

Dealing with Http2 exceptions can be confusing. Here's an example of a confusing error message that can manifest itself in code.

GRPC and Http/2 errors

GRPC can be opaque when you're dealing with errors. Using agents can bury the exceptions even deeper in your stack.

One issue we've seen is a runtime Exception such as:

org.apache.skywalking.apm.dependencies.io.grpc.StatusRuntimeException: INTERNAL: http2 exception

Caused by: org.apache.skywalking.apm.dependencies.io.netty.handler.codec.http2.Http2Exception: First received frame was not SETTINGS. Hex dump for first 5 bytes: 485454502f

A Github issue recommends ensuring the port is correctly assigned for HTTP2 traffic. While this does correspond to the Hex dump output, the actual error message isn't properly described which makes this difficult to diagnose.

We've found that often this is a case of missing certificates in the gRPC connection.

To fix this specific issue, please ensure the correct ca folders and certs are setup with your agents. For Apache Skywalking, they should be present as:

agent/ca/ca.crt

Subscribe to Antler - Applied Tracing

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe