GRPC can be opaque when you're dealing with errors. Using agents can bury the exceptions even deeper in your stack.
One issue we've seen is a runtime Exception such as:
org.apache.skywalking.apm.dependencies.io.grpc.StatusRuntimeException: INTERNAL: http2 exception
Caused by: org.apache.skywalking.apm.dependencies.io.netty.handler.codec.http2.Http2Exception: First received frame was not SETTINGS. Hex dump for first 5 bytes: 485454502f
A Github issue recommends ensuring the port is correctly assigned for HTTP2 traffic. While this does correspond to the Hex dump output, the actual error message isn't properly described which makes this difficult to diagnose.
We've found that often this is a case of missing certificates in the gRPC connection.
To fix this specific issue, please ensure the correct ca folders and certs are setup with your agents. For Apache Skywalking, they should be present as: