Using JDK 21 with virtual threads leads to missing cputimes #85
Comments
|
@pkutschera Thanks for the report. Can you share your test application to reproduce the problem? Also, on which platform (Windows, MacOS, Linux) did you run the test? |
|
Actually, it is quite easy. Just use a spring boot application with Java 21. In the application.yml you haveto enable virtual threads spring.threads.virtual.enabled=true I used MacOS |
|
@pkutschera This is a hard one - I have proposed a fix in this PR: Virtual Thread CPU Measurements In order to do this, I have extended the example applications in this repository to include one for the lowest JDK version (8) that we support and one for the latest LTS (21). The JDK 21 test now includes a test for virtual threads. As you can see in the PR description, the fix builds around the assumption that if a span on a virtual thread starts and ends on the same platform thread, we can allocate the whole Thread CPU time of the platform thread to the virtual thread. This might not always hold true but the alternative is to instrument the virtual thread scheduler, which I think is a lot more overhead for now. However, I will think about this a bit more, as this fix only works on Linux and Windows right but not not on MacOS. Furthermore, I think for memory allocations we need a similar extension, so I will keep the PR a bit open until these issues are resolved, but if you like, you can take a look already. I would also be looking for a contribution to do the same native measurements on MacOS as I have no test device handy. |
|
I support you to test the PR on MacOS. |
|
@pkutschera Thanks for your offer! I've updated the PR with support for native MacOS thread cpu time measurements, can you please test it with your example? Once you've checked out the PR, you can use the newly added test as a first step to see if everything works as expected: io.retit.opentelemetry.javaagent.extension.resources.macos.MacOSDataCollectorTest After that, you can verify the whole project using ./mvnw clean verify and use the extension/target/io.retit.opentelemetry.javaagent.extension.jar for your test application. Once you confirm that this works as expected, I will merge the PR. |
|
@brunnert Thank you for the quick fix, everything works fine with the new version of the extension. logger=infra.usagestats t=2025-01-31T15:03:14.602941881Z level=info msg="Usage stats are ready to report" |
* Move Examples to own Directory * Extend Tests for JDK8 and JDK21 * Add Test for Virtual Threads * #85 Add JDK21 Virtual Thread Handling * Fix PMD issues * Fix checkstyle issues * Publish native thread ids * Add notes about thread handling * Relax condition * Adding Mac Support * Fix Mac Support * Fix pom.xml * Fix PMD * Spotbugs Adjustments to Handle Reference-based C Code * Reduce Dependencies * Documentation Cleanup --------- Co-authored-by: brunnert <brunnert@hm.edu>
|
@pkutschera Thanks for your confirmation, I've merged the PR and created a new version including the fix: v0.0.13-alpha |
Virtual threads enabled
[otel.javaagent 2025-01-27 18:11:17:116 +0000] [tomcat-handler-0] INFO io.opentelemetry.exporter.logging.LoggingSpanExporter - 'GET /players/{id}' : 0b66ac03329c0f2f82c79475a95703c0 06e99885e2ae8dac SERVER [tracer: io.opentelemetry.tomcat-10.0:2.8.0-alpha] AttributesMap{data={http.response.status_code=200, thread.id=51, url.scheme=http, server.port=8088, thread.name=tomcat-handler-0, user_agent.original=IntelliJ HTTP Client/IntelliJ IDEA 2024.3.2.1, io.retit.endheapbyteallocation=-1, network.protocol.version=1.1, io.retit.endthread=51, io.retit.endcputime=-1, io.retit.startcputime=-1, network.peer.port=61234, server.address=localhost, client.address=172.19.0.1, io.retit.logsystemtime=1738001477116, network.peer.address=172.19.0.1, url.path=/players/1, http.request.method=GET, io.retit.startheapbyteallocation=-1, io.retit.startthread=51, http.route=/players/{id}}, capacity=128, totalAddedValues=21}
spring-rest-clean-football-modulith | [otel.javaagent 2025-01-27 18:11:21:830 +0000] [supportability_metrics_reporter] DEBUG io.opentelemetry.javaagent.shaded.instrumentation.api.internal.SupportabilityMetrics - Counter 'SqlStatementSanitizer cache miss' : 1
Virtual threads disabled
[otel.javaagent 2025-01-27 18:15:15:822 +0000] [http-nio-8088-exec-1] INFO io.opentelemetry.exporter.logging.LoggingSpanExporter - 'GET /players/{id}' : 02f8483bbd2da54b52fee72bd046151c 2599947fe41d1968 SERVER [tracer: io.opentelemetry.tomcat-10.0:2.8.0-alpha] AttributesMap{data={http.response.status_code=200, thread.id=47, url.scheme=http, server.port=8088, thread.name=http-nio-8088-exec-1, user_agent.original=IntelliJ HTTP Client/IntelliJ IDEA 2024.3.2.1, io.retit.endheapbyteallocation=34792440, network.protocol.version=1.1, io.retit.endthread=47, io.retit.endcputime=178593078, io.retit.startcputime=25276664, network.peer.port=59046, server.address=localhost, client.address=172.19.0.1, io.retit.logsystemtime=1738001715822, network.peer.address=172.19.0.1, url.path=/players/1, http.request.method=GET, io.retit.startheapbyteallocation=4045168, io.retit.startthread=47, http.route=/players/{id}}, capacity=128, totalAddedValues=21}
spring-rest-clean-football-modulith | [otel.javaagent 2025-01-27 18:15:20:206 +0000] [supportability_metrics_reporter] DEBUG io.opentelemetry.javaagent.shaded.instrumentation.api.internal.SupportabilityMetrics - Counter 'SqlStatementSanitizer cache miss' : 1
The text was updated successfully, but these errors were encountered: