Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark performance report published (and repeatable) #2577

Open
3 tasks
jsuereth opened this issue Jan 25, 2021 · 0 comments
Open
3 tasks

Benchmark performance report published (and repeatable) #2577

jsuereth opened this issue Jan 25, 2021 · 0 comments
Labels
Feature Request Suggest an idea for this project help wanted

Comments

@jsuereth
Copy link
Contributor

Is your feature request related to a problem? Please describe.
I'd like to very quickly be able to find performance characteristics of SDK components. The reasons are severalfold:

  • I'd like to understand if the Google-Cloud exporters are on-par with OTLP performance
  • I'd like to understand the CPU/Memory pressure caused via OTel's SDK and various mechanisms to alleviate it
  • I'd like to understand if we should be promoting different defaults for different envirornments, e.g. K8s vs. K-Native may be highly different in terms of what recommended setup we should promote.

Those are just top-of-mind. In general, it'd be nice if performance characteristics were as public as posisble (without requiring one to run the entire suite).

Describe the solution you'd like
I'd like to update the JMH hooks in the gradle build with some kind of task that will collect and generate markdown reports on that state of any given PR.

The goal would be to (one day) extend the performance monitoring-github action to autopublish performance dumps and over-time analysis of this SDK.

However, I see it as a multi-step process:

  • Create a task which pulls all JMH reports into one central report dump (can be multiple markdown files.
  • Enhance the task in a series of PRs and discussions until we're happy with its format.
  • Automate a GithubAction to generate performance reports when commits hit master (or every so often).
    Note: The Python SDK already does this, and it's pretty nice.

Describe alternatives you've considered
Keeping things as is. Didn't really consider many alternatives so happy to hear ideas.

Additional context
Python configuration: https://github.com/open-telemetry/opentelemetry-python/blob/master/.github/workflows/test.yml#L72
Python example: https://open-telemetry.github.io/opentelemetry-python/benchmarks/

@jsuereth jsuereth added the Feature Request Suggest an idea for this project label Jan 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature Request Suggest an idea for this project help wanted
Projects
None yet
Development

No branches or pull requests

2 participants