Release v1.29.0

Backend Changes

Breaking Changes

• Remove deprecated --badger.truncate CLI flag (@yurishkuro in #3410)

New Features

• Expose rackID option in ingester (@shyimo in #3395)

Bug fixes, Minor Improvements

• Fix debug image builds by installing build-base to enable GCC (@yurishkuro in #3400)
• Limit URL size in Elasticsearch index delete request (@jkandasa in #3375)

UI Changes

• UI pinned to version 1.19.0. The changelog is available here v1.19.0

Release v1.28.0

Backend Changes

• Add auth token propagation for metrics reader (@albertteoh in #3341)

New Features

• Add in-memory storage support for adaptive sampling (@lonewolf3739 in #3335)

Bug fixes, Minor Improvements

• Do not throw error on empty indices in Elasticsach rollover lookback (@jkandasa in #3369)
• Treat input throughput data as immutable (@rbroggi in #3360)
• Remove dependencies on unused tools, install tools explicitly instead of via go.mod (@rbroggi in #3355)
• Update mockery to version 2 and adapt to install-tools approach (@rbroggi in #3358)
• Control lightweight storage integration tests via build tags (@rbroggi in #3346)
• Remove package integration from coverage reports (@rbroggi in #3357)
• Remove outdated reference to cover.sh (@rbroggi in #3348)
• Update monitoring mixin (@jpkrohling in #3331)
• Update Jaeger chart link (@isbang in #3328)
• Fix args order in strings.Contains in es-rollover (@pavolloffay in #3324)
• Use (TB).TempDir instead of non-portable /mnt/* in Badger (@pavolloffay in #3325)
• Fix peer.service retrieval from Zipkin's MESSAGE_ADDR annotation (@Git-Jiro in #3312)

UI Changes

• UI pinned to version 1.18.0. The changelog is available here v1.18.0

Release v1.27.0

Backend Changes

New Features

• Add support for adaptive sampling with a Cassandra backend. (#2966, @joe-elliott, @Ashmita152)

Bug fixes, Minor Improvements

• Migrate elasticsearch rollover to go (#3242, @rubenvp8510)
• Add 'opensearch' as a supported value for SPAN_STORAGE_TYPE (#3255, @yurishkuro)
• Support graceful shutdown in grpc plugin (#3249, @slon)
• Enable gzip compression for collector grpc endpoint. (#3236, @slon)
• Use UTC in es-index-cleaner (#3261, @pavolloffay)
• Upgrade to alpine-3.14 (#3304, @nontw)
• refactor: move from io/ioutil to io and os package (#3294, @Juneezee)
• Changed sampling type env var and updated collector help text (#3302, @joe-elliott)
• Close #3270: Prevent rollover lookback from passing the Unix epoch (#3299, @ctreatma)
• Fixing otel configuration in docker compose (#3286, @Ashmita152)
• Added ability to pass config file to grpc plugin in integration tests (#3253, @EinKrebs)

UI Changes

• UI pinned to version 1.17.0. The changelog is available here v1.17.0

Release v1.26.0

Backend Changes

New Features

• Add cassandra v4 support (@Ashmita152 in #3225)
• Add es-index-cleaner golang implementation (@pavolloffay in #3204)
• Add CLI Option for gRPC Max Receive Message Size (@js8080 in #3214 and #3192)
• Automatically detect OpenSearch version (@pavolloffay in #3198)
• Add SendGetBodyAs on elasticsearch (@NatMarchand in #3193)
• Set lookback in ES rollover to distant past (@pavolloffay in #3169)

Bug fixes, Minor Improvements

• Check for invalid --agent.tags (@esnible in #3246)
• Replace old linters with golangci-lint (@albertteoh in #3237)
• Fix panic on empty findTraces query (@akuzni2 in #3232)
• Upgrade to Go v1.17 (@Ashmita152 in #3220)
• Add docker buildx make target (@pavolloffay in #3213)
• Fix the name for elasticsearch integration tests (@Ashmita152 in #3208)
• Upgrade ES images in integration tests (@pavolloffay in #3185)

UI Changes

• UI pinned to version 1.16.0 - https://github.com/jaegertracing/jaeger-ui/releases/tag/v1.16.0

Release v1.25.0

New Features

• Add query service with OTLP (#3086, @pavolloffay)
• Add ppc64le support on multiarch docker images (#3160, @krishvoor)

Bug fixes, Minor Improvements

• Fix base path in grpc gateway for api_v3 (#3139, @pavolloffay)
• Add /api prefix for /v3 API (#3178, @pavolloffay)
• Define http.Server.ErrorLog to forward logs to Zap (#3157, @yurishkuro)
• Add ATM dev environment docker-compose and API doc (#3171, @albertteoh)
• Log the source of sampling strategies (#3166, @yurishkuro)
• Pin elasticsearch-py to older version without elastic.co product check (#3180, @pavolloffay)

Release 1.24.0

Backend Changes

Breaking Changes

• Upgrade Badger from v1.6.2 to v3.2103.0 (#3096, @Ashmita152):
  • Deprecated --badger.truncate flag.
  • All badger related expvar prefix has changed from badger to badger_v3.

New Features

• Add docker images for linux/arm64 (#3124, @GaruGaru)
• Add s390x support on multiarch docker images (#2948, @kun-lu20)
• Add TLS support for Prometheus reader (#3096, @albertteoh)

Monitor tab for service health metrics

• Add HTTP handler for metrics querying #3095, @albertteoh)
• Add MetricsQueryService grcp handler #3091, @albertteoh)
• Hook up MetricsQueryService to main funcs (#3079, @albertteoh)
• Add metrics query capability to query service (#3061, @albertteoh)

Bug fixes, Minor Improvements

• Add build info metrics to Jaeger components (#3087, @Ashmita152)
• Upgrade gRPC to 1.38.x (#3096, @pavolloffay)

Release 1.23.0

Backend Changes

Breaking Changes

• Remove unused --es-archive.max-span-age flag (#2865, @albertteoh):
  • max-span-age is only relevant when searching for unarchived traces. Archived traces are searched with no look-back limit.

New Features

• Inject trace context to grpc metadata (#2870, @lujiajing1126)
• Passing default sampling strategies file as environment variable (#3027, @Ashmita152)
• [es] Add index rollover mode that can choose day and hour (#2965, @WalkerWang731)
• Add a TIMEOUT environment variable for es rollover (#2938, @ediezh)
• Allow the ILM policy name to be configurable (#2971, @jrRibeiro)
• [es] Add remote read clusters option for cross-cluster querying (#2874, @dgrizzanti)
• Enable logging in ES client (#2862, @albertteoh)

Bug fixes, Minor Improvements

• Fix jaeger-agent reproducible memory leak (#3050, @jpkrohling)
• Changed Range Query to use startTimeMillis date field instead of startTime field (#2980, @Sreevani871)
• Verify FindTraces() received a query (#2979, @esnible)
• Set Content-Type in healthcheck's http response (#2926, @logeable)
• Add jaeger-query HTTP handler diagnostic logging (#2906, @albertteoh)
• Fix es-archive namespace default values (#2865, @albertteoh)

UI Changes

Enhancements

• Make search panel more compact, button more visible (@meenal06 in #724)

Fixes

• Add null check for span.logs in search/filter-spans (@achesin in #734)
• Several updates to dependencies.

Release 1.22.0

Backend Changes

Breaking Changes

• Remove deprecated TLS flags (#2790, @albertteoh):

  • --cassandra.tls is replaced by --cassandra.tls.enabled
  • --cassandra-archive.tls is replaced by --cassandra-archive.tls.enabled
  • --collector.grpc.tls is replaced by --collector.grpc.tls.enabled
  • --collector.grpc.tls.client.ca is replaced by --collector.grpc.tls.client-ca
  • --es.tls is replaced by --es.tls.enabled
  • --es-archive.tls is replaced by --es-archive.tls.enabled
  • --kafka.consumer.tls is replaced by --kafka.consumer.tls.enabled
  • --kafka.producer.tls is replaced by --kafka.producer.tls.enabled
  • --reporter.grpc.tls is replaced by --reporter.grpc.tls.enabled

• Remove deprecated flags of Query Server --query.port and --query.host-port, please use dedicated HTTP --query.http-server.host-port (defaults to :16686) and gRPC --query.grpc-server.host-port (defaults to :16685) host-ports flags instead (#2772, @rjs211)

  • By default, if no flags are set, the query server starts on the dedicated ports. To use common port for gRPC and HTTP endpoints, the host-port flags have to be explicitly set

• Remove deprecated CLI flags (#2751, @LostLaser):

  • --collector.http-port is replaced by --collector.http-server.host-port
  • --collector.grpc-port is replaced by --collector.grpc-server.host-port
  • --collector.zipkin.http-port is replaced by --collector.zipkin.host-port

• Remove deprecated flags --health-check-http-port & --admin-http-port, please use --admin.http.host-port (#2752, @pradeepnnv)

• Remove deprecated flag --es.max-num-spans, please use --es.max-doc-count (#2482, @BernardTolosajr)

• Remove deprecated flag --jaeger.tags, please use --agent.tags (#2753, @yurishkuro)

• Remove deprecated Cassandra flags (#2789, @albertteoh):

  • --cassandra.enable-dependencies-v2 - Jaeger will automatically detect the version of the dependencies table
  • --cassandra.tls.verify-host - please use --cassandra.tls.skip-host-verify instead

• Remove incorrectly scoped downsample flags from the query service (#2782, @joe-elliott)

  • --downsampling.hashsalt removed from jaeger-query
  • --downsampling.ratio removed from jaeger-query

New Features

• Add TLS Support for gRPC and HTTP endpoints of the Query and Collector servers (#2337, #2772, #2798, @rjs211)

  • If TLS in enabled on either or both of gRPC or HTTP endpoints, the gRPC host-port and the HTTP host-port have to be different
  • If TLS is disabled on both endpoints, common HTTP and gRPC host-port can be explicitly set using the following host-port flags respectively:
    • Query: --query.http-server.host-port and --query.grpc-server.host-port
    • Collector: --collector.http-server.host-port and --collector.grpc-server.host-port

• Add support for Kafka SASL/PLAIN authentication via SCRAM-SHA-256 or SCRAM-SHA-512 mechanism (#2724, @WalkerWang731)

• [agent] Add metrics to show connections status between agent and collectors (#2657, @WalkerWang731)

• Add plaintext as a supported kafka auth option (#2721, @pdepaepe)

• Add ability to use JS file for UI configuration (#123 from jaeger-ui) (#2707, @th3M1ke)

• Support Elasticsearch ILM for managing jaeger indices (#2796, @bhiravabhatla)

• Push official images to quay.io, in addition to Docker Hub (#2783, @Ashmita152)

• Add status command (#2684, @sniperking1234)

  • Usage:

    $ ./cmd/collector/collector-darwin-amd64 status
    {"status":"Server available","upSince":"2021-02-19T17:57:12.671902+11:00","uptime":"25.241233383s"}

• Support configurable date separator for Elasticsearch index names (#2637, @sniperking1234)

Bug fixes, Minor Improvements

• Use workaround for windows x509.SystemCertPool() (#2756, @Ashmita152)
• Guard against mal-formed payloads received by the agent, potentially causing high memory utilization (#2780, @jpkrohling)
• Expose cache TTL for ES span writer index+service (#2737, @necrolyte2)
• Copy spans from memory store (#2720, @bobrik)
• [pkg/queue] Add StartConsumersWithFactory function (#2714, @mx-psi)
• Fix potential cross-site scripting issue (#2697, @yurishkuro)
• Updated gRPC Storage Plugin README with example (#2687, @js8080)
• Deduplicate collector tags (#2658, @Betula-L)
• Add latency metrics on collector HTTP endpoints (#2664, @dimitarvdimitrov)
• Fix collector panic due to sarama sdk (#2654, @Betula-L)
• Handle collector Start error (#2647, @albertteoh)
• [anonymizer] Save trace in UI format (#2629, @yurishkuro)

UI Changes

• UI pinned to version 1.13.0. The changelog is available here v1.13.0

Release 1.21.0

Backend Changes

New Features

• New debug Docker images for Jaeger backends are included in the distribution (#2545, @Ashmita152)
• A utility to anonymize a trace for sharing publicly (#2621, #2585, @Ashmita152)
• Sampling strategies file can be loaded from a URL, not just a file path (#2519, @goku321)
• Configuration parameters can be inspected at runtime via /debug/vars admin endpoint (#2496, @dstdfx)
• OTLP-proto encoding for Kafka supported in the OTEL Ingester (#2580, @XSAM)
• Display backend & UI versions in Jaeger UI
  • Inject version info into index.html (#2547, @yurishkuro)
  • Added jaeger ui version to about menu (#606, @alanisaac)

Bug fixes, Minor Improvements

• Update x/text to v0.3.4 (#2625, @objectiser)
• Update CodeQL to latest best practices (#2615, @jhutchings1)
• Bump opentelemetry-collector to v0.14.0 (#2617, @Vemmy124)
• Bump Badger to v1.6.2 (#2613, @Ackar)
• Fix sarama consumer deadlock (#2587, @albertteoh)
• Avoid deadlock if Stop is called before Serve (#2608, @chlunde)
• Return buffers to pool on network errors or queue overflow (#2609, @chlunde)
• Clarify deadlock panic message (#2605, @yurishkuro)
• fix: don't create tags w/ empty name for internal zipkin spans (#2596, @mzahor)
• TBufferedServer: Avoid channel close/send race on Stop (#2583, @chlunde)
• Bumped OpenTelemetry Collector to v0.12.0 (#2562, @jpkrohling)
• Disable Zipkin server if port/address is not configured (#2554, @yurishkuro)
• [hotrod] Add links to traces (#2536, @yurishkuro)
• OTel Cassandra/Elasticsearch Exporter queue defaults (#2533, @joe-elliott)
• [otel] Update jaeger-lib to v2.4.0 (#2538, @dstdfx)
• Remove unnecessary ServiceName index seek if tags query is available (#2535, @burmanm)
• Update static UI assets path in contrib doc (#2548, @albertteoh)

UI Changes

• UI pinned to version 1.12.0. The changelog is available here v1.12.0

Release 1.20.0

Backend Changes

Breaking Changes

• Configurable ES doc count (#2453, @albertteoh)

  The --es.max-num-spans flag has been deprecated in favour of --es.max-doc-count.
  --es.max-num-spans is marked for removal in v1.21.0 as indicated in the flag description.

  If both --es.max-num-spans and --es.max-doc-count are set, the lesser of the two will be used.

  The use of --es.max-doc-count (which defaults to 10,000) will limit the results from all Elasticsearch
  queries by the configured value, limiting counts for Jaeger UI:

  • Services
  • Operations
  • Dependencies (edges in a dependency graph)
  • Span fetch size for a trace

• The default value for the flag query.max-clock-skew-adjustment has changed to 0s, meaning that the clock skew adjustment is now disabled by default. See #1459.

New Features

• Grpc plugin archive storage support (#2317, @m8rge)
• Separate Ports for GRPC and HTTP requests in Query Server (#2387, @rjs211)
• Configurable ES doc count (#2453, @albertteoh)
• Add storage metrics to OTEL, metrics by span service name (#2431, @pavolloffay)

Bug fixes, Minor Improvements

• Increase coverage on otel/app/defaultconfig and otel/app/defaultcomponents (#2515, @joe-elliott)
• Use OTEL Kafka Exporter/Receiver Instead of Jaeger Core (#2494, @joe-elliott)
• Fix OTEL kafka receiver/ingester panic (#2512, @pavolloffay)
• Disable clock-skew-adjustment by default. (#2513, @jpkrohling)
• Fix ES OTEL status code (#2501, @pavolloffay)
• OTel: Factored out Config Factory (#2495, @joe-elliott)
• Fix failing ServerInUseHostPort test on MacOS (#2477, @albertteoh)
• Fix unmarshalling in OTEL badger (#2488, @pavolloffay)
• Improve UI placeholder message (#2487, @yurishkuro)
• Translate OTEL instrumentation library to ES DB model (#2484, @pavolloffay)
• Add partial retry capability to OTEL ES exporter. (#2456, @pavolloffay)
• Log deprecation warning only when deprecated flags are set (#2479, @pavolloffay)
• Clean-up Badger's trace-not-found check (#2481, @yurishkuro)
• Run the jaeger-agent as a non-root user by default (#2466, @chgl)
• Regenerate certificates to use SANs instead of Common Name (#2461, @albertteoh)
• Support custom port in cassandra schema creation (#2472, @MarianZoll)
• Consolidated OTel ES IndexNameProviders (#2458, @joe-elliott)
• Add positive confirmation that Agent made a connection to Collector (… (#2423, @BernardTolosajr)
• Propagate TraceNotFound error from grpc storage plugins (#2455, @joe-elliott)
• Use new ES reader implementation in OTEL (#2441, @pavolloffay)
• Updated grpc-go to v1.29.1 (#2445, @jpkrohling)
• Remove olivere elastic client from OTEL (#2448, @pavolloffay)
• Use queue retry per exporter (#2444, @pavolloffay)
• Add context.Context to WriteSpan (#2436, @yurishkuro)
• Fix mutex unlock in storage exporters (#2442, @pavolloffay)
• Add Grafana integration example (#2408, @fktkrt)
• Fix TLS flags settings in jaeger OTEL receiver (#2438, @pavolloffay)
• Add context to dependencies endpoint (#2434, @yoave23)
• Fix error equals (#2429, @albertteoh)

UI Changes

• UI pinned to version 1.11.0. The changelog is available here v1.11.0