Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to observe a dead loop. #4758

Open
tttoad opened this issue Dec 15, 2023 · 6 comments
Open

How to observe a dead loop. #4758

tttoad opened this issue Dec 15, 2023 · 6 comments
Labels
enhancement New feature or request

Comments

@tttoad
Copy link
Contributor

tttoad commented Dec 15, 2023

Problem Statement

func main(){
        // ....
	ctx, span := tracer.Start(
		ctx,
		"CollectorExporter-Example",
		trace.WithAttributes(commonAttrs...))
	defer span.End()
	for i := 0; i < 10; i++ {
		Work(ctx)
	}

	log.Printf("Done!")
}
func Work(ctx context.Context) {
	ctx, span := otel.Tracer("test-tracer").Start(ctx, "work")
	defer span.End()
	for i := 1; i < 10; i++ {
		// do something
		Work2(ctx)
	}
}

func Work2(ctx context.Context) {
	_, span := otel.Tracer("test-tracer").Start(ctx, "work2")
	defer span.End()
	for i := 1; i > 0; i++ {
		// do something
	}
}

If there's a bug in my code, it's a dead loop.
In this case the trace cannot find out where the program is looping.
Should a span timeout be added?
like Start(ctx,"name",Timeout(time.Minute )).

@tttoad tttoad added the enhancement New feature or request label Dec 15, 2023
@MrAlias
Copy link
Contributor

MrAlias commented Dec 15, 2023

	for i := 1; i > 0; i++ {
		// do something
	}

What is "do something"? If it does not include return or break this loop will never terminate.

Without the function returning you will never have your defered function run.

If you want to see the progress of an unterminated loop you need to create and end a span each iteration of the loop.

@tttoad
Copy link
Contributor Author

tttoad commented Dec 16, 2023

I mean how do I find out when my code never terminates?
Maybe add a timeout to automatically call the span.end() method when the Work2 function times out.

@dmathieu
Copy link
Member

Add a timeout to the loop itself, so it can't never terminate ?

@tttoad
Copy link
Contributor Author

tttoad commented Dec 18, 2023

Add a timeout to the loop itself, so it can't never terminate ?

Loop is just one possibility.Maybe it's a deadlock...
I'm not sure this is an issue that should be discovered by traces.
I had this problem, so I asked...
Other services send requests, but I don't see any span for my service...

@dmathieu
Copy link
Member

If you have infinite and unchecked loops or deadlocks, you'll end up with CPU/memory being used for nothing, and a leak. That should be tracked and fixed in your code, I'm not sure traces should be handling it.

Profiling could be something that helps you there.

@tttoad
Copy link
Contributor Author

tttoad commented Dec 20, 2023

I can't pinpoint the problem fast enough for the loop that won't terminate. ( Maybe the function is not being called, or not requested)
The problem was found to be more complex through Profiling.

timeout don't seem to be good.
In many cases, it is impossible to predict how long a function will run...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants