Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

suddenly the archival not showing anymore the completed workflows using S3 bucket #2128

Open
Marwan-Dalaty opened this issue May 30, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@Marwan-Dalaty
Copy link

Describe the bug
I have the archival connected to S3 bucket, it was working for few months and sudenly I cant find any archived wrkflow from the UI and it keeps loading for a long time after that show me no available archival. But If I check the archivals using tctl command I can find them.

I had the latest version temporal serevr 1.22.7, I thought the issue is from the version, so I upgraded also my server and UI to the latest version and issue still exist

To Reproduce
Steps to reproduce the behavior:

No Idea

Expected behavior
to show me the archived workflows

Screenshots

image
image

After loading for 1-2 minutes, it will give me the below message

image

@Marwan-Dalaty Marwan-Dalaty added the bug Something isn't working label May 30, 2024
@Marwan-Dalaty
Copy link
Author

when I cleared the S3 bucket, I was able to open again the archival and it was very fast.
So after troubleshooting, I found if your s3 bucket has more than 1k objects like archived workflows then you will face the above issue and it wont work.

I think it is related to the timeout between temporal and s3 and how much fast the temporal get the data from s3. It seems you may need to enhance the streadming data in a better way

@diseku
Copy link

diseku commented Jun 20, 2024

we're facing the same issue. could the query range be limited to fetch e.g. latest 100 records only?

@nimayu
Copy link

nimayu commented Dec 11, 2024

we have the same issue with our archival connected to a GCP bucket. Are there any updates on this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants