I had a pod restart for what I'm pretty sure was a OOM event but the only way I know this happened is because a running process terminated incorrectly and I could see the "Restarts: 1" in the log window. Nothing else notified me this happened or showed me the reason. Would be very helpful to be able to find the cause and know when this happens since this is unexpected behaviour that could be the result of an issue.
You are able to see these events under the 'Debug' section, in your situation it should show that a issue was detected and the reason for it.
If you have a suggestion where I could put a link/reference to this to make this more obvious, let me know!
Ah I indeed see this now: "Pkgtrends-scheduler previously ran out of memory and was restarted". Good to see my hunch was correct.
Just spitballing here not sure any of these suggestions are the correct ones but:
- When a "critical" issues occurs a e-mail notification would be great. Now I only know if I remember to check my debug tab.
- Maybe these issues can have some kind of "read' state so they can be surfaces with a counter in the tab to draw attention and if I have seen them the counter decreases
- When looking at the logs for a pod it might be nice to show X critical issues seen in the past X days or something so I can click through to the debug tab
When looking at the debug tab this list is also not very helpful, I would love to see a little more details or ways to see whats the issue instead of "Critical issue" triggered by "Issue detected". That says nothing to me so I need to click them open one by one to read a single line of text that explains it all 😅
I've added the main reason of the issue into the debug list. I've also added a notification so that every debug snapshot will be sent to your notification channels! To do this, first create a notification channel under your team settings, then per application create a notification channel with the type 'Error'. Once this has been setup a new critical issue detected will be notified to your notification channel.
Log reason for pod restart
-
Zander van der Meer moved item to board Live
1 month ago -
Zander van der Meer moved item to board Under review
1 month ago -
Alex Bouma moved item to project Applications
1 month ago -
Alex Bouma created the item
1 month ago