Searching mail logs hit & miss

[d19dotca]

Hello,

Is there any known issue or limitations to searching in the mail logs from the UI side of things? When I search a domain for example, nothing comes up in the results despite it being there if I'm not searching for it. But I'm sure this has worked before. This is not the first time I've seen this behaviour though so it almost seems to be sporadic in nature if I am recalling correctly. Not sure if it's because I restarted the mail server about 10 minutes prior, if that somehow prevents searching the logs even if the logs were generated since then for example? Not sure what's up with it. haha.

This leads me to not truly trust the search results of the mail logs. Of course I can look and download the logs and search that way, but it's inconvenient especially if needing to check something while out on a mobile phone.

Example of what I see right now... I'll see on the Mail page in the first page of the logs that there's an email to user@example.com and if I search example.com to find all relevant log entries fo that domain, no results show. Same if I go with just a username sometimes too.

Now most of the time it works fine, so I didn't want to make this a Support case yet as I'm not even sure if it's an issue. So I'm curious... does anyone else experience this behaviour from time to time at all? Or does it just always work for you when searching a partial or full term on the Mail page?

Lastly, I'd love to learn what specifically can be searched / is searchable. One example is I don't believe we can search for what messages were identified as spam (i.e. we can't search "spamStatus": "Yes (the partial term from the logs)). Just curious what specific components from the logs are actually searchable (or maybe easier to answer is what's not searchable).

[nebulon]

@d19dotca so the mail addon container has a sqlite3 db for event logs from the mailserver at /app/data/events/log.db this database will be queried with the search input as well as the type selected in the dropdown.

The SELECT query could possibly be improved if something is missing generally. But I think what you describe is some inconsistency when searching for the same thing at different times? This should not happen and could be a bug. In such cases, do you see any errors or timeouts in the browser console?

[d19dotca]

@nebulon That's good to know, thanks for the insight! And yes, my issue is more of an inconsistency where I can search "titan" for example which is a partial match to one of the domains I host, and it will pull up all the results. Then another day or later that day in the same browser and everything, I'll search "titan" again and then no results are found despite me knowing there are definitely results it should be returning. Then maybe 5-30 minutes later it's working fine again.

I didn't think at the time to check the browser console - I'll try and remember to do that next time I see it happen. I definitely don't recall seeing any errors or timeouts or anything in the front-end, but if that's only seen in the console then I never checked that yet. It's not super often (it works probably 98% of the time), but it's often enough that it can become a little annoying especially when there's no real good way to look at the logs on mobile other than that UI front for it. I'll try to watch for the next time it happens. Thanks for the advice.

[robi]

@d19dotca see what happens when you search and hit the error if you search for any other string, then search for 'titan' again.

[d19dotca]

@robi Oh yeah sorry I should have clarified. When I am in a position where it won't return any results, it won't return any results for any further queries for any other keywords. It's as if search is just dead for several minutes. So if I search and things are fine then suddenly it breaks when I go to search "titan", then if I immediately search something like "gmail" or "hotmail", it'll also return no results each time. Then maybe 10-15 minutes later I'll test it and it seems to be working again and it'll return results for every query I know should have returned results earlier.

[robi]

@d19dotca that just sounds like the browser connection to the server is timing out and isn't connected. At some point later it reconnects and things work again.

you try a browser page reload/refresh?

[d19dotca]

@robi Oh definitely tried refreshing the pages, yes. I'm not certain if it'd be a timeout / connection issue, but I'll definitely try to get more data when it happens again.

[nebulon]

@d19dotca since we had some recent timeout issue with the services view (and mail being a service here) it could be that also here the mail container or the server overall is busy during that time and thus the search queries timeout. If you see this happening again, a look into the system/box logs as well as mail service logs should reveal this then.