Chat logs lost after update from 1.19 -> 1.20

[infogulch]

All my TheLounge chat logs have been lost. After testing backups I found the first backup missing logs occurred in the update from 1.19 to 1.20 on 12/15. I have archived the last good backup so they're not completely destroyed.

Maybe related to the change mentioned in Lounge backups regularly stalling?

Cloudron should take these actions at minimum:

• Affected users should be notified (anyone running TheLounge app) and instructed to archive the last 1.19 backup ASAP
• Identify the cause of the lost logs (just a mishap in the transition to symlinked logs?)
• Validate that logs are still being backed up at all
• Come up with a strategy to merge logs from the backup with the logs generated between 12/15 and today.

[nebulon]

Do you mean chat history or which kind of logs are these? Log files are purged every now and then like in all packages.

Given that Lounge is an IRC client, isn't history stored on the IRC server side? I haven't used IRC in a long time though to remember the behavior.

[infogulch]

Sorry I didn't see your message until now.

Yes all chat logs were lost. I don't think it's a good idea to purge chat logs automatically. I have logs of dozens of channels over 10 years and it really doesn't take up that much space if you avoid channels with huge amounts of spam.

IRC has no mechanism for storing messages, the server only transmits them. If you're not present in the channel and store them yourself then you just miss those messages, including direct messages. This is the benefit and primary purpose of using an IRC bouncer, it stays connected so you can see and store chat history yourself.

[nebulon]

I see we are talking about different log types then. App logs are things like debug messages, those are logrotated. The chat history (logs) should not be purged if I would understand the app correctly. Possibly a bug then? I wonder how we can reproduce this.

[jdaviescoates]

yeah, @infogulch obviously isn't really talking about logs, but about chat message history.

I just checked my install too and can confirm that every channel I'm in now only scrolls back to 14 Dec, which is indeed when the update from Cloudron package 1.19.0 to 1.20.0 happened.

[infogulch]

My guess is that they are destroyed when upgrading to version 1.20 of the app, due to a change in the message history storage location described in the Lounge backups regularly stalling topic linked in the opening post of this thread. Maybe there's some mishap when 1.20+ is restoring files from a backup made at 1.19 or earlier.

This indicates a potential reproduction strategy:

1. Create a new The Lounge app at version 1.19
2. Log in and create some chat history
3. Stop the app, validate the logs are present on the fs, and create a backup
4. Upgrade the app to 1.20, observe the chat history is missing

---

@jdaviescoates In case you haven't already, I would strongly recommend that you pin the latest 1.19 backup so that it's not automatically cleaned up before this is sorted out.

[jdaviescoates]

@infogulch said in Chat logs lost after update from 1.19 -> 1.20:

@jdaviescoates In case you haven't already, I would strongly recommend that you pin the latest 1.19 backup so that it's not automatically cleaned up before this is sorted out.

Thanks, good idea! But in my case I'm really not bothered about this as there aren't really any important messages there.

[infogulch]

Ok well the new update loses chat logs again. I guess I can't use cloudron for irc.

[jdaviescoates]

@nebulon said in Chat logs lost after update from 1.19 -> 1.20:

I wonder how we can reproduce this.

Try what @infogulch suggested above?

[girish]

@infogulch @jdaviescoates the package incorrectly assumes that logs directory is app logs. It's actually the irc logs (messages). I am making a new package.

Usually, this is caught in our tests... But unfortunately, thelounge has no proper tests because this is not easy to test....

[girish]

New package is out. Unfortunately, the old versions still lose the irc logs

[infogulch]

Backups cover a lot of sins. I still have the archived backup from the last time this happened.

Now we need a process for restoring the chat logs. I guess we can start by cloning the app from an old backup, but I'm not sure how to copy files between containers. Any ideas?

[infogulch]

Oh. So because logs were assumed to be app logs and redirected to the ephemeral /run directory so they were not backed up.

How did this issue persist when we had a whole discussion about how logs are chat logs not app logs nearly 6 months ago?

[infogulch]

I updated to package version io.github.thelounge@1.20.1-1 and I still see that logs is a symlink to the ephemeral run directory:

[girish]

@infogulch said in Chat logs lost after update from 1.19 -> 1.20:

Oh. So because logs were assumed to be app logs and redirected to the ephemeral /run directory so they were not backed up.

yes, pretty much

How did this issue persist when we had a whole discussion about how logs are chat logs not app logs nearly 6 months ago?

I think this just fell through the cracks. We didn't investigate further and lost track.

[girish]

@infogulch said in Chat logs lost after update from 1.19 -> 1.20:

I updated to package version io.github.thelounge@1.20.1-1 and I still see that logs is a symlink to the ephemeral run directory:

My bad, that symlink has to get deleted. I pushed another package now, can you please check?

[infogulch]

It looks like it's working now.