App upgrade in an invalid state, app image upgraded and shows it is running new version under Info but under Updates it still believes an update is needed

[ChristopherMag]

Inside the apps table of the box database in the mysql server running on cloudron there is an updateInfoJson column that is set to null for apps that don't have a pending update but has a json value for apps that have a pending update.

It also looks like there is a config.json under the apps folder on the cloudron server located at /home/yellowtent/appsdata/e40353c4-d8bb-484c-8147-aeeb639c716a/config.json that contains a updateInfo key describing the update that is available.

Documenting what I am finding here while waiting for a more specific direction but it seems like maybe setting updateInfoJson to null for this app in the box database as well as setting it to null in the config.json might result in this no longer being stuck.

In that same vein though I would imagine that maybe that would be the case if I backup and clone the existing app, I am going to try that and see what happens and if that works it is a pain but maybe I could do that after hours to force the apps state to get fixed.

[ChristopherMag]

I took a backup and restored it and then ran check for updates and it doesn't detect any updates as needed for the restored version of paperless.

I would still like to understand how to fix this via another means as I am not 100% sure of all the effects doing this will have as I need the domain name to stay the same so I would likely have to move the existing app to a new domain, back it up, and then restore it to its original domain.

Is there any less impactful way to fix this, like updating that table to null the value of the updateInfoJson column for the app?

[ChristopherMag]

@robi Sorry I missed your message, when clicking the blue update button and then getting the red text error Downgrades are not permitted for apps installed from AppStore or Community. force to override there are no logs generated under the apps event log or the shared cloudron service's logs.

When the button is clicked the browser makes the following fetch:

fetch("https://redacted.com/api/v1/apps/e40353c4-d8bb-484c-8147-aeeb639c716a/update?access_token=redacted", {
  "headers": {
    "accept": "*/*",
    "accept-language": "en-US,en;q=0.9",
    "cache-control": "no-cache",
    "content-type": "application/json",
    "pragma": "no-cache",
    "priority": "u=1, i",
    "sec-ch-ua": "\"Not=A?Brand\";v=\"99\", \"Microsoft Edge\";v=\"151\", \"Chromium\";v=\"151\"",
    "sec-ch-ua-mobile": "?0",
    "sec-ch-ua-platform": "\"Windows\"",
    "sec-fetch-dest": "empty",
    "sec-fetch-mode": "cors",
    "sec-fetch-site": "same-origin"
  },
  "referrer": "https://redacted.com/",
  "body": "{\"skipBackup\":false,\"appStoreId\":\"com.paperlessng.cloudronapp@1.60.7\"}",
  "method": "POST",
  "mode": "cors",
  "credentials": "include"
});

And receives the following response:

{
  "status": "Bad Request",
  "message": "Downgrades are not permitted for apps installed from AppStore or Community. force to override"
}

[ChristopherMag]

This morning I changed the "location" of the paperless app to be a different host under the same domain, created an additional backup location of type filesystem on local storage (the multiple backup location feature is a lifesaver for things like this), created a backup, cloned it to the original location, and stopped the old copy that constantly wants to update but can't.

The new copy appears to be working fine and no longer indicates it has an unapplied update.

I will wait till there is another new update to paperless and then start the old one and see if with a subsequent update being available it finds a way to clear its stuck state and/or test out any other ways that end up being provided to fix this issue but for now things are back working again.

Independent of this I am going to try to not use a volume for the consume directory with paperless and instead try to have n8n watch the cifs file share and then pull documents and upload them directly to paperless without ever using the filesystem of paperelss in the first place.

[ChristopherMag]

After working through options, n8n has no easy way to communicate with a cifs file share without mounting it as a local path to the docker container which would reintroduce all the same issues that the paperless app had.

There are community libraries for n8n that enable it to access cifs shares but none of them implement CHANGE_NOTIFY so they would all be polling based which I don't want as I want the system to respond to files being written, not have to poll to see if files are there as that is the way it was working with paperless before.

In theory the smbclient command line tool could be used to do this but I have not been able to find any single binary source for this that would include the libraries it needs to function as installing it into the n8n app container via a package manager isn't an option unless I fork the n8n cloudron app.

I have hit a number of dead ends and at this point have gone back to adding the volume back to the paperless app and updating the configuration to use that volume for the consume directory.

Hopefully something can be done so that if an app pacakge upgrade completes but fails to start after the upgrade because of a volume not being accessible, it still properly removes the pending update so that it doesn't get stuck trying to update to an update that was already installed.

[robi]

For some reason while reading this I got the impression to use WebDAV with maybe Cubby. Would that work for your workflow?

[ChristopherMag]

@robi Unfortunately I don't have a way of using any other remote file transmission protocol aside from CIFS.

We use ScanSnap scanners (used to be Fujitsu, now owned by Ricoh) which have crummy support for anything other than local storage on the client endpoint or CIFS.

I would rather use another scanner vendor but in a comparison with Canon, various MFPs from Kycoera and Konica Minolta, HP, etc. the ScanSnap (like the ix 1600) consistently creates better pdfs with little or no configuration.

Better OCR (beats the OCR built into paperless), blank page removal, automatic rotation, image quality relative to the source (they do a bunch of dynamic tuning to adjust the dpi, contrast, etc. all dynamically and with better and more consistent results with less work than the alternatives), smaller size pdfs, etc.

It feels like the comparison of a canon DSLR camera from back in the day vs an iPhone where technically you have far more control and options with the canon but the iPhone for most circumstances does a better job overall without any extra expertise or configuration needed.

The downside of that is the that there is nothing like a twain driver for these ScanSnap scanners and so you have to use their not so great scan snap software which aside from local storage only supports scanning to CIFS shares unless, your willing to pass all your documents their their ScanSnap cloud which for compliance purposes we cannot do.

This leaves me in the crappy situation of either spending time working with all of those other scanner vendors and various software chains to try and recreate the processing chain built into a ScanSnap so that we can get the same quality of output with no additional work on behalf of the end user or dealing with the limitation of ScanSnap software and maintaining a CIFS share somewhere with the relative quirks that introduces when trying to get the files over into paperless.

I would love to have a better alternative and it feels like in the last 10 years we are getting closer to that point.

Hopefully one day I can treat a scanner as just a document feeder with an image sensor attached and everything else is handled with other open source software but I still haven't been able been able to accomplish that myself yet.

[robi]

Oh that explains it.. I'd think you could dump them locally then sync and cleanup, but if it's per user that's extra tooling for everyone.

So is the requirement that whatever you use be mountable as a drive letter like CIFS (WebDAV is too) or just a remote thing?

[ChristopherMag]

@robi Based on your prompting I have revisited the issue and now setup surfer with a shared volume and mapped a network drive to each client where the scan snap software runs via surfer's webdav url.

Now I can scan via scan snap, it writes to the local mounted drive letter, which goes through surfer to the volume attached to it and paperless, and paperless consumes the files written there in near real time.

No more CIFS mount as a part of the solution.

It means maintaining a mapped webdav directory on each client that needs to scan to paperless which isn't ideal but at this point I will take that over having to deal with any cifs based volumes.

Thank you for the questions and while not the solution I would prefer it is at least an incremental step better than where we were.

[timconsidine]

@ChristopherMag well done, good progress and definitely a ‘win’ (assuming that scan stations are relatively stable and not changing, the setup work seems tolerable).
Gives me some ideas also

[robi]

@ChristopherMag You're welcome.

I invite all of us to listen to our intuition more carefully.

[girish]

Late to the thread...

"body": "{"skipBackup":false,"appStoreId":"com.paperlessng.cloudronapp@1.60.7"}",

So this line indicates that the paperless app was in package version 1.60.7, which is the latest.

"message": "Downgrades are not permitted for apps installed from AppStore or Community. force to override"

The backend checks for updates periodically. It has a cache and passes on the cache to the frontend (for UI/display purposes). To prevent a race, eg you might have parallel tabs or another updated the app, the frontend passes back the version field to the backend before starting an update. What has happened here is that this cache is for some reason an older version and when it tries to update, the backend says that it's already updated to a more recent version.

This issue is mostly caused by either a frontend caching or the backend unable to reach the appstore. I don't think it's the latter because the cloudron-support diagnostics came out clean . Usually, you can also clear this cache by clicking 'Check for updates'.

I guess we have to debug further if someone gets this reproduced .

[ChristopherMag]

@girish We can close this for now, I don't think I am going to have the time in the near future to recreate this by doing the following:

• Install an older version of an app that is 1 version behind the most current version
• Create a volume on cloudron that is accessible
• Add that volume to the app
• Take some action so the volume not accessible any longer, (like a cifs/sshfs/nfs volume where the host has been shutdown
  • Confirm the volume still has a green status in Cloudron
• Update the app
• After the app fails to start up after the update remove the volume from the app
• Start up the app
• Confirm the app is running the updated package version
• Confirm that the updates tab still lists the same package version the app is running as an available version to install and that clicking update again fails with the downgrades not permitted error.

[ChristopherMag]

A new version of paperless was released and I started up the copy of the app that was unable to update as it kept trying to update to the version it was already on and when I did check updates it found the newest one and was able to install it.

Basically if an app gets in the weird state where it is already running the current version and it keeps trying to update to that same version it is already running and failing you can disable automatic updates until the next real update and the next real update will replace the one that was there and then it will be able to update normally again without backing up and restoring the app like I did to work around the issue sooner.

Separate from that, for or anyone coming later, the surfer solution is not working out as the password has to be reentered for the mapped drive each day.

I can see that the credential is saved in Windows Credential Manager but for whatever reason each new day it prompts for the credential with the remember my credentials box unchecked. Entering the password and checking the box will work for a day but then roughly by the next day it will prompt again.

If the user doesn't proactively navigate to this drive letter than scanning with the scanner just fails as it cannot write to that drive letter and it isn't obvious what is wrong.

It appears the issue is that windows won't resupply the authentication when the webdav server is using basic authentication.

This means that likely switching to the sftpgo app on cloudron that also offers webdav won't fix the issue as there is no shared authentication scheme, like kerberos, that both windows and cloudron can use that would allow this to work without resupplying the credential every day.

Microsoft's stated work around is a scheduled task that maps the drive new each time the computer boots and has the password hard coded in the task which is not good.

Going back to the drawing board of how to have access to a place save files from a windows host that can be accessed via standard windows file access libraries (drive letters, UNC paths).

Cloudron documentation does have a process for setting up an NFS share which I wouldn't have thought would be supported given that it installs packages and changes configuration on the host os but I am going to try that with the windows based nfs client and see if that works any better.

[nebulon]

If I read this correctly the issue that windows is not persisting the webdav password is not related to the update issue? Wherever you add the webdav mount in windows, maybe there is some "remember password" option or so? There is otherwise likely nothing we can do as that is a client side issue

[ChristopherMag]

@nebulon Aboslutely nothing you can do on your side about that, limitation of windows.

I tried to use NFS shares from the cloudron server but windows only supports NFSv3 not NFSv4 and that results in dynamic ports having to be opened on the firewall or a host of services have to be assigned static ports.

Even after all that the NFS mounts on windows also don't consistently remember their passwords (even when checking the boxes that indicates remember password).

I have given up and done what I didn't want to do which is install samba on the cloudron host and serve the directory via CIFS from there.

All the clients are connected via a samba user and it will persist accross reboots without issue.

The biggest downside is that I am now running something extra on the cloudron server that isn't intended to be there but until there is some way to host samba as an app or some other app on cloudron that supports cifs I don't see any way to work around the issue of needing windows clients to be able to write to a location via native filesystem apis that apps running on cloudron can also access without depending on a server other than cloudron itself to host those files.