App upgrade in an invalid state, app image upgraded and shows it is running new version under Info but under Updates it still believes an update is needed

[ChristopherMag]

@robi Sorry I missed your message, when clicking the blue update button and then getting the red text error Downgrades are not permitted for apps installed from AppStore or Community. force to override there are no logs generated under the apps event log or the shared cloudron service's logs.

When the button is clicked the browser makes the following fetch:

fetch("https://redacted.com/api/v1/apps/e40353c4-d8bb-484c-8147-aeeb639c716a/update?access_token=redacted", {
  "headers": {
    "accept": "*/*",
    "accept-language": "en-US,en;q=0.9",
    "cache-control": "no-cache",
    "content-type": "application/json",
    "pragma": "no-cache",
    "priority": "u=1, i",
    "sec-ch-ua": "\"Not=A?Brand\";v=\"99\", \"Microsoft Edge\";v=\"151\", \"Chromium\";v=\"151\"",
    "sec-ch-ua-mobile": "?0",
    "sec-ch-ua-platform": "\"Windows\"",
    "sec-fetch-dest": "empty",
    "sec-fetch-mode": "cors",
    "sec-fetch-site": "same-origin"
  },
  "referrer": "https://redacted.com/",
  "body": "{\"skipBackup\":false,\"appStoreId\":\"com.paperlessng.cloudronapp@1.60.7\"}",
  "method": "POST",
  "mode": "cors",
  "credentials": "include"
});

And receives the following response:

{
  "status": "Bad Request",
  "message": "Downgrades are not permitted for apps installed from AppStore or Community. force to override"
}

[ChristopherMag]

This morning I changed the "location" of the paperless app to be a different host under the same domain, created an additional backup location of type filesystem on local storage (the multiple backup location feature is a lifesaver for things like this), created a backup, cloned it to the original location, and stopped the old copy that constantly wants to update but can't.

The new copy appears to be working fine and no longer indicates it has an unapplied update.

I will wait till there is another new update to paperless and then start the old one and see if with a subsequent update being available it finds a way to clear its stuck state and/or test out any other ways that end up being provided to fix this issue but for now things are back working again.

Independent of this I am going to try to not use a volume for the consume directory with paperless and instead try to have n8n watch the cifs file share and then pull documents and upload them directly to paperless without ever using the filesystem of paperelss in the first place.

[ChristopherMag]

After working through options, n8n has no easy way to communicate with a cifs file share without mounting it as a local path to the docker container which would reintroduce all the same issues that the paperless app had.

There are community libraries for n8n that enable it to access cifs shares but none of them implement CHANGE_NOTIFY so they would all be polling based which I don't want as I want the system to respond to files being written, not have to poll to see if files are there as that is the way it was working with paperless before.

In theory the smbclient command line tool could be used to do this but I have not been able to find any single binary source for this that would include the libraries it needs to function as installing it into the n8n app container via a package manager isn't an option unless I fork the n8n cloudron app.

I have hit a number of dead ends and at this point have gone back to adding the volume back to the paperless app and updating the configuration to use that volume for the consume directory.

Hopefully something can be done so that if an app pacakge upgrade completes but fails to start after the upgrade because of a volume not being accessible, it still properly removes the pending update so that it doesn't get stuck trying to update to an update that was already installed.

[robi]

For some reason while reading this I got the impression to use WebDAV with maybe Cubby. Would that work for your workflow?

[ChristopherMag]

@robi Unfortunately I don't have a way of using any other remote file transmission protocol aside from CIFS.

We use ScanSnap scanners (used to be Fujitsu, now owned by Ricoh) which have crummy support for anything other than local storage on the client endpoint or CIFS.

I would rather use another scanner vendor but in a comparison with Canon, various MFPs from Kycoera and Konica Minolta, HP, etc. the ScanSnap (like the ix 1600) consistently creates better pdfs with little or no configuration.

Better OCR (beats the OCR built into paperless), blank page removal, automatic rotation, image quality relative to the source (they do a bunch of dynamic tuning to adjust the dpi, contrast, etc. all dynamically and with better and more consistent results with less work than the alternatives), smaller size pdfs, etc.

It feels like the comparison of a canon DSLR camera from back in the day vs an iPhone where technically you have far more control and options with the canon but the iPhone for most circumstances does a better job overall without any extra expertise or configuration needed.

The downside of that is the that there is nothing like a twain driver for these ScanSnap scanners and so you have to use their not so great scan snap software which aside from local storage only supports scanning to CIFS shares unless, your willing to pass all your documents their their ScanSnap cloud which for compliance purposes we cannot do.

This leaves me in the crappy situation of either spending time working with all of those other scanner vendors and various software chains to try and recreate the processing chain built into a ScanSnap so that we can get the same quality of output with no additional work on behalf of the end user or dealing with the limitation of ScanSnap software and maintaining a CIFS share somewhere with the relative quirks that introduces when trying to get the files over into paperless.

I would love to have a better alternative and it feels like in the last 10 years we are getting closer to that point.

Hopefully one day I can treat a scanner as just a document feeder with an image sensor attached and everything else is handled with other open source software but I still haven't been able been able to accomplish that myself yet.

[robi]

Oh that explains it.. I'd think you could dump them locally then sync and cleanup, but if it's per user that's extra tooling for everyone.

So is the requirement that whatever you use be mountable as a drive letter like CIFS (WebDAV is too) or just a remote thing?

[ChristopherMag]

@robi Based on your prompting I have revisited the issue and now setup surfer with a shared volume and mapped a network drive to each client where the scan snap software runs via surfer's webdav url.

Now I can scan via scan snap, it writes to the local mounted drive letter, which goes through surfer to the volume attached to it and paperless, and paperless consumes the files written there in near real time.

No more CIFS mount as a part of the solution.

It means maintaining a mapped webdav directory on each client that needs to scan to paperless which isn't ideal but at this point I will take that over having to deal with any cifs based volumes.

Thank you for the questions and while not the solution I would prefer it is at least an incremental step better than where we were.

[timconsidine]

@ChristopherMag well done, good progress and definitely a ‘win’ (assuming that scan stations are relatively stable and not changing, the setup work seems tolerable).
Gives me some ideas also

[robi]

@ChristopherMag You're welcome.

I invite all of us to listen to our intuition more carefully.

[girish]

Late to the thread...

"body": "{"skipBackup":false,"appStoreId":"com.paperlessng.cloudronapp@1.60.7"}",

So this line indicates that the paperless app was in package version 1.60.7, which is the latest.

"message": "Downgrades are not permitted for apps installed from AppStore or Community. force to override"

The backend checks for updates periodically. It has a cache and passes on the cache to the frontend (for UI/display purposes). To prevent a race, eg you might have parallel tabs or another updated the app, the frontend passes back the version field to the backend before starting an update. What has happened here is that this cache is for some reason an older version and when it tries to update, the backend says that it's already updated to a more recent version.

This issue is mostly caused by either a frontend caching or the backend unable to reach the appstore. I don't think it's the latter because the cloudron-support diagnostics came out clean . Usually, you can also clear this cache by clicking 'Check for updates'.

I guess we have to debug further if someone gets this reproduced .