Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved Paperless - indexing and archiving scanned documents

    App Wishlist
    14
    45
    3100
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • doodlemania2
      doodlemania2 App Dev @doodlemania2 last edited by

      @doodlemania2 just an fyi - this is working well, but the "consumption" directory isn't polling so still tinkering with that. I am able to upload manually and auto tagging and pdf extraction is working. so, we're about 95% there I think.

      scooke absnt_mind timconsidine 3 Replies Last reply Reply Quote 6
      • scooke
        scooke @doodlemania2 last edited by

        @doodlemania2 Keep up the good work!

        A life lived in fear is a life half-lived

        1 Reply Last reply Reply Quote 3
        • absnt_mind
          absnt_mind @doodlemania2 last edited by

          @doodlemania2 Very excited about your efforts on this! I appreciate you, sir! I'm a vanilla PHP and JS developer for the most part, so node, docker, manifests, build scripts, all of it gets a little sketchy if I'm the one on it. I would love to learn how to package Cloudron apps, though. Maybe I'll have a look at some of the material that's available on that here later.
          I have been aching for a good document management system with user permissions for probably the last 12 months or so.

          1 Reply Last reply Reply Quote 3
          • doodlemania2
            doodlemania2 App Dev @doodlemania2 last edited by

            @girish - I'm officially stuck. The app is ready and I think the tests might even work except for the consumption directory.

            Described my issue here: https://github.com/jonaswinkler/paperless-ng/issues/438.

            Tried moving the supervisors around, nothing.

            If I run the consume script from shell, works like a champ. Open to ideas.

            jimcavoli 1 Reply Last reply Reply Quote 2
            • jimcavoli
              jimcavoli App Dev @doodlemania2 last edited by

              @doodlemania2 Is that a periodic job? Even if not, could we just hack around it with a small caveat by using the shell command and scheduler add-on? It'd be a minor thing, I think, for the Cloudron package to run an "every minute" or something consumption

              doodlemania2 1 Reply Last reply Reply Quote 2
              • doodlemania2
                doodlemania2 App Dev @jimcavoli last edited by

                @jimcavoli That's a fabulous idea - I will give that a try - it uses inotify on the FS to monitor for changes (intended to always run), but this could be a fabulous workaround! ON IT!

                1 Reply Last reply Reply Quote 3
                • jimcavoli
                  jimcavoli App Dev last edited by

                  @doodlemania2 Sure thing! Also, didn't know it was inotify under the hood - that has a fraught history with dockerized runtimes, so this is likely your best option anyway 🙂

                  doodlemania2 1 Reply Last reply Reply Quote 5
                  • doodlemania2
                    doodlemania2 App Dev @jimcavoli last edited by

                    @jimcavoli alrighty - that seems to have worked (running it with cloudron scheduler). now just need someone to help me with tests and it can ship!

                    girish 1 Reply Last reply Reply Quote 6
                    • girish
                      girish Staff @doodlemania2 last edited by

                      @doodlemania2 I marked this as WIP now! Great progress.

                      ultraviolet 1 Reply Last reply Reply Quote 3
                      • ultraviolet
                        ultraviolet App Dev @girish last edited by

                        hi all,

                        I was actually working on this a few weeks ago. I forked @doodlemania2's repo and updated it to the latest version. I also enabled SFTP access I am also working to implement the proxyauth addon to pass though to the django app (still wip) as it does not support LDAP natively.

                        I don't have access to fork in the cloudron gitlab so I had to copy to my github account. You can find the repo here:
                        https://github.com/campbellmcgregor/paperless-ng-cloudron

                        thanks

                        doodlemania2 girish 2 Replies Last reply Reply Quote 2
                        • doodlemania2
                          doodlemania2 App Dev @ultraviolet last edited by

                          @ultraviolet would you send me a PR? I'd be delighted to merge those in!

                          ultraviolet 2 Replies Last reply Reply Quote 1
                          • ultraviolet
                            ultraviolet App Dev @doodlemania2 last edited by

                            @doodlemania2 sure next time I am at a desk I will send you a PR

                            1 Reply Last reply Reply Quote 1
                            • girish
                              girish Staff @ultraviolet last edited by

                              @ultraviolet said in Paperless - indexing and archiving scanned documents:

                              I don't have access to fork in the cloudron gitlab so I had to copy to my github account

                              Whoops, sorry about that. By default, accounts cannot fork since we get too much spam. I assume you are https://git.cloudron.io/admin/users/ultraviolet ? I have increased the project limit already.

                              ultraviolet 1 Reply Last reply Reply Quote 1
                              • ultraviolet
                                ultraviolet App Dev @girish last edited by

                                @girish yup that is me thanks!

                                1 Reply Last reply Reply Quote 0
                                • ultraviolet
                                  ultraviolet App Dev @doodlemania2 last edited by

                                  @doodlemania2 sent you a PR

                                  doodlemania2 1 Reply Last reply Reply Quote 1
                                  • doodlemania2
                                    doodlemania2 App Dev @ultraviolet last edited by

                                    @ultraviolet Merged! That's fantastic - great work!!!

                                    ultraviolet 1 Reply Last reply Reply Quote 1
                                    • ultraviolet
                                      ultraviolet App Dev @doodlemania2 last edited by

                                      @doodlemania2 thanks, I may send you another PR in a day or so hit a few walls with the proxyauth.

                                      1 Reply Last reply Reply Quote 2
                                      • girish
                                        girish Staff last edited by

                                        @ultraviolet @doodlemania2 Per https://github.com/jonaswinkler/paperless-ng/discussions/654, it seems if someone knows some python we can add proper ldap support.

                                        doodlemania2 1 Reply Last reply Reply Quote 2
                                        • doodlemania2
                                          doodlemania2 App Dev @girish last edited by

                                          @girish oooo shiny! @ultraviolet -- that you?

                                          1 Reply Last reply Reply Quote 2
                                          • timconsidine
                                            timconsidine App Dev @doodlemania2 last edited by

                                            @doodlemania2 great work, thank you for making this available.
                                            I built (very smooth) and installed on my Cloudron.
                                            As a noob to paperless-ng, I am not sure what is needed for "processing".
                                            I uploaded few documents to test and they seem stuck in 'processing' in dashboard view.
                                            A tag and correspondent don't seem to get auto-applied.
                                            Is this functionality working ?
                                            Do I need to do something ?

                                            timconsidine scooke 2 Replies Last reply Reply Quote 0
                                            • timconsidine
                                              timconsidine App Dev @timconsidine last edited by timconsidine

                                              @timconsidine Seems last was my noob issue.
                                              The dashboard clears eventually and tags can be retro-applied with

                                              root@<app-instance>:/app/code/src# python3 manage.py document_retagger -T
                                              

                                              New Q : I'm looking to add schedule tasks but cannot see in doc a list of task.function. Is there one available?

                                              doodlemania2 1 Reply Last reply Reply Quote 0
                                              • doodlemania2
                                                doodlemania2 App Dev @timconsidine last edited by

                                                @timconsidine Glad you got it working - I'm note sure how the scheduled tasks stuff works - since it does what i need out of the box, I just ignored those 🙂

                                                timconsidine 1 Reply Last reply Reply Quote 1
                                                • timconsidine
                                                  timconsidine App Dev @doodlemania2 last edited by

                                                  @doodlemania2 no problem, I understand, and likewise I have started using it as it does everything I currently want 'out of the box'.
                                                  I just don't like to miss a trick by getting even more value out of it by using 'advanced options'.

                                                  I have to say : thank you again. It's an awesome app, and solves a problem of coordinating docs between users (in my case purchase invoices needing review and input). Previously I have been using a shared folder in Nextcloud, and although that works, I have had issues when some user's sync to Nextcloud stops, and everyong starts shouting at each other because they are seeing different views of the world.

                                                  The only things not there which I can identify which would be nice :

                                                  • "tag tree" view, which I notice Teedy has. Just a UI advantage

                                                  • a way to remove tags w/o manually doing so, based on some conditions

                                                  But hey ho, just "nice to have" not essentials.

                                                  doodlemania2 1 Reply Last reply Reply Quote 1
                                                  • doodlemania2
                                                    doodlemania2 App Dev @timconsidine last edited by

                                                    @timconsidine outstanding! really glad it works for you

                                                    1 Reply Last reply Reply Quote 2
                                                    • scooke
                                                      scooke @timconsidine last edited by scooke

                                                      @timconsidine Hello! To get this installed, did you download the source, cd into the directory and run cloudron build then cloudron install? I only once ever managed to install a non-official app but I totally forget how I did it!

                                                      One place I get stuck on conceptually is at https://docs.cloudron.io/custom-apps/tutorial/, where it tells us to build:

                                                      # build the app
                                                      $ docker build -t username/nodejs-app:1.0.0 .
                                                      

                                                      What in the world is "username" supposed to be? Just a bit earlier it tells us to git clone https://git.cloudron.io/cloudron/tutorial-nodejs-app, so why wouldn't the build section just use cloudron as the username? And if it isn't cloudron but myself, does this mean it is going to be put in a directory in the directory I'm building in? Anyway, this stuff confuses the heck out of me.

                                                      A life lived in fear is a life half-lived

                                                      timconsidine 1 Reply Last reply Reply Quote 0
                                                      • timconsidine
                                                        timconsidine App Dev @scooke last edited by timconsidine

                                                        @scooke No I didn't use cloudron build
                                                        My process is :

                                                        • install cloudron cli (but I guess you did this)
                                                        • have your own docker repository : this is what is meant by username. I think it is best described as reponame. I am not so sophisticated as to have my own private repository system. I just use the free account on https://hub.docker.com/
                                                        • install Docker Desktop for your platform
                                                        • on local machine create a directory e.g. /cloudron-custom/appname and change into the appname directory
                                                        • download a zip file of the app source and unzip into the appname directory
                                                        • make sure you have Docker Desktop running (which logs you into the free hub.docker account)
                                                        • then build using docker build -t <reponame>/<appname>:<tag> . NB my change of terminology, and remember the trailing dot
                                                        • if I understand correctly, tag is only to distinguish different builds of the app. Initially i just ignored these, but probably good practice. But they can be anything (date, codeword)
                                                        • the built app is now just on your local machine so get it into the reppository system you chose, e.g. hub.docker.com using docker push <reponame>/<appname>:<tag>
                                                        • then it is available to cloudron cli so cloudron install --image <reponame>/<appname>:<tag> will ask you what location you want (e.g. appname.yourdomain.com)
                                                        • generally it is seamless from there, watch the pretty dots and when finished it should be in your Cloudron dashboard, ready to set access permissions or resources if you don't want the default.

                                                        I guess this is not a sophisticated process, it has multiple steps, but I find that easier when something goes wrong to debug.

                                                        I think my build of Paperless-NG is available publicly at tcmbp132021/paperless-ng:derek210709
                                                        I've never tried it but maybe you could save some effort and try just doing cloudron install --image tcmbp132021/paperless-ng:derek210709
                                                        Maybe there will be some permission issues, but it says it is public. If you try that, let me know, it will be useful to know.

                                                        I hope that's not too granular for anybody, personally I just like having step-by-step instructions. Good luck and shout if you run into problems. You will succeed !

                                                        scooke 1 Reply Last reply Reply Quote 1
                                                        • scooke
                                                          scooke @timconsidine last edited by

                                                          @timconsidine This is super helpful. I guess another thing that confuses me is, if I understand this process, the hub.docker.com site is going to end up (or already is) jam-packed with 1001 Docker images from everyone and their grandmother. Seems ineffecient and confusing to me.

                                                          I'll try skipping to the install step with the filled in <reponame>:<appname> details you gave. Thanks!

                                                          A life lived in fear is a life half-lived

                                                          timconsidine 1 Reply Last reply Reply Quote 0
                                                          • timconsidine
                                                            timconsidine App Dev @scooke last edited by

                                                            @scooke I think that's Docker's problem !
                                                            No different to any other SaaS-style hosted service. Providing performance on your account is OK, it's probably nothing to worry about.

                                                            Or alternatively set up a private repository. Isn't there a Cloudron app already for that ?

                                                            I've not been brave enough / had enough time to do that yet.

                                                            1 Reply Last reply Reply Quote 0
                                                            • scooke
                                                              scooke last edited by

                                                              @doodlemania2 With the help of @timconsidine I have this installed also. Looks and runs smooth! Thank you!

                                                              A life lived in fear is a life half-lived

                                                              1 Reply Last reply Reply Quote 4
                                                              • Referenced by  neurokrish neurokrish 
                                                              • nebulon
                                                                nebulon Staff last edited by

                                                                So this app is published as unstable in the paperless-ng flavor, thanks to @doodlemania2 for the majority of the initial packaging!

                                                                The forum section for further feedback and issues is now https://forum.cloudron.io/category/137/paperless-ng so I will lock this topic.

                                                                1 Reply Last reply Reply Quote 8
                                                                • Locked by  nebulon nebulon 
                                                                • First post
                                                                  Last post
                                                                Powered by NodeBB