Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Mailpiler - self hosted email archive

    App Wishlist
    10
    51
    2445
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • luckow
      luckow translator @marcusquinn last edited by

      @marcusquinn open please 🙂

      Pronouns: he/him | Primary language: German

      1 Reply Last reply Reply Quote 1
      • timconsidine
        timconsidine @vladimir.d last edited by

        @vladimir-d Feedback :
        I tried building the app from the git address
        I got error as below

        => ERROR [ 6/10] ADD config/nginx /app/data/nginx                                                                              0.0s
        ------
         > [ 6/10] ADD config/nginx /app/data/nginx:
        ------
        failed to compute cache key: "/config/nginx" not found: not found
        

        I fixed this by creating a directory 'nginx' in 'mailpiler-master/conf' and copied the 'nginx.conf from its location couple layers below
        Building then continued correctly.
        Other errors found which I am investigating

        1 Reply Last reply Reply Quote 1
        • timconsidine
          timconsidine @vladimir.d last edited by timconsidine

          @vladimir-d Feedback on building :
          After sorting out nginx.conf location, there is long list of building information until :

          #5 86.05 dpkg: error processing archive piler_1.3.11-focal-5c2ceb1_amd64.deb (--install):
          #5 86.05  package architecture (amd64) does not match system (arm64)
          #5 86.05 Errors were encountered while processing:
          #5 86.05  piler_1.3.11-focal-5c2ceb1_amd64.deb
          ------
          executor failed running [/bin/sh -c apt-get update &&     apt-get -y --no-install-recommends install        wget openssl sysstat php7.4-cli php7.4-cgi php7.4-mysql php7.4-fpm php7.4-zip php7.4-ldap        php7.4-gd php7.4-curl php7.4-xml php7.4-memcached catdoc unrtf poppler-utils nginx tnef sudo libzip5        libtre5 cron libmariadb-dev mariadb-client-core-10.3 python3 python3-mysqldb ca-certificates curl 	   crudini vim net-tools supervisor memcached &&     apt-get clean &&     rm -rf /var/lib/apt/lists/* &&     wget --no-check-certificate -q -O ${SPHINX_BIN_TARGZ} ${DOWNLOAD_URL}/generic-local/${SPHINX_BIN_TARGZ} &&     tar zxvf ${SPHINX_BIN_TARGZ} &&     sed -i '/session    required     pam_loginuid.so/c\#session    required     pam_loginuid.so' /etc/pam.d/cron && 	wget --no-check-certificate -q -O ${PACKAGE} https://bitbucket.org/jsuto/piler/downloads/${PACKAGE} &&     dpkg -i ${PACKAGE} &&     ln -sf /etc/piler/piler-nginx.conf /etc/nginx/sites-enabled/ &&     rm -f ${PACKAGE} ${SPHINX_BIN_TARGZ} /etc/nginx/sites-enabled/default /etc/piler/piler.key /etc/piler/piler.pem /etc/piler/config-site.php]: exit code: 1
          

          I can't see what is wrong but will continue looking.
          With such long combined commands ("&&"), it is difficult to identify on which command it is failing.

          Oh, BTW, I am building locally on a Mac Big Sur and using my docker registry.
          But didn't get as far as pushing it there yet.

          timconsidine 1 Reply Last reply Reply Quote 1
          • timconsidine
            timconsidine @timconsidine last edited by timconsidine

            I edited the Dockerfile to split the original multiple command line (#14) into multiple RUN lines to identify clearly where is the problem.
            Seems like it is :

             => ERROR [ 9/19] RUN dpkg -i piler_1.3.11-focal-5c2ceb1_amd64.deb                                                              0.2s 
            ------                                                                                                                               
             > [ 9/19] RUN dpkg -i piler_1.3.11-focal-5c2ceb1_amd64.deb:                                                                         
            #13 0.222 dpkg: error processing archive piler_1.3.11-focal-5c2ceb1_amd64.deb (--install):                                           
            #13 0.222  package architecture (amd64) does not match system (arm64)
            #13 0.226 Errors were encountered while processing:
            #13 0.226  piler_1.3.11-focal-5c2ceb1_amd64.deb
            ------
            executor failed running [/bin/sh -c dpkg -i ${PACKAGE}]: exit code: 1
            
            

            Not sure what to do here. Will have to think.
            Although I guess it has to be this on line #9

            PACKAGE="piler_1.3.11-focal-5c2ceb1_amd64.deb" \
            
            timconsidine 1 Reply Last reply Reply Quote 1
            • timconsidine
              timconsidine @timconsidine last edited by

              Ah. Thinks.
              Building this on Mac Big Sur running on Mac Mini with new Apple chip (not Intel)
              Is that the cause of the architecture issue ?

              timconsidine 1 Reply Last reply Reply Quote 1
              • timconsidine
                timconsidine @timconsidine last edited by

                Using the following seems to get past the platform architecture issue :
                Only needed perhaps if building on Mac with Apple chip

                docker buildx build --platform linux/amd64 -f Dockerfile --no-cache . -t <reponame>/mailpiler:cloudron-<date>
                
                timconsidine 1 Reply Last reply Reply Quote 1
                • timconsidine
                  timconsidine @timconsidine last edited by

                  Build process failed (for me) on this line in #nginx section

                  ln -sf /etc/piler/piler-nginx.conf /etc/nginx/sites-enabled/
                  

                  But i noticed preceding similar command succeeded

                  ln -s /app/data/nginx/conf/sites-enabled /etc/nginx/sites-enabled 
                  

                  So I amended problem line by removing the trailing "/"
                  That build process now runs to completion.

                  I hope people don't mind the segregated comments and 'running commentary'. Personally I find it easier to track problems and solutions.

                  timconsidine 1 Reply Last reply Reply Quote 2
                  • timconsidine
                    timconsidine @timconsidine last edited by

                    Successfully built and pushed to repository (docker in my case).
                    App installed on Cloudron w/o hassle .... but hangs in 'starting' mode.
                    So I uninstalled and tried cloudron install --image <repo>/<image>:<tag> without the --no-wait option.
                    This installs but similarly gets stuck

                     => Wait for health check ........................................................................................................................................................................................................................................................................................................................................................^C
                    

                    Out of my depth now. What to check / amend ?

                    timconsidine 1 Reply Last reply Reply Quote 2
                    • timconsidine
                      timconsidine @timconsidine last edited by

                      Blundering around in the dark, but I notice that start.sh has on line 53

                      local SSL_CERT_DATA="/C=US/ST=Denial/L=Springfield/O=Dis/CN=www.example.com"
                      

                      Should this not reference the Cloudron variable for location ?

                      vladimir.d 1 Reply Last reply Reply Quote 2
                      • girish
                        girish Staff last edited by

                        @marcusquinn @vladimir-d what is the status of the app package?

                        marcusquinn vladimir.d 2 Replies Last reply Reply Quote 0
                        • marcusquinn
                          marcusquinn @girish last edited by

                          @girish We have 2 self-installed working instances, not given it a thorough test but can't see anything obvious that wouldn't work either. Could perhaps do with some more experienced eyes and feedback on it though.

                          We're not here for a long time - but we are here for a good time :)
                          Jersey/UK
                          Work & Ecommerce Advice: https://brandlight.org
                          Personal & Software Tips: https://marcusquinn.com

                          1 Reply Last reply Reply Quote 1
                          • vladimir.d
                            vladimir.d @timconsidine last edited by

                            @timconsidine I've updated Dockerfile to fix all building issues you faced with. After that everything went just fine on a fresh cloned copy.

                            @timconsidine said in Mailpiler - self hosted email archive:

                            Blundering around in the dark, but I notice that start.sh has on line 53

                            local SSL_CERT_DATA="/C=US/ST=Denial/L=Springfield/O=Dis/CN=www.example.com"
                            

                            Should this not reference the Cloudron variable for location ?

                            When tls add-on is enabled in the manifest, the script doesn't generate a self signed certificate and uses the Cloudron certificate of the primary domain. I've amended it to use $CLOUDRON_APP_DOMAIN variable.

                            @timconsidine said in Mailpiler - self hosted email archive:

                            Successfully built and pushed to repository (docker in my case).
                            App installed on Cloudron w/o hassle .... but hangs in 'starting' mode.
                            So I uninstalled and tried cloudron install --image <repo>/<image>:<tag> without the --no-wait option.
                            This installs but similarly gets stuck

                             => Wait for health check ............................^C
                            

                            Out of my depth now. What to check / amend ?

                            It doesn't get stuck for me - I've tried to build&install the app with the recent changes.

                            I guess there is an issue with piler or nginx directories in /etc/, probably a wrong path or an invalid symlink or so.
                            You need to look at the app logs to see the issue.
                            Logs button in the app config console

                            timconsidine 2 Replies Last reply Reply Quote 2
                            • timconsidine
                              timconsidine @vladimir.d last edited by

                              @vladimir-d thank you !
                              I will give it another go in the morning.

                              1 Reply Last reply Reply Quote 0
                              • vladimir.d
                                vladimir.d @girish last edited by vladimir.d

                                @girish It needs to expose 25 port for piler-smtp daemon. it says 25 (and other smtp ports) are reserved.
                                For now I've added tcpPorts field to Cloudronmanifest.json and exposed it as 12525 but I guess it won't properly work in most cases.
                                Is there a workaround to expose 25 port for the app?

                                girish 1 Reply Last reply Reply Quote 1
                                • girish
                                  girish Staff @vladimir.d last edited by

                                  @vladimir-d Oh, interesting. Don't think an app can ever expose port 25 since the mail server requires port 25. Do you know why it requires port 25? Maybe I don't understand how the archiver works. I though it's pulling in all email via IMAP? Or is it that we should redirect all mail to mailpiler?

                                  luckow 1 Reply Last reply Reply Quote 1
                                  • luckow
                                    luckow translator @girish last edited by

                                    @girish According to my understanding of a mail archive solution, every incoming and outgoing mail must be forwarded to the archive solution. This is the only way to fulfil the requirements of the law. https://en.wikipedia.org/wiki/Email_archiving
                                    The mailpiler docs tells us:
                                    Postfix
                                    Add the following to main.cf then issue the postfix reload command:

                                    always_bcc = uuid@smtp.example.com

                                    For historical mails there is the import option
                                    https://docs.google.com/document/d/15F0fyb07etMqEXRJnMZcYc016UN-WDds-6zrZNjF6aU/edit#heading=h.58uixy8guf9y

                                    https://docs.google.com/document/d/1YK7zVbcohFWf2w8BpYhCcsAY6Q61JKXa8pqUDrU7nn4/edit#

                                    Both docs are for the enterprise edition. But the oss edition has similar docs.
                                    https://www.mailpiler.org/wiki/current:installation

                                    Pronouns: he/him | Primary language: German

                                    timconsidine 1 Reply Last reply Reply Quote 1
                                    • timconsidine
                                      timconsidine @luckow last edited by

                                      @luckow this makes sense ... except I don't understand why an archive solution would need SMTP for sending. It just needs its own incoming email capability, and the always_bcc means it will get a copy of all inbound and outbound emails from the primary 'working' email server.
                                      But maybe (as is often the case) I am not understanding the full picture.

                                      luckow 1 Reply Last reply Reply Quote 0
                                      • luckow
                                        luckow translator @timconsidine last edited by

                                        @timconsidine as I understand it, the smtp port of the archive solution is only relevant for receiving the bcc mails. The user uses the "normal" (in this case the Cloudron) smtp server for sending emails via webmail or the local mail client. And Cloudron smtp forwards/sends any incoming or outgoing mail via smtp to the archive solution. That is the magic in my understanding 🙂

                                        Pronouns: he/him | Primary language: German

                                        timconsidine 1 Reply Last reply Reply Quote 0
                                        • timconsidine
                                          timconsidine @luckow last edited by

                                          @luckow thank you
                                          I just didn't know inbound can even be done via SMTP. Either my bad knowledge, or non-standard config.

                                          luckow 1 Reply Last reply Reply Quote 0
                                          • luckow
                                            luckow translator @timconsidine last edited by

                                            @timconsidine worth reading https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol 😉

                                            Pronouns: he/him | Primary language: German

                                            1 Reply Last reply Reply Quote 1
                                            • timconsidine
                                              timconsidine @vladimir.d last edited by timconsidine

                                              @vladimir-d whoop whoop ! 🍾
                                              Just built, pushed and installed your updated package of Mailpiler.
                                              All went smoothly.
                                              Now I have to set it up and learn it.
                                              Thank you so much ! 👍

                                              marcusquinn 1 Reply Last reply Reply Quote 4
                                              • marcusquinn
                                                marcusquinn @timconsidine last edited by

                                                @timconsidine Nice, more the merrier!

                                                It's definitely another app that can help win mindshare for Cloudron as an enterprise considerate solution with this sort of auditing. Especially when it can be setup as a separate Cloudron instance too for further risk spreading.

                                                We're not here for a long time - but we are here for a good time :)
                                                Jersey/UK
                                                Work & Ecommerce Advice: https://brandlight.org
                                                Personal & Software Tips: https://marcusquinn.com

                                                timconsidine 1 Reply Last reply Reply Quote 3
                                                • timconsidine
                                                  timconsidine @marcusquinn last edited by

                                                  @marcusquinn agreed.
                                                  I totally agree. Have tried some archiving solutions previously and mailpiler looks nice.
                                                  But I having a "thick" morning :

                                                  • looking through app and their online docs, I can't what address emails should be BCC'ed to. Is it staring me in the face and I can't see it? Do I have to set it up ?

                                                  • importing from a server doesn't specify a port number to access IMAP on, so test connection always fails, and it doesn't specify SSL/STARTTLS

                                                  I'm embarrassed by these basic questions, but if someone can nudge me in the right direction, I'd appreciate it.

                                                  marcusquinn 1 Reply Last reply Reply Quote 0
                                                  • marcusquinn
                                                    marcusquinn @timconsidine last edited by

                                                    @timconsidine We haven't got to out setup and migration yet, working on 6 things at once, as always! Will bear in mind and answer when we do get to it.

                                                    We're not here for a long time - but we are here for a good time :)
                                                    Jersey/UK
                                                    Work & Ecommerce Advice: https://brandlight.org
                                                    Personal & Software Tips: https://marcusquinn.com

                                                    timconsidine 1 Reply Last reply Reply Quote 1
                                                    • timconsidine
                                                      timconsidine @marcusquinn last edited by

                                                      @marcusquinn no problem, stumbling through it
                                                      Testing some configs
                                                      Another Q for later : what is a "mapped domain" when setting up a domain? Can't find any references to it.
                                                      No rush.

                                                      1 Reply Last reply Reply Quote 0
                                                      • marcusquinn
                                                        marcusquinn last edited by

                                                        @jsuto You still with us here? Maybe someone can email him if the forum doesn't?

                                                        We're not here for a long time - but we are here for a good time :)
                                                        Jersey/UK
                                                        Work & Ecommerce Advice: https://brandlight.org
                                                        Personal & Software Tips: https://marcusquinn.com

                                                        necrevistonnezr 1 Reply Last reply Reply Quote 0
                                                        • necrevistonnezr
                                                          necrevistonnezr @marcusquinn last edited by

                                                          @marcusquinn
                                                          BTW does Mailpiler have any sort of compression? I recently tried to switch from the gold standard MailStore Home to another solution (in an attempt to ditch my Windows VM for good) - and my 30 GB MailStore Archive went up to 110 GB(!) in mbox files!

                                                          marcusquinn 1 Reply Last reply Reply Quote 1
                                                          • marcusquinn
                                                            marcusquinn @necrevistonnezr last edited by

                                                            @necrevistonnezr Don't know yet but will post some stats when I get to it. The project I have with it is for years of company email archives, so will have some good stats to compare from source to destination.

                                                            I might research file-system level deduplication, as I imagine there's a lot of duplicate files from email threads.

                                                            We're not here for a long time - but we are here for a good time :)
                                                            Jersey/UK
                                                            Work & Ecommerce Advice: https://brandlight.org
                                                            Personal & Software Tips: https://marcusquinn.com

                                                            necrevistonnezr 1 Reply Last reply Reply Quote 0
                                                            • M
                                                              mazarian last edited by

                                                              Just thought I'd chime in and say I'm very interested in a mail archive solution.

                                                              1 Reply Last reply Reply Quote 4
                                                              • Referenced by  luckow luckow 
                                                              • necrevistonnezr
                                                                necrevistonnezr @marcusquinn last edited by

                                                                @marcusquinn said in Mailpiler - self hosted email archive:

                                                                @necrevistonnezr Don't know yet but will post some stats when I get to it. The project I have with it is for years of company email archives, so will have some good stats to compare from source to destination.

                                                                I might research file-system level deduplication, as I imagine there's a lot of duplicate files from email threads.

                                                                Is this still in the works or in a beta stage to be installed?

                                                                marcusquinn 1 Reply Last reply Reply Quote 1
                                                                • Referenced by  necrevistonnezr necrevistonnezr 
                                                                • marcusquinn
                                                                  marcusquinn @necrevistonnezr last edited by

                                                                  @necrevistonnezr Hey, we have it working, need to ask @vladimir-d to commit to Cloudron's GitLab and let the team verify it's good to go for App Store inclusion.

                                                                  We're not here for a long time - but we are here for a good time :)
                                                                  Jersey/UK
                                                                  Work & Ecommerce Advice: https://brandlight.org
                                                                  Personal & Software Tips: https://marcusquinn.com

                                                                  1 Reply Last reply Reply Quote 5
                                                                  • Referenced by  necrevistonnezr necrevistonnezr 
                                                                  • Referenced by  scooke scooke 
                                                                  • N
                                                                    Neiluj last edited by

                                                                    Hi all,
                                                                    I am wondering if we are far away from an addition of mailpiler to the Cloudron app repository. It would be of great help in a couple of the situation I am facing atm.
                                                                    @vladimir-d Maybe you have had the chance to commit the working app version to Clourdon's GitLab already

                                                                    If I can help (within the remit of my abilities) or contribute in making this happen, I would be happy to do so.

                                                                    1 Reply Last reply Reply Quote 4
                                                                    • girish
                                                                      girish Staff last edited by

                                                                      I am happy to include this if we have something basic going . @marcusquinn @vladimir-d do you have any initial work here ?

                                                                      vladimir.d 1 Reply Last reply Reply Quote 1
                                                                      • vladimir.d
                                                                        vladimir.d @girish last edited by vladimir.d

                                                                        @girish here is some initial work.
                                                                        https://git.cloudron.io/vladimir.d/mailpiler

                                                                        TBH I didn't have a chance to complete it yet, I think the issue described here still needs to be addressed.

                                                                        1 Reply Last reply Reply Quote 1
                                                                        • First post
                                                                          Last post
                                                                        Powered by NodeBB