Mailpiler - self hosted email archive

[jsuto]

@marcusquinn Well, for the moment I'm not packaging it. First, I wanted to discover if there's an interest in the project as a cloudron app. Anyway, I'm new to cloudron, so I guess the community could do it much faster.

[marcusquinn]

@jsuto I'm interested if clients are, it's certainly a feature Cloudron doesn't already cover, lends itself better to being an app for multi-provider mail archiving, and is important for business compliance expectations.

Most are on either Microsoft Exchange or Google Vault for these things. Difficult to cost-compare when they obfuscate feature pricing by bundling but It certainly looks like you have a capable alternative.

For interest, I tested the Demo, and have the localcdn.org extension installed by default, so had to enable "HTML Filtering" for the UI to work, maybe something to look at, although I admit I'm a less common setup as most users aren't concerned about CDN privacy.

[jsuto]

@marcusquinn Er, I'm confused. What should I fix to make the localcdn plugin happy?

[marcusquinn]

@jsuto Not checked but it's probably something referenced to pull from a CDN than your own server.

Anything with one of these URLs in the code would be what that extension is designed to use a local version of:

ajax.googleapis.com
ajax.aspnetcdn.com
ajax.microsoft.com
cdnjs.cloudflare.com
code.jquery.com
cdn.jsdelivr.net
fonts.googleapis.com
yastatic.net
yandex.st
apps.bdimg.com
libs.baidu.com
cdn.staticfile.org
cdn.bootcss.com
mat1.gtimg.com
lib.sinaapp.com
upcdn.b0.upaiyun.com
stackpath.bootstrapcdn.com
maxcdn.bootstrapcdn.com
netdna.bootstrapcdn.com
use.fontawesome.com
ajax.cloudflare.com
akamai-webcdn.kgstatic.net
gitcdn.github.io
vjs.zencdn.net
cdn.plyr.io
sdn.geekzu.org
ajax.proxy.ustclug.org
unpkg.com
pagecdn.io
cdn.css.net
cdnjs.loli.net
ajax.loli.net
fonts.loli.net
lib.baomitu.com
cdn.bootcdn.net

Not a big deal as anyone using that extension would probabaly know to try again with it bypassed ("HTML Filtering"), just for your awareness really.

[marcusquinn]

@vladimir-d is working on packaging this for us all.

Expressions of interested and testing volunteers welcome.

[timconsidine]

@marcusquinn I'd be interested, if I can help

[vladimir.d]

Mailpiler app for cloudron is here, built based on the original docker configuration.
https://git.cloudron.io/vladimir.d/mailpiler

Comments are very welcome.

[marcusquinn]

@jsuto You still with us? Want to give this a spin?

I have a couple of questions on the app, shall we discuss in the open here for the benefit of the community that I'm sure will enjoy championing this addition from your great work for you.

[luckow]

@marcusquinn open please

[timconsidine]

@vladimir-d Feedback :
I tried building the app from the git address
I got error as below

=> ERROR [ 6/10] ADD config/nginx /app/data/nginx                                                                              0.0s
------
 > [ 6/10] ADD config/nginx /app/data/nginx:
------
failed to compute cache key: "/config/nginx" not found: not found

I fixed this by creating a directory 'nginx' in 'mailpiler-master/conf' and copied the 'nginx.conf from its location couple layers below
Building then continued correctly.
Other errors found which I am investigating

[timconsidine]

@vladimir-d Feedback on building :
After sorting out nginx.conf location, there is long list of building information until :

#5 86.05 dpkg: error processing archive piler_1.3.11-focal-5c2ceb1_amd64.deb (--install):
#5 86.05  package architecture (amd64) does not match system (arm64)
#5 86.05 Errors were encountered while processing:
#5 86.05  piler_1.3.11-focal-5c2ceb1_amd64.deb
------
executor failed running [/bin/sh -c apt-get update &&     apt-get -y --no-install-recommends install        wget openssl sysstat php7.4-cli php7.4-cgi php7.4-mysql php7.4-fpm php7.4-zip php7.4-ldap        php7.4-gd php7.4-curl php7.4-xml php7.4-memcached catdoc unrtf poppler-utils nginx tnef sudo libzip5        libtre5 cron libmariadb-dev mariadb-client-core-10.3 python3 python3-mysqldb ca-certificates curl 	   crudini vim net-tools supervisor memcached &&     apt-get clean &&     rm -rf /var/lib/apt/lists/* &&     wget --no-check-certificate -q -O ${SPHINX_BIN_TARGZ} ${DOWNLOAD_URL}/generic-local/${SPHINX_BIN_TARGZ} &&     tar zxvf ${SPHINX_BIN_TARGZ} &&     sed -i '/session    required     pam_loginuid.so/c\#session    required     pam_loginuid.so' /etc/pam.d/cron && 	wget --no-check-certificate -q -O ${PACKAGE} https://bitbucket.org/jsuto/piler/downloads/${PACKAGE} &&     dpkg -i ${PACKAGE} &&     ln -sf /etc/piler/piler-nginx.conf /etc/nginx/sites-enabled/ &&     rm -f ${PACKAGE} ${SPHINX_BIN_TARGZ} /etc/nginx/sites-enabled/default /etc/piler/piler.key /etc/piler/piler.pem /etc/piler/config-site.php]: exit code: 1

I can't see what is wrong but will continue looking.
With such long combined commands ("&&"), it is difficult to identify on which command it is failing.

Oh, BTW, I am building locally on a Mac Big Sur and using my docker registry.
But didn't get as far as pushing it there yet.

[timconsidine]

I edited the Dockerfile to split the original multiple command line (#14) into multiple RUN lines to identify clearly where is the problem.
Seems like it is :

 => ERROR [ 9/19] RUN dpkg -i piler_1.3.11-focal-5c2ceb1_amd64.deb                                                              0.2s 
------                                                                                                                               
 > [ 9/19] RUN dpkg -i piler_1.3.11-focal-5c2ceb1_amd64.deb:                                                                         
#13 0.222 dpkg: error processing archive piler_1.3.11-focal-5c2ceb1_amd64.deb (--install):                                           
#13 0.222  package architecture (amd64) does not match system (arm64)
#13 0.226 Errors were encountered while processing:
#13 0.226  piler_1.3.11-focal-5c2ceb1_amd64.deb
------
executor failed running [/bin/sh -c dpkg -i ${PACKAGE}]: exit code: 1

Not sure what to do here. Will have to think.
Although I guess it has to be this on line #9

PACKAGE="piler_1.3.11-focal-5c2ceb1_amd64.deb" \

[timconsidine]

Ah. Thinks.
Building this on Mac Big Sur running on Mac Mini with new Apple chip (not Intel)
Is that the cause of the architecture issue ?

[timconsidine]

Using the following seems to get past the platform architecture issue :
Only needed perhaps if building on Mac with Apple chip

docker buildx build --platform linux/amd64 -f Dockerfile --no-cache . -t <reponame>/mailpiler:cloudron-<date>

[timconsidine]

Build process failed (for me) on this line in #nginx section

ln -sf /etc/piler/piler-nginx.conf /etc/nginx/sites-enabled/

But i noticed preceding similar command succeeded

ln -s /app/data/nginx/conf/sites-enabled /etc/nginx/sites-enabled 

So I amended problem line by removing the trailing "/"
That build process now runs to completion.

I hope people don't mind the segregated comments and 'running commentary'. Personally I find it easier to track problems and solutions.

[timconsidine]

Successfully built and pushed to repository (docker in my case).
App installed on Cloudron w/o hassle .... but hangs in 'starting' mode.
So I uninstalled and tried cloudron install --image <repo>/<image>:<tag> without the --no-wait option.
This installs but similarly gets stuck

 => Wait for health check ........................................................................................................................................................................................................................................................................................................................................................^C

Out of my depth now. What to check / amend ?

[timconsidine]

Blundering around in the dark, but I notice that start.sh has on line 53

local SSL_CERT_DATA="/C=US/ST=Denial/L=Springfield/O=Dis/CN=www.example.com"

Should this not reference the Cloudron variable for location ?

[girish]

@marcusquinn @vladimir-d what is the status of the app package?

[marcusquinn]

@girish We have 2 self-installed working instances, not given it a thorough test but can't see anything obvious that wouldn't work either. Could perhaps do with some more experienced eyes and feedback on it though.

[vladimir.d]

@timconsidine I've updated Dockerfile to fix all building issues you faced with. After that everything went just fine on a fresh cloned copy.

@timconsidine said in Mailpiler - self hosted email archive:

Blundering around in the dark, but I notice that start.sh has on line 53

local SSL_CERT_DATA="/C=US/ST=Denial/L=Springfield/O=Dis/CN=www.example.com"

Should this not reference the Cloudron variable for location ?

When tls add-on is enabled in the manifest, the script doesn't generate a self signed certificate and uses the Cloudron certificate of the primary domain. I've amended it to use $CLOUDRON_APP_DOMAIN variable.

@timconsidine said in Mailpiler - self hosted email archive:

Successfully built and pushed to repository (docker in my case).
App installed on Cloudron w/o hassle .... but hangs in 'starting' mode.
So I uninstalled and tried cloudron install --image <repo>/<image>:<tag> without the --no-wait option.
This installs but similarly gets stuck

 => Wait for health check ............................^C

Out of my depth now. What to check / amend ?

It doesn't get stuck for me - I've tried to build&install the app with the recent changes.

I guess there is an issue with piler or nginx directories in /etc/, probably a wrong path or an invalid symlink or so.
You need to look at the app logs to see the issue.