Zabbix - Appdev Log

[BrutalBirdie]

Will post my findings and question here.

https://forum.cloudron.io/topic/1211/zabbix-network-monitoring-solution?_=1603281202737

Cloudron Gitlab:
MySQL Version: https://git.cloudron.io/BrutalBirdie/zabbix-app
PostgreSQL Version: https://git.cloudron.io/BrutalBirdie/zabbix-app/-/tree/feature/pgsql

[BrutalBirdie]

So far so good.
The guys at Zabbix did a very good job.

https://github.com/zabbix/zabbix-docker/tree/5.0/server-mysql/ubuntu

I will try to stay as close the the original Dockerfile and docker-entrypoint.sh to make it easier for future updates.

Also here their docker hub: https://hub.docker.com/r/zabbix/zabbix-server-mysql

They have this section: "Allowed volumes for the Zabbix server container"

Which should be all folders that need to be in /app/data/ including the /etc/zabbix/ folder since it has important configuration.

All the variables mentioned will be in a sample.env file and copied to /app/data/.env and the /app/data/zabbix/config/zabbix_server.conf will be auto generated by the docker-entrypoint.sh

[BrutalBirdie]

Error

2020-10-21T11:45:32.000Z + zcat /usr/share/doc/zabbix-server-mysql/create.sql.gz
2020-10-21T11:45:32.000Z ERROR 1071 (42000) at line 348: Specified key was too long; max key length is 3072 bytes

The create.sql excerpt of given error:

348: CREATE INDEX `items_1` ON `items` (`hostid`,`key_`(1021));

Makes no sense to me since 1021<=3072

Someone had this issue before:

https://www.zabbix.com/forum/zabbix-troubleshooting-and-problems/401826-database-upgrade-failed-specified-key-was-too-long-max-key-length-is-3072-bytes

TL;DR

I fixed my issue !

The solution is to convert every tables characters to utf8 and collation to utf8_bin. Not only the database default.

To do so I followed this method :
Code:
SELECT CONCAT('ALTER TABLE ',TABLE_SCHEMA,'.',TABLE_NAME,' CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;') FROM information_schema.TABLES WHERE TABLE_SCHEMA = 'databasename';

Another suggested fix was this

For me, the resolution was to upgrade the MySQL version from 5.6 to 5.8. After that, the database was able to upgrade.

Could this be a Cloudron MySQL Issue @appdev ?

[Lonkle]

Have you been able to try either of the suggestions you found? Is this an alternative to PHPMYAdmin or an alternative to MySQL?

[BrutalBirdie]

@Lonk said in Zabbix - Appdev Log:

Have you been able to try either of the suggestions you found? Is this an alternative to PHPMYAdmin or an alternative to MySQL?

Since this is the initial setup for zabbix, I would rather like it to work as intended instead of doing hotfixes in the initial installation

Feels just wrong.

Another Idea would be to switch to postgres instead of mysql.

https://github.com/zabbix/zabbix-docker/tree/5.0/server-pgsql/ubuntu

Postgres may be even the better option... hmm

@Lonk said in Zabbix - Appdev Log:

Have you been able to try either of the suggestions you found? Is this an alternative to PHPMYAdmin or an alternative to MySQL?

Wait? What? Sorry just re-read your question.

Is this an alternative to PHPMYAdmin or an alternative to MySQL?

You mean, is Zabbix is an alternative to PHPMYAdmin or an alternative to MySQL?

Answer is no see https://www.zabbix.com/

[BrutalBirdie]

@ruihildt
Since you just liked/upvoted.

Would you prefer postgres over mysql?

[ruihildt]

@BrutalBirdie It seems outside historical LAMP apps, most of the time Postgres is chosen as the more mature and stable RDBMS.

This is even not taking into account Mysql belonging to Oracle, which has spun of the MariaDB fork.

So yeah, I think you can't go wrong by going with Postgres instead of Mysql.

[Lonkle]

Sorry I just meant is there a way to use an alternative database protocol, but I try to keep original open source code as close to vanilla as possible for updates too. I’ll look into this again. Did you just start packaging it for Cloudron and the Cloudron SQL service is giving you issues?

[BrutalBirdie]

Next Issue now with PostgreSQL

 /usr/sbin/zabbix_server --foreground -c /app/data/zabbix/config/zabbix_server.conf
/usr/sbin/zabbix_server: error while loading shared libraries: libnetsnmp.so.30: cannot open shared object file: No such file or directory

Should be this package, but not existent in cloudron/base:2.0.0 image

libsnmp35

Edit - very simple why libsnmp35 is for focal Ubuntu 20.04 and Cloudron is not based on focal

https://packages.ubuntu.com/search?suite=default&section=all&arch=any&keywords=libsnmp&searchon=names

[BrutalBirdie]

@Lonk said in Zabbix - Appdev Log:

Sorry I just meant is there a way to use an alternative database protocol, but I try to keep original open source code as close to vanilla as possible for updates too. I’ll look into this again. Did you just start packaging it for Cloudron and the Cloudron SQL service is giving you issues?

MySQL and PostgreSQL are supported.

https://www.zabbix.com/download?zabbix=5.0&os_distribution=ubuntu&os_version=18.04_bionic&db=postgresql&ws=nginx

edit:
From Zabbix Docs - https://www.zabbix.com/documentation/current/manual/installation/requirements

MySQL	5.5.62 - 8.0.x	Required if MySQL is used as Zabbix backend database. InnoDB engine is required.
MariaDB (10.0.37 or later) also works with Zabbix.
We recommend using the MariaDB Connector/C library for building server/proxy regardless of whether you use a MySQL or MariaDB database server.

[ruihildt]

@BrutalBirdie Zabbix proposes specific build steps for different Ubuntu versions no? If so, changing the Ubuntu target should fix that error.

[BrutalBirdie]

@ruihildt said in Zabbix - Appdev Log:

@BrutalBirdie Zabbix proposes specific build steps for different Ubuntu versions no? If so, changing the Ubuntu target should fix that error.

Yes but sadly their Docker based setups are all based on Ubuntu:focal, see ->
https://github.com/zabbix/zabbix-docker/blob/5.0/server-mysql/ubuntu/Dockerfile

[Lonkle]

@BrutalBirdie said in Zabbix - Appdev Log:

@ruihildt said in Zabbix - Appdev Log:

@BrutalBirdie Zabbix proposes specific build steps for different Ubuntu versions no? If so, changing the Ubuntu target should fix that error.

Yes but sadly their Docker based setups are all based on Ubuntu:focal, see ->
https://github.com/zabbix/zabbix-docker/blob/5.0/server-mysql/ubuntu/Dockerfile

I wonder if they’re based on focal because that’s the latest or if they actually need features from focal. Maybe bionic would work as a target without any changes?

[BrutalBirdie]

@Lonk The 4.4 Version is based on Bionic

https://github.com/zabbix/zabbix-docker/blob/4.4/server-mysql/ubuntu/Dockerfile

I will have a look what the diff is compared to focal.

Edit - the answer is not much.

View a of a nice diff.
https://github.com/zabbix/zabbix-docker/compare/5.0...BrutalBirdie:feature/cloudron

Keep this local diff - because I will delete my fork later.

1c1
- FROM ubuntu:bionic
---
+ FROM cloudron/base:2.0.0@sha256:f9fea80513aa7c92fe2e7bf3978b54c8ac5222f47a9a32a7f8833edf0eb5a4f4
12,13d11
- ENV TINI_VERSION=v0.19.0
- 
15,22c13,21
-     addgroup --system --gid 1995 --quiet zabbix && \
-     adduser --quiet \
-             --system --disabled-login \
-             --ingroup zabbix --ingroup root \
-             --uid 1997 \
-             --home /var/lib/zabbix/ \
-         zabbix && \
-     usermod -G zabbix,dialout zabbix && \
---
+     # Not needed for cloudron
+     #addgroup --system --gid 1995 --quiet zabbix && \
+     #adduser --quiet \
+     #        --system --disabled-login \
+     #        --ingroup zabbix --ingroup root \
+     #        --uid 1997 \
+     #        --home /var/lib/zabbix/ \
+     #    zabbix && \
+     #usermod -G zabbix,dialout zabbix && \
40c39,41
-             curl \
---
+             # Cloudron changes - Not needed
+             #tini \
+             tzdata \
42,44d42
-             gpg \
-             dirmngr \
-             gpg-agent \
49a48
+             # Cloudron changes - Use right version for Ubuntu 18.04 LTS
52a52
+             # Cloudron changes - Use right version for Ubuntu 18.04 LTS
60,74d59
-     curl -L "https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini" -o /sbin/tini && \
-     curl -L "https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini.asc" -o /tmp/tini.asc && \
-     export GNUPGHOME="$(mktemp -d)" && \
-     for server in $(shuf -e ha.pool.sks-keyservers.net \
-                             hkp://p80.pool.sks-keyservers.net:80 \
-                             ipv4.pool.sks-keyservers.net \
-                             keyserver.ubuntu.com \
-                             keyserver.pgp.com \
-                             pgp.mit.edu) ; do \
-         gpg --keyserver "$server" --recv-keys 595E85A6B1B4779EA4DAAEC70B588DFF0527A9B7 && break || : ; \
-     done && \
-     gpg --batch --verify /tmp/tini.asc /sbin/tini && \
-     rm -r "$GNUPGHOME" /tmp/tini.asc && \
-     chmod +x /sbin/tini && \
-     apt-get -y purge curl gpg dirmngr gpg-agent && \
79,80c64,65
- ARG MAJOR_VERSION=4.4
- ARG ZBX_VERSION=${MAJOR_VERSION}.10
---
+ ARG MAJOR_VERSION=5.0
+ ARG ZBX_VERSION=${MAJOR_VERSION}.4
148c133,134
-     chown --quiet -R zabbix:root /etc/zabbix/ /var/lib/zabbix/ && \
---
+     # Cloudron changes 
+     chown --quiet -R cloudron:cloudron /etc/zabbix/ /var/lib/zabbix/ && \
172,174c158,159
- EXPOSE 10051/TCP
- 
- WORKDIR /var/lib/zabbix
---
+ # Cloudron changes - Comment out all we don't need
+ # EXPOSE 10051/TCP
176c161
- VOLUME ["/var/lib/zabbix/snmptraps", "/var/lib/zabbix/export"]
---
+ WORKDIR /app/code
178c163
- COPY ["docker-entrypoint.sh", "/usr/bin/"]
---
+ # VOLUME ["/var/lib/zabbix/snmptraps", "/var/lib/zabbix/export"]
180c165
- ENTRYPOINT ["/sbin/tini", "--", "/usr/bin/docker-entrypoint.sh"]
---
+ # COPY ["docker-entrypoint.sh", "/usr/bin/"]
182c167,209
- USER 1997
---
+ # ENTRYPOINT ["/usr/bin/tini", "--", "/usr/bin/docker-entrypoint.sh"]
+ 
+ # USER 1997
+ 
+ # CMD ["/usr/sbin/zabbix_server", "--foreground", "-c", "/etc/zabbix/zabbix_server.conf"]
+ COPY ["docker-entrypoint.sh", "sample.env", "/app/code/"]
+ 
+ # Cloudron - make all allowed volumes avaible in /app/data/ - see https://hub.docker.com/r/zabbix/zabbix-server-mysql > "Allowed volumes for the Zabbix server container"
+ RUN mv /usr/lib/zabbix/alertscripts /usr/lib/zabbix/alertscripts_default \
+     && ln -s /app/data/zabbix/alertscripts /usr/lib/zabbix/alertscripts \
+     #
+     && mv /usr/lib/zabbix/externalscripts /usr/lib/zabbix/externalscripts_default \
+     && ln -s /app/data/zabbix/externalscripts /usr/lib/zabbix/externalscripts \
+     #
+     && mv /var/lib/zabbix/modules /var/lib/zabbix/modules_default \
+     && ln -s /app/data/zabbix/modules /var/lib/zabbix/modules \
+     #
+     && mv /var/lib/zabbix/enc /var/lib/zabbix/enc_default \
+     && ln -s /app/data/zabbix/enc /var/lib/zabbix/enc \
+     #
+     && mv /var/lib/zabbix/ssh_keys /var/lib/zabbix/ssh_keys_default \
+     && ln -s /app/data/zabbix/ssh_keys /var/lib/zabbix/ssh_keys \
+     #
+     && mv /var/lib/zabbix/ssl/certs /var/lib/zabbix/ssl/certs_default \
+     && ln -s /app/data/zabbix/ssl/certs /var/lib/zabbix/ssl/certs \
+     #
+     && mv /var/lib/zabbix/ssl/keys /var/lib/zabbix/ssl/keys_default \
+     && ln -s /app/data/zabbix/ssl/keys /var/lib/zabbix/ssl/keys \
+     #
+     && mv /var/lib/zabbix/ssl/ssl_ca /var/lib/zabbix/ssl/ssl_ca_default \
+     && ln -s /app/data/zabbix/ssl/ssl_ca /var/lib/zabbix/ssl/ssl_ca \
+     #
+     && mv /var/lib/zabbix/snmptraps /var/lib/zabbix/snmptraps_default \
+     && ln -s /app/data/zabbix/snmptraps /var/lib/zabbix/snmptraps \
+     #
+     && mv /var/lib/zabbix/mibs /var/lib/zabbix/mibs_default \
+     && ln -s /app/data/zabbix/mibs /var/lib/zabbix/mibs \
+     #
+     && mv /var/lib/zabbix/export /var/lib/zabbix/export_default \
+     && ln -s /app/data/zabbix/export /var/lib/zabbix/export \
+     #
+     && mv /etc/zabbix /etc/zabbix_default \
+     && ln -s /app/data/zabbix/config /etc/zabbix
184c211
- CMD ["/usr/sbin/zabbix_server", "--foreground", "-c", "/etc/zabbix/zabbix_server.conf"]
\ Kein Zeilenumbruch am Dateiende.
---
+ CMD ["/app/code/docker-entrypoint.sh"]
\ Kein Zeilenumbruch am Dateiende.

My Dockerfile for version 5.0.4:

• https://git.cloudron.io/BrutalBirdie/zabbix-app/-/blob/master/Dockerfile

Dockerfile for 4.4:

• https://github.com/zabbix/zabbix-docker/blob/4.4/server-mysql/ubuntu/Dockerfile

[Lonkle]

@BrutalBirdie Perfect!

So, if you change:

FROM ubuntu:focal
to
FROM ubuntu:bionic

Does Docker itself still build and run it correctly? Cause that would be promising.

[BrutalBirdie]

@Lonk said in Zabbix - Appdev Log:

@BrutalBirdie Perfect!

So, if you change:

FROM ubuntu:focal
to
FROM ubuntu:bionic

Does Docker itself still build and run it correctly? Cause that would be promising.

Did not try that, but can try.

btw.
I refuse to provide ugly diffs so I abused github now

https://github.com/zabbix/zabbix-docker/compare/5.0...BrutalBirdie:feature/cloudron

[BrutalBirdie]

@Lonk said in Zabbix - Appdev Log:

@BrutalBirdie Perfect!

So, if you change:

FROM ubuntu:focal
to
FROM ubuntu:bionic

Does Docker itself still build and run it correctly? Cause that would be promising.

To build the official Dockerfile if I change to bionic these changes must be made for a successful build.

diff --git a/server-mysql/ubuntu/Dockerfile b/server-mysql/ubuntu/Dockerfile
index 421bceae..f604801a 100644
--- a/server-mysql/ubuntu/Dockerfile
+++ b/server-mysql/ubuntu/Dockerfile
@@ -1,4 +1,4 @@
-FROM ubuntu:focal
+FROM ubuntu:bionic
 
 LABEL org.opencontainers.image.title="Zabbix server (MySQL)" \
       org.opencontainers.image.authors="Alexey Pustovalov <alexey.pustovalov@zabbix.com>" \
@@ -35,7 +35,7 @@ RUN set -eux && \
     mkdir -p /usr/share/doc/zabbix-server-mysql && \
     apt-get -y update && \
     DEBIAN_FRONTEND=noninteractive apt-get -y --no-install-recommends install \
-            tini \
+            # tini \
             tzdata \
             ca-certificates \
             iputils-ping \
@@ -43,10 +43,10 @@ RUN set -eux && \
             fping \
             libcurl4 \
             libevent-2.1 \
-            libmysqlclient21 \
+            libmysqlclient20 \
             libopenipmi0 \
             libpcre3 \
-            libsnmp35 \
+            libsnmp30 \
             libssh-4 \
             libssl1.1 \
             libxml2 \

And I did these changes for the Cloudron version as well.

Here again my Cloudron Version Diff vs Offical Focal version:

1c1
- FROM ubuntu:focal
---
+ FROM cloudron/base:2.0.0@sha256:f9fea80513aa7c92fe2e7bf3978b54c8ac5222f47a9a32a7f8833edf0eb5a4f4
13,20c13,21
-     addgroup --system --gid 1995 --quiet zabbix && \
-     adduser --quiet \
-             --system --disabled-login \
-             --ingroup zabbix --ingroup root \
-             --uid 1997 \
-             --home /var/lib/zabbix/ \
-         zabbix && \
-     usermod -G zabbix,dialout zabbix && \
---
+     # Not needed for cloudron
+     #addgroup --system --gid 1995 --quiet zabbix && \
+     #adduser --quiet \
+     #        --system --disabled-login \
+     #        --ingroup zabbix --ingroup root \
+     #        --uid 1997 \
+     #        --home /var/lib/zabbix/ \
+     #    zabbix && \
+     #usermod -G zabbix,dialout zabbix && \
38c39,40
-             tini \
---
+             # Cloudron changes - Not needed
+             #tini \
46c48,49
-             libmysqlclient21 \
---
+             # Cloudron changes - Use right version for Ubuntu 18.04 LTS
+             libmysqlclient20 \
49c52,53
-             libsnmp35 \
---
+             # Cloudron changes - Use right version for Ubuntu 18.04 LTS
+             libsnmp30 \
129c133,134
-     chown --quiet -R zabbix:root /etc/zabbix/ /var/lib/zabbix/ && \
---
+     # Cloudron changes 
+     chown --quiet -R cloudron:cloudron /etc/zabbix/ /var/lib/zabbix/ && \
153c158,159
- EXPOSE 10051/TCP
---
+ # Cloudron changes - Comment out all we don't need
+ # EXPOSE 10051/TCP
155c161
- WORKDIR /var/lib/zabbix
---
+ WORKDIR /app/code
157c163
- VOLUME ["/var/lib/zabbix/snmptraps", "/var/lib/zabbix/export"]
---
+ # VOLUME ["/var/lib/zabbix/snmptraps", "/var/lib/zabbix/export"]
159c165
- COPY ["docker-entrypoint.sh", "/usr/bin/"]
---
+ # COPY ["docker-entrypoint.sh", "/usr/bin/"]
161c167,209
- ENTRYPOINT ["/usr/bin/tini", "--", "/usr/bin/docker-entrypoint.sh"]
---
+ # ENTRYPOINT ["/usr/bin/tini", "--", "/usr/bin/docker-entrypoint.sh"]
+ 
+ # USER 1997
+ 
+ # CMD ["/usr/sbin/zabbix_server", "--foreground", "-c", "/etc/zabbix/zabbix_server.conf"]
+ COPY ["docker-entrypoint.sh", "sample.env", "/app/code/"]
+ 
+ # Cloudron - make all allowed volumes avaible in /app/data/ - see https://hub.docker.com/r/zabbix/zabbix-server-mysql > "Allowed volumes for the Zabbix server container"
+ RUN mv /usr/lib/zabbix/alertscripts /usr/lib/zabbix/alertscripts_default \
+     && ln -s /app/data/zabbix/alertscripts /usr/lib/zabbix/alertscripts \
+     #
+     && mv /usr/lib/zabbix/externalscripts /usr/lib/zabbix/externalscripts_default \
+     && ln -s /app/data/zabbix/externalscripts /usr/lib/zabbix/externalscripts \
+     #
+     && mv /var/lib/zabbix/modules /var/lib/zabbix/modules_default \
+     && ln -s /app/data/zabbix/modules /var/lib/zabbix/modules \
+     #
+     && mv /var/lib/zabbix/enc /var/lib/zabbix/enc_default \
+     && ln -s /app/data/zabbix/enc /var/lib/zabbix/enc \
+     #
+     && mv /var/lib/zabbix/ssh_keys /var/lib/zabbix/ssh_keys_default \
+     && ln -s /app/data/zabbix/ssh_keys /var/lib/zabbix/ssh_keys \
+     #
+     && mv /var/lib/zabbix/ssl/certs /var/lib/zabbix/ssl/certs_default \
+     && ln -s /app/data/zabbix/ssl/certs /var/lib/zabbix/ssl/certs \
+     #
+     && mv /var/lib/zabbix/ssl/keys /var/lib/zabbix/ssl/keys_default \
+     && ln -s /app/data/zabbix/ssl/keys /var/lib/zabbix/ssl/keys \
+     #
+     && mv /var/lib/zabbix/ssl/ssl_ca /var/lib/zabbix/ssl/ssl_ca_default \
+     && ln -s /app/data/zabbix/ssl/ssl_ca /var/lib/zabbix/ssl/ssl_ca \
+     #
+     && mv /var/lib/zabbix/snmptraps /var/lib/zabbix/snmptraps_default \
+     && ln -s /app/data/zabbix/snmptraps /var/lib/zabbix/snmptraps \
+     #
+     && mv /var/lib/zabbix/mibs /var/lib/zabbix/mibs_default \
+     && ln -s /app/data/zabbix/mibs /var/lib/zabbix/mibs \
+     #
+     && mv /var/lib/zabbix/export /var/lib/zabbix/export_default \
+     && ln -s /app/data/zabbix/export /var/lib/zabbix/export \
+     #
+     && mv /etc/zabbix /etc/zabbix_default \
+     && ln -s /app/data/zabbix/config /etc/zabbix
163,165c211
- USER 1997
- 
- CMD ["/usr/sbin/zabbix_server", "--foreground", "-c", "/etc/zabbix/zabbix_server.conf"]
\ Kein Zeilenumbruch am Dateiende.
---
+ CMD ["/app/code/docker-entrypoint.sh"]
\ Kein Zeilenumbruch am Dateiende.

[Lonkle]

@BrutalBirdie When you make those changes, is there any issues? Cause if not, what’s holding you back from a full 5.0 upgrade? Just switching the database protocol? Did your previous Cloudron version not have any issues with MySQL?

Since the Cloudron Base Image is going to stay atubuntu:bionic for awhile according to the devs. Despite Cloudron itself supporting focal in 6.0.

[BrutalBirdie]

@Lonk said in Zabbix - Appdev Log:

@BrutalBirdie When you make those changes, is there any issues? Cause if not, what’s holding you back from a full 5.0 upgrade? Just switching the database protocol?

Since the Cloudron Base Image is going to stay atubuntu:bionic for awhile according to the devs. Despite Cloudron itself supporting focal in 6.0.

To make things clear, this is the initial packaging of the zabbix-app for cloudron.
It did not work once for now with cloudron.

I want the 5.0.4 LTS Version, on bionic or focal I don't care. But have to use bionic since cloudron base image yada.

[Lonkle]

@BrutalBirdie Yeah, best chance is to switch to Postegrl but I def understand you not wanting to do them because then upgrades will be harder to do. I’m the same way.

I personally hate converting the “small container” FROM alpine to Cloudron sometimes because so many DOCKERFILE lines have to be changed since they don’t use the same packaging system. But “small containers” are the new thing so I gotta get used to that when converting to Cloudron.