Error CONNECT ECONNREFUSED wrong internal IP

[rfg]

@girish thanks, restarting the server fixed it.

[rfg]

@girish It worked for a short time. Now is reporting same error.

What could be wrong?

Log:
https://www.dropbox.com/s/f5uelj2dsl5pa4m/mail.zip?dl=0

[nebulon]

@rfg hm I can see the mail service crashing a few times and thus restarting, so maybe this happens too often and docker reassigns IPs. The error from the logs is:

2022-02-17T12:33:00.000Z [INFO] [012E9F33-1E0E-4A1F-901A-C89B83E4FD2A.1794.1] [outbound] bouncing mail: 554 Failure g_imap_max_limit exceeded 199990/199990 folder new
2022-02-17T12:33:00.000Z [ERROR] [012E9F33-1E0E-4A1F-901A-C89B83E4FD2A.1794.1] [outbound] Unrecognized response from upstream server: \r
2022-02-17T12:33:00.000Z [INFO] [012E9F33-1E0E-4A1F-901A-C89B83E4FD2A.1794.1] [outbound] bouncing mail: Unrecognized response from upstream server: \r
2022-02-17T12:33:00.000Z [CRIT] [-] [core] Error: We are already running hooks! Fatal error!
2022-02-17T12:33:00.000Z [CRIT] [-] [core]     at Object.plugins.run_hooks (/app/code/haraka/plugins.js:431:15)
2022-02-17T12:33:00.000Z [CRIT] [-] [core]     at HMailItem._bounce (/app/code/haraka/outbound/hmail.js:1285:17)
2022-02-17T12:33:00.000Z [CRIT] [-] [core]     at HMailItem.bounce (/app/code/haraka/outbound/hmail.js:1274:14)
2022-02-17T12:33:00.000Z [CRIT] [-] [core]     at pluggableStream.<anonymous> (/app/code/haraka/outbound/hmail.js:699:22)
2022-02-17T12:33:00.000Z [CRIT] [-] [core]     at pluggableStream.emit (events.js:315:20)
2022-02-17T12:33:00.000Z [CRIT] [-] [core]     at pluggableStream.on_socket_data (/app/code/haraka/line_socket.js:25:20)
2022-02-17T12:33:00.000Z [CRIT] [-] [core]     at pluggableStream.emit (events.js:315:20)
2022-02-17T12:33:00.000Z [CRIT] [-] [core]     at TLSSocket.<anonymous> (/app/code/haraka/tls_socket.js:59:18)
2022-02-17T12:33:00.000Z [CRIT] [-] [core]     at TLSSocket.emit (events.js:315:20)
2022-02-17T12:33:00.000Z [CRIT] [-] [core]     at addChunk (internal/streams/readable.js:309:12)
2022-02-17T12:33:00.000Z [NOTICE] [-] [core] Shutting down

We have to investigate if this is something which requires a fix in haraka. Until that the previous temporary solution would have to be used

[rfg]

@nebulon do you think upgrading to Ubuntu 18.04 would help?

[nebulon]

@rfg I don't think it will make a difference here, since all those services run in their own container, not directly on the host system. However if you are still on Ubuntu 16, then yes it is still time to upgrade first to 18 and then ideally to 20 already https://docs.cloudron.io/guides/upgrade-ubuntu-18/

[rfg]

@nebulon ok I'll try that. Also I noticed this error

554 Failure g_imap_max_limit exceeded 199990/199990 folder new

It could be some big mailbox's folder?
Is there a way to inspect/delete/move the bigger ones?

[girish]

@rfg So, from what I see in the logs, there are multiple issues:

• There seems to be logs of bouncing happening. This is expected because EC2 does not allow mails to be sent out directly. You have to setup a relay - https://docs.cloudron.io/email/#relay-outbound-mails
• The mail container is in debug mode. Not sure if this was intentional? If you go to Services -> Mail -> Edit. Remove the debug mode. When debug mode is enabled, the mail container is not even running.
• There is also some crash in haraka (our mail server).

Can you try this:

• On the server, if you go to /home/yellowtent/boxdata/mail/haraka-queue . Do you see a lot files here? If so, these are the mails that failed to deliver and it's trying to deliver them forever (well, a very long time). You can just delete all the files inside that directory and then docker restart mail

• Remove the debug mode that I mentioned above.

• Send test mail. Does that work now? I think EC2 allows a very small amount of mails to be sent directly but I am not 100% sure about this. If you paste the logs when it tries to send the test mail, that will help. You can do this by opening the logs window in another browser tab and then clicking send test mail in another window.

[rfg]

@girish that was it!!!

After deleting a bunch of files in that path everything is working fine now.

Yes, I enabled the email debug mode.

Many, many thanks, I really appreciate your support.

[scooke]

@rfg What was IT? Which of the suggestions was the solution?

[rfg]

@scooke Solution was deleting a buch of files from path:

/home/yellowtent/boxdata/mail/haraka-queue

that files were created past Jan-06.

On that day we had an account hijacked that started sending out spam. I deactivated the account and it was working fine until past feb-16th, when email service started to crash.

HTH.