Backups failure, using tarball to external cloud wasabi.

[privsec]

Backups failure, using tarball to external cloud wasabi. It looks like the app in question specifically where it fails is my only office app.

When my backup job runs it keeps failing at backing up my onlyoffice app. I get the following

Unable to backup BoxError: Unexpected response code when piping https://path.to.file: 500 message: Internal Server Error filename: /home/yellowtent/appsdata/`app path`/postgresqldump
at /home/yellowtent/box/src/backups.js:1072:36
at wrapper (/home/yellowtent/box/node_modules/async/dist/async.js:268:20)
at iterateeCallback (/home/yellowtent/box/node_modules/async/dist/async.js:413:21)
at /home/yellowtent/box/node_modules/async/dist/async.js:321:20
at /home/yellowtent/box/src/services.js:1301:9
at f (/home/yellowtent/box/node_modules/once/once.js:25:25)
at Request.<anonymous> (/home/yellowtent/box/src/services.js:1312:53)
at Request.emit (events.js:315:20)
at Request.EventEmitter.emit (domain.js:467:12)
at Request.onRequestResponse (/home/yellowtent/box/node_modules/request/request.js:1059:10) {
reason: 'External Error',
details: {}

My backups have not been working now for 4 days. Can I please get some assistance with this?

Below is my backup settings

[robi]

check to see if all your services are green.

[d19dotca]

@privsec What @robi said. It looks like from the error this is possibly related to the PostgreSQL process not working correctly rather than the backup process itself.

[girish]

There is a postgresql issue where it starts to reject connections because it hits a connection limit. The workaround for now is to simply restart it. I have made a fix already in the next release where we limit the connection per database. This way it's easy to isolate which app is causing this connection limit issue. My guess is currently it's either nextcloud or grafana.

[privsec]

@robi
They are all green

The arrow denotes the service that the backup is failing for.

@girish said in Backups failure, using tarball to external cloud wasabi.:

There is a postgresql issue where it starts to reject connections because it hits a connection limit. The workaround for now is to simply restart it. I have made a fix already in the next release where we limit the connection per database. This way it's easy to isolate which app is causing this connection limit issue. My guess is currently it's either nextcloud or grafana.

It's likely nextcloud since nextcloud is using the only office instance

So how do I restart this?

[girish]

@privsec there is a button to the right of the postgresql item to restart the service.

[privsec]

@girish With them all being green though, do I need to?

[girish]

@privsec yes. The green signal is unfortunately not accurate for postgres. The healthcheck route doesn't take into account that no more connections can be made (the database is itself up but not accepting new connections).

[d19dotca]

@privsec Note, just because it's "running" doesn't mean it's running well.