n8n Web Socket Connection Failed

[igaudette]

Hi,

One of my n8n workflows gets a WebSocket connection error after 10-14 minutes.

Here is the error I'm seeing in my Chrome Console:

pushConnection.store-BrFt2YWH.js:54 WebSocket connection to 'wss://n8n.*************.com/rest/push?pushRef=abufwmz0xm' failed:
pushConnect @ pushConnection.store-BrFt2YWH.js:54

Here is what I tried:

1. Disable Cloudflare proxy
2. Add these variables to my .env file:
   export EXECUTIONS_TIMEOUT=3600
   export EXECUTIONS_TIMEOUT_MAX=7200
   export N8N_PUSH_BACKEND=websocket
3. Rebooting the server/app

I am running n8n on a Cloudron instance. See details:
App Title and Version: n8n 1.71.3
Package Version: io.n8n.cloudronapp@3.60.1
Installed At: 09/10/2024
Last Updated: 12/13/2024

Some additional information: My workflow sends multiple GET requests using the HTTP Request node to a website, I have successfully sent over 10k requests in another workflow without issue so I don't think the problem is here.

The workflow sends requests to 3 different websites, and the first 1k requests (first website) work fine, I noticed that the second website is slower as the requests take more time to be received. I think this is why the workflow fails and the WebSocket disconnects... Some sites return the data in just 1s while this site seems to take a few seconds to return, I haven't counted but probably 4-5s per request.. Could it cause the web socket to disconnect?

Anyway, I am sure there is a specific setting to adjust to allow more times for HTTP requests. Thanks for your help.

[joseph]

N8N_PUSH_BACKEND controls updates between the browser and backend, correct? The workflow itself (running in parallel) does not use websockets. Is that correct? What error is shown in browser console ? Maybe n8n is crashing/restarting and you have to give it more memory. Anything in n8n logs?

[igaudette]

@joseph I only see this in Chrome console:

pushConnection.store-BrFt2YWH.js:54 WebSocket connection to 'wss://n8n.*************.com/rest/push?pushRef=abufwmz0xm' failed:
pushConnect @ pushConnection.store-BrFt2YWH.js:54

My n8n instance is 8g and the graph history barely exceeds 2g.

I'm seeing this in the logs but I'm not sure if it's related to my workflow that crashes:

2024-12-20T05:32:35Z ----- Native stack trace -----
2024-12-20T05:32:35Z 10: 0x709bdded9ef6
2024-12-20T05:32:35Z 1: 0xb8ced1 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
2024-12-20T05:32:35Z 2: 0xf06460 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
2024-12-20T05:32:35Z 3: 0xf06747 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
2024-12-20T05:32:35Z 4: 0x11182e5  [node]
2024-12-20T05:32:35Z 5: 0x1130168 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
2024-12-20T05:32:35Z 6: 0x1106281 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
2024-12-20T05:32:35Z 7: 0x1107415 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
2024-12-20T05:32:35Z 8: 0x10e4a66 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
2024-12-20T05:32:35Z 9: 0x1540896 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
2024-12-20T05:32:35Z <--- JS stacktrace --->
2024-12-20T05:32:35Z <--- Last few GCs --->
2024-12-20T05:32:35Z FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
2024-12-20T05:32:35Z [1:0x12d66f70] 19549759 ms: Scavenge 1975.3 (2077.8) -> 1971.8 (2078.3) MB, 6.37 / 0.00 ms  (average mu = 1.000, current mu = 1.000) task;
2024-12-20T05:32:35Z [1:0x12d66f70] 19554444 ms: Scavenge 1977.4 (2079.7) -> 1974.6 (2081.7) MB, 7.13 / 0.01 ms  (average mu = 1.000, current mu = 1.000) task;
2024-12-20T05:32:35Z [1:0x12d66f70] 19556720 ms: Scavenge 1982.6 (2083.6) -> 1978.1 (2100.6) MB, 10.54 / 0.00 ms  (average mu = 1.000, current mu = 1.000) allocation failure;
2024-12-20T05:32:36Z => Ensure directories
2024-12-20T05:32:36Z => Loading configuration
2024-12-20T05:32:36Z => Setting permissions
2024-12-20T05:32:37Z => Starting N8N
2024-12-20T05:32:38Z Permissions 0644 for n8n settings file /app/data/user/.n8n/config are too wide. This is ignored for now, but in the future n8n will attempt to change the permissions automatically. To automatically enforce correct permissions now set N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true (recommended), or turn this check off set N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=false.
2024-12-20T05:32:38Z Loaded config overwrites from /app/data/configs/default.json
2024-12-20T05:32:38Z User settings loaded from: /app/data/user/.n8n/config
2024-12-20T05:32:40.000Z => Healtheck error: Error: connect ECONNREFUSED 172.18.18.84:5678
2024-12-20T05:32:40Z Last session crashed
2024-12-20T05:32:50.000Z => Healtheck error: Error: connect ECONNREFUSED 172.18.18.84:5678
2024-12-20T05:32:50Z Initializing n8n process
2024-12-20T05:32:51Z 
2024-12-20T05:32:51Z 
2024-12-20T05:32:51Z - N8N_CONFIG_FILES -> Please use .env files or *_FILE env vars instead.
2024-12-20T05:32:51Z The following environment variable is deprecated and will be removed in an upcoming version of n8n. Please take the recommended actions to update your configuration:
2024-12-20T05:32:51Z n8n ready on 0.0.0.0, port 5678

[igaudette]

I just tried to add this to my .env file:

NODE_OPTIONS=--max_old_space_size=8192

I will launch a new run and if this solves the problem, I'll keep you posted!

[joseph]

Those logs look like OOM from the javascript VM. maybe increase the app's memory limit as well.

[igaudette]

Adding max_old_space_size to my .env solved the issue yay 🥳