Bug Report: Umami's Nginx Configuration Sets An Incorrect X-Forwarded-For Header, Preventing Proper Operation with Proxied Analytic Events

[Sydney]

Umami uses the default nginx configuration for cloudron apps. Within the default configuration, the following proxy headers are added:

    ...
    proxy_set_header   Host               $host;
    proxy_set_header   X-Real-IP          $remote_addr;
    proxy_set_header   X-Forwarded-For    $remote_addr;
    proxy_set_header   X-Forwarded-Host   $host;
    proxy_set_header   X-Forwarded-Port   $server_port;
    proxy_set_header   X-Forwarded-Proto  https;
    proxy_set_header   X-Forwarded-Ssl    on;
    ...

This works fine out of the box, but unfortunately the proxy_set_header X-Forwarded-For $remote_addr; header breaks Umami's support for proxied tracker scripts (see link to Umami docs).

In a proxied tracker script setup, a user would setup Nginx or Cloudflare workers to proxy the Umami analytic javascript file and it's POST endpoint, so that the script and any tracker events are loaded from (or sent) to the website. In such a setup, the user would typically configure something like the following in the Nginx file of their website (i.e. the site that uses Umami for analytics):

   set $umami_script_url https://umami.example.com/script.js; # Here
   set $umami_event_url https://umami.example.com/api/send;  # Here

    location = /js/script.js {
        proxy_pass $umami_script_url;
        proxy_set_header Host umami.example.com;  # Here

        proxy_buffering on;

        add_header X-Cache $upstream_cache_status;
    }

    location = /js/api/send {
        proxy_pass $umami_event_url;
        proxy_set_header Host umami.example.com;  # Here
        proxy_buffering on;
        proxy_http_version 1.1;

        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
    }

Unfortunately, because the nginx config file of the Umami cloudron app overwrites the X-Forwarded-For header, all web visits and analytic events appear to come from the server running the website.

In order to fix this, a simple change is needed. Use $proxy_add_x_forwarded_for instead of $remote_addr in order to properly preserve the origin IP information.

    proxy_set_header   X-Forwarded-For    $proxy_add_x_forwarded_for;

This bug is a relatively minor one, and should be an easy fix.

[Sydney]

In general, I wonder if Cloudron should be using $proxy_add_x_forwarded_for instead of $remote_addr for all applications in the default cloudron configuration. $proxy_add_x_forwarded_for preserves the entire X-Forwarded-For chain, instead of overwriting the X-Forwarded-For header with the the final forwarder. It's a neater solution, and should not affect the vast majority of existing apps.

Right now, I imagine the only applications where this is an issue are analytics apps. Perhaps this bug would affect matamo as well.

[BrutalBirdie]

@staff this is interesting but might need further evaluation.

[joseph]

I think what you are looking is trust proxy . https://docs.cloudron.io/networking/#trusted-ips ? Set the cloudflare IP and then Cloudron's own reverse proxy will pick the X-Forwarded-For coming from the request .

[Sydney]

@joseph No, that is not the right solution. Indeed, I am not using cloudflare at all. The issue is that the default Nginx configuration for the Umami cloudron app is set to overwhite the X-Forwarded-For header with $remote_addr, which over-writes the header with the IP address of the forwarder. Setting trusted-ips will not do anything about it.

In my case, both the client app (e.g. a Ghost blog) and umami reside on Cloudron, so even if a trusted-IP will set it will not resolve the issue, since the forwarded events are coming from Cloudron itself. The only way to solve this for Umami is to change the nginx configuration as per my original bug report.

[joseph]

@Sydney said in Bug Report: Umami's Nginx Configuration Sets An Incorrect X-Forwarded-For Header, Preventing Proper Operation with Proxied Analytic Events:

Setting trusted-ips will not do anything about it.

It does. remote_addr changes to be based on X-Forwarded-For . See https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header

Can you explain your setup again? I think the post directly jumps to technical solution and problem. I don't think I understand what is trying to be solved.