Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor
  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum
Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Support
  3. Graphite service orange, spinning status seen. Can't load CPU or memory graphs.

Graphite service orange, spinning status seen. Can't load CPU or memory graphs.

Scheduled Pinned Locked Moved Support
3 Posts 1 Posters 87 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • d19dotcaD Offline
    d19dotcaD Offline
    d19dotca
    wrote last edited by
    #1

    I've noticed the Graphite service consistently seems to spin in circles no matter how long. This prevents the CPU and memory graphs on the System page from loading correctly.

    85e4e55b-8869-488b-8046-70b92d070259-image.png

    It seems like this started happening with 8.3.1, but it's unclear if that's the case as around the same time I also migrated to a new dedicated server with the OS upgrade from 22.04 to 24.04 as well. A lot changed in a short period of time, unfortunately which makes it harder to narrow down event-wise for me.

    It normally has 512 MB assigned to the service, but I did try giving it a few GB for troubleshooting, but no difference in behaviour.

    The logs for the service after rebooting my server are included below. I noticed a connection error, perhaps this is the issue? From the log: 2025-03-27T06:09:50Z [2025-03-27T06:09:50.529Z] ERROR [tcp] read error {"error": "read tcp 172.18.0.2:2003->172.18.0.1:53106: i/o timeout"}

    Full logs below:

    2025-03-27T06:07:40Z ==> Running graphite migrations
2025-03-27T06:07:42Z Operations to perform:
2025-03-27T06:07:42Z Apply all migrations: account, admin, auth, contenttypes, dashboard, events, sessions, tagging, tags, url_shortener
2025-03-27T06:07:42Z Running migrations:
2025-03-27T06:07:42Z No migrations to apply.
2025-03-27T06:07:42Z Your models in app(s): 'account', 'events' have changes that are not yet reflected in a migration, and so won't be applied.
2025-03-27T06:07:42Z Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
2025-03-27T06:07:42Z Starting graphite
2025-03-27T06:07:42Z 2025-03-27 06:07:42,629 INFO Included extra file "/etc/supervisor/conf.d/go-carbon.conf" during parsing
2025-03-27T06:07:42Z 2025-03-27 06:07:42,629 INFO Included extra file "/etc/supervisor/conf.d/gunicorn.conf" during parsing
2025-03-27T06:07:42Z 2025-03-27 06:07:42,629 INFO Included extra file "/etc/supervisor/conf.d/whisper-cleanup.conf" during parsing
2025-03-27T06:07:42Z 2025-03-27 06:07:42,630 INFO Set uid to user 0 succeeded
2025-03-27T06:07:42Z 2025-03-27 06:07:42,639 INFO RPC interface 'supervisor' initialized
2025-03-27T06:07:42Z 2025-03-27 06:07:42,639 CRIT Server 'inet_http_server' running without any HTTP authentication checking
2025-03-27T06:07:42Z 2025-03-27 06:07:42,640 INFO supervisord started with pid 1
2025-03-27T06:07:43Z 2025-03-27 06:07:43,643 INFO spawned: 'go-carbon' with pid 23
2025-03-27T06:07:43Z 2025-03-27 06:07:43,647 INFO spawned: 'gunicorn' with pid 24
2025-03-27T06:07:43Z 2025-03-27 06:07:43,649 INFO spawned: 'whisper-cleanup' with pid 25
2025-03-27T06:07:43Z ==> Cleanup old whisper databases...
2025-03-27T06:07:43Z ==> Cleanup empty directories...
2025-03-27T06:07:43Z ==> Wait for next run in 1h...
2025-03-27T06:07:43Z [2025-03-27T06:07:43.771Z] INFO [main] started {}
2025-03-27T06:07:43Z [2025-03-27 06:07:43 +0000] [24] [INFO] Starting gunicorn 23.0.0
2025-03-27T06:07:43Z [2025-03-27 06:07:43 +0000] [24] [INFO] Listening at: http://0.0.0.0:8000 (24)
2025-03-27T06:07:43Z [2025-03-27 06:07:43 +0000] [24] [INFO] Using worker: sync
2025-03-27T06:07:43Z [2025-03-27 06:07:43 +0000] [36] [INFO] Booting worker with pid: 36
2025-03-27T06:07:44Z [2025-03-27 06:07:44 +0000] [37] [INFO] Booting worker with pid: 37
2025-03-27T06:07:45Z 2025-03-27 06:07:45,057 INFO success: go-carbon entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-03-27T06:07:45Z 2025-03-27 06:07:45,057 INFO success: gunicorn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-03-27T06:07:45Z 2025-03-27 06:07:45,057 INFO success: whisper-cleanup entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-03-27T06:08:43Z [2025-03-27T06:08:43.771Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.runtime.GOMAXPROCS", "value": 4}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.771Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.runtime.NumGoroutine", "value": 19}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.771Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.size", "value": 0}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.771Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.metrics", "value": 3}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.771Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.maxSize", "value": 1000000}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.notConfirmed", "value": 4096}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queries", "value": 0}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.tagsNormalizeErrors", "value": 0}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.overflow", "value": 0}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queueBuildCount", "value": 598}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queueBuildTimeMs", "value": 3}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queueWriteoutTime", "value": 0}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.droppedRealtimeIndex", "value": 0}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.tcp.metricsReceived", "value": 761}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.tcp.active", "value": 1}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.tcp.errors", "value": 260}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.updateOperations", "value": 761}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.committedPoints", "value": 761}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.pointsPerUpdate", "value": 1}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.created", "value": 0}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.oooDiscardedPoints", "value": 0}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.maxUpdatesPerSecond", "value": 0}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.workers", "value": 8}
2025-03-27T06:08:43Z [2025-03-27T06:08:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.extended", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.runtime.GOMAXPROCS", "value": 4}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.runtime.NumGoroutine", "value": 19}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.size", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.metrics", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.maxSize", "value": 1000000}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.notConfirmed", "value": 4096}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queries", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.tagsNormalizeErrors", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.overflow", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queueBuildCount", "value": 597}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queueBuildTimeMs", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queueWriteoutTime", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.droppedRealtimeIndex", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.tcp.metricsReceived", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.773Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.tcp.active", "value": 1}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.773Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.tcp.errors", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.773Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.updateOperations", "value": 24}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.773Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.committedPoints", "value": 24}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.773Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.pointsPerUpdate", "value": 1}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.773Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.created", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.773Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.oooDiscardedPoints", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.773Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.maxUpdatesPerSecond", "value": 0}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.773Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.workers", "value": 8}
2025-03-27T06:09:43Z [2025-03-27T06:09:43.773Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.extended", "value": 0}
2025-03-27T06:09:50Z [2025-03-27T06:09:50.529Z] ERROR [tcp] read error {"error": "read tcp 172.18.0.2:2003->172.18.0.1:53106: i/o timeout"}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.runtime.GOMAXPROCS", "value": 4}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.runtime.NumGoroutine", "value": 17}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.size", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.metrics", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.maxSize", "value": 1000000}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.notConfirmed", "value": 4096}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queries", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.tagsNormalizeErrors", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.overflow", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queueBuildCount", "value": 597}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queueBuildTimeMs", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.queueWriteoutTime", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.cache.droppedRealtimeIndex", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.tcp.metricsReceived", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.tcp.active", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.tcp.errors", "value": 1}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.updateOperations", "value": 24}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.committedPoints", "value": 24}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.pointsPerUpdate", "value": 1}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.created", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.oooDiscardedPoints", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.maxUpdatesPerSecond", "value": 0}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.workers", "value": 8}
2025-03-27T06:10:43Z [2025-03-27T06:10:43.772Z] INFO [stat] collect {"endpoint": "local", "metric": "carbon.agents.graphite.persister.extended", "value": 0}

    --
    Dustin Dauncey
    www.d19.ca

    1 Reply Last reply
    1
    • d19dotcaD Offline
      d19dotcaD Offline
      d19dotca
      wrote last edited by
      #2

      It seems like it's working again, but it's unclear why this is.

      --
      Dustin Dauncey
      www.d19.ca

      1 Reply Last reply
      1
      • d19dotcaD Offline
        d19dotcaD Offline
        d19dotca
        wrote last edited by
        #3

        I wonder if this is perhaps related to the issue being troubleshot here: https://forum.cloudron.io/post/104263

        --
        Dustin Dauncey
        www.d19.ca

        1 Reply Last reply
        0
        Reply
        • Reply as topic
        Log in to reply
        • Oldest to Newest
        • Newest to Oldest
        • Most Votes

        • Login
        • Don't have an account? Register
        • Login or register to search.
        • First post
          Last post
        0
        • Categories
        • Recent
        • Tags
        • Popular
        • Bookmarks
        • Search