Solved Graphs error with Volumes
@girish Huh weird...
Now I see this on the Disk usage tab under "System Info". My mounts seem to work just fine as I can now access the volumes on them and everything in the apps work.
@atridad Can you try restarting graphite under Services and wait for 5-10 minutes (to collect the disk information) ? But yeah, ideally it should not barf like that....
@girish kk trying that. Its not functionally causing issues but just thought I should report.
On a side note, I changed the text for this page now from this "Cloudron in the wild" to
"You are seeing this page because this domain points to Cloudron server IP but no app is installed on that domain" (which is the only reason you should see this page).
@girish Nice, thanks! Oh and it seems that error is back.
Seems like it has an issue with my hard drive mount??? sdc1 is the partition I am using on my 2TB HDD (where my music, movies, and shows live)
@atridad Oh, I think I know the issue and it's probably related to https://git.cloudron.io/cloudron/box/-/issues/786 . Let me test this and check if that's the bug or not. Basically, collectd profiles for volumes are not created on restore. I just left this as a known bug in 6.3 but looks to be worthwhile fixing.
Of course a workaround is to delete the volume and create it again.
@girish oh so I actually don't even have the drive set up as a volume. My setup is as follows:
Main 1TB SSD
2TB SSD manually mounted to /mnt/ssd, and has my next cloud instances on it
2TB HDD manually mounted to /mnt/ssd
Volumes for music, movies, and songs pointing to /mnt/hdd/media/<type of media>
@atridad Do you still get this error? I think I will have to debug on your server to know what the problem is. If it's a problem, can you write to us at firstname.lastname@example.org?
@girish yeah still a problem. And sure I can send in a ticket.
The full stack trace is:
16/07/2021 17:58:24 :: [console] Error writing to collectd.localhost.df-sdc1.df_complex-used: Unable to read header (/var/lib/graphite/whisper/collectd/localhost/df-sdc1/df_complex-used.wsp) 16/07/2021 17:58:24 :: [console] Unhandled Error Traceback (most recent call last): File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda> inContext.theWork = lambda: context.call(ctx, func, *args, **kw) File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext return func(*args,**kw) File "/usr/lib/python3/dist-packages/carbon/writer.py", line 189, in writeForever writeCachedDataPoints() --- <exception caught here> --- File "/usr/lib/python3/dist-packages/carbon/writer.py", line 165, in writeCachedDataPoints state.database.write(metric, datapoints) File "/usr/lib/python3/dist-packages/carbon/database.py", line 124, in write whisper.update_many(path, datapoints) File "/usr/lib/python3/dist-packages/whisper.py", line 740, in update_many return file_update_many(fh, points, now) File "/usr/lib/python3/dist-packages/whisper.py", line 747, in file_update_many header = __readHeader(fh) File "/usr/lib/python3/dist-packages/whisper.py", line 294, in __readHeader raise CorruptWhisperFile("Unable to read header", fh.name) whisper.CorruptWhisperFile: Unable to read header (/var/lib/graphite/whisper/collectd/localhost/df-sdc1/df_complex-free.wsp)
The last line gives a hint that the graphite file is corrupt. So, I removed all the whisper files in
/home/yellowtent/platformdata/graphite/whisper/collectd/localhost/df-sdc1and graphs seems to work after that.
@atridad Maybe a good idea to check the health of the hard disk ? Also, since we hit the systemd issue on the same server.