Mongo DB not starting
-
Hey there,
at some point the mongo DB seems to have broken on my installation. I just noticed it, because the SSL certificates did not update automatically. Furthermore the app updates are stuck updating now. I checked the service tab, where the mongo DB stays on yellow and does not come up as it should.In the logs I get the following:
I tried rebooting, updating Ubuntu to 20.04 and worked my way through some troubleshooting steps found here without luck. How can I get this fixed, without losing my data like here..?
Kind regards
-
Is there any error shown in the mongodb service itself? There is a logviewer button in the services view itself for each service.
If others read this, generally rebooting the server is usually not helping much, but may put still running apps at risk to be down aftertwards as well, without understanding why something does not start.
-
N nebulon marked this topic as a question on
-
Indeed, I just restartet the mongodb service, to get fresh logs and following showed (can't share the entire log, since it is to long and I am not allowed to upload txt files, so here what seemed most important):
Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.655+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1689665604:655246][14:0x7ff7eea7acc0], file:local/collection-10-4461053802942612146.wt, txn rollback_to_stable: __rollback_to_stable_btree_apply, 1530: file:local/collection-10-4461053802942612146.wt: unable to open handle: WT_ERROR: non-specific WiredTiger error"}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.657+00:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1689665604:657723][14:0x7ff7eea7acc0], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 1, snapshot max: 1 snapshot count: 0, oldest timestamp: (1681933318, 1) , meta checkpoint timestamp: (1681933323, 1) base write gen: 16574542"}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.661+00:00"},"s":"I", "c":"STORAGE", "id":4795906, "ctx":"initandlisten","msg":"WiredTiger opened","attr":{"durationMillis":996}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.661+00:00"},"s":"I", "c":"RECOVERY", "id":23987, "ctx":"initandlisten","msg":"WiredTiger recoveryTimestamp","attr":{"recoveryTimestamp":{"$timestamp":{"t":1681933323,"i":1}}}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.663+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":0,"message":"[1689665604:663636][14:0x7ff7eea7acc0], file:sizeStorer.wt, WT_SESSION.open_cursor: __wt_block_read_off, 286: sizeStorer.wt: potential hardware corruption, read checksum error for 4096B block at offset 32768: block header checksum of 0x47873a4d doesn't match expected checksum of 0x7a76f50f"}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.663+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":0,"message":"[1689665604:663763][14:0x7ff7eea7acc0], file:sizeStorer.wt, WT_SESSION.open_cursor: __wt_bm_corrupt_dump, 140: {0: 32768, 4096, 0x7a76f50f}: (chunk 1 of 4): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 33 00 00 00 0b 00 00 00 01 00 00 01 00 10 00 00 4d 3a 87 47 01 00 00 00 e2 f5 1a 80 cf c0 e2 1f c0 80 81 e3 fc b1 d9 e2 6f c0 00 00 00 00 00 00 c4 90 65 6e 63 72 79 70 74 69 6f 6e 3d 28 6b 65 79 69 64 3d 2c 6e 61 6d 65 3d 29 2c 62 6c 6f 63 6b 5f 6d 65 74 61 64 61 74 61 5f 65 6e 63 72 79 70 74 65 64 3d 66 61 6c 73 65 2c 62 6c 6f 63 6b 5f 6d 65 74 61 64 61 74 61 3d 5b 61 63 63 65 73 73 5f 70 61 74 74 65 72 6e 5f 68 69 6e 74 3d 6e 6f 6e 65 2c 61 6c 6c 6f 63 61 74 69 6f 6e 5f 73 69 7a 65 3d 34 4b 42 2c 61 70 70 5f 6d 65 74 61 64 61 74 61 3d 2c 61 73 73 65 72 74 3d 28 63 6f 6d 6d 69 74 5f 74 69 6d 65 73 74 61 6d 70 3d 6e 6f 6e 65 2c 64 75 72 61 62 6c 65 5f 74 69 6d 65 73 74 61 6d 70 3d 6e 6f 6e 65 2c 72 65 61 64 5f 74 69 6d 65 73 74 61 6d 70 3d 6e 6f 6e 65 29 2c 62 6c 6f 63 6b 5f 61 6c 6c 6f 63 61 74 69 6f 6e 3d 62 65 73 74 2c 62 6c 6f 63 6b 5f 63 6f 6d 70 72 65 73 73 6f 72 3d 2c 63 61 63 68 65 5f 72 65 73 69 64 65 6e 74 3d 66 61 6c 73 65 2c 63 68 65 63 6b 73 75 6d 3d 75 6e 63 6f 6d 70 72 65 73 73 65 64 2c 63 6f 6c 6c 61 74 6f 72 3d 2c 63 6f 6c 75 6d 6e 73 3d 2c 64 69 63 74 69 6f 6e 61 72 79 3d 30 2c 65 6e 63 72 79 70 74 69 6f 6e 3d 28 6b 65 79 69 64 3d 2c 6e 61 6d 65 3d 29 2c 66 6f 72 6d 61 74 3d 62 74 72 65 65 2c 68 75 66 66 6d 61 6e 5f 6b 65 79 3d 2c 68 75 66 66 6d 61 6e 5f 76 61 6c 75 65 3d 2c 69 64 3d 32 2c 69 67 6e 6f 72 65 5f 69 6e 5f 6d 65 6d 6f 72 79 5f 63 61 63 68 65 5f 73 69 7a 65 3d 66 61 6c 73 65 2c 69 6e 74 65 72 6e 61 6c 5f 69 74 65 6d 5f 6d 61 78 3d 30 2c 69 6e 74 65 72 6e 61 6c 5f 6b 65 79 5f 6d 61 78 3d 30 2c 69 6e 74 65 72 6e 61 6c 5f 6b 65 79 5f 74 72 75 6e 63 61 74 65 3d 74 72 75 65 2c 69 6e 74 65 72 6e 61 6c 5f 70 61 67 65 5f 6d 61 78 3d 34 4b 42 2c 6b 65 79 5f 66 6f 72 6d 61 74 3d 75 2c 6b 65 79 5f 67 61 70 3d 31 30 2c 6c 65 61 66 5f 69 74 65 6d 5f 6d 61 78 3d 30 2c 6c 65 61 66 5f 6b 65 79 5f 6d 61 78 3d 30 2c 6c 65 61 66 5f 70 61 67 65 5f 6d 61 78 3d 33 32 4b 42 2c 6c 65 61 66 5f 76 61 6c 75 65 5f 6d 61 78 3d 30 2c 6c 6f 67 3d 28 65 6e 61 62 6c 65 64 3d 74 72 75 65 29 2c 6d 65 6d 6f 72 79 5f 70 61 67 65 5f 69 6d 61 67 65 5f 6d 61 78 3d 30 2c 6d 65 6d 6f 72 79 5f 70 61 67 65 5f 6d 61 78 3d 35 4d 42 2c 6f 73 5f 63 61 63 68 65 5f 64 69 72 74 79 5f 6d 61 78 3d 30 2c 6f 73 5f 63 61 63 68 65 5f 6d 61 78 3d 30 2c 70 72 65 66 69 78 5f 63 6f 6d 70 72 65 73 73 69 6f 6e 3d 66 61 6c 73 65 2c 70 72 65 66 69 78 5f 63 6f 6d 70 72 65 73 73 69 6f 6e 5f 6d 69 6e 3d 34 2c 73 70 6c 69 74 5f 64 65 65 70 65 6e 5f 6d 69 6e 5f 63 68 69 6c 64 3d 30 2c 73 70 6c 69 74 5f 64 65 65 70 65 6e 5f 70 65 72 5f 63 68 69 6c 64 3d 30 2c 73 70 6c 69 74 5f 70 63 74 3d 39 30 2c 76 61 6c 75 65 5f 66 6f 72 6d 61 74 3d 75 2c 76 65 72 73 69 6f 6e 3d 28 6d 61 6a 6f 72 3d 31 2c 6d 69 6e 6f 72 3d 31 29 2c 63 68 65 63 6b 70 6f 69 6e 74 3d 28 57 69 72 65 64 54 69 67 65 72 43 68 65 63 6b 70 6f 69 6e 74 2e 31 30 39 37 39 31 39 3d 28 61 64 64 72 3d 22 30 31 38 31 38 31 65 34 61 65 34 63 37 37 34 39 38 32 38 31 65 34 31 35 34 36 62 64 31 36 38 33 38 31 65 34 66 63 64 66 33 31 33 64 38 30 38 30 38 30 65 "}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.664+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":0,"message":"[1689665604:664002][14:0x7ff7eea7acc0], file:sizeStorer.wt, WT_SESSION.open_cursor: __wt_bm_corrupt_dump, 140: {0: 32768, 4096, 0x7a76f50f}: (chunk 2 of 4): 32 36 66 63 30 63 66 63 30 22 2c 6f 72 64 65 72 3d 31 30 39 37 39 31 39 2c 74 69 6d 65 3d 31 36 38 31 38 35 39 30 38 37 2c 73 69 7a 65 3d 38 31 39 32 2c 6e 65 77 65 73 74 5f 73 74 61 72 74 5f 64 75 72 61 62 6c 65 5f 74 73 3d 30 2c 6f 6c 64 65 73 74 5f 73 74 61 72 74 5f 74 73 3d 30 2c 6e 65 77 65 73 74 5f 74 78 6e 3d 30 2c 6e 65 77 65 73 74 5f 73 74 6f 70 5f 64 75 72 61 62 6c 65 5f 74 73 3d 30 2c 6e 65 77 65 73 74 5f 73 74 6f 70 5f 74 73 3d 2d 31 2c 6e 65 77 65 73 74 5f 73 74 6f 70 5f 74 78 6e 3d 2d 31 31 2c 70 72 65 70 61 72 65 3d 30 2c 77 72 69 74 65 5f 67 65 6e 3d 31 36 35 36 35 31 33 34 2c 72 75 6e 5f 77 72 69 74 65 5f 67 65 6e 3d 31 36 35 36 35 31 33 32 29 29 2c 63 68 65 63 6b 70 6f 69 6e 74 5f 62 61 63 6b 75 70 5f 69 6e 66 6f 3d 2c 63 68 65 63 6b 70 6f 69 6e 74 5f 6c 73 6e 3d 28 39 33 2c 34 36 30 38 29 5d c0 fb 63 68 65 63 6b 70 6f 69 6e 74 3d 28 57 69 72 65 64 54 69 67 65 72 43 68 65 63 6b 70 6f 69 6e 74 2e 31 30 39 39 31 35 34 3d 28 61 64 64 72 3d 22 30 31 38 35 38 31 65 34 30 32 37 35 30 35 62 35 38 36 38 31 65 34 34 32 34 31 30 37 62 34 38 30 38 30 38 30 38 30 38 30 38 30 65 32 36 66 63 30 63 66 63 30 22 2c 6f 72 64 65 72 3d 31 30 39 39 31 35 34 2c 74 69 6d 65 3d 31 36 38 31 39 33 33 32 30 34 2c 73 69 7a 65 3d 38 31 39 32 2c 6e 65 77 65 73 74 5f 73 74 61 72 74 5f 64 75 72 61 62 6c 65 5f 74 73 3d 30 2c 6f 6c 64 65 73 74 5f 73 74 61 72 74 5f 74 73 3d 30 2c 6e 65 77 65 73 74 5f 74 78 6e 3d 30 2c 6e 65 77 65 73 74 5f 73 74 6f 70 5f 64 75 72 61 62 6c 65 5f 74 73 3d 30 2c 6e 65 77 65 73 74 5f 73 74 6f 70 5f 74 73 3d 2d 31 2c 6e 65 77 65 73 74 5f 73 74 6f 70 5f 74 78 6e 3d 2d 31 31 2c 70 72 65 70 61 72 65 3d 30 2c 77 72 69 74 65 5f 67 65 6e 3d 31 36 35 36 38 38 35 36 2c 72 75 6e 5f 77 72 69 74 65 5f 67 65 6e 3d 31 36 35 36 35 31 35 32 29 29 9a 01 85 81 e4 02 75 05 b5 86 81 e4 42 41 07 b4 80 80 80 80 80 80 e2 6f c0 cf c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.664+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":0,"message":"[1689665604:664402][14:0x7ff7eea7acc0], file:sizeStorer.wt, WT_SESSION.open_cursor: __wt_bm_corrupt_dump, 140: {0: 32768, 4096, 0x7a76f50f}: (chunk 3 of 4): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.664+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":0,"message":"[1689665604:664537][14:0x7ff7eea7acc0], file:sizeStorer.wt, WT_SESSION.open_cursor: __wt_bm_corrupt_dump, 140: {0: 32768, 4096, 0x7a76f50f}: (chunk 4 of 4): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.664+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1689665604:664552][14:0x7ff7eea7acc0], file:sizeStorer.wt, WT_SESSION.open_cursor: __wt_block_read_off, 295: sizeStorer.wt: fatal read error: WT_ERROR: non-specific WiredTiger error"}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.664+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31804,"message":"[1689665604:664559][14:0x7ff7eea7acc0], file:sizeStorer.wt, WT_SESSION.open_cursor: __wt_block_read_off, 295: the process must exit and restart: WT_PANIC: WiredTiger library panic"}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.664+00:00"},"s":"F", "c":"-", "id":23089, "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":50853,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp","line":481}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.664+00:00"},"s":"F", "c":"-", "id":23090, "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.665+00:00"},"s":"F", "c":"CONTROL", "id":4757800, "ctx":"initandlisten","msg":"Writing fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}} Jul 18 09:33:24 {"t":{"$date":"2023-07-18T07:33:24.787+00:00"},"s":"I", "c":"CONTROL", "id":31431, "ctx":"initandlisten","msg":"BACKTRACE: {bt}","attr":{"bt":{"backtrace":[{"a":"556C2CAB354A","b":"556C29C9B000","o":"2E1854A","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1EA"},{"a":"556C2CAB4FD9","b":"556C29C9B000","o":"2E19FD9","s":"_ZN5mongo15printStackTraceEv","s+":"29"},{"a":"556C2CAB2366","b":"556C29C9B000","o":"2E17366","s":"_ZN5mongo12_GLOBAL__N_116abruptQuitActionEiP9siginfo_tPv","s+":"66"},{"a":"7FF7EF9B3520","b":"7FF7EF971000","o":"42520","s":"__sigaction","s+":"50"},{"a":"7FF7EFA07A7C","b":"7FF7EF971000","o":"96A7C","s":"pthread_kill","s+":"12C"},{"a":"7FF7EF9B3476","b":"7FF7EF971000","o":"42476","s":"raise","s+":"16"},{"a":"7FF7EF9997F3","b":"7FF7EF971000","o":"287F3","s":"abort","s+":"D3"},{"a":"556C2ABEF8E4","b":"556C29C9B000","o":"F548E4","s":"_ZN5mongo25fassertFailedWithLocationEiPKcj","s+":"12B"},{"a":"556C2A8C0BAF","b":"556C29C9B000","o":"C25BAF","s":"_ZN5mongo12_GLOBAL__N_141mdb_handle_error_with_startup_suppressionEP18__wt_event_handlerP12__wt_sessioniPKc.cold.1052","s+":"16"},{"a":"556C2ADCDF8B","b":"556C29C9B000","o":"1132F8B","s":"__eventv","s+":"3FB"},{"a":"556C2A8D2C14","b":"556C29C9B000","o":"C37C14","s":"__wt_panic_func","s+":"10C"},{"a":"556C2A8E05FD","b":"556C29C9B000","o":"C455FD","s":"__wt_block_read_off.cold.5","s+":"85"},{"a":"556C2AF01546","b":"556C29C9B000","o":"1266546","s":"__wt_block_extlist_read","s+":"96"},{"a":"556C2AF01A6B","b":"556C29C9B000","o":"1266A6B","s":"__wt_block_extlist_read_avail","s+":"2B"},{"a":"556C2AEFC6B0","b":"556C29C9B000","o":"12616B0","s":"__wt_block_checkpoint_load","s+":"1E0"},{"a":"556C2AEF4A37","b":"556C29C9B000","o":"1259A37","s":"__bm_checkpoint_load","s+":"37"},{"a":"556C2AE237AA","b":"556C29C9B000","o":"11887AA","s":"__wt_btree_open","s+":"D9A"},{"a":"556C2AD4C6A2","b":"556C29C9B000","o":"10B16A2","s":"__wt_conn_dhandle_open","s+":"8D2"},{"a":"556C2ADCC889","b":"556C29C9B000","o":"1131889","s":"__wt_session_get_dhandle","s+":"E9"},{"a":"556C2ADCCF32","b":"556C29C9B000","o":"1131F32","s":"__wt_session_get_dhandle","s+":"792"},{"a":"556C2ADCD224","b":"556C29C9B000","o":"1132224","s":"__wt_session_get_btree_ckpt","s+":"154"},{"a":"556C2AD6447A","b":"556C29C9B000","o":"10C947A","s":"__wt_curfile_open","s+":"5A"},{"a":"556C2ADC786B","b":"556C29C9B000","o":"112C86B","s":"__session_open_cursor_int","s+":"2DB"},{"a":"556C2ADC7528","b":"556C29C9B000","o":"112C528","s":"__wt_open_cursor","s+":"58"},{"a":"556C2AEB648E","b":"556C29C9B000","o":"121B48E","s":"__wt_curtable_open","s+":"3BE"},{"a":"556C2ADC79E0","b":"556C29C9B000","o":"112C9E0","s":"__session_open_cursor_int","s+":"450"},{"a":"556C2ADC81AE","b":"556C29C9B000","o":"112D1AE","s":"__session_open_cursor","s+":"71E"},{"a":"556C2AD29097","b":"556C29C9B000","o":"108E097","s":"_ZN5mongo12_GLOBAL__N_111_openCursorEP12__wt_sessionRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKcPP11__wt_cursor.constprop.929","s+":"37"},{"a":"556C2AD295E7","b":"556C29C9B000","o":"108E5E7","s":"_ZN5mongo17WiredTigerSession15getCachedCursorERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEmPKc","s+":"67"},{"a":"556C2ACE4B02","b":"556C29C9B000","o":"1049B02","s":"_ZN5mongo16WiredTigerCursorC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEmbPNS_16OperationContextE","s+":"172"},{"a":"556C2AD2CA43","b":"556C29C9B000","o":"1091A43","s":"_ZNK5mongo20WiredTigerSizeStorer4loadEPNS_16OperationContextENS_10StringDataE","s+":"153"},{"a":"556C2AD18300","b":"556C29C9B000","o":"107D300","s":"_ZN5mongo21WiredTigerRecordStoreC1EPNS_18WiredTigerKVEngineEPNS_16OperationContextENS0_6ParamsE","s+":"6C0"},{"a":"556C2AD187C4","b":"556C29C9B000","o":"107D7C4","s":"_ZN5mongo29StandardWiredTigerRecordStoreC1EPNS_18WiredTigerKVEngineEPNS_16OperationContextENS_21WiredTigerRecordStore6ParamsE","s+":"C4"},{"a":"556C2ACF851D","b":"556C29C9B000","o":"105D51D","s":"_ZN5mongo18WiredTigerKVEngine21getGroupedRecordStoreEPNS_16OperationContextENS_10StringDataES3_RKNS_17CollectionOptionsENS_8KVPrefixE","s+":"3FD"},{"a":"556C2B5F8104","b":"556C29C9B000","o":"195D104","s":"_ZN5mongo17StorageEngineImpl11loadCatalogEPNS_16OperationContextE","s+":"1344"},{"a":"556C2B5FB18C","b":"556C29C9B000","o":"196018C","s":"_ZN5mongo17StorageEngineImplC1EPNS_8KVEngineENS_20StorageEngineOptionsE","s+":"25C"},{"a":"556C2ACDF0B5","b":"556C29C9B000","o":"10440B5","s":"_ZNK5mongo12_GLOBAL__N_117WiredTigerFactory6createERKNS_19StorageGlobalParamsEPKNS_21StorageEngineLockFileE","s+":"1A5"},{"a":"556C2B605AC7","b":"556C29C9B000","o":"196AAC7","s":"_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE","s+":"4F7"},{"a":"556C2AC8B1C7","b":"556C29C9B000","o":"FF01C7","s":"_ZN5mongo12_GLOBAL__N_114_initAndListenEPNS_14ServiceContextEi.isra.1585","s+":"3F7"},{"a":"556C2AC8F3FA","b":"556C29C9B000","o":"FF43FA","s":"_ZN5mongo12_GLOBAL__N_111mongoDbMainEiPPc","s+":"A5A"},{"a":"556C2ABFEFB9","b":"556C29C9B000","o":"F63FB9","s":"main","s+":"9"},{"a":"7FF7EF99AD90","b":"7FF7EF971000","o":"29D90","s":"__libc_init_first","s+":"90"},{"a":"7FF7EF99AE40","b":"7FF7EF971000","o":"29E40","s":"__libc_start_main","s+":"80"},{"a":"556C2AC8A5FE","b":"556C29C9B000","o":"FEF5FE","s":"_start","s+":"2E"}],"processInfo":{"mongodbVersion":"4.4.19","gitVersion":"9a996e0ad993148b9650dc402e6d3b1804ad3b8a","compiledModules":[],"uname":{"sysname":"Linux","release":"5.4.0-153-generic","version":"#170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023","machine":"x86_64"},"somap":[{"b":"556C29C9B000","elfType":3,"buildId":"0A9DC1A0996DF21B80723D4010693B05ED2C96B2"},{"b":"7FF7EF971000","path":"/lib/x86_64-linux-gnu/libc.so.6","elfType":3,"buildId":"69389D485A9793DBE873F0EA2C93E02EFAA9AA3D"}]}}}} ...
Jul 18 09:33:25 /app/code/start.sh: line 18: 14 Aborted (core dumped) $mongod --noauth 2> /dev/null
Thanks, good to know, seems I got lucky nothing further broke in my case
-
@logo you can try something like:
- mv /home/yellowtent/platformdata/mongodb/ /home/yellowtent/platformdata/mongodb-save
- mkdir /home/yellowtent/platformdata/mongodb/
- systemctl restart box
Then, mongodb should come up or go to Services -> Restart Mongodb.
If you have apps that used mongodb, then go into each app -> backups -> restore from latest backup.
-
@logo both don't use mongodb.
Though @nebulon asked, It's not straightforward to know if an app uses mongodb
You have to look into the package code at https://git.cloudron.io/cloudron/<appname>-app . Inside, each repo there is CloudronManifest.json file. Inside that file , it will list under the addons section whether the app needs mongodb or not.
-
G girish has marked this topic as solved on
-
G girish referenced this topic on