@techcoderx witness node crashed [Update: fixed]

View this thread on: d.buzz | hive.blog | peakd.com | ecency.com
·@techcoderx·
0.000 HBD
@techcoderx witness node crashed [Update: fixed]
<center>![witness update log 27.png](https://files.peakd.com/file/peakd-hive/techcoderx/23tGcDBq3TJ33GrGcLL9obUebgJVnTqi6Dt5dVejM1xqSkNHrz8ucq5SAPta52bG5FUCQ.png)</center>

Just an update post for the record that @techcoderx Hive witness node have crashed unexpectedly, and right now in the process of replaying the blockchain up to the head block.

The witness node is currently hosted with Privex using the pre-replayed VPS located in Sweden, and I have been using it for over a year without issues until this got hit 2 days ago. For reference, the previous downtime was scheduled on 7 November 2021 when the disk had to be resized. It is the only other downtime with this provider.

The pre-replayed image was 3 weeks old hence it needs to catch up with the network, and unfortunately this is running slow as it replays at a rate of ~10b/s. As expected, this is due to disk I/O bottleneck as the VPS only has 8GB of RAM to work with. The CPU scored poorly on [Geekbench 5](https://browser.geekbench.com/v5/cpu/9438035) as well.

This remains an ongoing incident and will update here as the story develops.

In the worst case scenario (and probably long term as well), I may have to move this server off to a dedicated instance that might cost more, and likely spin up a backup hived node on a NUC that I bought for another blockchain. It might even help with the ridiculously high block time offsets that I often see on techcoderx.com node even with v1.26 optimizations.

|![](https://cdn.discordapp.com/attachments/956143542839939143/1000978112839942245/IMG_0140.jpg)|![IMG_0148.jpg](https://files.peakd.com/file/peakd-hive/techcoderx/EoyPa5VBkoJagduQa2W7ZzpVfFTEQvnvQmssfFc8Xh7HTpzCMSYViSXfLn6ieX8xjSR.jpg)|
|-|-|

**Update 1 (19/8/2022 08:48 UTC):** Caught up with the network, price feed is also now up to date.

**Update 2 (19/8/2022 09:25 UTC):** Subsequently a [block](https://hiveblocks.com/b/67150492) has been produced successfully.

**Update 3 (20/8/2022 04:47 UTC):** It is now techcoderx.com's [turn](https://cdn.discordapp.com/attachments/958354847818723388/1010409828230893608/Screenshot_2022-08-20_at_12.47.21_PM.png).

<hr>

# Server resource statistics

#### hived (v1.26, d821f1aa, all plugins)
`block_log` file size (compressed): 323 GB
`block_log.artifacts` file size: 1.6 GB
`shared_memory.bin` file size: 20 GB
`blockchain` folder size: 345 GB

#### HAF db

Output of `SELECT pg_size_pretty( pg_database_size('block_log') );`
Database size: 2,976 GB

#### hivemind (v1.25.3, a68d8dd4)

Output of `SELECT pg_size_pretty( pg_database_size('hive') );`
Database size: 533 GB
Compressed: 206 GB

#### Overall Postgres database
RAM usage: 4.5 GB
Compressed disk usage: 1.4 TB

<hr>

<center>[![Hive witness footer 2.png](https://ipfs.io/ipfs/QmVTNReuWzvLtY5sucfunQ2m9JpLWroYrXrcztpVsZq3sG)](https://hivesigner.com/sign/account-witness-vote?witness=techcoderx&approve=1)</center>
👍 , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,