Oh my…. I haven’t written at all in this blog this year…
Anyhow…. I will report about a number of technical problems…
They always seem to come at the same time, don’t they?
Let’s see, my raspberry pi server, down again. So… what I did in the past, I moved the root file system to an external hard disk, since I was suffering from file system corruption on the SD card. This week, file system corruption also occured on that external hard disk. Last week, the raspberry pi crashed, and when I rebooted it didn’t come up. It couldn’t find the root file system. Powercycling the hard disk containing the root file system fixed the problem. Seeing the problem I had last week, I blaim faulty hardware for the problems I’ve been experiencing. Now, I come to a point where I suspect the SD card I used before to be faulty as well. I am thinking ways to test that SD card. Perhaps I could create a script that writes random data to random positions, both on the SD card and in an image file, and comparing both in the end.
My dedicated server, which hosts this blog, also encountered a problem. A problem simular to a problem I’ve been experiencing with my previous server. It ran out of memory and killed amavis, virtually disabling the email subsystem. Now, I have configured the OOM to be less likely to kill amavisd-new or clamd, to be less likely to be killed in the event the system runs out of memory. For now, only a single instance. Let’s have a look at the /proc filesystem. In there we see a directory per running process.
Say, the pid of clamd is 12345, then we can set the killing likelyness from -16 (unlikely) to 15 (very likely) like this:
# pidof clamd 12345 # echo -15 > /proc/12345/oom_adj #
Well… having that said, the issue still is, the system ran out of memory. Now, I have to say, this machine was pre-installed by OVH using their default ISPConfig configuration. The machine has 3 GB of RAM installed, and was configured with 1 GB swap. Now, the rule of thumb is to have twice as much swap then ram. So, I guess that’s the cause of the problem. Now, I would like to increase the swap, but I cannot repartition a running system. And even if that wasn’t an issue, the system is running a software raid.
# blkid /dev/sda3: UUID="8199facf-617b-475a-9874-79f017193cce" TYPE="swap" /dev/sdb3: UUID="366beea7-8e2f-457b-b184-78c811df7d07" TYPE="swap" /dev/sda1: UUID="9cf67138-a39d-3a3f-a4d2-adc226fd5302" TYPE="linux_raid_member" /dev/sda2: UUID="f8058e35-2dec-e859-a4d2-adc226fd5302" TYPE="linux_raid_member" /dev/sdb1: UUID="9cf67138-a39d-3a3f-a4d2-adc226fd5302" TYPE="linux_raid_member" /dev/sdb2: UUID="f8058e35-2dec-e859-a4d2-adc226fd5302" TYPE="linux_raid_member" /dev/md2: LABEL="/var" UUID="f8893706-4d3c-45aa-af84-b302221f3bd7" TYPE="ext4" /dev/md1: LABEL="/" UUID="4903b0b6-3e50-4fc9-bb0b-1ab3e6c505b7" TYPE="ext4"
So, I guess I’ll solve it by creating a swap file instead. The system is paritioned to have a small root file system, and a huge /var file system. Therefore I’ll create the swap file in /var
# fallocate -l 6G /var/swapfile
# chmod 600 /var/swapfile
# mkswap /var/swapfile
mkswap: /var/swapfile: warning: don't erase bootbits sectors
on whole disk. Use -f to force.
Setting up swapspace version 1, size = 6291452 KiB
no label, UUID=74989c54-dc14-4b65-8577-13b356ca79c8
# swapon /var/swapfile
And add the swap file to the fstab.
Checking the swap status:
# swapon -s Filename Type Size Used Priority /dev/sda3 partition 525308 254816 -1 /dev/sdb3 partition 525308 16488 -2 /var/swapfile file 6291452 0 -3
Well… I guess this solves the memory problem. I might have a closer look at some logs, but so far I haven’t seen any task using suspecious amount of resources. I guess 3 GB of RAM + 1 GB of swap just wasn’t enough.