Yesterday, I tried logging in to my Raspberry Pi server from my phone using ConnectBot, an Android SSH client. It was disconnected immediately. Something is wrong. When being back at home, I tried logging in from my desktop, with the same problem. Pre-authentication disconnect.
It appears, when being booted headless, the Raspberry Pi enables a console at its composite port. Being located behind my tv, it’s easy to plug in a cable. systemctl status sshd
indicated an error state, and it refused to start again. It would immediately get killed by a sigbus. A sigbus??? wtf?
The logfiles show the following error:
Mar 9 17:55:19 rpi-server sshd[21742]: Inconsistency detected by ld.so: dl-version.c: 224: _dl_check_map_versions: Assertion `needed != ((void *)0)' failed!
I assumed some upgrade I ran earlier upgraded some libraries, conflicting with some in-momory code, therefore I decided to reboot the Pi.
It comes up with the Raspberry logo, saying welcome to ArchLinux ARM, but then it just hangs. Nothing else happens. Nothing.
To verify if it’s the Pi, I decided to boot it up using the SD card from my other Pi. Since it boots fine, there is something wrong with the content of the rpi-server’s SD card, which seems to be confirmed by
Mar 9 17:55:14 rpi-server kernel: [3514985.125660] EXT4-fs error (device mmcblk0p2): __ext4_ext_check_block:475: inode #699: comm systemctl: bad header/extent: inval id extent entries - magic f30a, entries 223, max 340(340), depth 0(0)
I have made an image if the SD card’s current state. I can restore the SD card with an earlier image I’ve made when installing rpi-server. I think I have changed some things since I’ve made that image, but the basic functionality, such as NFS, CUPS and SANE should be on there. (As I’ve mentioned I was going to make this backup in a post mentioning sane.
So, this seems to be a case of file system corruption. The question remains what caused this corrupt file system? Is there something wrong with the Pi? the SD card? Is there a bug in the kernel used by ArchLinuxARM? Was the system possibly hacked?
I ran fsck -y /dev/mmcblk0p2
on the SD card. I have a backup anyways, so the -y won’t hurt. I have mentioned, even though I marked it as “should be checked on boot” in my fstab, it doesn’t appear to be doing so for the rootfs. (It checks the bootfs during boot) But since this system is supposed to be always up, a check during boot shouldn’t make much difference. And in principe, the file system should not get corrupted. So…. why did it become corrupted?
Anyhow, getting multiple claimed blocks. Therefore I say it’s no use to continue, I should just write the old image back and that be it…. thinking about what I’ve installed afterwards… one of the things would be apache I guess…
My laptop’s SD slot stopped working due overheating. The samn /etc/cpufreq-bench.conf
was set to governor ondemand
. I have set it to powersave
a dozen times, but it keeps popping back to ondemand
, probably during the installation of updates. I don’t know why, but I need to clock down my laptop to the lowest possible speed or else it will overheat. The SD slot is the first thing to give issues when this happens.
Running on the image I put back, and performed an upgrade, I see these messages in my logs again:
[ 1417.529973] EXT4-fs error (device mmcblk0p2): ext4_ext_check_inode:462: inode #55228: comm pacman: bad header/extent: invalid magic - magic f34a, entries 1, max 4(0), depth 0(0)
I did the upgrade immediately and rebooted, so it is still either the SD card or the current kernel.
Hello Andre,
same problem here. I tried it with different SD-Cards, I don’t overclock anything. After it happened the 4th time, I used the SD-Card only as boot partition, all the rest that was before on /dev/mmcblk0p2 i put an USB pendrive partiton. But that didtn’t help either, file system corruption was just a matter of time.
I also made a memory check and I couldn’t find any errors.
Since 3 days I’m now testing NetBSD and did a lot of installing, writing operations and stress testing. Until now it works flawlessly. Let’s see if it will stay that way.
Best Wishes,
Royskatt
P.S. Your mathematical bot check writes the first number in dutch words, makes it a little harder for people who don’t speak that language ;)
The strangest part is, it worked fine for months, and then suddenly this started happening.
But if it also corrupts its root file system if it sits on an USB stick…. weird… I was thinking about making an initramfs.
I believe having no initramfs is why it doesn’t fsck the root file system during boot. That shouldn’t be the fix, but at least it would make me notice the problem earlier.
I haven’t noticed any corruptions on my external hard disks yet… perhaps I should give them an extra check. (I got a corrupted file system om them when using my BeagleBoard)
I’ll have a look at that capcha plugin. I guess I’ll change the global language of my blogging software.
Same problem here, after flashing the latest image of ArchLinux ARM today.
So, is it a kernel issue then?
Hi again,
I don’t think that’s a kernel problem. I tried Linux, FreeBSD and NetBSD on it. Always the same after a while => filesystem curruption. I guess the hardware is just crap with lots of models (mine is made in England). I read a lot about people who have the same problem.
It’s very sad because generally I like the Raspberry. But we need to faith the truth, it’s crap.
Maybe I’m going to buy a “hackberry”.
Regards,
Royskatt
I’m thinking about putting the rootfs on an external HD. The system destroyed itself again yesterday.
I haven’t spend much time playing with my ARM boards lately…. for months that is.
I also should have a more serious look at my CubieBoard. Well…. time to get at least my Raspberry Pi up running again, at it’s playing NAS for my music and videos, and printer server.
I recall, from the early days, there were problems with certain SD cards. Might this still be the case, an incompatibility with this specific SD card? But if that’s the case…. it has been running for months without problems, and now, within a few weeks they emmerge again.
After setting the captcha to english…. it’s just making it easy for spambots. So much spam I might accidently trash a real comment… so sorry if I did.
Nevermind, I understand some basic dutch ;).
Let me give you one advice. Don’t waste your time on the Raspberry anymore and try the CubieBoard.
That looks really cool. I think I’ll buy it soon.
Maybe give it a try with an external HD, but as I said. I tried it already with putting it on on USB pendrive. Maybe a real HD will be more robust, but actually it was one of the main reasons to avoid moving parts when running such a device like Raspberry.
I spent too much time already on fixing this sh*it. No more Raspberry for me!
Just before buying a new raspberry and trying it again, I found this:
http://cxcv.de/post/34356721648/fixing-raspberry-pi-sd-card-issues
Will try this now, curious if this can fix it….