Archive for June, 2014


Raspberry Pi and SD card file system corruption. I have been experiencing these issues and I can’t really pinpoint the source of the trouble. I decided to take one of my Raspberry Pi’s, install Raspbian on it, and then ISPConfig, to have a test server, with a similar configuration then the server hosting this blog, –production server– as you may. — anyhow — during the installation of some packages, the file system already got corrupted — therefore this was a never finished project.

At this point, I started to suspect the specific Pi. (I have two Model B revision 1 Pi’s) But, I cannot be sure. I have run the “Overclock stability test” script on both my Pi’s, on two different SD cards. Note: I am not overclocking, but it is a stability test, that reads and writes from the SD card. Everything seemed stable. (Also when I *did* overclock a bit)

One thing to notice about this test, it performs a series of sequential reads and writes. Perhaps that’s the difference. File system corruption occurs when writing to files. Keep in mind that SD cards are actually microcontrollers, and their programming might contain some FAT specific optimisations which cause corruption when running another file system. Keeping this in mind — the random writes method I have been suggesting earlier might not trigger the error. However… I am just speculating. In the end — it could be the power supply problem some people have been suggesting.

Anyway, enough about Pi’s, now it’s time for my ODROID U3 to do something: replacing my Raspberry Pi as NFS server. The Raspberry Pi has bad performance as an NFS server anyways. Therefore, in the past, I was about to replace it with my CubieBoard. Initial tests were promising, throughput was, –as far as I can recall– double the Pi’s. I went to sleep, to discover the next day, the CubieBoard was down. I assumed it crashed for some reason, and powercycled it. It didn’t come back up. No…. since that moment, I haven’t been able to get it to do anything. No boot from NAND or µSD card, nothing on the serial interface: it’s dead.

So… now it’s about time to replace my 8530w laptop — temporary playing server — with a more permanent solution: my UDROID-U3. I am not going to discuss the entire configuration. I have discussed configuring NFS servers before. However, there are a few things I would like to discuss. I problem I have encountered before: using new nfs-utils versions with older linux kernels. The kernel for the UDROID is 3.8.13.23-3-ARCH, which is rather old.

The problem is, when starting the nfs daemon, it requests the versions on NFS to use from the kernel. Starting at nfs-utils 1.2.9, it will include 4.1 in this request. Older kernels will not recognise this and generate an error. On my Pi, I reverted to nfs-utils 1.2.8, but now, I have a more permanent solution.

The problem:

[root@odroid andre]# systemctl status nfsd
● nfsd.service - NFS Server Daemon
   Loaded: loaded (/usr/lib/systemd/system/nfsd.service; enabled)
   Active: failed (Result: exit-code) since Fri 1999-12-31 18:00:31 MST; 14 years 5 months ago
     Docs: man:rpc.nfsd(8)
  Process: 294 ExecStopPost=/usr/bin/exportfs -a -u (code=exited, status=0/SUCCESS)
  Process: 291 ExecStop=/usr/bin/rpc.nfsd 0 (code=exited, status=0/SUCCESS)
  Process: 260 ExecStartPost=/usr/bin/exportfs -a (code=exited, status=1/FAILURE)
  Process: 233 ExecStart=/usr/bin/rpc.nfsd $NFSD_OPTS $NFSD_COUNT (code=exited, status=0/SUCCESS)
 Main PID: 233 (code=exited, status=0/SUCCESS)

Dec 31 18:00:31 odroid rpc.nfsd[233]: rpc.nfsd: Setting version failed: errno 22 (Invalid argument)
Dec 31 18:00:31 odroid exportfs[260]: exportfs: Failed to stat /srv/nfs4/grassroot: No such file or directory
Dec 31 18:00:31 odroid systemd[1]: nfsd.service: control process exited, code=exited status=1
Dec 31 18:00:31 odroid systemd[1]: Failed to start NFS Server Daemon.
Dec 31 18:00:31 odroid systemd[1]: Unit nfsd.service entered failed state.


Note to audience: I have no battery connected to the RTC, so “Dec 31 18:00:31” is a faulty date before it contacted an NTP server.
Note to self: image has an American default timezone: fix it! timedatectl set-timezone CET

The solution:
The configuration file for the nfs server is /etc/conf.d/nfs-server.conf. When you open time file, you see nothing that would suggest setting a version. No, the file only specifies command line options to nfs daemon. Putting NFSD_OPTS="-V 4" does the trick. You can specify multiple versions by comma separating them, for example NFSD_OPTS="-V 2,3,4", which would enable NFS version 2, 3 and 4.

Wishing everyone a happy midsummer!

From my previous post, you’re been able to read I am trying to cross compile and set up an AVR32 system. I have been trying some things and learning about cross compiling. I realise, last post was written from my notes, and may appear a little messy. I have make some progress, and learned how to do things, and realise I should have dome some things differently then my first attempt.

I have decided to create two kinds of posts for this topic. First will be a how-to guide. Just step-by-step guide to re-create the cross-compiler, the grasshopper root-fs, and possibly what I might create in my next steps. I am trying to set up a distcc based cross compile system, using ArchLinux’ pacman package manager and its package build utility makepkg, which has native distcc support. But it is still a long road before that works.

Apart from that, I will create a separate post with pitfalls. During this project I have come across certain error messages, which, of course, I have asked Google. Therefore I think it’s important to post those errors so it can be found.

Quite a while ago, I bought a Grasshopper. (a.k.a. ICnova AP7000 Base) I bought the thing and never even touched it… until recently. This little board has an AP7000 CPU. It’s architecture is AVR32. Not the most common thing. That’s why I bought it ;)

Well…. I have been playing around with it for the past week, or rather, some preparation to play with it, such as, a cross compiler. Since it’s a less common architecture, there are some oddities to overcome, but let’s begin. I will be using crosstool-ng to build the cross compiler. When building a cross compiler for ARM, it’s just setting the target and build, but for AVR32, the build process is not that straight forwards.

The first thing to encounter is the fact that “config.sub” doesn’t recognise the architecture, and so the build doesn’t even start. A simple “hack”, by replacing the content of “lib/ct-ng.1.19.0/scripts/config.sub” with

#!/bin/bash
echo avr32-unknown-linux-uclibc

The second thing to encounter, upstream binutils has never heard of the AVR32 architecture. Have I mentioned it’s a less common architecture?
Specifically, the Binary File Descriptor library, libbfd, misses the definitions for AVR32. ATMEL provides a
patch against binutils 2.22. It’s not the most recent version… but ok.

GCC also doesn’t know anything about the AVR32 architecture. I have found GCC 4.4.7 at the ATMEL site. There seems to be a binutils 2.23, but that gave compilation errors, so I stuck with 2.22 (at least, for now. I just want a working cross compiler)

The GNU C Library also doesn’t know anything about the AVR32 architecture. I have to use the uClibc C library instead. The crosstool-NG integration isn’t complete, as I need to provide an external configuration file for uClibc. But here I ran into a number of problems as well. First of all. I am using version 0.9.31. I haven’t got a later version to work.

During stage 2 of the C compiler compilation

[ALL  ]    In file included from /home/andre/x-tools-avr32/avr32-unknown-linux-gnu/avr32-unknown-linux-gnu/sysroot/usr/include/bits/uClibc_mutex.h:15,
[ALL  ]                     from /home/andre/x-tools-avr32/avr32-unknown-linux-gnu/avr32-unknown-linux-gnu/sysroot/usr/include/bits/uClibc_stdio.h:107,
[ALL  ]                     from /home/andre/x-tools-avr32/avr32-unknown-linux-gnu/avr32-unknown-linux-gnu/sysroot/usr/include/stdio.h:72,
[ALL  ]                     from /home/andre/cross/bin/.build/src/gcc-custom/libgcc/../gcc/tsystem.h:87,
[ALL  ]                     from /home/andre/cross/bin/.build/src/gcc-custom/libgcc/../gcc/libgcc2.c:29:
[ERROR]    /home/andre/x-tools-avr32/avr32-unknown-linux-gnu/avr32-unknown-linux-gnu/sysroot/usr/include/pthread.h:227: error: expected ')' before '*' token

The default threading implementation is “nptl”, when I set it to “linuxthreads”, the problem is fixed. The next problem:

[ALL  ]    avr32-unknown-linux-uclibc-ranlib libgcov.a
[ALL  ]    # If the gcc directory specifies which extra parts to
[ALL  ]    # build for this target, and the libgcc configuration also
[ALL  ]    # specifies, make sure they match.  This can be removed
[ALL  ]    # when the gcc directory no longer holds libgcc configuration;
[ALL  ]    # it is useful when migrating a target.
[ALL  ]    Configuration mismatch!
[ALL  ]    Extra parts from gcc directory: crtbegin.o crtbeginS.o crtend.o crten
dS.o crti.o crtn.o
[ALL  ]    Extra parts from libgcc: crtbegin.o crtbeginS.o crtbeginT.o crtend.o 
crtendS.o

requires yet another fix in a build script. Actually, I just commented out the part that checked for this configuration mismatch. The file that needs to be patched is “libgcc/Makefile.in” in the GCC files from the ATMEL site. (comment away lines 839-851)

I also ran in some problems with messages like

Error: .size expression with symbol `_init' does not evaluate to a constant
Error: .size expression with symbol `_fini' does not evaluate to a constant

For wich I found a solution here. Even though the solution at the said site is for PowerPC, the solution should apply to all architectures. The solution consists of removing two lines from a file. “.build/src/uClibc-0.9.31/libc/sysdeps/linux/$ARCH/crtn.S”, “.size _init, .-_init” and “.size _fini, .-_fini”.

With those patches, I managed to get a working cross compiler. From there, I was able to cross compile a linux kernel version 3.15. Compiling some other programs like busybox gave a few problems with the uClibc. As I mentioned, I need to feed an uClibc config file to the crosstool-NG. I had to enable certain options to make certain functionally available.

One of the problems I ran into was

loginutils/lib.a(getty.o): In function `getty_main':
getty.c:(.text.getty_main+0x42a): undefined reference to `usleep'

This site mentioned, I need to enable “SuSv3 legacy functions” to enable the build of usleep. As I was at it, I’ve enabled SuSv3 legacy macro’s and SuSv4 legacy functions as well. Just in case anything needs them. Anyhow…. I will stop writing here, as I am running into another subject. I will write more soon, as I have more to tell about the grasshopper, including issues regarding NFS booting. I needed this cross-compiled busybox, which gave me the SuSv3 problem, to use in the NFS root…. so strictly spoken, this last remark belongs in the next post :P

A friend of mine asked if I had a spare PC. Now, I had this PC, an old AMD Athlon, sitting in my storage. The PC had a Debian installation, installed for an event a few years ago, so, it didn’t contain any personal information. Anyways, I decided to re-install the machine. I decided to install Xubuntu. A “n00b-friendly” distro but with a sensible Desktop Environment.

While under Debian the machine worked fine, under Ubuntu, the first issue were graphical glitches and artefacts. The second problem is tha Flash refused to work. Firefox reported the plugin was installed, but it didn’t work. Chromium even refused to start up when the pepper version of the flash plugin was installed. *sigh*

So, I am installing Debian again. It’s a newer then the version that was previously installed, of course, but since Debian worked before, I guess it should work again.
Well…. nope… Under Debian I am having simular issues. Well… Ubuntu and Debian are related. Perhaps I should try something else. I mean… personally I would install ArchLinux, but since I will not be the user for that installation, it should be something n00b-friendly. However…. I might try my Arch installation from another machine, just as a reference. Just to rule our a hardware issue….