Tag Archive: nfs


Raspberry Pi and SD card file system corruption. I have been experiencing these issues and I can’t really pinpoint the source of the trouble. I decided to take one of my Raspberry Pi’s, install Raspbian on it, and then ISPConfig, to have a test server, with a similar configuration then the server hosting this blog, –production server– as you may. — anyhow — during the installation of some packages, the file system already got corrupted — therefore this was a never finished project.

At this point, I started to suspect the specific Pi. (I have two Model B revision 1 Pi’s) But, I cannot be sure. I have run the “Overclock stability test” script on both my Pi’s, on two different SD cards. Note: I am not overclocking, but it is a stability test, that reads and writes from the SD card. Everything seemed stable. (Also when I *did* overclock a bit)

One thing to notice about this test, it performs a series of sequential reads and writes. Perhaps that’s the difference. File system corruption occurs when writing to files. Keep in mind that SD cards are actually microcontrollers, and their programming might contain some FAT specific optimisations which cause corruption when running another file system. Keeping this in mind — the random writes method I have been suggesting earlier might not trigger the error. However… I am just speculating. In the end — it could be the power supply problem some people have been suggesting.

Anyway, enough about Pi’s, now it’s time for my ODROID U3 to do something: replacing my Raspberry Pi as NFS server. The Raspberry Pi has bad performance as an NFS server anyways. Therefore, in the past, I was about to replace it with my CubieBoard. Initial tests were promising, throughput was, –as far as I can recall– double the Pi’s. I went to sleep, to discover the next day, the CubieBoard was down. I assumed it crashed for some reason, and powercycled it. It didn’t come back up. No…. since that moment, I haven’t been able to get it to do anything. No boot from NAND or µSD card, nothing on the serial interface: it’s dead.

So… now it’s about time to replace my 8530w laptop — temporary playing server — with a more permanent solution: my UDROID-U3. I am not going to discuss the entire configuration. I have discussed configuring NFS servers before. However, there are a few things I would like to discuss. I problem I have encountered before: using new nfs-utils versions with older linux kernels. The kernel for the UDROID is 3.8.13.23-3-ARCH, which is rather old.

The problem is, when starting the nfs daemon, it requests the versions on NFS to use from the kernel. Starting at nfs-utils 1.2.9, it will include 4.1 in this request. Older kernels will not recognise this and generate an error. On my Pi, I reverted to nfs-utils 1.2.8, but now, I have a more permanent solution.

The problem:

[root@odroid andre]# systemctl status nfsd
● nfsd.service - NFS Server Daemon
   Loaded: loaded (/usr/lib/systemd/system/nfsd.service; enabled)
   Active: failed (Result: exit-code) since Fri 1999-12-31 18:00:31 MST; 14 years 5 months ago
     Docs: man:rpc.nfsd(8)
  Process: 294 ExecStopPost=/usr/bin/exportfs -a -u (code=exited, status=0/SUCCESS)
  Process: 291 ExecStop=/usr/bin/rpc.nfsd 0 (code=exited, status=0/SUCCESS)
  Process: 260 ExecStartPost=/usr/bin/exportfs -a (code=exited, status=1/FAILURE)
  Process: 233 ExecStart=/usr/bin/rpc.nfsd $NFSD_OPTS $NFSD_COUNT (code=exited, status=0/SUCCESS)
 Main PID: 233 (code=exited, status=0/SUCCESS)

Dec 31 18:00:31 odroid rpc.nfsd[233]: rpc.nfsd: Setting version failed: errno 22 (Invalid argument)
Dec 31 18:00:31 odroid exportfs[260]: exportfs: Failed to stat /srv/nfs4/grassroot: No such file or directory
Dec 31 18:00:31 odroid systemd[1]: nfsd.service: control process exited, code=exited status=1
Dec 31 18:00:31 odroid systemd[1]: Failed to start NFS Server Daemon.
Dec 31 18:00:31 odroid systemd[1]: Unit nfsd.service entered failed state.


Note to audience: I have no battery connected to the RTC, so “Dec 31 18:00:31” is a faulty date before it contacted an NTP server.
Note to self: image has an American default timezone: fix it! timedatectl set-timezone CET

The solution:
The configuration file for the nfs server is /etc/conf.d/nfs-server.conf. When you open time file, you see nothing that would suggest setting a version. No, the file only specifies command line options to nfs daemon. Putting NFSD_OPTS="-V 4" does the trick. You can specify multiple versions by comma separating them, for example NFSD_OPTS="-V 2,3,4", which would enable NFS version 2, 3 and 4.

Yaourt

Yaourt is a wrapper for the pacman package manager, which allows installing from the Arch User Repository, AUR. It also supports installing from a normal user, and then prompts for a password to install through su or sudo. Yaourt is not available in the standard repositories. However, it’s available in the ArchLinux France repository:
Adding the archlinuxfr repository and install yaourt.

Network

In order to have a dynamic network configuration, for example, to easily connect to a WiFi, NetworkManager is a convenient tool.

[andre@8570w ~]$ yaourt -S openntpd networkmanager dnsmasq bluez network-manager-applet gnome-keyring
[root@8570w andre]# systemctl enable NetworkManager 
[root@8570w andre]# systemctl enable NetworkManager-wait-online

Network was previously configures to use DHCP on the ethernet connection, disable this to prevent conflicts:

[root@8570w andre]# systemctl disable dhcpcd@enp0s25

In order to start openntpd once the connection has been established, look here.

NFS

Installing NFS support and enabling the daemons:

[andre@8570w ~]$ yaourt -S nfs-utils
[root@8570w andre]# systemctl enable rpc-statd
[root@8570w andre]# systemctl enable rpc-gssd

NFS3 vs NFS4

On the old laptop, I was running NFS3 as NFS4 gave me some problems. There was a detail I’ve overlooked. Back then, I was like, I’ll come back to this issue later, and never looked into it again. The difference between mounting NFS3 and NFS4 is the fact NFS3 required the full path on the server, while NFS4 requires a relative path to the NFS root. Therefore the corresponding lines in the fstab are

NFS3:

192.168.178.49:/srv/nfs4/1000   /mnt/1000   nfs defaults,user,noauto 0 0

NFS4:

192.168.178.49:/1000   /mnt/1000   nfs4 defaults,user,noauto 0 0

Mounting from thunar

In order to mount removable media, for example, an SD card, “USB stick” or external hard disk through Thunar, we need the gvfs

[andre@8570w ~]$ yaourt -S gvfs

Before I installed gvfs, I had my user mountable NFS mounts from my fstab listed in Thunar, after installing they were gone. To get them back, add x-gvfs-show to the mount options in fstab

192.168.178.49:/1000   /mnt/1000   nfs4 defaults,user,noauto,x-gvfs-show 0 0

Touchpad

This laptop has a huge touchpad. By default it was set to “two finger scrolling”. Really annoying. This is not a Mac for God’s sake! After a day, I’m getting RSI pains. No good… but fortunately, this is just a configuration option, and can be set to normal edge scrolling.

woohoo --- i can use edge scrolling

Localisation

At first, I didn’t configure a language. Just running in English was fine, but what I didn’t realise is that this meant I was running in “C”, meaning it wasn’t using UTF-8 encoding, which prevented me from listening to Russian music, as their file names contain non-ASCII characters.

[andre@8570w ~]$ echo $LANG
C

Before setting any locale, it has to be generated. Uncomment the desired locales in /etc/locale-gen and run locale-gen

[root@8570w andre]# locale-gen

So, I am going to configure a global setting of UK English with UTF-8,

[root@8570w andre]#  localectl set-locale LANG=en_GB.UTF-8

and set my own profile to Dutch by adding the following to ~/.bashrc

export LANG=nl_NL.UTF-8

Now, let’s get started with NFS, right? Well… first things first… this configuration is supposed to replace the BeagleBoard. As mentioned, I have occidentally overwritten its SD card. This also means I cannot access its current configuration.

I cannot login to the device anymore over ssh or serial connection due incorrect content of the root filesystem. However the NFS server is still serving perfectly fine. Since I cannot login, I cannot gracefully unmount the filesystems. The point is, I am listening to music. The music is on the USB hard disks connected to the BeagleBoard, which I will have to shut down in order to connect them to the Raspberry Pi to configure the mount points. If I could access the old configureation, I could just copy over the entries in /etc/fstab.

I will be using UUIDs to identify the partition to mount, as, when using USB, I cannot predict which device name it will have. I mean, if I disconnect, and reconnect the hard disks on a different USB port, I wish not to have to change the configuration. Therefore I will use UUIDs so no matter what device name it gets, it will work. So, let’s delay configuring the mount points to a later point. We will configure the mount points (same as currently on the BeagleBoard):

/mnt/1000 : 1000 GB USB HARD DISK, DATA
/mnt/1500 : 1500 GB USB HARD DISK, DATA
/home : 250 GB USB HARD DISK, HOME PARTITION

If I just create the directories, the rest of the procedure should work, and the actual hard disks can be added later. Let’s install the NFS software. Also… let’s install vim, the editor I am used to, as it isn’t installed by default in an ArchLinux installation (vi and nano are)

# pacman -S nfs-utils vim

For security reasons, we will set up an nfs root

# mkdir /srv/nfs4
# mkdir /srv/nfs4/1000
# mkdir /srv/nfs4/1500
# mkdir /srv/nfs4/andre

and add the following to /etc/fstab

/mnt/1000 /srv/nfs4/1000 none bind 0 0
/mnt/1500 /srv/nfs4/1500 none bind 0 0
/home/andre /srv/nfs4/andre none bind 0 0

and set up the exports in /etc/exports

/srv/nfs4/ 192.168.178.1/24(rw,fsid=0,no_subtree_check)
/srv/nfs4/1000 192.168.178.1/24(rw,no_subtree_check,nohide)
/srv/nfs4/1500 192.168.178.1/24(rw,no_subtree_check,nohide)
/srv/nfs4/andre 192.168.178.1/24(rw,no_subtree_check,nohide)

To start the nfs server, just now

# systemctl start nfsd.service rpc-idmapd.service rpc-mountd.service rpcbind.service

Verifying the exports from my desktop machine:

$ showmount -e 192.168.178.49
Export list for 192.168.178.49:
/srv/nfs4/andre 192.168.178.1/24
/srv/nfs4/1500 192.168.178.1/24
/srv/nfs4/1000 192.168.178.1/24
/srv/nfs4 192.168.178.1/24

Since this looks fine, I can enable startup during boot using

# systemctl enable nfsd.service rpc-idmapd.service rpc-mountd.service rpcbind.service

This is the new way of configuring services at boot, as compared to the old /etc/rc.conf method.

So, I guess it’s about time to shut down the BeagleBoard and replace it by the Raspberry Pi. Even though there are some more services to be configured, such as my IPv6 tunnel, printer server, scanner server, and so on. As I am unable to gracefully shut the BeagleBoard down, and due its instability, I guess I’d better fsck all the external hard disks before putting them online in the new configuration.

One thing to verify before proceeding is to verify the Raspberry Pi boots up headless, and I’ve heard this might be an issue. However, the system comes up perfectly fine.

In order to listen some music while these operations are in progress, I will revert to vinyl. The real way to listen to music ;)
PS. I bought this record last wednesday. It’s awesome to see new music appear on vinyl only ;)