Boot Raspberry Pi over NFS

The Raspberry Pi is typically booted from an SD card which contains the bootloader and the root partition. This can be limiting from a space and speed perspective,and writes to the card will slowly cause it’s death. SD media doesn’t typically use the same wear leveling technology that goes into a solid state drive. Today I’ll take a quick look at how you can boot your Pi from an NFS server.

The Pi can have it’s root partition on an NFS share, however you must still use the SD card to hold the firmware and bootloader. Think of this as the equivalent of having your BIOS stored on the SD, and the network share for all other storage. The catch here is that you’ll need a 2GB+ card to be able to get the unit booting normally before moving the OS over to the network share.

Before beginning if you haven’t installed Raspbian check out my article on it. Also, I recommend you install rpi-update on the Pi and get your firmware up to date ( sudo rpi-update) as mentioned in the previous article. Note that if you are on a 2GB SD card you may want to hold off on any updates until after the move to larger storage.

Prepare the NFS Share

Create a directory on the NFS server where you would like the Pi’s operating system to be stored. Share this directory by modifying /etc/exports with the proper information:

Replace  /nfsshares/rpi  with the directory where your share will be located on the server, and replace  <pi-ip>  with the IP address of the Pi (or the subnet that the Pi is on if you want this accessible to more than one device.)

Once your share is setup restart the NFS service using the following (for Debian)

This concludes the server end of the setup, now onto the Pi itself.

Mount NFS Share on the Pi

On the Pi, mount the share using the following command (the /mnt directory needs to exist before we do this.)

Where  <nfs-server-ip>  is the address of the NFS server. This will mount the NFS share under /mnt  on the Pi. Now that this is done we will setup the OS.

Copy Root Filesystem to NFS

To do this we will copy the files over with one easy command

Wasn’t that easy? Now we’ve got our filesystem on the NFS server and we’re ready to configure the Pi to boot from it.

Edit the Pi’s fstab

Now we’re going to configure the Pi to stop trying to mount the root partition stored on the SD card. This will be handled by the firmware when we configure it to look at the NFS share.

We need to edit the /etc/fstab which resides on the NFS share, so this would be  /mnt/etc/fstab  from the Pi itself or /nfsshares/rpi/etc/fstab  from the NFS server. Delete the line which mounts  /dev/mmcblk0p2  on  /  or comment it out. We don’t need this line.

Modify the Pi’s Boot Configuration

To modify the boot configuration of the Pi we need to edit  /boot/cmdline.txt  on the first partition of the SD card. This partition is located at  /dev/mmcblk0p1  on the SD.

We need to change a few settings and add a few. First change:

Then add:

Turbo mode is a fix to prevent kernel panics during high network load. If the Pi’s IP is to be assigned by DHCP, that is good. If manually assigned use the following.

Make sure you only include one or the other.

Reboot the Pi

We are done setting things up. Reboot your Pi and you should now be loading off of the NFS server if everything was done correctly. I recommend you leave the root partition intact on the SD card so you can revert to it easily if you ever need to boot the Pi without the NFS server available. Keep in mind it will not stay in sync with changes made on the NFS share.

Move Swap

By default Raspbian uses a swap file on the local SD card which is not recommended as it will wear your card. We’re going to move this swap file using a loop device over to the network share.

First, clean the old swap out

Now create a new swap file and setup the loop device, then activate the new swap

This has created a 1GB swap file in the /var directory. Check your new swap with the command free -m. This should tell you how much swap space is available. To make these changes permanent, edit the  /etc/rc.local  file and add the following before  exit 0 .

There you have it. Your network swap file will be active on boot now.

Recommended Adjustments

Here’s a couple of tweaks to system files to do some fine tuning. First, edit  /etc/default/rcS  and add ASYNCMOUNTNFS=no. Now edit  /etc/sysctl.conf  and add or change

This will set aside 12MB of free RAM to prevent the risk of a kernel panic. You may set this value lower if you wish, but on a 512MB Pi it’s not a huge concern.

Finished

Now you’ve got your Pi all setup and tweaked to efficiently boot off of an NFS server. This should give you much more options for total space and provide much more reliable storage. Of course, the first thing I did was test out the speed (which by the way is not that much better than SD simply because of the USB-based 10/100 NIC that the Pi uses.) Here’s the command to do a speed test:

This will write a 100MB file to the share and give you the time it took. My Pi was maxing at about 8.4MB/sec which is almost twice the speed of my SD card. Don’t forget to clean up the test file afterwards, and now would probably be a good time to get Raspbian all up to date as well.

That’s all folks. Enjoy your new expanded storage capabilities for the Raspberry Pi!

46 thoughts on “Boot Raspberry Pi over NFS

  1. K.Saravanan

    Thanks for the idea and for giving clear steps to do. Following the instructions i am able to mount the root file system though NFS . But i am not able to login , instead “login incorrect” message is displayed. Please help.

    1. Brian Post author

      I’m not sure where you’re seeing this. Are you saying your logs on the NFS server are showing this, or the client is showing this after booting from NFS? Are you sure you are entering the correct credentials?

      1. K.Saravanan

        The client is showing this after booting from NFS and when i try to log in the shell. The credentials are correct.

        1. Brian Post author

          That’s a strange issue. I’m wondering if something got corrupted on the move from local filesystem to NFS. Is this error repeatable (if you don’t mind trying the process from start to end again)? Can you still login as root (if that’s not what you’re already using)?

  2. PB

    Same issue — need to use sudo when issuing the cp commands otherwise permissions on system files get messed up, which prevents login.

    1. Brian Post author

      Thanks for the update! That’s my bad, I frequently use root when needed instead of sudo, and forgot to prepend sudo when writing the entry. I wrote this after having completed it myself, I should have “ate my dog food” after writing it, before posting to the web. Fixed now.

      1. PB

        I should have caught it when running the commands anyway. Your guide still saved me tons of time — thanks for the excellent write-up!

  3. david

    Hi, what is time that your RPI takes to mount NFS shares? Mine needs ~50 seconds so I’m thinking that its strange…

    1. Brian Post author

      To mount the shares or to boot? I haven’t timed it but I’d say it’s more like 50 seconds for the whole boot process. Try checking /var/log/messages on the nfs server, or on the Pi (if it boots correctly) and see if you find anything related to why it might be stalling. Does this happen when you mount the share from another linux box?

  4. Lex

    Tank you very much, works like a charm!!!
    I need to mount the NFS share with nfsvers=3 from my Synology NAS.

  5. Matt

    I’m new to this and was wondering if this is a way to set up many RPI’s.. we have a project where we have to set up 1200 RPI’s exactly the same… If we set up the first one copied that image to the NFS could we use this system to copy to multiple RPI’s at the same time?

    I’m also a bit unclear you say the RPI is booting using the NFS? So we would need to keep the RPI connected or else it will not boot?

    1. Brian Post author

      You could use this to boot many Pis. If you were to copy the folder structure on the NFS server for all the different machines you could boot them all this way. You would need to do one NFS share per Pi. You could do it with one common share for all the Pis, but that is beyond the scope of this article. Keep in mind you would have to duplicate the SD card (which contains /boot) and modify the share location in /boot/cmdline.txt in each to it’s unique share.

      The RPi would have to stay connected, it needs communication to the NFS server or it will fail to boot. Management on 1200 RPis could get interesting, this may not be the best solution for that (are all 1200 Pi’s on the same network?)

  6. Glenn

    Thanks for this – I get an error “failed to preserve ownership” when I execute “sudo cp -axv /. /mnt/.” The files copy to the NFS share on my Synology NAS. Any thoughts on why that might be / if it matters?

  7. Jekl

    Hi,

    First thanks for the great article!
    I got mine raspberry working and booting from NFS yesterday. As I tried to boot it today, it stopped in the boot proces at the dhcp-request. It keeps on trying to get an IP but somehow it doesn’t get one.
    What is going wrong ? Nothing changed in the setup.

  8. jekl

    mmmmm…. it did finally boot, after 57000 seconds!! (according to message on console)
    Dont know what caused the huge delay, I think I use a fixed IP now.

  9. jekl

    never mind previous messages….
    I booted the raspberry pi the night before with a sd-card with XBMC and dhcp. The lease time was 1 day, I guess the lease was still active when I booted with whezzy from nfs, so no ip-address was given…. now it is working ok, I changed dhcp lease time to 1 hour.

    1. Brian Post author

      Hisashi: x says to stay on the file system. Basically, if anyone has mounted a USB drive or something like that, it will not copy that over to the NFS share with that command. As for -ra, the ‘a’ flag implies recursive so the ‘r’ is pointless, and for -rp the ‘a’ flag already applies both of these. Think of -a as using -dpr with one flag.

      1. hisashi

        Brian: thank you for your clear explanation of the -x flag. I have one more question. Is there any idea to speed up file transfer from an SD card to NFS? It took several hours. I could not wait. So I pluged out the card and copied files from the usb port of the NFS server. But It could not boot it.

        1. Brian Post author

          Copying locally on the NFS server should not be a problem. As long as you maintain file ownership and permissions everything should be fine. Without knowing more I can’t really help, but this method is indeed possible. Check if the files appear when mounting the NFS share on another already booted client. Check /var/log/messages for anything that might indicate what’s going on.

          1. christian

            if your NFS server is a full fledged linux box you can mount the wheezy image directly there and “cp” at HDD speed (about 2 minutes or so), i just had to offset the mount point for the loop device (the offset is blocksize times sectors, 512 x 122880 in my case):

            # mkdir -p /mnt/raspian
            # fdisk -lu /path-to-your-wheezy-img/2014-01-07-wheezy-raspbian.img

            Disk 2014-01-07-wheezy-raspbian.img: 0 MB, 0 bytes
            255 heads, 63 sectors/track, 0 cylinders, total 0 sectors
            Units = sectors of 1 * 512 = 512 bytes

            Device Boot Start End Blocks Id System
            2014-01-07-wheezy-raspbian.img1 8192 122879 57344 c W95 FAT32 (LBA)
            Partition 1 does not end on cylinder boundary.
            2014-01-07-wheezy-raspbian.img2 122880 5785599 2831360 83 Linux
            Partition 2 does not end on cylinder boundary.

            #mount -o loop,offset=$((512*122880)) /path-to-your-wheezy-image/2014-01-07-wheezy-raspbian.img /mnt/raspbian
            #df -h
            /media/storage2/2014-01-07-wheezy-raspbian.img
            2.7G 2.0G 586M 78% /mnt/raspbian

          2. hisashi

            Thank you Brian. It successfuly booted from NFS server. I copied /media /mmcblk0p2/* with axv option to a shared holder. It took only 5 minitues.

  10. Juergen

    Big THX for this perfect information – saves me a lot of time !
    Now my Pi is running with a old 128MB SD Card

  11. Benjamin

    It appears to be important to have the order of the lines in the cmdline.txt right:

    ip=dhcp nfsroot=IP.OF.YOUR.NAS:/SHARENAME_ON_YOUR_NAS,udp,nfsvers=3 smsc95xx.turbo_mode=N
    root=/dev/nfs rootfstype=nfs

    otherwise it won’t find the root

  12. Jesse

    Hi, I have a question about moving swap, after booting from nfs root, /var/swap is already in nfsshare, not on SD card. Is it still necessary to rm /var/swap and purge dphys-swapfile and create a new /var/swap?
    Thanks.

    1. Brian Post author

      Now that you mention it, I agree this is probably the case. I don’t have my Pi handy at the moment to check, but I’d bet that /var/swap is indeed on the network share after the move to NFS without removing the dphys-swapfile package and setting up manually.

  13. Rolf

    Hi,

    Maybe someone can help on this: When I follow the instructions I get to the point where the Pi boots from the NFS server for the first time and shows the login.

    When I login wih user pi and password raspberry I get the following message:

    unable to cd /home/pi

    And I get the login prompt again.

    What is wrong?

    Thx for your help!

    Regards,
    Rolf

    1. Brian Post author

      Do you see anything in the logs on the nfs server? Is the Pi mounting properly as far as you can tell from that end?

      1. Rolf

        Brian,

        I am not sure in what logs files I have to look. (I’m quite new to Linux)

        During the setup, the Pi mounted smoothly on the NFS server. I copied all the files /. to /mnt/.
        However, during the copying I got a message that the rights could not be preserved or something like that.

        Regars,
        Rolf

  14. Rolf

    I got it to work!!! It was an issue with the user rights on the NFS server.

    I executed the following commands on the NFS server (Debian) to make it work:

    find /nfsshares/rpi -type d -exec chmod 777 {} \;
    find /nfsshares/rpi -type f -exec chmod 777 {} \;

    Still I have to figure out what went wrong in the first place.

  15. Rolf

    Hi,

    I still experience difficulties when using the instructions above:

    I use a Raspberry Pi B+ and a Debian Linux server with NFS server enabled. I manage to share a NFS Folder and mount this folder on the Pi.

    However, when I start copying the files from the Pi to the NFS server using the following command

    sudo cp -axv /. /mnt/.

    I get the message:

    cp: failed to preserve ownership for …………. invalid argument

    What is wrong?

    Regards,
    Rolf

    1. Rolf

      I finally found what was wrong. I needed to start rpcbind and nfs-common before copying the files, after that it worked as expected.

      this is how I did it:

      /etc/init.d/rpcbind start
      /etc/init.d/nfs-common start

  16. Nuno

    Dear *,

    I’m redoing my RPi configuration, from having chosen distro solely in the SD but split in between the SD and a Synology NAS server. An this is how I found this post.

    My question is.
    With the setup proposed in this post, does auto-update of Raspbian distro works ok? I assume there’s such a functionality.

    I’m asking this because from another post OpenELEC NFS Booting How-To, where an OpenELEC distro is split into the SD and NFS NAS shared space, it is said that such auto-update feature does not work anymore, and that current SD cards do not suffer anymore from the previous writ e problems and thus corruption is not anymore a big issue.

    Of course Raspbian is not OpenELEC, nonetheless was just trying to understand if booting from an NFS is still a good and robust choice. In short, I see advantages having the storage at the NFS, not so sure if NFS boot is still the way to go.

    Thank you,
    N.

  17. John Macdowall

    Excellent instructions! Worked out of the box! Be care that you update the /etc/fstab in the nfs share after you copy your OS over. Don’t let the nfs version try to remount the local /.

    Question: now that everything’s working and I’m making copies on the nfs directories for more raspberries to boot from — what files/directories/boot images do I need to maintain on the SD card/s? I want to clean up the SD card and then image copy it to multiple SD cards for the rest of the fleet of raspberries…

  18. Griff

    I was able to get this working after some fiddling, seems the modifications I made to fstab didn’t stick. I had to hook a monitor up to the Pi to see what was going on.

    I’m booting from a Synology DS213. I’m not sure if this is related, but I had to add a “-0 nolock” to the mount commands in the preparatory steps to mount my NFS share – otherwise the Pi threw a whole bunch of complaints.

  19. Steve

    Hi excellent guide, could you help with this issue:

    Pi is booting from the NFS server fine however i can no longer startx or get to a desktop envirnoment.

    startx

    Fatal server error:
    no screens found

    I’ve tried apt-get install xserver-xorg-video-all and if I boot from the SD card rather than the NFS share startx works fine. My /etc/exports: /nfsshares/rpi *(rw,sync,no_root_squash,no_all_squash,no_subtree_check)

    Any idea’s?

    Thanks

  20. Dirk

    Thanks for your great tutorial. Got it working at once. I was so tired of it after my raspberry did crash again, hope it keeps running now.

  21. David Glover

    It’s unnecessary to copy the contents of /dev. /dev is mounted as a devtmpfs filesystem and is reconstructed on every boot. Anything in /dev on the nfs fileserver will be ignored.

      1. Brian Post author

        Agreed, /dev and /proc are not necessary. Thank you both for pointing that out. As I no longer use my Pi for much I’ve let this article rot a bit too long. I should really take another look and update it.

    1. Brian Post author

      Thanks for sharing. I no longer us my Pi for much and haven’t looked at this in ages. I have not tested with any of the newer distro releases available.

Comments are closed.