diskless clients

Intro

My aim is to provide support of two diskless clients in the local network. These clients should be booted up from LAN without any additional devices (flash, floppy, etc.). There is FreeBSD 4.7 server in the network.

FreeBSD 4.7 is not the same as current FreeBSD versions but differences in configuration are negligible (anyway, I'll mention these differences).

I'm sure that there are a lot of "diskless" guides but my experience could be useful too.

Environment

I have backup image with FreeBSD 4.7 system (dump(8)).

FreeBSD has apache, MySQL and some handmade services installed. It is important do not break work of these services. Upgrade to any modern FreeBSD version is not allowed (the main reason is that existing configuration is a handmade solution which is unlikely to work after upgrade).

Realtech 8139 NIC is used because it supports PXE network booting.

Restore backup

1. Write backup image to the flash drive (my 2Gb flash is enough for small 300Mb backup image).
2. Boot from FreeBSD 6.1 disk1 (I do not have other boot CDs at the moment).
3. Go to Configure, then create slice using Fdisk and partition using Label.
4. Select Fixit, then select CDROM/DVD.
5. Then mount flash drive:
   
   

   # mkdir /flash
   # mount_msdosfs /dev/da0s1 /flash
   

6. Then make new file system and mount disk drive:

   # newfs -O1 /dev/ad0s1a
   
   # mount_msdosfs /dev/ad0s1a /mnt
   

   UFS (not UFS2) is necessary because backup image contains FreeBSD 4.7.

7. Then restore image:
   

   # cd /mnt
   # cat /flash/ad0s1a.dump|restore -xf -
   
   

It doesn't take too much time because my backup image is relatively small, otherwise restore operation might have some time.

Boot

Intel's PXE boot works in the following way:

1. PXE sends bootp request on the network (it is similar to DCHP broadcasting). It expects the following information:
   • IP address / network mask
   • Default router
   • Information about where to get boot loader (the path name of the root device and boot loader name)
2. tftp(1) is used in order to provide boot loader. It is usual FreeBSD boot loader with PXE support and either nfs or tftp support.
3. The kernel starts. DHCP server provides root-path information and boot loader tries to mount this root device (tftp or nfs).

It is possible to use nfs share or memory disk as the root device. The advantages of nfs are:

• on-demand fetching of files
• nfs is more flexible as root device because allows different roots for different clients

The main advantage of memory device is simplicity: memory device is the same for all clients, memory device is always loaded into RAM, so it works faster. However, this simplicity imposes big limitation: memory disk size is limited with the amount of client's RAM. As compared with nfs tftp is fast but not so flexible.




It looks like memory device is much more appropriate for me but my final choice is nfs. Why? - I'm thinking ahead and nfs looks more appropriate for some development of server configuration.

System configuration

This FreeBSD 4.7 stay untouched since 2002. I have strong requirement do not break the entire system so I decided do not touch it at all (of course I've checked that system is configured properly). Additional argument is that this server is a local server without external connections.

DCHP

First of all it is necessary to set up DHCP server. Configuration file dhcpd.conf should contain:

  
#global
  ddns-update-style none;

  subnet 192.168.2.0 netmask 255.255.255.0 {
  range 192.168.2.1 192.168.2.254;
  default-lease-time 3600;
  max-lease-time 86400;

  option domain-name "domain.com";
  option routers 192.168.2.200;
}

# diskless
group {
  use-host-decl-names on;
  filename "boot/pxeboot";
  option root-path "192.168.2.200:/var/FreeBSD";

  host client1 {
    hardware ethernet 00:00:00:00:00:00;
  }
  host client2 {
    hardware ethernet 11:11:11:11:11:11;
  }
}
  

Then enable DHCP server by adding the following line:

  
dhcpd_enable="YES"
  

Depending on DHCP configuration other options may be required.

TFTP

I decided to use the pxeboot loader with tftp support.


In order to enable tftp make sure that your /etc/inetd.conf contains the following string:

  

tftp  dgram  udp  wait  root  /usr/libexec/tftpd  tftpd  -l -s /var/tftp
  

/var/tftp is a directory where boot loader will reside. In order to start inetd make sure that your /etc/rc.conf contains the following:

  
inetd_enable="YES"
  

Installation of boot loader:

  
# cd /usr/src/sys/boot
# make -DLOADER_TFTP_SUPPORT=YES
# cp i386/pxeldr/pxeboot /var/tftp/boot/
# cp i386/boot0/boot0 /var/tftp/boot/
# cp i386/boot2/boot1 /var/tftp/boot/
# cp i386/boot2/boot2 /var/tftp/boot/
# cp i386/mbr/mbr /var/tftp/boot/
  

It is a bit tricky because you need to copy some files manually.

NFS

tftp is used for kernel loading but root filesystem will be mounted using nfs. Make sure that nfs is properly enabled in /etc/rc.conf:

  

nfs_server_enable="YES"
mountd_enable="YES"
mountd_flags="-r"
portmap_enable="YES"
  

NOTE!:

• portmap is no longer used. Starting from 5.x FreeBSD uses rpcbind utility. So in order to configure nfs on FreeBSD 5.x/6.x you should replace portmap with rpcbind.
• Also mountd on FreeBSD 4.7 doesn't support '-p' option. This option could be used to bind mountd to a specific port. For example port 59 could be used, according to the /etc/services port 59 is "any private file service".

As I don't need anything except bare system, so my /etc/exports is very simple:

  
/var/FreeBSD -ro -maproot=root:wheel -network 192.168.2.0 -mask 255.255.255.0
  

However, is is possible to use more complex configuration. For example you could use separate /var, /tmp.

Kernel

For diskless system your kernel should support mounting of the root file system via nfs. Also it is good to have BOOTP support:

  
options         NFSCLIENT
options         NFS_ROOT
options         BOOTP
options         BOOTP_COMPAT
  

Before building your kernel make sure that your are compiling kernel of the same FreeBSD version. Otherwise you should perform make buildworld before building kernel.

  
# cd /usr/src
# make -DLOADER_TFTP_SUPPORT=YES buildkernel
# mkdir -p /var/tftp/boot/defaults
# make DESTDIR=/var/tftp installkernel
# cp /usr/src/sys/i386/conf/GENERIC.hints /var/tftp/boot/device.hints
# cp /usr/src/sys/boot/forth/loader.conf /var/tftp/boot/defaults/
# cp /usr/src/sys/boot/forth/loader.4th /var/tftp/boot/defaults/
# cp /usr/src/sys/boot/forth/support.4th /var/tftp/boot/defaults/
  

Then it is necessary to update boot/loader.rc:

  
include /boot/loader.4th
start
  

Base system

The diskless client loads kernel and modules using tftp. The root file system is mounted using nfs. The system will be installed in /var/FreeBSD. This configuration doesn't use separate /var and /tmp directories, so they will be created as memory file systems. It means that they will be cleaned up during reboot and also they will take some amount of memory.


In order to create base system it is necessary:

  
# mkdir /var/FreeBSD
# cd /usr/src
# make buildworld
# make DESTDIR=/var/FreeBSD installworld
# cd etc
# make DESTDIR=/var/FreeBSD distribution
  

On FreeBSD 5.x/6.x make distribution should be executed in /usr/src.



Of course you should adjust /etc/make.conf in order disable some unnecessary options.

Configuration

The further configuring of the base system is standard:

  
# chroot /var/FreeBSD /bin/csh
  

Then adjust the following files:

• /etc/rc.conf:
  

  sendmail_enable="NONE"
  cron_enable="NONE"
  update_motd="NO"
  root_rw_mount="NO"
  background_fsck="NO"
  newsyslog_enable="NO"
    
  
  

  The main idea is to avoid writing to the disk because anyway it is read-only.

• /etc/syslog.conf:
  

  *.*     @192.168.2.200
    
  

  Syslog should send logs to the server machine.

• /etc/fstab:
  

  192.168.2.200 /     nfs     ro       0       0
  /proc               procfs  rw       0       0

  Mount root file system via nfs.

Also it might be useful to modify /etc/resolv.conf and /etc/nsswitch.conf.

NOTES

Investigate /etc/rc.diskless1 and /etc/rc.diskless2 scripts (modern FreeBSD system has /etc/rc.initdiskless script).

I was compelled to edit one of these scripts: it was necessary to let ssh write host information to the known_hosts file. For that I've modified startup script to create /home as RAM disk and mount in RW mode.


If you allow "diskless" users some actions that may require writing to the file system, then you could modify startup scripts to let them create writable /home, /var etc. It's not so hard because /etc/rc.initdiskless already has functions (FreeBSD 4.7 also has similar functions):

  
# Create a generic memory disk
#
mount_md() {
    /sbin/mdmfs -S -i 4096 -s $1 -M md $2
}

# Create the memory filesystem if it has not already been created
#
create_md() {
    if [ "x`eval echo \\$md_created_$1`" = "x" ]; then
        if [ "x`eval echo \\$md_size_$1`" = "x" ]; then
            md_size=10240
        else
            md_size=`eval echo \\$md_size_$1`
        fi
        mount_md $md_size /$1
        /bin/chmod 755 /$1
        eval md_created_$1=created
    fi
}
  

All that you need is to create necessary disk and copy all necessary files (or may be perform some other operation).