Freed from storage failure by FreeNAS

Today I’ve been freed from storage failure by FreeNAS.

From time to time technology is a real pain in the backside. We’re all more reliant on it than we’ve ever been before and when it goes wrong these days, generally it means you’ll end up suffering. Storage failure is always the worst kind of technology issue because it generally means losing something that’s important to you, e.g. family photographs.

Today though in a first for me, storage failure reared its ugly head but was slapped down by FreeNAS. Keeping a long story short, one dying drive + one next day delivery replacement drive + one set of FreeNAS wiki instructions = No need to restore from backups and NO LOSS OF DATA!

Now whilst some people might attribute all this to the components that make up FreeNAS (e.g. FreeBSD, ZFS) without the work done by the FreeNAS team to wrap these all up in a friendly, usable package, surviving a storage failure like this would have remained a pipe dream for me.

Thank you FreeNAS team!

FreeNAS Jail with Bittorrent Sync

Here are my instructions for creating a FreeNAS Jail with Bittorrent Sync (BTsync) running inside it. This uses no FreeNAS plugins whatsoever and takes full advantage of all the power that comes with having a FreeBSD core running at the heart of FreeNAS 9.1. This has been done on FreeNAS-9.1.1-RELEASE-x64.

  1. Login to your FreeNAS systems web interface.
  2. Click: Jails
  3. Click: Add Jail
    0001
  4. For Jail Name use btsync and click ok. Let FreeNAS do its thing.
    0002
  5. The jail will automatically start. You now need to SSH into your FreeNAS box as root. In the alternative SSH in as a regular user and then change to root with su.
  6. Run: jls
  7. Make a note of the number to the left of the btsync name used earlier.
  8. Run: jexec # csh
    [where # is the number from item 7. above]
  9. Run: mkdir /usr/local/btsync
  10. Run: cd /usr/local/btsync
  11. Run: fetch http://download-lb.utorrent.com/endpoint/btsync/os/FreeBSD-x64/track/stable
  12. Run: tar zxvf stable
  13. Run: rm stable
  14. Run: ./btsync –dump-sample-config > btsync.conf
  15. Run: pkg_add -rv nano
  16. Run: nano /usr/local/btsync/btsync.conf
  17. Change the following:
    1. Replace: “device_name”: “My Sync Device”, with a friendly name for your BTsync service, e.g. “device_name”: “Fred”,
    2. Replace: “storage_path” : “/home/user/.sync”, with “storage_path” : “/usr/local/btsync/.sync”,
    3. Replace: “password” : “password” with “password” : “XXXXXXXXXX” where XXXXXXXXXX is a password of your choosing.
  18. Close Nano and save the changes: Ctrl +X -> Y -> Enter
  19. Run: nano /etc/rc.d/btsync
  20. Insert the following:
    #!/bin/sh
    #
    # PROVIDE: btsync
    # REQUIRE: LOGIN DAEMON NETWORKING
    # KEYWORD: shutdown
    #
    # To enable BTSync, add this line to your /etc/rc.conf:
    #
    # btsync_enable="YES"
    #
    # And optionally these line:
    #
    # btsync_user="username" # Default is "root"
    # btsync_bin="/path/to/btsync" # Default is "/usr/local/sbin/btsync"
    
    . /etc/rc.subr
    
    name="btsync"
    rcvar="btsync_enable"
    
    load_rc_config $name
    
    required_files=$btsync_bin
    
    : ${btsync_enable="NO"}
    : ${btsync_user="root"}
    : ${btsync_bin="/usr/local/sbin/btsync"}
    : ${btsync_config="/usr/local/btsync/btsync.conf"}
    
    command=$btsync_bin
    command_args="--config ${btsync_config}"
    run_rc_command "$1"
  21. Close Nano and save the changes: Ctrl +X -> Y -> Enter
  22. Run: chmod 555 /etc/rc.d/btsync
  23. Run: nano /etc/rc.conf
  24. Insert the following:
    # BitTorrent Sync
    btsync_enable="YES"
    btsync_user="nobody"
  25. Close Nano and save the changes: Ctrl +X -> Y -> Enter
  26. Run: chown -R nobody:nogroup /usr/local/btsync
  27. Run: mv /usr/local/btsync/btsync /usr/local/sbin/
  28. Run: chown root:wheel /usr/local/sbin/btsync
  29. Login to your FreeNAS systems web interface.
  30. Click: Jails
  31. Stop the btsync jail and start it again. Bittorrent Sync should now be running.
  32. Click: Edit Jail
  33. Make a note of IP.
  34. Login to your Bittorrent Sync web interface via: XXX.XXX.XXX.XXX:8888
    [Where XXX.XXX.XXX.XXX is the IP from Item 34]

That should about do it.

Some things to remember:

  • You can use the FreeNAS admin interface to mount storage into folders that exist outside of the jail into the jail.
  • Files and folders should all be set to nobody:nogroup as a user/group combination. This killed off 99.9% of problems I was having.
  • Bittorrent Sync does not seem to update itself at the moment, at least it hasn’t thus far in this configuration. This could be because it just doesn’t do it on FreeBSD or because I’ve done something stupidly wrong. Either way you’ll have to watch the version numbers.

FreeNAS 9.1.0 Beta

Brilliant news, FreeNAS 9.1.0 Beta has been released

Will be a while before I get to play with this what with the wedding preparations, house move, new job, etc, etc. but that said I’m really looking forward to:

Improved Plugin Jail subsystem which supports multiple jails and an enhanced UI including enhancements from PC-BSD Warden.

Hopefully this will lead to some serious fun experimenting which will let me separate my current jail (which contains Serviio and other applications) into separate compartments that I can then turn on and off as necessary.

FreeNAS Jail with Transmission

Here are my instructions for creating a FreeNAS Jail with Transmission running inside it. This uses no FreeNAS plugins whatsoever and takes full advantage of all the power that comes with having a FreeBSD core running at the heart of FreeNAS 8.3.

Please note that if you already followed my guide for installing Serviio you can start from item 8.9. below.

  1. Install a plugin Jail as per the instructions available on the FreeNAS wiki here and start the Jail running.
  2. Create a user on your FreeNAS for yourself, e.g. MYUSER
  3. Enable SSH on your FreeNAS and SSH into your FreeNAS install using your client of choice.
  4. Execute the following commands:
    1. SU root
    2. jexec 1 csh
    3. mkdir /home
    4. adduser
  5. When prompted by adduser follow the instructions on-screen to create a user that matches the user you created in Step 2. Make sure that the Uid matches the user you created in Step 2. If you are not sure what the Uid number is then look it up in the FreeNAS control panel. Here’s the example:
    1. Username: MYUSER
      Full name: MYUSER
      Uid (Leave empty for default): 1111
      Login group [MYUSER]:
      Login group is MYUSER. Invite MYUSER into other groups? []: wheel
      Login class [default]:
      Shell (sh csh tcsh nologin) [sh]: csh
      Home directory [/home/MYUSER]:
      Home directory permissions (Leave empty for default):
      Use password-based authentication? [yes]:
      Use an empty password? (yes/no) [no]:
      Use a random password? (yes/no) [no]:
      Enter password:
      Enter password again:
      Lock out the account after creation? [no]:Username : MYUSER
      Password : *****
      Full Name : MYUSER
      Uid : 1111
      Class :
      Groups : MYUSER wheel
      Home : /home/MYUSER
      Home Mode :
      Shell : /bin/csh
      Locked : no
      OK? (yes/no): yes
      adduser: INFO: Successfully added (MYUSER) to the user database.
      Add another user? (yes/no): no
      Goodbye!
  6. Now we need to leave the jail and create a bridge between the FreeNAS internals and the jail. So logout of your SSH client and then turn off the jail in FreeNAS control panel. Then navigate to Services -> Plugins -> Management -> Mount Points and add a new mount point. Map a directory you can access regularly through FreeNAS as the directory /home/MYUSER within the jail. Turn the jail back on and restart your FreeNAS.
  7. SSH into your FreeNAS install using your client of choice.
  8. Execute the following commands [NB: This section will take some time to complete so set aside half an hour]:
    1. SU root
    2. jexec 1 csh
    3. portsnap fetch extract update
    4. cd /usr/ports/ports-mgmt/portmaster/ && make install clean
      Be sure to use the appropriate options for your system. If you’re not sure, check all the boxes.
      Portmaster
    5. echo ‘WITH_PKGNG=yes’ >> /etc/make.conf
    6. rehash
    7. pkg2ng
    8. portmaster -G editors/nano
    9. rehash
    10. cd /usr/ports/ftp/curl
      1. Set the options so that they match the screenshot below:
      2. curl
    11. portmaster -G net-p2p/transmission-daemon
    12. mkdir /home/MYUSER/transmission
    13. mkdir /home/MYUSER/transmission/watch
    14. mkdir /home/MYUSER/transmission/complete
    15. nano /etc/rc.conf and add lines:
      transmission_enable=”YES”
      transmission_watch_dir=”/home/MYUSER/transmission/watch”
      transmission_conf_dir=”/home/MYUSER/transmission”
      transmission_download_dir=”/home/MYUSER/transmission/complete”
      transmission_user=”MYUSER”
    16. rehash
    17. service transmission start
  9. You’re done. Transmission will now start-up on every boot. Login to it remotely and set up the directories you need as necessary!

FreeNAS Jail with Serviio 1.1

Here are my instructions for creating a FreeNAS Jail with Serviio 1.1 running inside it. This uses no FreeNAS plugins whatsoever and takes full advantage of all the power that comes with having a FreeBSD core running at the heart of FreeNAS 8.3.

  1. Install a plugin Jail as per the instructions available on the FreeNAS wiki here and start the Jail running.
  2. Create a user on your FreeNAS for yourself, e.g. MYUSER
  3. Enable SSH on your FreeNAS and SSH into your FreeNAS install using your client of choice.
  4. Execute the following commands:
    1. SU root
    2. jexec 1 csh
    3. mkdir /home
    4. adduser
  5. When prompted by adduser follow the instructions on-screen to create a user that matches the user you created in Step 2. Make sure that the Uid matches the user you created in Step 2. If you are not sure what the Uid number is then look it up in the FreeNAS control panel. Here’s the example:
    1. Username: MYUSER
      Full name: MYUSER
      Uid (Leave empty for default): 1111
      Login group [MYUSER]:
      Login group is MYUSER. Invite MYUSER into other groups? []: wheel
      Login class [default]:
      Shell (sh csh tcsh nologin) [sh]: csh
      Home directory [/home/MYUSER]:
      Home directory permissions (Leave empty for default):
      Use password-based authentication? [yes]:
      Use an empty password? (yes/no) [no]:
      Use a random password? (yes/no) [no]:
      Enter password:
      Enter password again:
      Lock out the account after creation? [no]:Username : MYUSER
      Password : *****
      Full Name : MYUSER
      Uid : 1111
      Class :
      Groups : MYUSER wheel
      Home : /home/MYUSER
      Home Mode :
      Shell : /bin/csh
      Locked : no
      OK? (yes/no): yes
      adduser: INFO: Successfully added (MYUSER) to the user database.
      Add another user? (yes/no): no
      Goodbye!
  6. Now we need to leave the jail and create a bridge between the FreeNAS internals and the jail. So logout of your SSH client and then turn off the jail in FreeNAS control panel. Then navigate to Services -> Plugins -> Management -> Mount Points and add a new mount point. Map a directory you can access regularly through FreeNAS as the directory /home/MYUSER within the jail. Turn the jail back on and restart your FreeNAS.
  7. SSH into your FreeNAS install using your client of choice.
  8. Execute the following commands [NB: This section will take some time to complete so set aside a couple of hours]:
    1. SU root
    2. jexec 1 csh
    3. portsnap fetch extract update
      1. Update 02 February 2013
      2. To ensure that you have the new pkg program installed run: pkg -v
      3. If you don’t get a response of 1.0.7 or higher then run: cd /usr/ports/ports-mgmt/pkg/ && make install clean
    4. cd /usr/ports/ports-mgmt/portmaster/ && make install clean
      Be sure to use the appropriate options for your system. If you’re not sure, check all the boxes.
      Portmaster
    5. echo ‘WITH_PKGNG=yes’ >> /etc/make.conf
    6. rehash
    7. pkg2ng
    8. portmaster -G editors/nano
    9. rehash
    10. cd /usr/ports/multimedia/ffmpeg1
    11. make config
      1. Set the options so that they match the information shown in the Serviio wiki for building ffmpeg. This is complicated so follow the screenshots below:
      2. ffmpeg1-page1
      3. ffmpeg1-page2
      4. ffmpeg1-page3
      5. press ok
    12. cd /usr/ports/graphics/frei0r/
    13. make config
    14. check Enable MMX CPU instructions then press ok
    15. portmaster -G multimedia/ffmpeg1
    16. portmaster -G net/serviio
    17. nano /etc/hosts and add line [replacing Jailed with the name of your jail from the FreeNAS control panel]:
      127.0.0.1 Jailed
    18. nano /etc/rc.conf and add lines:
      serviio_enable=”YES”
      serviio_args=”-Dserviio.remoteHost=192.168.0.100″
    19. service serviio start
  9. You’re done. Serviio will now start-up on every boot. Login to it remotely and set up the directories you need as necessary! I heartily recommend using ServiiDroid for Android which is available here.