nas

by sarbyn on DIY

Install KODI on raspiNAS

My raspiNAS is based on a raspberry pi 2, with 1GB of ram and a 4core ARM processor, a more powerful board able to run a NAS and also a full HD media center. So in order to simplify my home setup and remove a raspberry I’ve decided to merge the KODI mediacenter (based on OSMC) into the raspiNAS board.

We have to create the group “input” if it doesn’t exist.

sudo addgroup --system input

Edit the file

sudo nano /etc/udev/rules.d/99-input.rules

enter the following text and save it:

SUBSYSTEM==input, GROUP=input, MODE=0660
KERNEL==tty[0-9]*, GROUP=tty, MODE=0660

Create & edit the following file:

sudo nano /etc/udev/rules.d/10-permissions.rules

enter this text and save it:

# input
KERNEL=="mouse*|mice|event*",   MODE="0660", GROUP="input"
KERNEL=="ts[0-9]*|uinput",     MODE="0660", GROUP="input"
KERNEL==js[0-9]*,             MODE=0660, GROUP=input
# tty
KERNEL==tty[0-9]*,            MODE=0666
# vchiq
SUBSYSTEM==vchiq,  GROUP=video, MODE=0660

Run the following commands for user pi (if you haven’t changed your user name):

sudo usermod -a -G audio pi
sudo usermod -a -G video pi
sudo usermod -a -G input pi
sudo usermod -a -G dialout pi
sudo usermod -a -G plugdev pi
sudo usermod -a -G tty pi

To play full HD video in Kodi, you have to set

gpu_mem=160

in /boot/config.txt (or higher) and reboot the board.

Now install kodi with:

sudo apt-get install kodi

Edit /etc/default/kodi in order to start kodi at boot:

# Set this to 1 to enable startup
ENABLED=1

# The user to run Kodi as
USER=pi

# Adjust niceness of Kodi (decrease for higher priority)
NICE=-5

Source: https://www.raspberrypi.org/forums/viewtopic.php?t=99866

 
by sarbyn on DIY

RaspiNAS – DIY nas with a Raspberry

After more than one year with my NAS build over an ALIX, I bought a Raspberry pi 2 and I want to check if the “USB POWER BUG” is resolved now.

I don’t need an high speed NAS so the speed issue with the shared PCI link between USB hub and network adapter is not so important.

Advantages of a Raspberry pi 2 setup are:

  • Less power consumption: raspberry drain half the power of an Alix card
  • Flexibility: 4 USB ports instead of 2
  • Smaller: Raspberry Pi 2 is smaller than Alix board

So this is a copy&paste from my previous post, adapted for the new environment

Install Raspbian lite

Go to Raspbian download page and download the lite version

Install VIM
# apt-get install vim
Add new user
# useradd -m -s /bin/bash sarbyn
# passwd sarbyn
Add user to sudo
# visudo --- add user
Install NTFS-3G
# apt-get install ntfs-3g
Fstab and volumes
# mkdir /mnt/TimeMachine
# mkdir /mnt/MUSIC
# chown sarbyn:sarbyn /mnt/MUSIC
# chown sarbyn:sarbyn /mnt/TimeMachine

UUID=206606FF6606D606 /mnt/MUSIC ntfs-3g   rw,defaults     0       0
UUID=03c90579-88f0-4478-ae8d-adc64972a07d /mnt/TimeMachine ext4 defaults 0 0
Samba
# apt-get install samba samba-common-bin
# service samba stop
# mv /var/lib/samba /var/tmp/samba
# ln -s /var/tmp/samba /var/lib/samba
# smbpasswd -a sarbyn

[mnt]
path = /mnt
read only = Yes
guest only = Yes
guest ok = Yes

[mnt_RW]
path = /mnt
read only = No
valid user = sarbyn
browseable = yes
writable = yes
SSH
$ mkdir .ssh
$ chmod 0700 .ssh/
$ cd .ssh
$ ls
$ vim authorized_keys
$ chmod 600 authorized_keys

Disable root login and plaintext login in /etc/ssh/sshd_config file

Transmission
# apt-get install transmission-daemon
# service transmission-daemon start
FIX cracklib config

I don’t know if it is required with new Jessie debian version…but I’ve repeated it

# apt-get install cracklib-runtime
# create-cracklib-dict /usr/share/dict/*

Before
root@voyage:/var/cache/cracklib# ls -la
-rw-r--r-- 1 root root 1024 Oct 4 13:28 cracklib_dict.hwm
-rw-r--r-- 1 root root 16 Oct 4 13:28 cracklib_dict.pwd
-rw-r--r-- 1 root root 16 Oct 4 13:28 cracklib_dict.pwi
-rw-r--r-- 1 root root 65 Oct 4 13:17 src-dicts

AFTER
root@voyage:/var/cache/cracklib# ls -la
-rw-r--r-- 1 root root 1024 Oct 7 22:46 cracklib_dict.hwm
-rw-r--r-- 1 root root 412696 Oct 7 22:46 cracklib_dict.pwd
-rw-r--r-- 1 root root 22968 Oct 7 22:46 cracklib_dict.pwi
-rw-r--r-- 1 root root 65 Oct 4 13:17 src-dicts

# cd /usr/lib/
# root@voyage:/usr/lib# ln -s  /var/cache/cracklib/cracklib_dict.hwm
# root@voyage:/usr/lib# ln -s  /var/cache/cracklib/cracklib_dict.pwd
# root@voyage:/usr/lib# ln -s  /var/cache/cracklib/cracklib_dict.pwi
Netatalk

*DON’T USE NETATALK FROM DEBIAN REPOS*
The older version of netatalk are VERY…VERY unstable. Download the sources from Netatalk site and follow —> this tutorial <—

# apt-get install build-essential libevent-dev libssl-dev libgcrypt11-dev libkrb5-dev libpam0g-dev libwrap0-dev libdb-dev libtdb-dev libmysqlclient-dev avahi-daemon libavahi-client-dev libacl1-dev libldap2-dev libcrack2-dev systemtap-sdt-dev libdbus-1-dev libdbus-glib-1-dev libglib2.0-dev tracker libtracker-sparql-1.0-dev libtracker-miner-1.0-dev 
# apt-get install avahi-daemon

# insserv avahi-daemon
# insserv netatalk
# /etc/init.d/avahi-daemon start
# /etc/init.d/netatalk start

Edit “/usr/local/etc/afp.conf”.

[Global]
; Global server settings
uam list = uams_dhx.so,uams_dhx2.so
set password = yes

[MusicAFP]
    path = /mnt/MUSIC

[TimeMachineAFP]
    path = /mnt/TimeMachine
    time machine = yes

Add user to Netatalk (using the SHORT password)

# afppasswd -an sarbyn
FIX BOOT

The NTFS volume does not mount at boot, so add some lines on /etc/rc.local

# vim /etc/rc.local

service transmission-daemon stop
mount -a
swapon /mnt/MUSIC/swapfile
service transmission-daemon start

# chmod +x /etc/rc.local
NOIP client
# wget http://www.no-ip.com/client/linux/noip-duc-linux.tar.gz
# tar xzf noip-duc-linux.tar.gz
# cd no-ip-2.1.9
# make
# make install
# sudo noip2 -U 60
Install RPI monitor

Rpi monitor is a small web application that shows usage statistics of a raspberry pi board. Install rpi monitor following this link
Configure /etc/rpimonitor/storage.conf in order to setup storage monitoring and edit the /etc/rpimonitor/data.conf in order to include the storage.conf entry.

Optional – MiniDLNA support

If you need miniDLNA support follow this link

Final step – do a MicroSD backup

Now is all setted up but it’s better to do a MicroSD backup using dd

# dd if=DISK of=raspinas-backup.img bs=1m
 
by sarbyn on DIY, Examples

Rsync : daily backup

This is a simple reminder.
I’m doing a lot of changes in my NAS and my “domotic” RaspBerry (One of these days I’m going to write down something about the Raspberry project) and I need a simple backup solution for a running linux machine.

It’s very simple, use this script and add to /etc/cron.daily

rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} --delete /* /path/to/backup/folder
 
by sarbyn on Random things

Cloud backup using CrashPlan

My photo archive on my MacBook is huge and my worst nightmare is loosing all my photo.
Yes, I have a time machine drive configured on my Nas.
Yes, I have also a rsync folder on my Nas.
But I want also an online solution in case of earthquake….fire…alien invasion…and so on…
Yes…I’m paranoid.

My first choice was Google drive with the 1TB storage plan (9$ / month). It was perfect: a reliable service with an acceptable upload speed and feature set. It has only a problem: no external drive support.
My photo library is growing very fast and my internal 500gb SSD is becoming too small. So I decided to move all my library on an external drive…but with the official Google Drive client I can’t sync my library anymore.

After a little search I found Insync (15$ one time). A third part Google Client with external drive support. So beautiful!!! I started the free trial and I started to upload my library (about 100gb and 30k+ files). The first bad news was that upload speed was very….very….very slow. It took about 140 hours (5+ days) to upload all files using a 100mbit Internet connection….argh!!!
The second and worst bad news was that after some days the folder simply disappears from Google drive!!! 140 hours and then … no file on the cloud.
Insync support told me that the upload speed issue will be fixed in next release…but the lost folder was a signal: Insync is not reliable…

At the end my needs are simple:
A cloud backup solution (with unlimited space?). Mac, windows and mobile client. External drive support. Not so expensive.

My first try was with Just Cloud. In one word: s**t. The unlimited plan is not truly unlimited. No automatic video backup. No automatic backup for file larger than 1gb. A lot of restriction. No such cheaper (after the first year it will cost about 150 euros every years). After some complains with support service about the not-really-unlimited plan they give me a full money refund. Kudos to customer service. But I’ll never suggest JustCloud as a solution.

My last chance is a choice between two similar services: Backblaze and Crashplan.
Both have unlimited plan and external drive support, the monthly fee is similar ( Backblaze 50$ for 1 year – Crashplan 59.99$ for 1 year ), they have multi platform and mobile client support.

So I started the free trials (30 days for Crashplan, 15 days for Backblaze) and these are some thoughts about my experience (on internet you can find a lot of more exaustive comparison):

  • Backblaze is simpler to configure but you cannot select a single folder to backup. The Italian translation is horrible. The upload speed is faster than Crashplan but the restore options have only the “download zip” choice (100gb???) or shipped hard disk drive. No tablet (iPad and android tablet) support.
  • Crashplan has a lot of possible customization via application settings. You can select any folder to backup easily. The Mac OS X client has a good Italian translation and a better UI. The restore action can be done for any folder or file using the client, file by file. Full tablet support (and also Linux client).

It has only a great drawback: the upload speed…. I notice that it use only few bandwith: at home with a 8mbit connection the upload speed was about 400kbps. At office with a 100mbit connection the top upload speed is about 2mbps.

I tried to configure the CPU usage and upload buffer in order to speed up the upload but nothing changes: seems that this is a known issue and it is very annoying especially with the first huge backup of data. There is also the possibility to receive from Crashplan an external drive, do the initial backup and send back the drive with all your data, but for my needs I can wait some days while the client finish the upload.

Finally I found the solution and now my upload speed is about 30-40mbps (UPDATE: today is 70mbps!).

  1. First of all, for the first backup disable encryption, compression and data de-duplication from the advanced backup settings (from CrashPlan support site)
  2. Then use this link in order tu speed-up the backup
  3. If you have again speed issue try to change CrashPlan destination server using this guide and repeat until your speed grows. It works for me after 4 or 5 attempts

Finally my backup is now completed. 150GB – 40k files uploaded in about 16 hours (2 working day on a 100mbit internet connection).

I send also a mail to Code42 support team regarding my speed issue: this is its reply

Thank you for contacting Code42 support. We are a shared bandwidth service, so the speeds are not going to match your ISP’s advertised speeds. That being said, the changes you reference (http://networkrockstar.ca/2013/09/speeding-up-crashplan-backups/ and http://lifehacker.com/speed-up-crashplan-backups-and-free-up-cpu-power-with-t-1658888517) are unsupported changes to the application XML, and we highly discourage making these changes. I would suggest taking a look here:

http://support.code42.com/CrashPlan/Latest/Configuring/Unsupported_Changes_To_CrashPlan_De-Duplication_Settings

This will give you an idea and what exactly is being effected by this change, not including any other unknown consequences. Furthermore, we can’t really troubleshoot unsupported changes to the application, so I would first suggest removing the current installation and reinstalling it with a supported version, and if you continue to have issues let us know.

http://support.code42.com/CrashPlan/Latest/Troubleshooting/Uninstalling_And_Reinstalling_CrashPlan

Keep in mind that our minimum threshold is around 1Mbps, and while most users experience speeds well above this, others see speeds only marginally above this. My point is that we cannot guarantee your speeds will improve, but we can at least then look to make sure nothing is going wrong.

By the way my configuration now works fine, so I will not change it and let’s see if something goes wrong. Otherwise I will revert my config, but with the initial backup done and a lot of time saved.

Only two thing are missing: a “photo gallery” feature for mobile client and web access, and folder sharing option (two features available with Google Drive). But this is a cloud backup solution, and it simply does a cloud backup. I hope that in the next releases something will be implemented 🙂

 
by sarbyn on DIY

DIY Home NAS: add MiniDLNA support

A little addendum about the Alix + Voyage linux guide. I forget to add a little service on my NAS, useful with my Samsung smart TV: a DLNA server.

Install MiniDLNA
# sudo apt-get install minidlna
# sudo apt-get clean
Configure MiniDLNA

Now we need to configure the MUSIC and FILM folders and let MiniDLNA scans and populates its database. Minidlna config file is /etc/minidlna.conf. Stop MiniDLNA with # service minidlna stop and edit the config file. Remember to change db_dir and log_dir options (because Voyage filesystem is in read-only mode, you need to put the MiniDLNA database on your external USB drive. For example

# Path to the directory that should hold the database and album art cache.
db_dir=/mnt/MUSIC/minidlna

# Path to the directory that should hold the log file.
log_dir=/mnt/MUSIC/minidlna/logs

You need also to define the video and audio folders, for example

media_dir=V,/mnt/MUSIC/SERIE_TV
media_dir=V,/mnt/MUSIC/FILM
media_dir=A,/mnt/MUSIC/mp3
Some others minor configurations

You can also change the friendly_name option and model_name if you want to rename your MiniDLNA server name.

Now you can remountro your filesystem and start MiniDLNA server. The server will scan all your media folder and will be ready for sharing your media files with any DLNA client.

 
by sarbyn on DIY

Alix.2D + Voyage Linux = a DIY home NAS

This is my 3th attempt to build a DIY home nas.
The first attempt with a Raspberry PI was a disaster: the well-known “USB POWER BUG” makes my NAS (with 2 USB DISK) pretty unstable.
The second attempt with an ALIX board + voyage linux crashes without any reason, but I admit that the initial setup was a mess…

The goal is:

  • An home NAS with 2 external USB: a little one EXT-4 for my mac TimeMachine (using Netatalk) and a bigger one NTFS as a generic storage (using Samba)
  • TrasmissionDaemon with WEB interface, as thin torrent client

Now is up and running. But if it will die again I think that I will buy a QNAP 🙂

Install VOYAGE LINUX

Read the Voyage Linux README

Install VIM
# apt-get install vim
Fix sudo
# chown -R root:root /usr/bin/sudo
# chmod u+s /usr/bin/sudo
# chown root:root /usr/lib/sudo/sudoers.so
# chown root:root /etc/sudoers
# chown root:root /etc/sudoers.d/
# chown root:root /etc/sudoers.d/README
# chown root:root /var/lib/sudo

visudo --- add user
Add new user
# useradd -m -s /bin/bash sarbyn
# passwd sarbyn
Install NTFS-3G
# apt-get install ntfs-3g
Fstab and volumes
# mkdir /mnt/TimeMachine
# mkdir /mnt/MUSIC
# chown sarbyn:sarbyn /mnt/MUSIC
# chown sarbyn:sarbyn /mnt/TimeMachine

UUID=206606FF6606D606 /mnt/MUSIC ntfs-3g   rw,defaults     0       0
UUID=03c90579-88f0-4478-ae8d-adc64972a07d /mnt/TimeMachine ext4 defaults 0 0
Enable swap file
# mkswap /mnt/MUSIC/swapfile
# swapon /mnt/MUSIC/swapfile
Samba
# apt-get install samba samba-common-bin
# service samba stop
# mv /var/lib/samba /var/tmp/samba
# ln -s /var/tmp/samba /var/lib/samba
# smbpasswd -a sarbyn

[mnt]
path = /mnt
read only = Yes
guest only = Yes
guest ok = Yes

[mnt_RW]
path = /mnt
read only = No
valid user = sarbyn
browseable = yes
writable = yes

Next add these directories to the file etc/default/voyage-util so it now contains the line:
VOYAGE_SYNC_DIRS=”/var/cache”

SSH
$ mkdir .ssh
$ chmod 0700 .ssh/
$ cd .ssh
$ ls
$ vim authorized_keys
$ chmod 600 authorized_keys

Disable root login and plaintext login in /etc/ssh/sshd_config file

Transmission
# apt-get install transmission-daemon
# service transmission-daemon stop
# cp settings.json /etc/transmission-daemon/
# service transmission-daemon start

Next add transmission directories to the file etc/default/voyage-util so it now contains the line:
VOYAGE_SYNC_DIRS=”/var/cache /var/lib/transmission-daemon”

FIX cracklib config
# apt-get install cracklib-runtime
# create-cracklib-dict /usr/share/dict/*

Before
root@voyage:/var/cache/cracklib# ls -la
-rw-r--r-- 1 root root 1024 Oct 4 13:28 cracklib_dict.hwm
-rw-r--r-- 1 root root 16 Oct 4 13:28 cracklib_dict.pwd
-rw-r--r-- 1 root root 16 Oct 4 13:28 cracklib_dict.pwi
-rw-r--r-- 1 root root 65 Oct 4 13:17 src-dicts

AFTER
root@voyage:/var/cache/cracklib# ls -la
-rw-r--r-- 1 root root 1024 Oct 7 22:46 cracklib_dict.hwm
-rw-r--r-- 1 root root 412696 Oct 7 22:46 cracklib_dict.pwd
-rw-r--r-- 1 root root 22968 Oct 7 22:46 cracklib_dict.pwi
-rw-r--r-- 1 root root 65 Oct 4 13:17 src-dicts

# cd /usr/lib/
# root@voyage:/usr/lib# ln -s  /var/cache/cracklib/cracklib_dict.hwm
# root@voyage:/usr/lib# ln -s  /var/cache/cracklib/cracklib_dict.pwd
# root@voyage:/usr/lib# ln -s  /var/cache/cracklib/cracklib_dict.pwi
Netatalk

*DON’T USE NETATALK FROM DEBIAN REPOS*
The older version of netatalk are VERY…VERY unstable. Download the sources from Netatalk site and follow —> this tutorial <—

# apt-get install build-essential libssl-dev libgcrypt11-dev libkrb5-dev libpam0g-dev libwrap0-dev libdb-dev libmysqlclient-dev libavahi-client-dev libacl1-dev libldap2-dev libcrack2-dev systemtap-sdt-dev libdbus-1-dev libdbus-glib-1-dev libglib2.0-dev tracker libtracker-sparql-0.14-dev libtracker-miner-0.14-dev

Next add /usr/local/var/netatalk to the file etc/default/voyage-util so it now contains the line:
VOYAGE_SYNC_DIRS=”/var/cache /var/lib/transmission-daemon /usr/local/var/netatalk”

# apt-get install avahi-daemon

# insserv avahi-daemon
# insserv netatalk
# /etc/init.d/avahi-daemon start
# /etc/init.d/netatalk start

Edit “/usr/local/etc/afp.conf”.

[Global]
; Global server settings
uam list = uams_dhx.so,uams_dhx2.so
set password = yes

[MusicAFP]
    path = /mnt/MUSIC

[TimeMachineAFP]
    path = /mnt/TimeMachine
    time machine = yes

Add user to Netatalk (using the SHORT password)

# afppasswd -an sarbyn
FIX BOOT

The NTFS volume does not mount at boot, so add some lines on /etc/rc.local

# vim /etc/rc.local

service transmission-daemon stop
mount -a
swapon /mnt/MUSIC/swapfile
service transmission-daemon start

# chmod +x /etc/rc.local
Fix timezone
# ln -sf /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime
NOIP client
# wget http://www.no-ip.com/client/linux/noip-duc-linux.tar.gz
# tar xzf noip-duc-linux.tar.gz
# cd no-ip-2.1.9
# make
# make install

Next add /usr/local/etc/ to the file etc/default/voyage-util so it now contains the line:
VOYAGE_SYNC_DIRS=”/var/cache /var/lib/transmission-daemon /usr/local/var/netatalk /usr/local/etc”

PUT APT cache on external disk
# remountrw
# cd /var/cache
# rm apt
# ln -s /mnt/MUSIC/APT_CACHE/ apt
# remountro
Clear APT cache
# apt-get clean
Final step – do a CompactFlash backup

Now is all setted up but it’s better to do a CompatFlash backup using dd

# dd if=DISK of=voyage-backup.img bs=1m
EXTRA – Mount image done with DD
# apt-get install kpartx
# kpartx -a MUSIC/backup_voyage_2014_10_09.iso
# mount /dev/mapper/loop0p1 /mnt/cf -o loop,ro