Revision as of 00:23, 22 October 2013

Hardware

The cluster was built from parts ordered from Newegg. The important components of the system are as follows:

PSU: (1x) Corsair RM850 850W 92% Efficiency Active-PFC
Mobo: (1x) Gigabyte GA-EP45-UD3P (rev 1.1)
CPU: (1x) Intel Core 2 Quad Q8300 2.5GHz
RAM: (4x) Corsair XMS2 DDR2 800 (PC2 6400) 2GB
Gfx: (1x) NVIDIA GeForce 9500 GT PCI-E 2.0 x8
RAID: (1x) 3ware 9650SE-4LMPL PCI-E 2.0 x1
RAID: (4x) WD 1TB 7200RPM SATA-II HDD

Operating System

The cluster currently runs Debian 7.0 amd64. The recommended method of installing the base operating system is to download the Small CD ISO from Debian and burn it to a CD/DVD.

The (MBR) partitioning of the main 1TB disk should be:

Num.	Size	Filesystem	Mount point
1	256 MB	ext2	/boot
2	317 GB	ext4	/
3	2.5 GB	swap	swap

The 2.7TB RAID5 array should already be partitioned (GPT). At all costs, maintain the integrity of the data already contained on the RAID5 array. Keep frequent backups of the array to at least one external source.

Other important settings to choose during the installation are summarized below: FIXME

Software

The software on the cluster consists of packages installed through the Debian repositories and third-party software installed into /opt. It is highly recommended that the software in this section be installed in the order it is listed here, if possible.

Debian Packages

Install Debian packages before third-party software, if at all possible. Many of these commands will require mounting the administrator flash drive, in which case this documentation assumes it will be mounted to /mnt/flash.

Initial configuration

We don't want all the ports on our precious cluster to be out there flapping in the breeze. Install some files that loosely restrict which daemons can provide services to which addresses:

install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/hosts.allow /etc/
install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/hosts.deny /etc/

Install a file that defines WAN and LAN IP-hostname mappings:

install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/hosts /etc/

Add support for hosting both 32-bit and 64-bit packages on this 64-bit system:

dpkg --add-architecture i386
apt-get update

Now install the the following packages using apt-get:

libc6-i686
libx11-6:i386
libxext6:i386

The groundwork

Some miscellaneous software packages need to be installed as boilerplate. Install the following packages using apt-get...

First, install some required shells:

tcsh
bash-completion

Lay down a decent text editor:

vim

Add a few extra utilities:

zip
nmap
links
rsync
screen
gnuplot
imagemagick

Now install some more language support:

pkg-config
build-essential
libtool
texlive
texi2html
ruby
gfortran
perl

Python

A few Python packages are standard fare for programmers in the group. Install the following packages using apt-get:

python3
python2.7
python3.2
python-numpy
python-sklearn
python-matplotlib

R

The R language is useful to some people, I guess.

r-base
r-recommended

Sudo

Install the sudo package using apt-get. Then add the following line to the /etc/sudoers file:

%bionmr ALL=(ALL:ALL) ALL

OpenSSH

The OpenSSH daemon allows users to log into the cluster from anywhere in the world using their RSA private key. First, install the following packages using apt-get:

openssh-client
openssh-server

Then install the /etc/ssh/sshd_config file from the administrator flash drive and restart the SSH daemon:

install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/ssh/sshd_config /etc/ssh/
service ssh restart

NFS

The NFS server allows users to access their files on campus from Debian Linux workstations. First, install the following packages using apt-get:

nfs-kernel-server
portmap

Then install the /etc/exports file from the administrator flash drive and restart the NFS daemon:

install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/exports /etc/
service nfs-kernel-server restart

Samba

The Samba server allows users to access their files on campus from Windows workstations. First, install the following packages using apt-get:

cifs-utils
samba

Then install the Samba configuration and password files and restart the Samba daemon:

install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/samba/smb.conf /etc/samba/
install -v -m 600 -o root -g root /mnt/flash/admin-tools/linux/cluster/var/lib/samba/passdb.tdb /var/lib/samba/
service samba restart

MySQL

FIXME

mysql-client
mysql-server

Apache

The Apache server hosts the group website, plain and simple. Install the following packages using apt-get:

apache2
apache2-doc

Then install the password file and an updated site configuration file and restart the Apache daemon:

install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/apache2/htpasswd /etc/apache2/
install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/apache2/sites-available/default /etc/apache2/sites-available
service apache2 restart

PHP

The PHP interpreter is required to host the website using the UNL university style templates. Install the following packages using apt-get:

php5
php5-mysql
php5-sqlite
libapache-mod-php5

Then download and install Pyrus (PEAR2) using the following commands:

wget http://pear2.php.net/pyrus.phar
install -v -m 644 -o root -g root pyrus.phar /usr/bin/
rm pyrus.phar

Open a new file in vi called /usr/bin/pyrus and add the following text:

#!/bin/bash
/usr/bin/php -dphar.readonly=0 /usr/bin/pyrus.phar $*

Save the file and make it executable:

chmod 755 /usr/bin/pyrus

Now run pyrus for the first time. Tell it yes to initialize the configuration. Tell it to install packages to /usr/share/pear by default, when it asks. Now install the UNL template package:

pyrus channel-discover pear.unl.edu
pyrus install -o pear.unl.edu/UNL_Templates-beta

Finally, install updated configuration files for PHP, restart the Apache server and remove the placeholder html page:

install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/php5/apache2/php.ini /etc/php5/apache2/
install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/php5/cli/php.ini /etc/php5/cli/
service apache2 restart
rm /var/www/index.html

DHCP

The cluster LAN is managed by a DHCP server on the second Ethernet port. Install the following package using apt-get:

isc-dhcp-server
bridge-utils
dnsmasq

Install the configuration files for the DHCP server:

install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/dhcp/dhcpd.conf /etc/dhcp/
install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/default/isc-dhcp-server /etc/default/
install -v -m 644 -o root -g root /mnt/flash/admin-tools/linux/cluster/etc/network/interfaces /etc/network/

Bring up the second Ethernet interface and restart the DHCP server:

ifup eth1
service isc-dhcp-server restart

Run the following commands:

brctl addbr br0
brctl addif br0 eth0 eth1
ifup br0

Octave

The Octave programming environment provides users with mathematical routines that simplify data handling. Install the following packages using apt-get:

octave
liboctave-dev
octave-audio
octave-control
octave-data-smoothing
octave-fixed
octave-fpl
octave-general
octave-geometry
octave-image
octave-io
octave-linear-algebra
octave-odepkg
octave-optim
octave-optiminterp
octave-pkg-dev
octave-plot
octave-quaternion
octave-signal
octave-specfun
octave-statistics
octave-strings

Third-party Software