Installation Guide of Abacus (Legacy)

Content

0. Inroduction

1. Hardware setup

1.1 Front-end node hardware setup

1.2 Client node hardware setup

2. Software Setup

2.1 Front-end node setup

2.1.1 Installing Red-Hat Linux 6.2 Operating System

2.1.1.1 Pre-Install preparation

2.1.1.2 Disk partition configuration

2.1.1.3 Root Password and Create User

2.1.1.4 Network parameter configuration

2.1.1.5 Select Software Package

2.1.1.6 Configurations for the system environment

2.1.1.7 Add basic Group / User

2.1.2 Configurating Network Information Service (NIS) server
2.1.3 Configurating Network File System (NFS) server
2.1.4 Installing MySQL database server
2.1.5 Installing Message Passing Interface (MPI) system
2.1.5.1 Installation procedure

2.2.5.2 Banechmarking procedure

2.2 Client node setup

2.2.1 Installing Red-Hat Linux 6.2 Operating System

2.2.1.1 Pre-Install preparation

2.2.1.2 Disk partition configuration

2.2.1.3 Root Password and Create User

2.2.1.4 Network parameter configuration

2.2.1.5 Select Software Package

2.2.1.6 Configurations for the system environment

3. System migration from previous system

3.1 Database migration

3.2 Image Retrieval System migration

@

0. Introduction

"Abcus" is a powerful caculating tool in the Chinese history, which use small rods connect with movable counters (eg. beads) to perform caculation together, and it can help to perform very fast caculation.

"Abacus" comes from the idea of Abacus, it is also a "Beowulf Class" supercomputer, which is defined by a Cluster of workstation (beads) which can perform parallel computing through the network (rod).

The Abacus contains a front-end node and 16 client nodes, the front-end node is a dual Intel Pentium III 500MHz machine the client nodes are dual Intel Pentium III 450MHz machines (34 CPUs in total), all the machines are connected to a 100Mbps / 1Gbps Switch, which enables high speed network based parallel computing.

The Abacus use Message Passing Interface (MPI) to enable parallel computing, so the Linux system is used because LINUX have a high speed computation, and stable network implementation, also the Network Information Service (NIS) and Network File System (NFS) enables a convinent environment to manage the system.

1. Hardware Setup

1.1 Front-end node

The fron-end node configuration:

CPU : Dual Intel Pentium III processor 500 MHz with 512Kb Level 2 cache

Memory : 256MB of SDRAM

Storage : 3 x 9GB Ultra Wide SCSI Harddisk running at RAID 5

CD-ROM : SCSI CD-ROM drive

Network : 100 Mbps Ethernet Adapter connected to CSC backbone

1Gbps Ethernet Adapter (Intel Pro 1000) connect to a 3Com

100 / 1000 Mbps switch

1.2 Client node

The client node configuration:

CPU : Dual Intel Pentium III processor 450 MHz with 512Kb Level 2 cache

Memory : 128MB of SDRAM

Storage : 10GB IDE UDMA Harddisk

CD-ROM : Maxell 40x CD-ROM drive

Network : 100Mbps Ethernet Adapter (Intel Pro 100) connect to a 3Com

100 / 1000 Mbps switch

 

2. Software setup

2.1 Front-end node setup

2.1.1 Installing Red-Hat Linux 6.2 Operating System

Note: Version 6.2 of Red-Hat Linux have a very impressive change on

install process, it is fairly easy to install a usable system

2.1.1.1 Pre-Install preparation

Make sure you have enabled CD-ROM bootup in BIOS setup

and enabled RAID on the machine.

If done, reboot the system with RedHat 6.2 CDROM for installation, just press

<Enter> when you see the "LILO:" prompt for GUI install.

2.1.1.2 Disk partition configuration

Use the following:

Mount point Size Type

/boot 32MB Linux Native

swap 512MB Linux Swap

/ "grow to fill" Linux Native

-The /boot stores the kernel and should be seperated

-The swap provide "Virtual Memory" and in standard

configuration is double the Phisical Memory Size.

-The / sotre all ohter files including user files and

programs.

2.1.1.3 Root Password and Create User

Fill the both password and confirm box with same password

Do not forget the password, there is NO WAY to get back

the password.

Do not create user now.

 

2.1.1.4 Network parameter configuration

We have 2 Ethernet Adapter, alised as 'eth0' and 'eth1'

Identify which is connected to switch or (outside)

Assuming eth0 is connect to switch

In the page of eth0, fill the following:

IP : 192.168.0.100

Netmask : 255.255.255.0

Network : (leave it blank)

boardcast: (leave it blank)

eth1

IP : 144.214.130.188

Netmask : 255.255.0.0

Network : (leave it blank)

boardcast: (leave it blank)

hostname : abacus.testdomain.com (would have a better name)

gateway : 144.214.130.254

DNS : 144.214.2.32

Press "Next"

 

Check the box Enable NIS

NIS domain : testdomain.com

NIS Server : 192.168.0.1 (must be IP number)

2.1.1.5 Select Software Package

Choose Server Install

Disable the following :

Windows / DOS Connectivity

Multimedia Support

Enable the following :

NIS Server

NFS Server

Development

Kernel Development

Check the box "Select Individual Package"

Make Sure the following have choosen:

-System Environment -> Daemons -> telnet-server

-System Environment -> Daemons -> rsh-server

Wait the system install the packages, when finished

the install program will prompt for reboot, take out

the installation CD-ROM before reboot.

When reboot some error about NIS would comeout, but

don't worry, it will be fixed in the following.

2.1.1.6 Configurations for the system environment

Thanks to the configuration tools from RedHat linux, most

configurations can be done with visual tools (you just need

fill the table), but some configuration still need to do in

old way (change the configuration file).

- Login as root

Almost configuartion change need root privilege

- edit the file /etc/hosts by "pico" (or "vi")

Step 1: pico /etc/hosts

(edit the configuration file because the

"netconf" have some problem on edit hostnames)

(Tip: you can use <Ctrl-K> <Ctrl-U> to "Cut" and "Copy")

Edit the file to same as the following:

127.0.0.1 localhost.localdomain localhost

144.214.130.188 abacus.ee.cityu.edu.hk abacus

192.168.0.100 abacus.testdomain.com abacus

192.168.0.1 node01.testdomain.com node01

192.168.0.2 node02.testdomain.com node02

192.168.0.3 node03.testdomain.com node03

192.168.0.4 node04.testdomain.com node04

192.168.0.5 node05.testdomain.com node05

192.168.0.6 node06.testdomain.com node06

192.168.0.7 node07.testdomain.com node07

192.168.0.8 node08.testdomain.com node08

192.168.0.9 node09.testdomain.com node09

192.168.0.10 node10.testdomain.com node10

192.168.0.11 node11.testdomain.com node11

192.168.0.12 node12.testdomain.com node12

192.168.0.13 node13.testdomain.com node13

192.168.0.14 node14.testdomain.com node14

192.168.0.15 node15.testdomain.com node15

192.168.0.16 node16.testdomain.com node16

Step 2: Press <Ctrl-O> to save the file.

Step 3: Press <Ctrl-X> to exit.

- Type a file /etc/hosts.equiv

Step 1: pico /etc/hosts.equiv

Edit the file to same as the following:

abacus.testdomain.com

node01.testdomain.com

node02.testdomain.com

node03.testdomain.com

node04.testdomain.com

node05.testdomain.com

node06.testdomain.com

node07.testdomain.com

node08.testdomain.com

node09.testdomain.com

node10.testdomain.com

node11.testdomain.com

node12.testdomain.com

node13.testdomain.com

node14.testdomain.com

node15.testdomain.com

node16.testdomain.com

Step 2: Press <Ctrl-O> to save the file.

Step 3: Press <Ctrl-X> to exit.

 

- Check the configuration of network adapters

Step 1: netconf

Step 2: choose basic host information and check the following:

(Press <Ctrl-C> if you done something wrong and want to cancel)

(Press <tab> to switch between fields)

hostname : node01 (or node01.testdomain.com)

Adapter 1

check the box "enabled"

Config Mode : Manual

Primary name+Domain : abacus.testdomain.com

Aliases : abacus

IP : 192.168.0.1

Net device : eth0 (press <Ctrl-X> to choose)

Kernel module : e1000

Adapter 2

check the box "enabled"

Config Mode : Manual

Primary name+Domain : abacus.ee.cityu.edu.hk.com

Aliases : (blank)

IP : 144.214.130.188

Net device : eth0 (press <Ctrl-X> to choose)

Kernel module : eepro100

 

Step 3: Choose Quit

Choose "Activate Changes".

(you may need to wait few seconds)

 

2.1.1.7 Add basic accounts and groups

Step 1: userconf

Step 2: choose Group definition

Step 3: choose Add and fille the following

Group Name: mpiuser

Choose Accept

Choose Quit (To main menu of userconf)

Step 4: Choose User accounts

Step 5: Choose Add and fill the following

login name : carlos (or anything you like)

group : mpiuser (press <Ctrl-X> to choose)

Choose Accept

Choose Quit (To main menu of userconf)

Step 6: Choose Quit

Choose "Activate Changes".

(you may need to wait few seconds)

@

2.1.2 Configurating Network Information Service (NIS) server

Step 1: login as root

Step 2: chkconfig --level 3 ypserv on

chkconfig --level 4 ypserv on

chkconfig --level 5 ypserv on

Step 3: /usr/libyp/makedbm -c

Step 4: chkconfig --level 3 ypbind on

chkconfig --level 4 ypbind on

chkconfig --level 5 ypbind on

Step 5: pico /etc/sysconfig/network

find the line NISDOMAIN="xxx"

change to NISDOMAIN="testdomain.com"

if not exists, append a new line

When you finish:

Press <Ctrl-O> to save the file.

Press <Ctrl-X> to exit.

Step 6: pico /etc/yp.conf

except the remarked line (# on beginning)

it should have the following:

ypserver 192.168.0.1

ypdomain testdomain.com

When you finish:

Press <Ctrl-O> to save the file.

Press <Ctrl-X> to exit.

 

Step 7: cd /var/yp

pico ypservers

type the following:

abacus.testdomain.com

node01.testdomain.com

node02.testdomain.com

node03.testdomain.com

node04.testdomain.com

node05.testdomain.com

node06.testdomain.com

node07.testdomain.com

node08.testdomain.com

node09.testdomain.com

node10.testdomain.com

node11.testdomain.com

node12.testdomain.com

node13.testdomain.com

node14.testdomain.com

node15.testdomain.com

node16.testdomain.com@

When you finish:

Press <Ctrl-O> to save the file.

Press <Ctrl-X> to exit.

Step 8: make

(after each tim you change user information

you need to preform "make" in /var/yp)

@

2.1.3 Configurating Network File System (NFS) server

Step 1: login as root

Step 2: chkconfig --level 3 nfs on

chkconfig --level 4 nfs on

chkconfig --level 5 nfs on

Step 3: pico /etc/exports

type the following:

/home *.testdomain.com(rw, no_root_squash)

/usr/local *.testdomain.com(rw, no_root_squash)

2.1.4 Install MySQL Database Server

Step 1: Download the source tarball from:

http://www.mysql.com/Downloads/MySQL-3.22/mysql-3.22.32.tar.gz

Step 2: To unzip the source:

tar zxfv mysql-3.22.32.tar.gz

cd mysql-3.22.32

Step 3: To comiple and install: (make sure you are "root")

./configure -prefix=/usr/local

make

make install

Step 4: To install basic database "mysql":

cd scripts

mysql_install_db

Step 5: To run the server daemon:

/usr/local/share/mysql/mysql.server

Step 6: To set "root" password:

/usr/local/bin/mysqladmin -u root -p password 'new-pw'

(note: the program will prompt for password, just

press <Enter>)

Step 7: To use the mysql client:

mysql -p

Step 8: To test mysql console and basic database install:

use mysql

(note: the mysql console will display database changed

if success)

Step 9: To quit:

\q <Enter>

Step 10: Make the mysql server runs when boot up:

append a line in the file /etc/rc.d/rc.local:

/usr/local/share/mysql/mysql.server start

2.1.5 Installing Message Passing Interface (MPI) system

2.1.5.1 Installation procedure

Step 1: Download the source tarball:

ftp://ftp.mcs.anl.gov/pub/mpi/mpich.tar.gz

Step 2: Unzip the source:

tar zxfv mpich.tar.gz

cd mpich

Step 3: Install the package:

./configure -prefix=/usr/local

make

make install

Step 4: Specify the machines that use MPI

edit the file: /usr/local/share/machines.LINUX

(assume we have 3 nodes with dual CPU)

abacus

node01

node02

abacus

node01

node02

Step 5: Testing (use a user account to login)

/usr/local/sbin/tstmachines

(note: if ok, NO message will show)

2.1.5.2 Banechmarking procedure

Use Linpack benchmark (will be avaliable soon)

 

2.2 Client node installation

2.2.1 Installing Red-Hat Linux 6.2 Operating System

2.2.1.1 Pre-Install preparation

Make sure you have enabled CD-ROM bootup in BIOS setup

and enabled RAID on the machine.

If done, reboot the system with RedHat 6.2 CDROM for installation, just press

<Enter> when you see the "LILO:" prompt for GUI install.

2.2.1.2 Disk partition configuration

Use the following:

Mount point Size Type

/boot 64MB Linux Native

swap 256MB Linux Swap

/ "grow to fill" Linux Native

2.2.1.3 Root Password and Create User

Fill the both password and confirm box with same password

Do not create user now.

 

2.2.1.4 Network parameter configuration

In the page of eth0, fill the following:

IP : 192.168.0.x

Netmask : 255.255.255.0

Network : (leave it blank)

boardcast: (leave it blank)

hostname : nodex.testdomain.com (would have a better name)

gateway : 192.168.0.100

DNS : 144.214.2.32

Press "Next"

Check the box Enable NIS

NIS domain : testdomain.com

NIS Server : 192.168.0.100 (must be IP number)

 

2.2.1.5 Select Software Package

Choose Server Install

Disable the following :

Windows / DOS Connectivity

Multimedia Support

Enable the following :

NIS Server

NFS Server

Development

Kernel Development

Check the box "Select Individual Package"

Make Sure the following have choosen:

-System Environment -> Daemons -> telnet-server

-System Environment -> Daemons -> rsh-server

Wait the system install the packages, when finished

the install program will prompt for reboot, take out

the installation CD-ROM before reboot.

@

2.2.1.6 Configurations for the system environment

- Login as root

- edit the file /etc/hosts by "pico" (or "vi")

Step 1: pico /etc/hosts

(edit the configuration file because the

"netconf" have some problem on edit hostnames)

(Tip: you can use <Ctrl-K> <Ctrl-U> to "Cut" and "Copy")

Edit the file to same as the following:

127.0.0.1 localhost.localdomain localhost

192.168.0.100 abacus.testdomain.com abacus

192.168.0.1 node01.testdomain.com node01

192.168.0.2 node02.testdomain.com node02

192.168.0.3 node03.testdomain.com node03

192.168.0.4 node04.testdomain.com node04

192.168.0.5 node05.testdomain.com node05

192.168.0.6 node06.testdomain.com node06

192.168.0.7 node07.testdomain.com node07

192.168.0.8 node08.testdomain.com node08

192.168.0.9 node09.testdomain.com node09

192.168.0.10 node10.testdomain.com node10

192.168.0.11 node11.testdomain.com node11

192.168.0.12 node12.testdomain.com node12

192.168.0.13 node13.testdomain.com node13

192.168.0.14 node14.testdomain.com node14

192.168.0.15 node15.testdomain.com node15

192.168.0.16 node16.testdomain.com node16

Step 2: Press <Ctrl-O> to save the file.

Step 3: Press <Ctrl-X> to exit.

- Type a file /etc/hosts.equiv

Step 1: pico /etc/hosts.equiv

Edit the file to same as the following:

abacus.testdomain.com

node01.testdomain.com

node02.testdomain.com

node03.testdomain.com

node04.testdomain.com

node05.testdomain.com

node06.testdomain.com

node07.testdomain.com

node08.testdomain.com

node09.testdomain.com

node10.testdomain.com

node11.testdomain.com

node12.testdomain.com

node13.testdomain.com

node14.testdomain.com

node15.testdomain.com

node16.testdomain.com

Step 2: Press <Ctrl-O> to save the file.

Step 3: Press <Ctrl-X> to exit.

- Check the configuration of network adapters

Step 1: netconf

Step 2: choose basic host information and check the following:

(Press <Ctrl-C> if you done something wrong and want to cancel)

(Press <tab> to switch between fields)

("xx" is the node number)

hostname : nodexx (or nodexx.testdomain.com)

Adapter 1

check the box "enabled"

Config Mode : Manual

Primary name+Domain : nodexx.testdomain.com

Aliases : nodexx

IP : 192.168.0.xx

Net device : eth0 (press <Ctrl-X> to choose)

Kernel module : eepro100

Step 3: Choose Quit

Choose "Activate Changes".

(you may need to wait few seconds)

- Configure NFS client

Step 1: pico /etc/fstab

Step 2: add these lines:

abacus:/usr/local /usr/local nfs defaults 1 1

abacus:/home /usr/local nfs defaults 1 1

Step 3: save the file

 

3. System migration from previous system

Since the system would be very different, it would be only possible to give a guidance instead of step by step procedures.

3.1 Image Database migration

Since the Image Database in the previous system use miniSQL, and our new system uses MySQL, we need to "dump" the old database, and "import" into new database.

Step 1: Dump the database (database name is image)

msqldump image > imgdb_dump

(note: you will have a file imgdb_dump now)

Setp 2: Edit the dump file to satisify MySQL

The main difference of miniSQL and MySQL is that the declaration

of UNIQUE is different, after change these UNIQUE declaration,

it would properly imported into MySQL

MySQL:

CREATE TABLE type (

typeid int(11) DEFAULT '0' NOT NULL,

typename char(50),

filename char(120),

UNIQUE type_index (typeid)

);

miniSQL:

CREATE TABLE type (

typeid INT,

typename CHAR(50),

filename CHAR(120)

) \g

CREATE UNIQUE INDEX type_index ON type (

typeid

) \g

Step 3: Create the database for Import

Step 4: Import the dump file into new database

mysql -p image < imgdb_dump_mysql

(note: if somethings goes wrong just drop the database and

create again)

3.2 Image Retrieval System migration

This part is for change the CGI programs to use MySQL instead of miniSQL.

The datatype is very different:

miniSQL

MySQL
int *db_name; MYSQL db_name;
int *table_name; Not need
m_result result; MYSQL_RES result;
char** data_row; MYSQL_ROW data_row;

The basic functions that do the same things

miniSQL MySQL
msqlConnect(); mysql_real_connect();
msqlSelectDB(); mysql_select_db();
mysqlClose(); mysql_close();
mysqlFetchRow(); mysql_fecth_row();

(note:since the datatypes are different the usage of functions is not the same)

By Carlos, Wong Ka Man

Image Processing Lab, City University of Hong Kong, 2001