1.1 Front-end node hardware setup
1.2 Client node hardware setup
2.1 Front-end node setup
2.1.1 Installing Red-Hat Linux 6.2 Operating System
2.1.1.1 Pre-Install preparation
2.1.1.2 Disk partition configuration
2.1.1.3 Root Password and Create User
2.1.1.4 Network parameter configuration
2.1.1.5 Select Software Package
2.1.1.6 Configurations for the system environment
2.1.1.7 Add basic Group / User
2.1.2 Configurating Network Information Service (NIS) server
2.1.3 Configurating Network File System (NFS) server
2.1.4 Installing MySQL database server
2.1.5 Installing Message Passing Interface (MPI) system
2.1.5.1 Installation procedure2.2.5.2 Banechmarking procedure
2.2 Client node setup
2.2.1 Installing Red-Hat Linux 6.2 Operating System
2.2.1.1 Pre-Install preparation
2.2.1.2 Disk partition configuration
2.2.1.3 Root Password and Create User
2.2.1.4 Network parameter configuration
2.2.1.5 Select Software Package
2.2.1.6 Configurations for the system environment
3.1 Database migration
3.2 Image Retrieval System migration
¡@
"Abcus" is a powerful caculating tool in the Chinese history, which use small rods connect with movable counters (eg. beads) to perform caculation together, and it can help to perform very fast caculation.
"Abacus" comes from the idea of Abacus, it is also a "Beowulf Class" supercomputer, which is defined by a Cluster of workstation (beads) which can perform parallel computing through the network (rod).
The Abacus contains a front-end node and 16 client nodes, the front-end node is a dual Intel Pentium III 500MHz machine the client nodes are dual Intel Pentium III 450MHz machines (34 CPUs in total), all the machines are connected to a 100Mbps / 1Gbps Switch, which enables high speed network based parallel computing.
The Abacus use Message Passing Interface (MPI) to enable parallel computing, so the Linux system is used because LINUX have a high speed computation, and stable network implementation, also the Network Information Service (NIS) and Network File System (NFS) enables a convinent environment to manage the system.
The fron-end node configuration:
CPU : Dual Intel Pentium III processor 500 MHz with 512Kb Level 2 cache
Memory : 256MB of SDRAM
Storage : 3 x 9GB Ultra Wide SCSI Harddisk running at RAID 5
CD-ROM : SCSI CD-ROM drive
Network : 100 Mbps Ethernet Adapter connected to CSC backbone
1Gbps Ethernet Adapter (Intel Pro 1000) connect to a 3Com
100 / 1000 Mbps switch
The client node configuration:
CPU : Dual Intel Pentium III processor 450 MHz with 512Kb Level 2 cache
Memory : 128MB of SDRAM
Storage : 10GB IDE UDMA Harddisk
CD-ROM : Maxell 40x CD-ROM drive
Network : 100Mbps Ethernet Adapter (Intel Pro 100) connect to a 3Com
100 / 1000 Mbps switch
Note: Version 6.2 of Red-Hat Linux have a very impressive change on
install process, it is fairly easy to install a usable system
Make sure you have enabled CD-ROM bootup in BIOS setup
and enabled RAID on the machine.
If done, reboot the system with RedHat 6.2 CDROM for installation, just press
<Enter> when you see the "LILO:" prompt for GUI install.
Use the following:
Mount point Size Type
/boot 32MB Linux Native
swap 512MB Linux Swap
/ "grow to fill" Linux Native
-The /boot stores the kernel and should be seperated
-The swap provide "Virtual Memory" and in standard
configuration is double the Phisical Memory Size.
-The / sotre all ohter files including user files and
programs.
Fill the both password and confirm box with same password
Do not forget the password, there is NO WAY to get back
the password.
Do not create user now.
We have 2 Ethernet Adapter, alised as 'eth0' and 'eth1'
Identify which is connected to switch or (outside)
Assuming eth0 is connect to switch
In the page of eth0, fill the following:
IP : 192.168.0.100
Netmask : 255.255.255.0
Network : (leave it blank)
boardcast: (leave it blank)
eth1
IP : 144.214.130.188
Netmask : 255.255.0.0
Network : (leave it blank)
boardcast: (leave it blank)
hostname : abacus.testdomain.com (would have a better name)
gateway : 144.214.130.254
DNS : 144.214.2.32
Press "Next"
Check the box Enable NIS
NIS domain : testdomain.com
NIS Server : 192.168.0.1 (must be IP number)
Choose Server Install
Disable the following :
Windows / DOS Connectivity
Multimedia Support
Enable the following :
NIS Server
NFS Server
Development
Kernel Development
Check the box "Select Individual Package"
Make Sure the following have choosen:
-System Environment -> Daemons -> telnet-server
-System Environment -> Daemons -> rsh-server
Wait the system install the packages, when finished
the install program will prompt for reboot, take out
the installation CD-ROM before reboot.
When reboot some error about NIS would comeout, but
don't worry, it will be fixed in the following.
Thanks to the configuration tools from RedHat linux, most
configurations can be done with visual tools (you just need
fill the table), but some configuration still need to do in
old way (change the configuration file).
- Login as root
Almost configuartion change need root privilege
- edit the file /etc/hosts by "pico" (or "vi")
Step 1: pico /etc/hosts
(edit the configuration file because the
"netconf" have some problem on edit hostnames)
(Tip: you can use <Ctrl-K> <Ctrl-U> to "Cut" and "Copy")
Edit the file to same as the following:
127.0.0.1 localhost.localdomain localhost
144.214.130.188 abacus.ee.cityu.edu.hk abacus
192.168.0.100 abacus.testdomain.com abacus
192.168.0.1 node01.testdomain.com node01
192.168.0.2 node02.testdomain.com node02
192.168.0.3 node03.testdomain.com node03
192.168.0.4 node04.testdomain.com node04
192.168.0.5 node05.testdomain.com node05
192.168.0.6 node06.testdomain.com node06
192.168.0.7 node07.testdomain.com node07
192.168.0.8 node08.testdomain.com node08
192.168.0.9 node09.testdomain.com node09
192.168.0.10 node10.testdomain.com node10
192.168.0.11 node11.testdomain.com node11
192.168.0.12 node12.testdomain.com node12
192.168.0.13 node13.testdomain.com node13
192.168.0.14 node14.testdomain.com node14
192.168.0.15 node15.testdomain.com node15
192.168.0.16 node16.testdomain.com node16
Step 2: Press <Ctrl-O> to save the file.
Step 3: Press <Ctrl-X> to exit.
- Type a file /etc/hosts.equiv
Step 1: pico /etc/hosts.equiv
Edit the file to same as the following:
abacus.testdomain.com
node01.testdomain.com
node02.testdomain.com
node03.testdomain.com
node04.testdomain.com
node05.testdomain.com
node06.testdomain.com
node07.testdomain.com
node08.testdomain.com
node09.testdomain.com
node10.testdomain.com
node11.testdomain.com
node12.testdomain.com
node13.testdomain.com
node14.testdomain.com
node15.testdomain.com
node16.testdomain.com
Step 2: Press <Ctrl-O> to save the file.
Step 3: Press <Ctrl-X> to exit.
- Check the configuration of network adapters
Step 1: netconf
Step 2: choose basic host information and check the following:
(Press <Ctrl-C> if you done something wrong and want to cancel)
(Press <tab> to switch between fields)
hostname : node01 (or node01.testdomain.com)
Adapter 1
check the box "enabled"
Config Mode : Manual
Primary name+Domain : abacus.testdomain.com
Aliases : abacus
IP : 192.168.0.1
Net device : eth0 (press <Ctrl-X> to choose)
Kernel module : e1000
Adapter 2
check the box "enabled"
Config Mode : Manual
Primary name+Domain : abacus.ee.cityu.edu.hk.com
Aliases : (blank)
IP : 144.214.130.188
Net device : eth0 (press <Ctrl-X> to choose)
Kernel module : eepro100
Step 3: Choose Quit
Choose "Activate Changes".
(you may need to wait few seconds)
Step 1: userconf
Step 2: choose Group definition
Step 3: choose Add and fille the following
Group Name: mpiuser
Choose Accept
Choose Quit (To main menu of userconf)
Step 4: Choose User accounts
Step 5: Choose Add and fill the following
login name : carlos (or anything you like)
group : mpiuser (press <Ctrl-X> to choose)
Choose Accept
Choose Quit (To main menu of userconf)
Step 6: Choose Quit
Choose "Activate Changes".
(you may need to wait few seconds)
¡@
Step 1: login as root
Step 2: chkconfig --level 3 ypserv on
chkconfig --level 4 ypserv on
chkconfig --level 5 ypserv on
Step 3: /usr/libyp/makedbm -c
Step 4: chkconfig --level 3 ypbind on
chkconfig --level 4 ypbind on
chkconfig --level 5 ypbind on
Step 5: pico /etc/sysconfig/network
find the line NISDOMAIN="xxx"
change to NISDOMAIN="testdomain.com"
if not exists, append a new line
When you finish:
Press <Ctrl-O> to save the file.
Press <Ctrl-X> to exit.
Step 6: pico /etc/yp.conf
except the remarked line (# on beginning)
it should have the following:
ypserver 192.168.0.1
ypdomain testdomain.com
When you finish:
Press <Ctrl-O> to save the file.
Press <Ctrl-X> to exit.
Step 7: cd /var/yp
pico ypservers
type the following:
abacus.testdomain.com
node01.testdomain.com
node02.testdomain.com
node03.testdomain.com
node04.testdomain.com
node05.testdomain.com
node06.testdomain.com
node07.testdomain.com
node08.testdomain.com
node09.testdomain.com
node10.testdomain.com
node11.testdomain.com
node12.testdomain.com
node13.testdomain.com
node14.testdomain.com
node15.testdomain.com
node16.testdomain.com¡@
When you finish:
Press <Ctrl-O> to save the file.
Press <Ctrl-X> to exit.
Step 8: make
(after each tim you change user information
you need to preform "make" in /var/yp)
¡@
Step 1: login as root
Step 2: chkconfig --level 3 nfs on
chkconfig --level 4 nfs on
chkconfig --level 5 nfs on
Step 3: pico /etc/exports
type the following:
/home *.testdomain.com(rw, no_root_squash)
/usr/local *.testdomain.com(rw, no_root_squash)
2.1.4 Install MySQL Database Server
Step 1: Download the source tarball from:
http://www.mysql.com/Downloads/MySQL-3.22/mysql-3.22.32.tar.gz
Step 2: To unzip the source:
tar zxfv mysql-3.22.32.tar.gz
cd mysql-3.22.32
Step 3: To comiple and install: (make sure you are "root")
./configure -prefix=/usr/local
make
make install
Step 4: To install basic database "mysql":
cd scripts
mysql_install_db
Step 5: To run the server daemon:
/usr/local/share/mysql/mysql.server
Step 6: To set "root" password:
/usr/local/bin/mysqladmin -u root -p password 'new-pw'
(note: the program will prompt for password, just
press <Enter>)
Step 7: To use the mysql client:
mysql -p
Step 8: To test mysql console and basic database install:
use mysql
(note: the mysql console will display database changed
if success)
Step 9: To quit:
\q <Enter>
Step 10: Make the mysql server runs when boot up:
append a line in the file /etc/rc.d/rc.local:
/usr/local/share/mysql/mysql.server start
2.1.5.1 Installation procedure
Step 1: Download the source tarball:
Step 2: Unzip the source:
tar zxfv mpich.tar.gz
cd mpich
Step 3: Install the package:
./configure -prefix=/usr/local
make
make install
Step 4: Specify the machines that use MPI
edit the file: /usr/local/share/machines.LINUX
(assume we have 3 nodes with dual CPU)
abacus
node01
node02
abacus
node01
node02
Step 5: Testing (use a user account to login)
/usr/local/sbin/tstmachines
(note: if ok, NO message will show)
2.1.5.2 Banechmarking procedure
Use Linpack benchmark (will be avaliable soon)
2.2.1.1 Pre-Install preparation
Make sure you have enabled CD-ROM bootup in BIOS setup
and enabled RAID on the machine.
If done, reboot the system with RedHat 6.2 CDROM for installation, just press
<Enter> when you see the "LILO:" prompt for GUI install.
2.2.1.2 Disk partition configuration
Use the following:
Mount point Size Type
/boot 64MB Linux Native
swap 256MB Linux Swap
/ "grow to fill" Linux Native
2.2.1.3 Root Password and Create User
Fill the both password and confirm box with same password
Do not create user now.
2.2.1.4 Network parameter configuration
In the page of eth0, fill the following:
IP : 192.168.0.x
Netmask : 255.255.255.0
Network : (leave it blank)
boardcast: (leave it blank)
hostname : nodex.testdomain.com (would have a better name)
gateway : 192.168.0.100
DNS : 144.214.2.32
Press "Next"
Check the box Enable NIS
NIS domain : testdomain.com
NIS Server : 192.168.0.100 (must be IP number)
2.2.1.5 Select Software Package
Choose Server Install
Disable the following :
Windows / DOS Connectivity
Multimedia Support
Enable the following :
NIS Server
NFS Server
Development
Kernel Development
Check the box "Select Individual Package"
Make Sure the following have choosen:
-System Environment -> Daemons -> telnet-server
-System Environment -> Daemons -> rsh-server
Wait the system install the packages, when finished
the install program will prompt for reboot, take out
the installation CD-ROM before reboot.
¡@
2.2.1.6 Configurations for the system environment
- Login as root
- edit the file /etc/hosts by "pico" (or "vi")
Step 1: pico /etc/hosts
(edit the configuration file because the
"netconf" have some problem on edit hostnames)
(Tip: you can use <Ctrl-K> <Ctrl-U> to "Cut" and "Copy")
Edit the file to same as the following:
127.0.0.1 localhost.localdomain localhost
192.168.0.100 abacus.testdomain.com abacus
192.168.0.1 node01.testdomain.com node01
192.168.0.2 node02.testdomain.com node02
192.168.0.3 node03.testdomain.com node03
192.168.0.4 node04.testdomain.com node04
192.168.0.5 node05.testdomain.com node05
192.168.0.6 node06.testdomain.com node06
192.168.0.7 node07.testdomain.com node07
192.168.0.8 node08.testdomain.com node08
192.168.0.9 node09.testdomain.com node09
192.168.0.10 node10.testdomain.com node10
192.168.0.11 node11.testdomain.com node11
192.168.0.12 node12.testdomain.com node12
192.168.0.13 node13.testdomain.com node13
192.168.0.14 node14.testdomain.com node14
192.168.0.15 node15.testdomain.com node15
192.168.0.16 node16.testdomain.com node16
Step 2: Press <Ctrl-O> to save the file.
Step 3: Press <Ctrl-X> to exit.
- Type a file /etc/hosts.equiv
Step 1: pico /etc/hosts.equiv
Edit the file to same as the following:
abacus.testdomain.com
node01.testdomain.com
node02.testdomain.com
node03.testdomain.com
node04.testdomain.com
node05.testdomain.com
node06.testdomain.com
node07.testdomain.com
node08.testdomain.com
node09.testdomain.com
node10.testdomain.com
node11.testdomain.com
node12.testdomain.com
node13.testdomain.com
node14.testdomain.com
node15.testdomain.com
node16.testdomain.com
Step 2: Press <Ctrl-O> to save the file.
Step 3: Press <Ctrl-X> to exit.
- Check the configuration of network adapters
Step 1: netconf
Step 2: choose basic host information and check the following:
(Press <Ctrl-C> if you done something wrong and want to cancel)
(Press <tab> to switch between fields)
("xx" is the node number)
hostname : nodexx (or nodexx.testdomain.com)
Adapter 1
check the box "enabled"
Config Mode : Manual
Primary name+Domain : nodexx.testdomain.com
Aliases : nodexx
IP : 192.168.0.xx
Net device : eth0 (press <Ctrl-X> to choose)
Kernel module : eepro100
Step 3: Choose Quit
Choose "Activate Changes".
(you may need to wait few seconds)
- Configure NFS client
Step 1: pico /etc/fstab
Step 2: add these lines:
abacus:/usr/local /usr/local nfs defaults 1 1
abacus:/home /usr/local nfs defaults 1 1
Step 3: save the file
Since the system would be very different, it would be only possible to give a guidance instead of step by step procedures.
Since the Image Database in the previous system use miniSQL, and our new system uses MySQL, we need to "dump" the old database, and "import" into new database.
Step 1: Dump the database (database name is image)
msqldump image > imgdb_dump
(note: you will have a file imgdb_dump now)
Setp 2: Edit the dump file to satisify MySQL
The main difference of miniSQL and MySQL is that the declaration
of UNIQUE is different, after change these UNIQUE declaration,
it would properly imported into MySQL
MySQL:
CREATE TABLE type (
typeid int(11) DEFAULT '0' NOT NULL,
typename char(50),
filename char(120),
UNIQUE type_index (typeid)
);
miniSQL:
CREATE TABLE type (
typeid INT,
typename CHAR(50),
filename CHAR(120)
) \g
CREATE UNIQUE INDEX type_index ON type (
typeid
) \g
Step 3: Create the database for Import
Step 4: Import the dump file into new database
mysql -p image < imgdb_dump_mysql
(note: if somethings goes wrong just drop the database and
create again)
This part is for change the CGI programs to use MySQL instead of miniSQL.
The datatype is very different:
|
miniSQL |
MySQL |
| int *db_name; | MYSQL db_name; |
| int *table_name; | Not need |
| m_result result; | MYSQL_RES result; |
| char** data_row; | MYSQL_ROW data_row; |
The basic functions that do the same things
| miniSQL | MySQL |
| msqlConnect(); | mysql_real_connect(); |
| msqlSelectDB(); | mysql_select_db(); |
| mysqlClose(); | mysql_close(); |
| mysqlFetchRow(); | mysql_fecth_row(); |
(note:since the datatypes are different the usage of functions is not the same)
By Carlos, Wong Ka Man
Image Processing Lab, City University of Hong Kong, 2001