Write

Read

Analysis

The first and easiest conclusion that comes to me is that a USB connected PATA drive is just slow, so it is pretty much out of the running right there. With write speeds 9-50% faster and reads 99-127% faster, iSCSI and NFS just win.

The choice between the remaining two is more difficult since they each win in different areas. NFS disk writes are a fair amount faster than iSCSI while iSCSI has a mild lead on reads. There is an issue with both iSCSI and NFS when reading from a disk, and that issue is CPU usage. Both use about 85% of our arm box’s CPU during heavy use resulting in likely slower build times then you would otherwise get.

Now that all 3 data storage systems are set up i am very curious to see real builds done using each medium, rather than synthetic benchmarks.

So which one should we go with, to be honest i am not sure as it depends on what is more important for our cluster of koji builders, disk writes or reads.

To Be Continued…

Version 1.03e ——Sequential Output—— –Sequential Input- –Random-

-Per Chr- –Block– -Rewrite- -Per Chr- –Block– –Seeks–

Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP

fedora-arm 2G 4923 82 31503 30 1827 3 5537 96 59147 84 2514 44

——Sequential Create—— ——–Random Create——–

-Create– –Read— -Delete– -Create– –Read— -Delete–

files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP

16 11785 92 +++++ +++ 21841 99 12533 98 +++++ +++ 21201 100

fedora-arm,2G,4923,82,31503,30,1827,3,5537,96,59147,84,2514.5,44,16,11785,92,+++++,+++,21841,99,12533,98,+++++,+++,21201,100

Write
31,503 KB/s @ 9% Cpu Usage
Read
59,147 KB/s @ 84% Cpu Usage
]]> http://dmchisho.wordpress.com/2010/04/11/performance-tests-update-iscsi/feed/ 2 dmchisho http://dmchisho.wordpress.com/2010/03/31/performance-testing-nfs-vs-usb-pata-vs-iscsi/ http://dmchisho.wordpress.com/2010/03/31/performance-testing-nfs-vs-usb-pata-vs-iscsi/#comments Wed, 31 Mar 2010 14:59:44 +0000 dmchisho http://dmchisho.wordpress.com/?p=27 ]]> We need performance tests on what data transfer/storage system would be most efficient to get data to and from all the machine on the network. There are many ways to do this, some requiring some system changes which just are not worth the effort, and some simply requiring the use of 1 command, the latter is what I chose.

The application used is called Bonnie++. It is a simple drive benchmarking tool that has the convenience of being able to be run on any folder on the system, thus making network mounts testable. It uses some simple switches to be used including the directory to be tested, the size of the file to test with (4x the system memory is recommended to avoid system buffering of the files), and then the user to test with (this is required to run the test as root.)

The final commands I went with are:

NFS: bonnie++ -d /share -s 2048 -u root

PATA:  bonnie++ -d /hd1 -s 2048 -u root

Here are the results returned.

To install NFS in Fedora Core 12 I used the following steps.

1. Install the “nfs-utils” and “rpcbind” packages using yum.

2. Added the line “/var/www/html/yum/arm_built *(rw,no_root_squash)” to “/etc/exports” to tell nfs which folder is to be shared and with what permissions.

3. Added iptables incoming/outgoing traffic access on port  2049 on the 192.168.*.* network.

With any luck you should be able to start the “rpcbind” and “nfs” services and there should be no further configuring on the server side of the nfs share.

Next are the instructions to connect to the share on any hosts in the 192.168.122.* network.

1. Edit your /etc/fstab and change the following lines file system to “auto” from “nfs”
“/dev/root / nfs defaults 1 1″

2. Use yum to install “rpcbind” and “nfs-utils” and make sure that the “rpcbind” and “nfs” services are started at boot.

3. Create a folder where you would like to mount the NFS share. In these examples i will be using “/nfsshare” as the mounting point.

4. Run the command “mount 192.168.122.1:/var/www/html/yum/arm_built /nfsshare” to test that you can successfully mount the share. Where ever I use this directory, replace it with the mount point of your choice

5. If it mounts correctly it will give no output. Run the “mount” command to check that it has been successful.

6. Add the following line to “/etc/fstab” to auto mount the share every time the VM boots.

“192.168.122.1:/var/www/html/yum/arm_built /nfsshare nfs defaults 0 0″

7. Reboot your vm to make sure the nfs share is properly mounted at boot. To restart your vm log into hongkong as root and run the following commands. As an example i will use arm-001-002, replace this with your VM.

#virsh destory arm-001-002
#virsh start arm-001-002

With that out the of way ill just throw out a couple comments about settings this share up.

NFS is usually a very very simple thing to install and get working. Grab the package manually or through some sort of package manage such as YUM, add the files you would like to me shared int /etc/exports and then make any configuration changes you need.

The biggest issues I ran into was Fedora 12′s firewall blocking the service. In all honesty I have no idea what ended up making it work due to the fact that i do not remember making any changes from the time it wouldn’t mount on arm-001-002 to the time it would successfully.

From the hosts end it is a simple process to get the shares mounted and usable. Install a couple packages, run a couple services and add a line to /etc/fstab to auto mount it at boot.

Done and Done.

1. Lack of understanding on the part of the re-packager (unfortunately i fall under this)

2. Lack of sufficient documentation withing the source of the package, making it very difficult to figure out exactly what is needed to get these programs to compile properly.

So as an introductory package to well repackage (does that sound right? ) I decided to use the relatively simple htop, available from the following link.

http://downloads.sourceforge.net/project/htop/htop/0.8.3/htop-0.8.3.tar.gz?use_mirror=iweb

With a rather large amount of help from CTyler i was able to get the SPEC file in working order. Which currently passes rpmlint and mock without any errors or warnings.
For those interested here is the contents of the SPEC file. Any comments on improvements and the like are always welcome.

————————————————————————–

Name:   htop
Version:        0.8.3
Release:        1%{?dist}
Summary:        CLI Resource Monitor

Group:          Applications/System
License:        GPLv2
URL:            http://htop.sourceforge.net/
Source0:        http://downloads.sourceforge.net/project/htop/htop/0.8.3/htop-0.8.3.tar.gz
BuildRoot:      %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX)

BuildRequires: ncurses-devel

%description
A CLI based resource monitor.

%prep
%setup -q

%build
%configure
make %{?_smp_mflags}

%install
rm -rf $RPM_BUILD_ROOT
make install DESTDIR=$RPM_BUILD_ROOT

%clean
rm -rf $RPM_BUILD_ROOT

%files
%defattr(-,root,root,-)
%doc AUTHORS COPYING INSTALL NEWS README TODO
%{_bindir}/*
%{_mandir}/*/*
%{_datadir}/pixmaps/*
%{_datadir}/applications/*

%changelog

* Fri Feb 05 2010 David Chisholm <dmchisho@learn.senecac.on.ca> – 0.8.3-1.fc12
- Rebuilt for SBR600.

* Fri Jul 24 2009 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> – 0.8.3-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_12_Mass_Rebuild

* Tue Jun 23 2009 RafaÃ
Psota – 0.8.3-1
- update to 0.8.3

* Fri Jun 12 2009 RafaÃ
Psota <rafalzaq@gmail.com> – 0.8.2-2
- “htop aborts after hitting F6 key” fixed (#504795)

* Tue Jun 02 2009 RafaÃ
Psota <rafalzaq@gmail.com> – 0.8.2-1
- update to 0.8.2

* Tue Feb 24 2009 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> – 0.8.1-4
- Rebuilt for https://fedoraproject.org/wiki/Fedora_11_Mass_Rebuild

* Thu Feb 05 2009 RafaÃ
Psota <rafalzaq@gmail.com> – 0.8.1-3
- “Tree view doesn’t work with threads hidden” fixed (#481072)

* Tue Nov 18 2008 RafaÃ
Psota <rafalzaq@gmail.com> – 0.8.1-2
- non-printable character filter patch (#504144)

* Tue Oct 14 2008 RafaÃ
Psota <rafalzaq@gmail.com> – 0.8.1-1
- update to 0.8.1

* Thu Jul 31 2008 RafaÃ
Psota <rafalzaq@gmail.com> – 0.8-1
- update to 0.8

* Sun Apr 27 2008 RafaÃ
Psota <rafalzaq@gmail.com> – 0.7-2
- desktop file fix

* Mon Feb 11 2008 RafaÃ
Psota <rafalzaq@gmail.com> – 0.7-1
- update to 0.7

* Sat Dec  9 2006 Dawid Gajownik – 0.6.5-1
- Update to 0.6.5

* Wed Oct  4 2006 Dawid Gajownik <gajownik[AT]gmail.com> – 0.6.4-1
- Update to 0.6.4

* Sat Sep 16 2006 Dawid Gajownik <gajownik[AT]gmail.com> – 0.6.3-2
- Rebuild for FE6

* Sun Jul 30 2006 Dawid Gajownik <gajownik[AT]gmail.com> – 0.6.3-1
- Update to 0.6.3
- Correct e-mail address in ChangeLog
- Replace tabs with spaces

* Sat May 20 2006 Dawid Gajownik <gajownik[AT]gmail.com> – 0.6.2-1
- Update to 0.6.2

* Wed May 10 2006 Dawid Gajownik <gajownik[AT]gmail.com> – 0.6.1-2
- Add missing BR: desktop-file-utils

* Wed May 10 2006 Dawid Gajownik <gajownik[AT]gmail.com> – 0.6.1-1
- Update to 0.6.1

* Tue Feb 14 2006 Dawid Gajownik <gajownik[AT]gmail.com> – 0.6-3
- Rebuild for Fedora Extras 5

* Wed Dec 28 2005 Dawid Gajownik – 0.6-2
- Rebuild with updated tarball

* Wed Dec 28 2005 Dawid Gajownik <gajownik[AT]gmail.com> – 0.6-1
- Version 0.6

* Fri Nov 11 2005 Dawid Gajownik <gajownik[AT]gmail.com> – 0.5.4-2
- Don’t use superflous CFLAGS variable (Dmitry Butskoy)
- Don’t include AUTHORS and NEWS files

* Thu Nov 10 2005 Dawid Gajownik <gajownik[AT]gmail.com> – 0.5.4-1
- Initial RPM release.

—————————————————————

I still have a fair amount of reading to do to really understand the process of doing this quickly and efficiently, but I definitely have a greater grasp then a few short days ago.

I chose to compile Apache from source. The testing methodology is as follows.

1. Downloaded source
2. ./configure
3. time make -j#
4. make clean
5. repeat steps 3-4

The system i used was a VirtualBox install of Fedora Core 12 32-Bit. It was given 4 cores and 2048MB of memory. The Host machine’s system specs are as follows.

Intel Core i7 920 @ 2.80Ghz
6GB G.Skill Perfect Storm DDR3 memory @ 1600Mhz 8-8-8-21-2t
Evga x58 3xSLI Classified (e760)
Windows 7 64-Bit

As will be apparent with the results, things didn’t exactly remain consistent through out the testing. I believe this is due to the fact that the Intel Core i7 is hyperthreaded, which as far as Windows is concerned has 8 physical cores to work with, when in actuality there are 4. The 4 cores given to the VM during the test could have been hyperthreads, real cores, or any combination of the two, i have no way to know.

That said, here are my results. I also include the average cpu usage observed in windows. It includes the VM overhead, the compile being done by said VM, and various episodes of House.

j1
avgcpu    25%
real    7m38.268s
user    2m5.876s
sys    8m20.100s

j2
avgcpu    32%
real    7m43.762s
user    3m38.455s
sys    15m39.675s

j3
avgcpu    40%
real    12m21.833s
user    6m8.897s
sys    30m41.856s

j4
avgcpu 40%
real    11m4.554s
user    6m8.402s
sys    31m23.809s

j5
avgcpu 41%
real    11m24.975s
user    6m24.835s
sys    33m34.616s

j6
avgcpu 40%
real    11m51.639s
user    6m41.291s
sys    35m21.778s

Due to my odd results i cannot come to a conclusion to what j-value is optimal for my system. I may in the future temporarily disable hyperthreading and give the VM 3 actual cores and see how the results differ. But that is something for another day.

First off is NLed.

The install process was easy. Found the source code on http://cdot.senecac.on.ca/software/nled/.

Downloaded it, and unzipped it. Ran make and copied the nled file to /usr/bin. All seems to work fine!!!

Next off is IRSSI.

Downloaded the Irssi from http://www.irssi.org/download. Version 0.8.14

In a similar process, unzipped it, ran the configure, make, and makestall and everything worked properly. Nice little irc client if your into the CLI.

Links

http://zenit.senecac.on.ca/wiki/index.php/User:Dmchisho

https://fedoraproject.org/wiki/User:Dmchisho

IRC Snippet:

[14:36] <dmchisho> aren’t we all