Wednesday, May 11, 2016

CIS(theta) 2015-2016 May Meeting: openMPI (LAST MEETING)!

CIS(theta) 2015-2016 
May Meeting: openMPI (LAST MEETING)!
OMG! How upsetting is this? openMPI-bin is suddenly back in the Ubuntu repository. So now we could actually finish our project like this:

InstantCluster Step 5: Software Stack II
We then installed openMPI (we had a lot less dependencies this year with Natty 11.04 64bit) and tested multi-core with flops. Testing the cluster as a whole will have to wait until the next meeting when we scale the cluster! We followed openMPI install instructions for Ubuntu from
These instructions say to use sudo and run run apt-get install openmpi-bin openmpi-doc libopenmpi-dev However, the way our firewall is setup at school, I can never update my apt-get sources files properly. So, I used and installed openmpi-bin, gfortran and libopenmpi-dev. That's it!
Then we used the following FORTRAN code to test multi-core. FORTRAN, really? I haven't used FORTRAN77 since 1979! ...believe it or don't!
We compiled flops.f on the Master Node (any node can be a master):
mpif77 -o flops flops.f
and tested openmpi and got just under 800 MFLOPS using 2 cores (one PC):
mpirun -np 2 flops
Next, we generated a "machines" file to tell mpirun where all the nodes (Master and Workers) are (2 PCs or nodes with 2 cores each for example):
mpirun -np 4 --hostfile machines flops
Every node has the same "machines" text file in /home/jobs listing all the IPs, one per line. Every node has the same "flops" executable file (or whatever your executable will be) in /home/jobs. Every node has the same "authorized_keys" text file with all 25 keys in /home/jobs/.ssh
So, what's all this good for aside from making a Fractal Zoom or Shrek Movie?
Protein Folding
SETI Signals Analysis
Scientific Computing
Computational Physics
Computational Chemistry
Computational Astronomy
Computer Aided Design (CAD)
Computer Algebra Systems (CAS)
These are but a few examples of using Computer Science to solve problems in Mathematics and the Sciences (STEAM). In fact, many of these applications fall under the heading of Cluster Programming or Super Computing. These problems typically take too long to process on a single PC, so we need a lot more horse power. Next time, maybe we'll just use Titan!

CIS(theta) 2015-2016 
April Meeting: openMPI!
It was finally time to install openMPI and all the MPI compilers this month. In years past, this was a simple issue of going to the Ubuntu Software Center or or going the the command line and entering the following command.

sudo apt-get install openmpi-bin

This year, however, said package was missing in the Ubuntu 64bit 14.10 repositories. So, next meeting we'll be compiling from source:

CIS(theta) 2015-2016 
Feb+Mar Meetings: Public Keys!
We decided to install a native openMPI stack over our Ubuntu OS on all the hard drives in our cluster and skip using the pelicanHPC DVD. The idea is to login from home and ssh to any node to login on the cluster as needed! We took our queue from from the 2013-2015 CIS(theta) team summary (se below). We already covered steps 1-3. We're on step 4, installing Public Key Authenticated SSH!

InstantCluster Step 1: 
Infrastructure - Power, Wiring and AC

InstantCluster Step 2: 
Hardware - PCs

InstantCluster Step 3: 
Firmware - Ubuntu

InstantCluster Step 4: 
Software Stack I - openSSH:

01) Install openSSH-server from USC or

02) Create a the same new user on every box of the cluster

03) login as the new user, we used 
userid: jaeger, passwd: galaga

04) If you have no .ssh directory in your home directory, ssh to some other machine in the lab; then Ctrl-d to close the connection, creating .ssh and some related files. 

05) From your home directory, make .ssh secure by entering:
chmod 700 .ssh

06) Next, make .ssh your working directory by entering:
cd .ssh

07) To list/view the contents of the directory, enter:
ls -a [we used ls -l]

08) To generate your public and private keys, enter:
ssh-keygen -t rsa

The first prompt is for the name of the file in which your private key will be stored; press Enter to accept the default name (id_rsa).The next two prompts are for the password you want, and since we are trying to avoid entering passwords, just press Enter at both prompts, returning you to the system prompt.

09) To compare the previous output of ls and see what new files have been created, enter:
ls -a [we used ls -l]
You should see id_rsa containing your private key, and containing your public key.

10) To make your public key the only thing needed for you to ssh to a different machine, enter:
cat >> authorized_keys

NOTE: The Linux boxes on our LAN, soon to be cluster, have IPs ranging from to So, we copied each file to temp01-temp24 and uploaded these files via ssh to the teacher station. Then we just ran cat tempnn >> authorized_keys for each temp file to generate one master authorized_keys file for all nodes that we could just download to each node's .ssh dir.

[optional] To make it so that only you can read or write the file containing your private key, enter:
chmod 600 id_rsa 

[optional] To make it so that only you can read or write the file containing your authorized keys, enter: 
chmod 600 authorized_keys

InstantCluster Step 5: 
Software Stack II - openMPI

InstantCluster Step 6: 
Coding I - Quadrature

InstantCluster Step 7: 
Coding II - Mandelbrot

InstantCluster Step 8: 
Coding III - Mandel Zoom

InstantCluster Step 9: 
Coding IV - POVRay

InstantCluster Step 10: 
Coding V - Blender

InstantCluster Step 11: 
Coding VI - 3D Animation

CIS(theta) 2015-2016 
January Meeting: Teacher PC Install!
The Mother Of All TEACHER PC Installs!
(Please note: this was not an actual meeting. This is just an update to the November meeting when we installed all the Student PCs.)

As you can see below, installing the Teacher PC for SmartBoarding, ScreenCasting and Mirroring starts off much as the Student PC Installs from November. But, then I add A LOT of tweaks!

(01)Download the latest nonLTS Ubuntu.
(02)Burn the ISO file to DVD with Brasero.
(03)Reboot PC with DVD.
(04)Install to Linux Partition by script.
(05)Reboot Linux Partition.
(06)Tweak Appearance
(07)Tweak Brightness
(08)Configure Network
Method: Manual
Proxy: Manual/ (optional)
(09)Configure Printers (laser printer) (color laser printer)
(10)sudo chmod 700 /usr/games/*
(11)sudo apt-get update
(12)sudo apt-get upgrade
(13a)install WINE for the emulators (USC)
(13b)install Kate for color laser printer (USC)
(14a)copy WABBIT directory to Desktop
(14b)copy TI84C.rom to WABBIT directory
(14c)copy KARMTI directory to Desktop
(14d)copy boot string into KARMTI 

BTW, I set up the Server PC for my AP Computer Science students totally differently last January! 

Midterm Week has become the traditional time for me to update my Teacher PC. With a little time on my hands between grading and proctoring midterms and Regents exams, I decided to throw off the yoke of SmartNotebook and 32bit Linux once and for all! Also, I have a few days to test and debug my installation before classes start anew.

Here's the big question:
Can I run an Intel i7 octo-core, 1 TB hdd, 8GB RAM Lenovo ThinkCentre PC with 64bit Ubuntu 15.10 and still use the SmartBoard without installing SmartNotebook? 

The grand experiment! 
I left my current install as the default boot partition in case another teacher, say a sub, needs it and then I installed the 64bit and 32bit Ubuntu 15.10 ISOs on 2 new partitions. Primarily, I'm hoping to use the 64bit installation, but the 32bit one is there just in case I FUBAR everything!

The big SNAFU I keep running into, whenever I reinstall thTeacher PC, is that Smart Notebook for Linux was only released for 32bit operating systems, has not been updated since version 11 and is getting a bit buggy to use even in a 32bit install. I even tried installing Smart Notebook with linux32 or even with WINE but nothing worked. In fact, linux32 completely trashed my dependency table, so I had to start the install over again from scratch!

Now, the idea is to install the 64bit partition without any software from SmartTech at all! "What if the SmartBoard goes out of alignment," I hear you cry? Good question! That's why I always install Smart Notebook. I don't need the Notebook pre se, I can use xournal, webwhiteboard or classflow instead (carefull, classflow hangs the PC if you goto full-screen mode with F11). I thought that I always needed the drivers from SmartTech to re-calibrate the board. Knock on wood, every time I rebooted the 64bit bit partition this week, the board has maintained alignment ... so far. If push came to shove, I suppose I could reboot into 32bit, re-calibrate and reboot into 64bit or stay on the 32bit desktop.

A new issue is that Screencast-o-matic is not multi-platform anymore. Version 2.0 works as a down-loadable app in Windows only. Version 1.0 was much more flexible in that it would work in any Firefox compatible browser with JRE installed.  Now, I'm using Simple Screen Recorder which works right from the DeskTop and renders small mp4 files on the fly that I can easily upload to YouTube!

So, on the plus side, I don't have to bother installing JRE in Firefox anymore, not even for SAGE. 3D graphs in SAGE also needed JRE, but sagecell doesn't need this anymore! 

Also, once I installed chrome on the 32bit partition, I started getting notifications that Google will stop updates for 32bit versions soon!

After all was said and done, my 64bit partition seemed ready for prime time! I'll be testing it in class next week. I tested out everything I could think of this week, and all seems well. Further, I'm using: 
1) Xournal instead of Smart Notebook (SmartBoarding and saving PDFs)
2) SimpleScreenRecorder instead of Screencast-o-matic
(ScreenCasting for YouTube)
3) Teamviewer instead of Splashtop (Mirroring Desktop to Tablet)

BTW, I use my Samsung  Galaxy NotePro 12.2" Tablet as a remote SmartBoard (even if I only have a PC Projector) when speaking at a conference or when I'm a guest lecturer at the local college (whether I'm ScreenCasting or not). That's what Mirroring apps like Teamviewer are for. So, if the alignment in the 64bit install becomes an issue, I can always use my S-Pen instead of a SmartBoard Pen! Gotta love the BlueTooth Keyboard, BlueTooth Headset, S-Pen and S-Mouse!

In fact, my room is now completely automated:
1) SmartBoard: auto shutdown (30 min)
2) Student PCs: auto hibernate (60 min)
3) Teacher PC: auto logout (60 min)
4) Room Lights: auto off (60 min)
5) Room Lights: auto on (motion sensor)
6) A/C: 68degF thermostat

Further, I reboot into 64bit every morning (grub selection) and reboot into the "guest" 32bit partition (grub default) every evening!

So, my Teacher PC install fest really went like this:

Back up files and folders you need to preserve from the current Desktop and  Home directories to a USB stick.

I downloaded the latest Ubuntu ISO. For the Student PCs I usually take the latest LTS ISO. 15.04 was not an LTS release, so we used 14.04 Trusty. For the Teacher PC install this time I went with 15.10 Wily. BTW, I think we've been using Ubuntu Linux since Lucid! Before that we used several versions of Knoppix and before that Slackware. We've even used QuantianScientificMOSIXpelicanHPC and BCCD.

So, I downloaded the ISO, burned the DVD and rebooted the Teacher PC with this new installation DVD. When running the installation script, I chose to "install alongside existing linux" which then prompted me to resize the current partition to make room for a new one. eventually installed 2 new partitions alongside the existing one making each about 333GB. 

There is ONE MAJOR SECURITY ISSUE to keep in mind, however. On the original single partition installation, I had 2 users defined. The default user was set to boot up to the Desktop without any passwd. I created the 2nd user so I could keep my files secure. The second user requires a passwd to access and has it's home directory permissions changed: 

chmod 700 /home/*

When we have more than one partition THIS IS NOT ENOUGH SECURITY! When you are logged into one partition, you can navigate to another partition and view any files in the home dir even if the permissions are reset as above! So, when the installation script asks if you want it passwd protected, say yes! I did not do this on the primary partition so any teacher could come in and use the Teacher PC without a passwd.  

WAIT! THIS IS STILL NOT SECURE! You must also encrypt the home directory! Now if you are running another partition, you can navigate to any encrypted partition's home dir, but you will not be able to view the contents on said dir. 

BTW, I don't see any performance issues when using an encrypted partition except at boot, it takes a little while to decrypt the home dir.

Everytime you install a new partition, it becomes the primary boot partition in GRUB. So, for my old partition to be accessible for other teachers I edited /etc/default/grub setting GRUB_DEFAULT=4. Also, the resolution at boot-up was not visible to choose another partition from the GRUB menu, so I uncommented GRUB_GFXMODE=640x480. Just for fun, I also uncommented INIT_GRUB_TUNE="480 440 1" as follows:

sudo nano /etc/default/grub
sudo update-grub 

This IS identical to the Student PC, except for a new step 07a to make sure the Desktop shows on both the SmartBoard and the PC Monitor:
(06)Tweak Appearance
(07)Tweak Brightness
(07a)Tweak Display Resolution
(08)Configure Network
Method: Manual
Proxy: Manual/ (optional)
(09)Configure Printers (laser printer) (color laser printer)

Now I copy the contents of my Desktop and Home dirs I copied to USB from the old partition to this new one. Part of this step includes putting the WABBIT and KARMTI emulators on the Desktop. For these to work, I need WINE. Let me throw in Kate (for the color printer and syntax highlighting in AP Computer Science class) and Printer stuff for good measure:
(11)sudo apt-get update
(12)sudo apt-get upgrade
(13a)install WINE for the emulators (USC)
(13b)install Kate for color laser printer (USC)
(14a)copy WABBIT directory to Desktop
(14b)copy TI84C.rom to WABBIT directory
(14c)copy KARMTI directory to Desktop
(14d)copy boot string into KARMTI

I have a number of programs locked in my sidebar for quick access in class:
FIREFOX - this one is here by default
CHROME - from 
XOURNAL - install from USC
SMARTNOTEBOOK - (32bit only)
Install 32bit *.deb files in this order:
SOUND - run from launcher
PDF READER - run from launcher
VLC - install from USC
TERMINAL - run from launcher
add ppa, then:

sudo apt-get update
sudo apt-get install simplescreenrecorder 

GEDIT - run from launcher
KATE - run from launcher
DRAW - run from launcher
WRITE - this one is here by default
CALC - this one is here by defaul
IMPRESS - this one is here by default
USC - this one is here by default
DROPBOX - install from USC

1) SplashTop only has support for Ubuntu 12.04 and 14.04. 
2) SplashTop Dependency libx264-142 cannot be met in Ubuntu 15.
3) In Linux, I use TeamViewer instead of SplashTop
4a) When I'm a guest lecturer, using someone else's Windows PC, I'll still use SplashTop if I can install it.
4b) When I'm a guest lecturer, using someone else's Windows PC, I'll still use Screencast-o-matic.
5) I installed ubuntu-restricted-extras at this point so I can play movie DVDs.
6) Smart Notebook activation key:

use a number of tabs locked in my browser for quick access in class: "what most schools..."  

use a number of tabs locked in my browser for quick access in class:
CURRENT EXTRAS FOR YOUTUBE WED: "William Noel" "ignite archimedes"

CIS(theta) 2015-2016 
December Meeting: pelicanHPC!
Today we downloaded the pelicanHPC ISO, burned 1 DVD and booted up the whole room via PXE. We set the BIOS on each node to netBoot before booting from the hdd. 

This was the first year that our network cooperated on the first try! We never could even boot up the DVD, controling the whole room, from the Teacher PC before. Today, we finally did! Each of the 24 Student PCs have quad-cores and the Teacher PC has an octo-core. So, we actually have 104 64-bit cores available!

We scanned all 25 nodes and ran flops.f to see how fast our cluster is running. We managed to fire up 100 cores. Each core ran over 500 MFLOPS, so the cluster ran over at a rate of 50 GFLOPS. I don't think we ever got this result so early in the year before! 

This is how we compile flops.f:
mpif77 -o flops flops.f

This is how we run flops:
mpirun -np 100 --hostfile ~/tmp/bhosts flops

bhosts is a file containing the IP addresses for each node as assigned by our DHCP server.

BTW, we're doing this to see how we can add openSSH and openMPI to our Ubuntu installation and run cluster programs. 

CIS(theta) 2015-2016 
Oct+Nov Meeting: Linux Install Fest!
Installing STUDENT PCs
(01)Download the latest LTS Ubuntu.
(02)Burn the ISO file to DVD with Brasero.
(03)Reboot PC with DVD.
(04)Install to Linux Partition by script.
(05)Reboot Linux Partition.
(06)Tweak Appearance
(07)Tweak Brightness
(08)Configure Network
Method: Manual
Proxy: Manual/ (optional)
(09)Configure Printers (laser printer) (color laser printer)
(10)sudo chmod 700 /usr/games/*
(11)sudo apt-get update
(12)sudo apt-get upgrade
(13a)install WINE for the emulators (USC)
(13b)install Kate for color laser printer (USC)
(14a)copy WABBIT directory to Desktop
(14b)copy TI84C.rom to WABBIT directory
(14c)copy KARMTI directory to Desktop
(14d)copy boot string into KARMTI
(15a)add firefox tab for SAGE 
(15b)add firefox tab for HOC 

CIS(theta), 2015-2016 
September MeetingAdministrativa!
(1) Wreath of the Unknown Server: We visited our first ssh server, Colossus, which is still in the switch room, though dormant. I set it up for the first time in 1995 running Slackware Linux. Colossus ran for 12 years straight, 24x7 never having to shut down, reboot or even have anything re-installed!

(2) Planning: We have to wait another 2 weeks at which point Ubuntu 15.10 Desktop Edition should be available for a mini install fest. After that, we will look into installing our Linux Cluster using openMPI and MPI4py!

(3) Display Case Unveiled: We took down a ton of fractal prints and ray tracings from Room 429 to the 2 display cases on the 1st floor near the art wing. We decorated both display cases as best we could and left before anyone saw us. Must have been gremlins. BTW, we're looking into getting an HDTV with DVD player built in to show case student work in these display cases.

(4) NCSHS: We're going to continue our chapter of the National Computer Science Honor Society. We talked about the requirements for membership and how we started a chapter. Each chapter is called "Zeta Omicron something." We're "Zeta Omicron NY Hopper." This is a pretty new honor society. The first few chapters were called Zeta Omicron Alpha and Omicron Zeta Beta. We have the first NYS chapter!


Membership (alphabetic by first name):
CIS(theta) 2015-2016: 
BenR(11), BrandonL(12), DavidZ(12), EvanF(12), GabeT(12), HarrisonD(11), HunterS(12), JacksonC(11), SafirT(12), TimL(12)

CIS(theta) 2014-2015: 
BryceB(12), CheyenneC(12), CliffordD(12), DanielP(12), DavidZ(12), GabeT(11), KeyhanV(11), NoelS(12), SafirT(11)

CIS(theta) 2013-2014: 
BryanS(12), CheyenneC(11), DanielG(12), HarineeN(12), RichardH(12), RyanW(12), TatianaR(12), TylerK(12)

CIS(theta) 2012-2013: 
Kyle Seipp(12)

CIS(theta) 2011-2012: 
Graham Smith(12), George Abreu(12), Kenny Krug(12), LucasEager-Leavitt(12)

CIS(theta) 2010-2011: 
David Gonzalez(12), Herbert Kwok(12), Jay Wong(12), Josh Granoff(12), Ryan Hothan(12)

CIS(theta) 2009-2010: 
Arthur Dysart(12), Devin Bramble(12), Jeremy Agostino(12), Steve Beller(12)

CIS(theta) 2008-2009: 
Marc Aldorasi(12), Mitchel Wong(12)

CIS(theta) 2007-2008: 
Chris Rai(12), Frank Kotarski(12), Nathaniel Roman(12)

CIS(theta) 1988-2007: 
A. Jorge Garcia, Gabriel Garcia, James McLurkin, Joe Bernstein, ... too many to mention here!

Well, that's all folks,
A. Jorge Garcia
Applied Math, Physics and CS
2015 NYS Secondary Math PAEMST Nominee

Happy Linux Clustering, 
PasteBin SlideShare 
Sage Ebay

No comments:

Post a Comment