Archive for March, 2008

The clarity of Python vs. the cloud of Perl

I am learning the Python programming language right now, for many reasons. I need to hone in on a language so that I can write up a script in minutes to do pretty much anything that I need. Things ranging from text file processing, web CGI scripting to generate graphs from data, numerical analysis, and so on. This leaves me with a large number of options such as C++, Java, MATLAB, Python, Perl, and others.

While I am not a programming newbie, my skill level sits somewhere in the moderate area as I have been exposed to many different programming languages at the beginner level. So when I look for programming tutorials, it leaves me somewhere between the beginner books and websites that assume that you barely know how to “download your camera to your PC” and the other side of the spectrum of which the tutorial looks about as exciting as a book of log tables:

logTables
(Flickr user quimby)

That being said, that leaves us moderate programmers who want to learn – stuck at a good and bad part of our learning experience. The part where you need to practice about every day writing real-world scripts over and over and over. So, after many weeks of lagging through with Python and putting off script writing, I attacked my first real Python program. It seems really trivial and would probably be one of the first few homework problems assigned in a programming course – but I will document my learning process nonetheless for the other moderate programmers that are out there.

Why Python? I chose Python after dabbling in each language and reading way too much information on each one and finally just trying them out for myself. Which language would be able to match my high-level idealistic mind but still be practical enough to have some power and force behind it? Well, take a look at my example program below. I wanted a program that would take in values from a CSV (comma-separated value) file, loop through a template file, and output new text files with the data from the CSV file’s rows in each output file.

For the more visually oriented:

pythonrep.png

I actually had my roommate last summer help me out with a Perl version of this program, and for comparison, here it is:

#!/usr/bin/perl

if(@ARGV < 2)
{
print "Usage: extractData <csv> <template>\n";
exit;
}</template></csv>

my ($file, $templateFile) = @ARGV;
my $lastTest = "";
my $templateString = `cat $templateFile`;
open DF, "&lt; $file";

while(<df>)
{
my $line = $_;
next unless($line =~ m/(^IT)|(^\,)/);
my ($test, undef, undef, $tray, undef, $cabletype) = split /\,/, $line;
$test = $lastTest if($test eq "");
$test =~ s/IT//g;</df>

$lastTest = $test;

print "test: $test; tray: $tray.\n";

$test = sprintf("%02d", $test);

(my $toPrint = $templateString) =~ s/TEST/$test/g;
$toPrint =~ s/TRAY/$tray/g;
$toPrint =~ s/CABLETYPE/$cabletype/g;

my $outFile = "CAROLFIRE_IT_" . $test . "_Tray_" . $tray . ".fds";

open OF, "&gt; $outFile";
print OF $toPrint;
close OF;
}

close DF;

Then, here is the version that I wrote last night using Python:

"""Module docstring.
Usage: python fdscsv.py <csv> <template> <output>
"""</output></template></csv>

import csv, sys, os, re

arguments = sys.argv
input = csv.reader(open(arguments[1],"r"))
template = open(arguments[2], "r")
lines = template.readlines()
counter = 1

for i, j, k in input:
output = open(arguments[3] + str(counter) + ".fds", "w")
for line in lines:
output.write(line.replace("IREP,JREP,KREP",(str(i) + "," + str(j) + "," + str(k))))
counter += 1
output.close()

Now, I realize that the function of the scripts are slightly different while the primary CSV functionality that I illustrated above still remains. I am not going for a line-by-line comparison here. I do want you to just look over the code and see which one makes more sense to your mind. For me, the Python is so easy to read and almost natural to understand while the Perl takes some serious brainpower for me to decode.In conclusion, I just wanted to show where I am at in learning the Python language. It really is enjoyable at this point for me when compared to learning Perl, which was just painful for me. So I hope to add to the resounding praise of Python by posting these examples for other programmers who may be stuck in the intermediate phase of their learning and need a little push of motivation to continue on.

Finally, if you are interested in what the Python code is actually doing, here is my commented version. Thanks for reading.

"""Module docstring.
Usage: python fdscsv.py <csv> <template> <output>
"""</output></template></csv>

import csv, sys, os, re

# Reads the arguments into a list
arguments = sys.argv

# Reads in the input csv file using the module csv
input = csv.reader(open(arguments[1],"r"))

# Reads in the template file
template = open(arguments[2], "r")

# Splits the template file into lines
lines = template.readlines()

counter = 1

# Labels the columns for the data read from the csv and loops through the lines in the csv
for i, j, k in input:

# Opens a new file with the user-input name plus a counter and an fds extension
output = open(arguments[3] + str(counter) + ".fds", "w")

# Loops through each line in the template file
for line in lines:

# Replaces strings in the template file with numbers from the csv file
output.write(line.replace("IREP,JREP,KREP",(str(i) + "," + str(j) + "," + str(k))))

# Increments the counter for the filename
counter += 1

output.close()

Comments (2) »


Updates on welcomed unstability

Been thinking a lot lately. Exerting myself mentally. Not so much physically. But exercising my mind daily is what I love. If it was this “easy” to exercise physically, well, I would be in well shape. But something drives me to work on projects. Some meaningful, some just practicing and moving my brain cells along the line.

So, this is nothing other than an updating type of post, a snapshot of where my neurons are firing.

1) Been actually practicing typing and using Python everyday. Did you know that the language Python was named after Monty Python’s Flying Circus? Neither did I until finally doing the official tutorial. Why am I practicing typing? I have been using computers since I was about 5, and still I cannot proper type. Sort of embarrassing. Not that though. I just want to type about thinking about typing. This post was written without proper typing. I have finally completed the typing tutor deal on my laptop. And now just need the practice part. I can type at 90+ WPM with this improper style, but the memory obstacle is still there.

That moves me on to Python. Why am I learning yet another language? Well, let me clarify. I am not the master of any one language. Python just seems so sensible to me. Good language to finally master. It comes on every Mac and Linux install. It can do all of my automation tasks that I dearly not need waste time on. Filling out FDS files based on numbers in a CSV file. Searching through PDF files for a server. Or making a post-processor for FDS output files. I love the direct-ness of the language. To print something is “print ‘hello world”. And that’s it. No braces, no output specifiers, no weird containing characters, no declarations, and so on.

2) Finished the Four-Hour Work Week. Amazing book. It is the first book that I am going to buy ever since giving away a huge percentage of my books about six months ago. I am now tasked with actually making something tangible that the rest of the world can use (and will pay for). The book is not just a business book, but a book about lifestyle, travel, and the philosophy of modern culture.

I now have to think about what I can provide to benefit masses of people. Perhaps the biggest thinking outside experiment that I have ever done. This can be difficult for my idealistic mind, but is totally possible. Since I retired in the month of August in 2006, this is certainly a key element in keeping myself financially supported – but avoiding the traps of an 80-hour per week self business. I do hope to develop my website with more FDS videos and info. This website or another, I am not sure. Hobby or “business”, I am not yet sure. Time and experiences will tell.

Rack sprinkler

3) Travel. Need more travel and exploring. The above points will actually help with this in my abstract mind.

4) Graduate school politics. My fire science hero has a terrible relationship with my future grad. school advisor, department, and school. Yay. All I want is meaningful work, and I hope that they all understand that. To think that my next two years of work might be thrown away is 1000x times worse than saying I would never be paid for the work. I will do my best, as always. My future advisor posed the question of why should an entity contribute to an open-source worldwide project who did not fund them directly. Why should he post the “results” for free. This violated one of my primary values in life. I immediately thought of Linux, KDE, Google, and tons of other open source projects that were not directly funded by those agencies themselves but have benefited the lives of millions based on contributed work.

Hell, I have been contributing to the FDS project for “free” for the past three years in whatever way that I can, every single day. I would do it for free – because it is one project of many in the world that is meaningful and benefits the world. It even has the bonus of saving lives through better fire protection design. So, I took that comment with a hard heart and looked back on the past 2 years as I have been earning a McDonald’s salary to teach, research, and contribute to a meaningful project. Priorities.

AFD warehouse

To leave on a good note, as I should, I have emailed several contacts around the US and the world about future travel and research opportunities. We will see what happens! Have a fantastic week.

Leave a comment »