If you remember, after 9/11 the government dropped the Patriot Act within about 20 days and it was passed. I mean, the Patriot Act is huge. I remember someone asking a Justice Department official, how did they write such a large statute so quickly, and of course the answer was that it’s been sitting in the drawers of the Justice Department for the last 20 years waiting for the event that would allow them to drop it.

Of course, the Patriot Act is filled with all sorts of insanity about changing the way civil rights are not protected anymore in the United States. So I was having dinner once and Richard Clarke was sitting at the table and I said to him, “Is an equivalent, is there a Patriot Act — an i-Patriot Act — just sitting waiting for some substantial event for them to come and have an excuse to radically change the way the internet works?”

And he said, “Of course there is.” And I swear this is what he said, and quote, “Vint Cerf is not going to like it very much.”

So this is the big terror. They’re just sitting there waiting for the inevitable to happen, and then ‘Slam!’

It’s surprisingly simple to persist your socket connections while restarting a Ruby program. The key lies in understand how Ruby handles file descriptors. A file descriptor is a number that represents an open file that is buffered in the OS Kernel. Now I’ve only tried this technique on Linux so I don’t know if it works on Windows, so chime in on the comments if it does or does not work. Let’s start by exploring file descriptors a little bit so we can improve our Linux knowledge. If you want immediate gratification you can skip down to the bottom-most snippet.


irb(main):001:0> require 'socket'
=> true
irb(main):002:0> sock = TCPSocket.open("google.com",80)
=> #<TCPSocket:0xb7c73c44>
irb(main):003:0> sock.fileno
=> 3


note that the file number of the socket is 3. Now look at the output of lsof:


ryan@rtmlap:~$ lsof -c irb
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
irb 5645 ryan cwd DIR 8,2 12288 1441793 /home/ryan
irb 5645 ryan rtd DIR 8,1 4096 2 /
irb 5645 ryan txt REG 8,1 3564 1177574 /usr/bin/ruby1.8
irb 5645 ryan mem REG 8,1 67408 539926 /lib/tls/i686/cmov/libresolv-2.7.so
irb 5645 ryan mem REG 8,1 38412 539915 /lib/tls/i686/cmov/libnss_files-2.7.so
irb 5645 ryan mem REG 8,1 254076 1208067 /usr/lib/locale/en_US.utf8/LC_CTYPE
irb 5645 ryan mem REG 8,1 190584 522320 /lib/libncurses.so.5.6
irb 5645 ryan mem REG 8,1 196560 522360 /lib/libreadline.so.5.2
irb 5645 ryan mem REG 8,1 17884 539913 /lib/tls/i686/cmov/libnss_dns-2.7.so
irb 5645 ryan mem REG 8,1 40016 1210577 /usr/lib/ruby/1.8/i486-linux/socket.so
irb 5645 ryan mem REG 8,1 1364388 539898 /lib/tls/i686/cmov/libc-2.7.so
irb 5645 ryan mem REG 8,1 149328 539906 /lib/tls/i686/cmov/libm-2.7.so
irb 5645 ryan mem REG 8,1 38300 539902 /lib/tls/i686/cmov/libcrypt-2.7.so
irb 5645 ryan mem REG 8,1 9684 539904 /lib/tls/i686/cmov/libdl-2.7.so
irb 5645 ryan mem REG 8,1 112354 539924 /lib/tls/i686/cmov/libpthread-2.7.so
irb 5645 ryan mem REG 8,1 787660 1177653 /usr/lib/libruby1.8.so.1.8.6
irb 5645 ryan mem REG 8,1 7552 522335 /lib/libnss_mdns4_minimal.so.2
irb 5645 ryan mem REG 8,1 25700 98159 /usr/lib/gconv/gconv-modules.cache
irb 5645 ryan mem REG 8,1 15312 1207882 /usr/lib/ruby/1.8/i486-linux/readline.so
irb 5645 ryan mem REG 8,1 109152 522259 /lib/ld-2.7.so
irb 5645 ryan 0u CHR 136,1 3 /dev/pts/1
irb 5645 ryan 1u CHR 136,1 3 /dev/pts/1
irb 5645 ryan 2u CHR 136,1 3 /dev/pts/1
irb 5645 ryan 3u IPv4 1897889 TCP rtmlap.local:45854->jc-in-f99.google.com:www (ESTABLISHED)


Alright, so now here’s the snippet that shows persistent sockets:

#!/usr/bin/ruby
#simple_connector.rb
require 'socket'

puts "Started."

if ARGV[0] == "restart"
  sock = IO.open(ARGV[1].to_i)
  puts sock.read
  exit
else
  sock = TCPSocket.new('google.com', 80)
  sock.write("GET /\n")
end

Signal.trap("INT") do
  puts "Restarting..."
  exec("ruby simple_connector.rb restart #{sock.fileno}")
end

while true
  sleep 1
end

When the program is first run it prints ‘Started.’ Then it opens a socket to Google and requests the homepage. Then it waits indefinitely until it receives the INT signal (CTRL-C or kill -term PID). After that it runs exec which spawns a new program in the place of the current one. It spawns the same program, with additional command line arguments “restart #{fileno}.” When the program starts again these additional arguments make it follow the other path, which uses IO.open(file_descriptor) to open the file descriptor from the file descriptor number that we passed in on the exec line. Then it reads the socket Kernel buffer, which already contains the Google homepage.

And there you have it, a socket that persists beyond a single program.

lsof is a linux command that lists the open file descriptors on your system. I like to use the following options to help reduce spam:

lsof -c command-name
lsof -p port-number

This will tell you how to win at Scrabulous using a few simple Linux commands and a working knowledge of Regular Expressions. The basic principle behind this approach is to quickly write regular expressions to search for every possible word you can make a specific part of the board. By cating a scrabble word list into a pipe and using egrep with regular expressions, you can quickly and easily match patterns against your tiles and the state of the board.

Quick example:

$ wget http://www.ryantm.com/scrabble-word-list.tar.gz
$ tar -xvzf scrabble-word-list.tar.gz
$ cat scrabble-word-list | egrep "^A[EPABVT]{4}$"
ABATE

We’ve found a valid 5 letter scrabble word that starts with an A (^ means start of line), contains 4 of the set E,P,A,B,V,T ($ means end of word). Expanding on this concept you can do all sorts of awesome patterns.

Say you want to make a word that goes from what you currently have to a triple word score 5 tiles away that connects to a D. You just do something like this:

$ cat scrabble-word-list | egrep "^[EABTTE]*D$"
ABATED
BATTED
BETTED
TABBED
TATTED
TEATED

Then suddenly you’ve got a list of words that work for what you want. I did * here instead of {4} * means match as many characters as you can. It’s usually a good idea to start with * and narrow it down with {number} if you need to. You might notice that one of them doesn’t work (TABBED), because you don’t have two B’s. The regular expression matcher doesn’t care how many times you put a letter inside a set ([]) and it will match it as many times as it can. You can filter these words quickly yourself though, because you know you don’t have two B’s.

The General Principle

1. Take what letters you have an form at set: [BZTQEAI]
2. Find a starting or ending place: ^ACE or Y$
3. Put it together with the number of letters you want to use: “^ACE[BZTQEAI]*”
4. Double check the word with reality
5. Repeat until you are satisfied with your word score.

Hank Williams over at the Why Does Everything Suck blog is saying Yahoo Browser Plus is a Game Changer, but I don’t think so. While I agree that Yahoo Browser Plus is more well put together than Google Gears, both in terms of downloading and installing modules, and that their model for designing new modules is more open-source and organic, they need to get their software on computers.

Developers, Developers, Developers, Developers

Getting this Browser++ software on computers has already been shown to be quite difficult. Until these add-ons get close to the market penetration of flash they are going to have a particularly tough time attracting developers. I was also surprised that while Yahoo Browser Plus seems to be better than Google Gears in a lot of ways, they have no version targeting Ubuntu Linux, something that Google Gears has been serious about since the beginning. If you want to attract good developers to your product, you need to run on Linux.

Socket Applications Won’t Suck as Much

A web application that I’m working on would particularly benefit from the widespread penentration of the Browser++ software. I’m making a browser based MUD client. Right now the only way to reasonably do this is to have a persistent socket server on the web server that is hooked up to the browser via AJAX. This works pretty well, but an version built similarly to Yahoo’s IRC chat, would shuttle most of my server load to the users computers, where they have typically resided for socket based clients in the past. This seems like it would be a great candidate for a Browser++ application but there are a couple of hangups so far.

Future of Browser++

I am particularly hesitant to use Browser++ tools to develop my web applications, because the web is supposed to work everywhere. If the software doesn’t have enough penetration it doesn’t make sense to build a solution that you want to work everywhere on these tools. The unfortunate problem with this is that it is a chicken and egg problem. Macromedia solved this problem by integrating Flash over many many years. They didn’t just release Flash and then say “okay now the game has changed, everyone can run streaming video on the web.” This happened over time. The game hasn’t changed, it is changing.

When I read that Drupal 6 supported OpenID, I was happy about it. I use OpenID for logins whenever I can, but I have it set up in a sort of novel way. My novel way is to have my open id be www.ryantm.com and then have that website delegate the authentication to another company. In my case, I picked this good OpenID providor. I figured out how to do this by watching Simon Willison’s google video talk. All you do is add these few lines to the head element of your OpenID url, but replace ‘ryantm’ with your username:

<!– / Open Id –>
<link href=‘http://www.myopenid.com/server’ rel=‘openid.server’ />
<link href=‘http://ryantm.myopenid.com’ rel=‘openid.delegate’ />
<meta content=‘http://www.myopenid.com/xrds?username=ryantm.myopenid.com’ http-equiv=‘X-XRDS-Location’ />

The great part about this is that it works with any OpenID providor. So if at some point myopenid.com crosses my path, I can just delegate my OpenID authentication to another providor, and all of my OpenID logins will still work. In a sense, I’ve given myself complete flexibility with no chance of lock-in.

Unfortunately Drupal 6’s OpenID support isn’t up to snuff to handle my OpenID Delegation. It seems that while they seem claim they’ve implemented the entire OpenID spec by the wording on their marketing pages it doesn’t work for me. I tried www.ryantm.com and http://www.ryantm.com and it keeps rudely telling me that’s not a valid OpenID. I was under the impression that any url was a valid OpenID.

Anyway, I’ve created a bug report for this so you can follow what’s going on. Please add comments to the bug if you are having this problem as well.

HAML 2.0 is out. I’ve been using HAML since back when I made my Assassins Game Hosting Service in 2007. If you are still using ERB or some other templating language for your web applications here is why you should given HAML a try.

HAML gets rid of end tags. By using Python like indentation it removes the need for you to type all of those pesky HTML endtags. Also, it lets you quickly see the nesting of tabs simply by the indentation of the line you are on. This quick look at nesting is espcially helpful when your HTML files start to get longer than 80 lines or so.

HAML generates pretty source. I am anal about how my HTML souce looks. I want it to be indented with spaces at every nesting point. If you’ve ever seen the HTML source for a webpage generated in on of the terrible, horrible WYSIWYG editors, you know exactly what I don’t want. Every time I look at the source of pages generated by those tools a little piece of me dies. HAML is the complete opposite of this. It generates the cleanest possible HTML document it can. The file format convinces you to write cleanly, but it also smooths over some errors you could make. For instance, if you include unnecessary newlines in your source file, HAML removes them, leaving on the effectful newlines.

HAML is integrated with Ruby. ERB’s Ruby integration is a joke. All it does it execute Ruby between <% %> with a little extra handling for loops and if statements. HAML’s Ruby integration is like a Red Corvette. You can use = for ruby code with output, or == for ruby strings, or - for Ruby code without output. Also, now with HAML 2.0 error handling for Ruby and HAML is great a well.

HAML installation is easy. You can try HAML out right away with little effort, and no conflict with your current Rails installs just do the following:

gem install haml --no-ri

haml --rails path/to/app

I should also mention that HAML is used in my favorite static HTML generator, Staticmatic.

Have you ever wanted to remind yourself of an email? People frequently remind themselves of emails by leaving those emails in their inbox. If you’re busy, like me, your inbox reminders will grow uncontrollably. That’s why I developed ReplyLater.com. ReplyLater.com lets you schedule email reminders by simply sending emails. Let me show you what I mean.

Say I want to send an email to a company about a business proposal I have. I want to make sure they reply to me within a certain timeframe, or I want to followup again. The way I used to do this is by sending the mail to myself and leaving it in my inbox until I have the time to look through my inbox for things to do. Instead now I just address the email like this:

BusinessDevelopmentGuy@google.com, 2weeks@replylater.com

Good, I’m giving BusinessDevelopmentGuy 2 weeks to get back to me, but after that I’m going to get a email reminding me about this.

There are plenty of other applications to this as well. You can remind yourself about future events and obligations, like birthdays, anniversaries, little league games, vacations, etc.

ReplyLater.com is free to use, so theres no reason you can’t go to the website and check it out and start using it now! Any comments or suggestions are appreciated.

So Google App Engine(GAE) came out and it has generated quite a buzz in the blagosphere especially surrounding the idea of sharecropping. But that isn’t want I want to talk about. I want to talk about the growing amount of evidence that strikes down the idea that programmers are smarter than the average person.

There is a certain mythos surrounding the idea that some of us can “talk to the machines,” while others cannot. This misunderstanding has been capitalized by many programming types to act in a condescending manner against the people who do not know about computers. This has led to the mistaken belief that programmers are smarter than the average person. This is not the case. Programmers are humans like everyone else who have chosen to study an area very deeply. Many other humans have chosen to study other areas deeply as well, for instance, have you ever met a sculptor, or a advertiser, or a politician? Every area where people are engaged they are studying things at a deeper level than average. This deeper level of study does not make them smarter, it just means they have more contextual information. Now that I’ve introduced this flawed belief of programmer intelligence, I’d like to offer some evidence that caused me to write this article.

When Google release their App Engine, they made a Google Code issue tracker for it. This tracker has since, at the suggestion of Google employees, become a dumping ground for feature requests for GAE. One feature, which I was particularly interested about–being a ruby programmer–, is Ruby support. Now Google Code ranks issues on the issue page by the number of people that have starred an issue and starring an issue lets you receive updates when the issue’s status changes ( i.e. the feature is implemented). Unfortunately this also means you receive comments every time someone comments to the thread, and unfortunately as well, there are a lot of average Ruby programmers that showed up on this issue thread. The result is that since I have starred the issue, I’ve received over 300 emails from people who have written comments that say:

+1

Yep that’s right, that’s all it says. I’ve been receiving them at what seems like two a minute, though that’s an exaggeration. People have tried to quell the mob of idiots by numerous posts telling people to stop, and that it only counts if you star it, but no one listens. They just keep posting “+1″ by the hundreds. While these actions were surprising to me, they should not have been. People are going to keep following the crowd and writing plus one because they are people. Programmers aren’t different or better. Programmers are still humans.

One last thing I’d like to say that is probably the reason programmers have thought so highly of themselves and in some cases acted condescendingly to non-programmers. I believe that computer programming is an essential skill, along the lines of literacy, and speaking. The reason I categorize it with these is because, just like writing, reading and speaking, programming is game changing. With computer programming people can take an idea and propagate it massively and automatically. Properly programming something can change the way every other human who interacts with that program behaves and lives. This is powerful stuff that shouldn’t be relegated to one group of skilled persons.

Anyone with programming skills can work to automate their job of doing menial tasks. For example, when I worked at Caterpillar as an engineering intern, they had about 3 people who’s primary job was to make charts and give presentations to management about these charts. The charts graphed the quality of the machines we were shipping and the manufacturing defects that were caught. I noticed that these 3 people were spending at least 3 hours a day using excel to generate these charts. So I created a Visual Basic program (the only programming language I had available to me) to generate these charts automatically in a fraction of the time. The reason I bring this up is not that I think these people should have been writing programs to do their graphs for them. I bring it up because what if they could? What if they could sit down and automate the task in front of them, and then move on to doing other useful activities for the company.

This behavior of automation of jobs at the lowest level is why I say programming changes the game like reading, writing and speaking. When people at the lowest level learned to read and write it changed the game. Literate people can communicate much more effectively with their peers and get their ideas across to a lot of people. This idea automation through literacy is paralleled by task automation through computer programming. Look at what this means, just as you can’t have an elite group of scribes that does all your information dissemination, you can’t have an elite group of programmers to do your job automation. The idea of a programmer as a skill or trade needs to be broken down before we can see the true gains in productivity that computers promise.

So I kinda jumped around a bit here, I first cut down programmers to average, then I said, “wouldn’t it be great if the average person knew how to program.” I guess those ideas a bit hypocritical but they both serve to explain why I think that programmers aren’t as elite as they signal to others, and that others need to learn programming. I can hear it every time someone goes, “ooh I’m not good with computers.”

Google Accounts is inadequate for my usage of Google’s services. First, I have a google apps account for my email, I’d like this to be my primary account. Unfortunately there are certain Google applications that only work with a Google Account and Not a Google Apps for your Domain Account. These services include 4 that I know of: Google Reader, Google Adsense, Google Adwords, and Google Checkout. What is most disappointing to me is that Adsense and Adwords are supposed to be Google’s flagship moneymaker products, the ones they should be caring most about. The only google application to be responsible about this is Google Analytics, it lets you add other email accounts to have access to the application, and because of this it works properly with my account setup.

In order to use these second class Google citizens, I have to log out of one Google account and into another. This leads to confusion between my tabs; ever left Google Reader open while checking your ad revenue, too bad, you can’t!

Here’s a special gripe about Google Reader. They have this nifty feature that lets you email your friends an item that you particularly like, but it just assumes that it will check your Gmail account contacts. If your login to Google Reader is ryan@ryantm.com and ryantm.com is hosted google app, then it should check your contacts over there. Instead I get a blank auto-completer even though my auto-completer works in my email account.

Anyway, Google, get your act together. You should have been able to figure out accounts better than this, instead you’re doing accounts in the same crappy way as everyone else: total chaos