Probably Programming

Angie has been slaving away working on Company-Y lately, and I think it’s coming along quite nicely.

Yesterday I decided to give her a break by helping with the coloring of the latest page “I love it when a plan comes together”, and I realized that the coloring process could be greatly sped up if we could simply flood fill the various areas, rather than having to draw around the outline of the area before flood filling it.

The comic is drawn in The GIMP. The lines are drawn on an transparent layer, and then the coloring is done on another layer below, so as not to mess up the lines. Since there are no boundaries on the lower layer, flood filling doesn’t work.

So I tossed together a quick TinyLisp add-on which takes the current layer, duplicates it, thresholds on alpha level (gets rid of any pixels that aren’t within a certain opacity level), and puts the new layer below the current one. After that, flood filling can be done in the lower layer without any issues related to the antialiased/feathered edges of the lines.

It’s a fairly simple script, though it took me a while to get it all working correctly (why the heck do all the GIMP built-ins return lists, making me have to (car …) everywhere, instead of just the primitives?)

Anyway, I was inspired, and came up with some ideas for some more time-saving add-ons to write, so I’ve started a Comic-Fu project at GitHub

If you want to install it, simply download make-fillable.scm and put it into your .gimp-2.6/scripts directory. It should then be in the Filters->Comic-Fu menu the next time you start up the GIMP.

p.s. I’m totally counting this as one of my programs for my 500 Programming Languages thing. I’ll do a write-up of TinyScheme at a later point.

I recently discovered the unicode xscreensaver hack, and have had it running on my secondary monitor for the heck of it. I found out that there are dice face symbols, and I was inspired to do something with them.

So, I made a simple Unicode-dice roller in JavaScript. I know the same thing could be done in JS, but this way you can roll some numbers to paste into Twitter or something, if you feel like it.

Enter number of Dice to roll:

Result:  

The code is extremely simple, calling Math.floor on the result of Math.random n times, then using a small table of the codepoints from U+2680 to U+2685 to build a string.

Here is the code for anyone interested:

<script type="text/javascript">
function roll(n) {
    var dice = ['\u2680', '\u2681', '\u2682', '\u2683', '\u2684', '\u2685'];
    var results = '';
    for (var i=0; i<n; i++) {
        results += dice[Math.floor(Math.random()*6)] + ' ';
    }
    return results;
}
 
function doRoll(countElem, resultElem) {
    document.getElementById(resultElem).innerHTML = 
        roll(parseInt(document.getElementById(countElem).value));
}
</script>
 
Enter number of Dice to roll:<br><input type="text" id="dice-roll-number" />
<input type="button" onclick="doRoll('dice-roll-number', 'dice-roll-result')" value="Roll!" />
 
Result: <span style="font-size: 64px; line-height: normal" id="dice-roll-result"></span>

(image source)

I read a lot of web comics, and one of the things that happens when reading through the archives of a comic is that I lose my place in the history, so when I come back, I have to spend a lot of time searching to find where I left off.

I wrote this simple set of bookmarklets to help me in this situation. The “save place” bookmarklet stuffs the current URL into a cookie (cookies are per site, so there will be one cookie for each site you use this on), and the “restore place” bookmarklet takes the URL out of the cookie (if it exists) and loads that page. The “clear place” bookmarklet clears the cookie.

To use them, just drag the bookmarklets to your bookmark toolbar, or right-click on them and select “bookmark this link”.

Save Place | Restore Place | Clear Place

For those who are interested, here’s the code in function form:

function savePlace() {
    var date = new Date();
    document.cookie='ppdc_saved_location='+escape(document.location)
        +'; expires='+date.setTime(date.getTime()+22896000000)+'; path=/';
}
 
function restorePlace() {
    document.location = (function() {
         var ca = document.cookie.split(';');
         for(var i=0;i<ca.length;i++) {
             var c=ca[i];
             while (c.charAt(0)==' ') c = c.substring(1,c.length);
             if (c.indexOf('ppdc_saved_location=') == 0) 
                 return unescape(c.substring(20,c.length));
         }
         return document.location;
    })();
}
 
function clearPlace() {
    document.cookie='ppdc_saved_location=; expires=-1; path=/';
}

I based my code off the createCookie and readCookie functions from QuirksMode

I recently read about the CSS transform property, and I was inspired to give it a try in a fun and mostly useless way.

After a bit of fidding, I present to you the “Rotate!” bookmarklet (click it several times if it doesn’t work the first time):

Rotate!

Simply drag the above link to your bookmarks toolbar, and whenever you get bored with a webpage, click that link, and see what happens!

The code is pretty simple. It just gets a list of all the divs on the page, picks one of them at random, and starts rotating it. Click it multiple times, and it will rotate multiple times.

Here’s the code, formatted for your viewing pleasure:

(function() {
  var ds = document.getElementsByTagName('div');
  var d = ds[Math.round(Math.random()*ds.length)];
  function rotateHeaders(r) {
    d.style['WebkitTransform'] = d.style['MozTransform'] = 'rotate(' + r + 'deg)';
    setTimeout(function() {rotateHeaders(++r % 360);}, 100);
  }
  rotateHeaders(1);
})()

Unfortuneately, this only works in Firefox and Webkit-based browsers at the moment..wait, no, that’s pretty much ok. You IE folks out there are used to things not working in your browser by now.

With a couple small changes, you can make it scale instead: Scale!

A managed to sneak in a little time recently and write a simple Lisp s-expression parser in Python.

I started out with a version that just understands parentheses, atoms, and strings, but it was just a couple of extra lines to add in support for the quote “operator”, that I went ahead and threw that in there, too.

I’ll go ahead and go through it bit-by-bit here, though I think it’s pretty straightforward.

from string import whitespace
 
atom_end = set('()"\'') | set(whitespace)

When any of these things are encountered while reading an atom, the atom should be ended. This means no need for whitespace betwen an atom and opening or closing parentheses or quotes.

def parse(sexp):
    stack, i, length = [[]], 0, len(sexp)
    while i < length:

Rather than write this recursively, I am using a list as a stack and a simple while-loop to go through the contents of the string. I start with an empty list on the stack, which will hold the entire parsed sexp when the parsing is done.

        c = sexp[i]
        reading = type(stack[-1])

Each character is put into c and the type of the current item being read is checked. Rather than using some variable to keep track of the current state of the parser, I just used a distinct Python type for each of the possible lisp types to be read: a list for a list, a string for a string, and a string inside a tuple for an atom. Using this system, I just have to check what type the top element in the stack is, and I’ll know what I’m currently reading.

        if reading == list:
            if   c == '(': stack.append([])
            elif c == ')': 
                stack[-2].append(stack.pop())
                if stack[-1][0] == ('quote',): stack[-2].append(stack.pop())
            elif c == '"': stack.append('')
            elif c == "'": stack.append([('quote',)])
            elif c in whitespace: pass
            else: stack.append((c,))

When reading a list:

• If an opening parenthesis is encountered, add a new list to the stack
• If a closing parenthesis is encountered, take the list from the top of the stack and append it to the next list down in the stack. Check if the last thing read was a quote, and if so, add this list as the parameter to the quote call.
• If an opening double-quote is encountered, add an empty string to the stack.
• If a single-quote is found, add a list with a quote symbol at the beginning of it.
• If c is whitespace, ignore it.
• Otherwise, start an atom with whatever other character was found.

        elif reading == str:
            if   c == '"': 
                stack[-2].append(stack.pop())
                if stack[-1][0] == ('quote',): stack[-2].append(stack.pop())
            elif c == '\\': 
                i += 1
                stack[-1] += sexp[i]
            else: stack[-1] += c

When reading a string:

• If a closing double-quote is found, end the string, and check for a previous quote call like above (it doesn’t really make sense to quote a string, but this isn’t really the place to check for that, is it?)
• If a backslash is found, put whatever the following character is in the string. This is a simplistic way to at least allow double-quotes to be escaped, but I don’t have anything allowing for more complex escape-strings in here yet.
• Anything else, add to the string.

        elif reading == tuple:
            if   c in atom_end: 
                atom = stack.pop()
                if atom[0][0].isdigit(): stack[-1].append(eval(atom[0]))
                else: stack[-1].append(atom)
                if stack[-1][0] == ('quote',): stack[-2].append(stack.pop())
                continue
            else: stack[-1] = ((stack[-1][0] + c),)

When reading an atom:

• If the end of the atom has been reached, check if the atom is a number (this code just assumes that if the first character of the atom is numeric, that it’s a number). If it is, I cheat and use eval to get its value, if it’s not, I leave the atom as a tuple-wrapped string.
• Anything else, add as a character to the atom.

        i += 1
    return stack.pop()

At the end here, I just take whatever’s on top of the stack (which should be the only thing on the stack if a proper s-expression was passed in), and return that.

I’m going to continue working on and improving this parser, for use in another project of mine, so I will be adding more functionality.

I was thinking of doing a s-expression to Python bytecode compiler, but someone already wrote one, and they even took the name I was going to use: sexpy.

I think the first thing I’ll be adding is the capability for the parser to yield sub-sexps so the entire thing doesn’t need to be parsed in one go before returning anything. Another thing I’ll most likely add is some checking for invalid sexps, something I left out for clarity in my initial version.

For those who are interested, here’s the complete version of sexp.py

If you don’t specify type=”submit” for a <button> element, IE7 (and probably IE6, too) will do nothing when that button is clicked (well, unless you have an onclick handler on that element, of course).

The W3C HTML standard states that the default for a <button> element should be “submit”, but if you want it to work with IE7 or earlier, you’ll have to be explicit about it.

I lost some precious time on this fun bug. I was doing some form validation via the great jQuery Validation plugin and for some reason, the form wasn’t validation in any IE earlier than IE 8.

Of course, since it’s such a pain to debug IE, I had a heckuva time figuring out the source of the bug, because I just assumed it was some weird JavaScript quirk, and didn’t even consider that it could be something as simple as a “missing” element attribute.

Ah well. Hopefully this will help somebody else avoid the same problem.

I’ve been neglecting both walking and posting on here, though I’ve neglected posting more, obviously.

I’ve been very busy with work and side projects and my house trying to fall apart on me, so I’ve been out of free time lately.

Anyway, my total is now up to 45 miles of walking. Not too much more than last time, really. I’ve gone on some walks around the neighborhood with Angie, and I’m not counting them, mostly because I don’t know how long they are, really.

I decided I wanted to start a “chain,” where every day I do something to work toward a goal I’ve set for myself (yes, I mean other than the walking, though that can be a chain as well).

Fortunately enough for me, I found Don’t Break The Chain, where you can keep track of one or more “chains” of things you want to do every day.

The Twitter Client Chain

So I’m creating a web-based Twitter client, as a project for me to try out all the different techniques and technologies that I want to use, but don’t have the chance to work with as much as I’d like during my day job. Also, ideally, it will at some point become a supplementary source of income.

That’s a quite a way off until I have more free time in a month or two, but for now, I will satisfy myself with making some small amount of progress on it each day (hopefully).

I’m writing this client in Erlang, using the awesome Nitrogen web framework with the equally awesome MochiWeb web server. My goals for this project include using test driven development and continuous deployment and fully embracing the mantra of “release early, release often”.

The plan is to have something up and running on a server as soon as possible. By something, I mean even less than a full-featured twitter client. But as soon as something is up, I can start getting feedback, and see what I should work on next. If I’m creating a product for users, there’s no use in developing what I want.

I need to develop what my users want.

I started this chain up about a week ago, and broke it after the first day. Now I’m trying again.

Tests and pre-commit hooks

During my previous, one-day chain, I managed to get a test harness (using the EUnit Erlang unit test framework) set up which will find any modules ending with _tests, stopping at the first error.

At the moment, I have a shell script which runs the tests. When it starts to take too long, I will switch over to an Erlang module which loops through, so I don’t get the startup times of the Erlang interpreter for each test. Or if anyone knows an easier way (perhaps there’s already a module that does that somewhere?).

#!/bin/bash
 
for modfile in `ls ebin/*_tests.beam`
do
    mod=`basename $modfile .beam`
    echo Running $mod:
    erl -noshell -pa ./ebin -s runtest main ${mod%.*}
    [ $? -eq 0 ] || exit 1
done
[ $mod ] || exit 1

The runtest module consists of the following:

-module(runtest).
-export([main/1]).
 
main(Mod) ->
    case eunit:test(Mod) of
        error -> halt(1);
        _ -> init:stop()
    end.

It’s all kind of thrown together, but the point is incremental improvement, so I’ll get back to it when/if I need to. Like I said, there’s probably something else out there that will do a better job at this than I did.

So, today, my “chain link” was partially taking that script and setting it up to run from the GIT pre-commit script, and partially writing this blog post.

Getting it set up to run as a pre-commit script just involved making sure it returns non-zero whenever there is a test error (or if no tests were run, which happens when make hasn’t been run yet).

The pre-commit script itself (placed in .git/hooks/pre-commit and set executable) is pretty simple:

#!/bin/sh
./tests.sh
if [ $? != 0 ]
then
        echo "===================================="
        echo " Commit stopped due to test failure"
        echo "===================================="
        exit 1
fi

Chain length: 1!

What’s next?

Next up, I’m acutally going to start on the meat of the project. Since I haven’t fully decided on what GUI library I want to use, nor what GUI testing framework I want to use, I’m going to start on the backend with code to manage user login info and queries (timeline searches, DMs, etc.).

I’ve got some great ideas (I think) on ways to enhance the way people have conversations over Twitter (and eventually other mediums), so you can be sure that there will be more posts about this in the future.

I walked at least 4 miles today. I’m not sure if I got too far over that, though, because a thunderstorm came through and killed the power before I could look at the exact amount on the treadmill. The last time I looked it was 4, so I’ll count it as that.

Thunderstorm

I was excited when I first saw the clouds rolling in, because we haven’t gotten a good rain for months now, and our lawn (as I mentioned in my last entry) is getting pretty brown.

However, when lightning started striking across the street from me, I started to get a little worried. That, and the fact that my wall-mounted monitor was swaying back and forth. Fun fact: speakers make a nice crackly noise when lighting strikes close by.

I don’t think I’d seen lightning strike before. It was quite a sight to see. A bright flash, then a sort of green glow and a halo of sparks that drifted slowly away from the strike point. And it happened in the same place, three times in a row, a few minutes in between. “Never strikes the same place twice” my ass!

After the storm had passed, we went outside to see how bad the damage was. Our South and street facing section of fence had been blown over into our yard, all of the fence posts simply snapped off. It takes a pretty strong wind to break a healthy 4×4, I think. We also lost the corrugated plastic roofing over half our back patio, and worst of all, the planter box with my ginger plants in it got blown over! Three out of four of the plants seem to be okay, but one of them got snapped off at the base of the stalk. We also found some roofing shingles in the back yard, so we’ll probably need to take a look at the roof when it’s light outside.

We were lucky compared to some neighbors. One guy down the street got a huge tree (somewhere around 3 feet in diameter) through the roof of his station wagon, and partway through his PT Cruiser (it hit the station wagon first, so that took the brunt of the damage). He seemed in relatively good spirits despite the damage, telling us that he was going to convert the station wagon into some sort of el Camino-type thing by cutting the damaged back half of the roof off. Perhaps all the beer he and his friend were drinking was helping him to stay in a good mood about it.

500 languages slowdown

I’ve been very busy with work and personal projects recently, and so I haven’t had much time for the 500 programming language project. I started learning about Nu, a lispy, interpreted, object-oriented language built in objective-c. I got the interpreter compiled and working on my 64-bit Ubuntu machine, which took more work than I would have liked due to the fact that most of the development of Nu takes place on Macs. Anyway, there’s a webserver for Nu, called Nunja and a Nu markdown processor, so I was thinking I’d write a simple blog engine as my Nu program. Nu is looking like a really interesting programming language so far, so it might be one I come back and spend some more time on.

But as I was saying, I haven’t had as much free time lately, so the posts have slowed down. Don’t let that fool you, though. I haven’t given up on the challenge, I’ve just accepted that it might be a while before I can be pumping out 5 languages a week. And it might, in fact, take me more than the 2 years I was originally thinking (dreaming) that it would take.

I’m starting a walking journal, to keep track of how far I walk each day, and to give me something to post about more frequently. Despite the fact that most of my posts on here are Probably about Programming, this is still, in fact, a personal blog, so I think posting personal things on here pretty much makes sense to me.

I’ll also use the walking journal as an actual journal, to talk about things other than walking. I can justify this by noting that I will, most likely be writing a lot of the journal while walking, so the name still works. I wrote most of this one while walking, so it works!

Walking journal

I’ve walked 5 miles since deciding to start this morning, so that’s a good start, I suppose.

Perhaps I’ll do a 100200 miles to a chumby-style thing with this. Though I have an advantage with my walking desk, because I can walk and be working at the same time. If I walked during my entire workday, I’d be able to get 16 or so miles a day without a problem, and I’d have 100 miles in no time , so maybe I’ll make my goal about losing weight instead of a specific distance. 20 pounds to a Pandora?

I’ve redoubled my efforts at weight loss, and I’ve started both walking daily and tracking my weight again.

You can join in on the fun and track my progress on my Physics Diet profile

It looks like I’ve been slowly on an upward gain, but it’s just because I weighed myself before dinner, rather than before breakfast as I always did before, and it filled in the blanks since February, since I haven’t weighed myself since then. I’ve actually been holding pretty steady in weight, I think, so it’ll be good to break through this plateau and get my weight lower.

School’s started again in Round Rock

School is back in session. I guess no more sleeping in since the children gather outside our bedroom window in the morning to wait for the bus.

On the plus side, I can once again go out on my front lawn in a bathrobe and yell at those “damn kids” to “get off my lawn!”

I could have done that during the summer, but I would have looked a little silly when all that was out there was dead grass.

Teeth are expensive

I had a 9am (super early for me) dentist appointment today to get an impression for a crown taken. Good god these teeth are costing me a lot of money.

I’ve used up all my COBRA dental insurance from my last job, and from what I hear it’s nearly impossible to get private dental insurance, so it’s going to all be out-of-pocket from now on.

Since with my current job I’m technically a “private contractor,” because they have no offices in the US, they are just paying me what would cover insurance (though it might not cover personal insurance after my COBRA is over).

Perhaps the way to go is really personal insurance in the sense that I will put away that same money every month in case of medical emergency.

Or I could just have all my teeth removed and switch to an all-liquid diet. I’m sure Angie would love having a toothless 25-year-old husband. For sure.

I just put up a live list of the most popular Twitter clients. The contents of that page are updated every 60 seconds (the length of time twitter caches their public timeline).

For some reason I woke up at 7 this morning, unable to get back to sleep, and I was randomly wondering how many Twitter clients were out there, and which were the most popular. So I decided to find out.

After a bit of hacking around, I came up with the following Python script:

from urllib import urlopen
from contextlib import closing
import pickle
import json
 
try:
    clientlist = pickle.load(open('clientlist.pickle'))
except:
    clientlist = {}
 
with closing(urlopen('http://twitter.com/statuses/public_timeline.json')) as f:
    json_str = f.read()
 
tweets = json.loads(json_str)
 
for tweet in tweets:
    source = tweet['source']
    clientlist[source] = clientlist.get(source, 0) + 1
 
with open('clientlist.pickle', 'w') as f:
    pickle.dump(clientlist, f)
 
total = 0
for count in clientlist.itervalues():
    total += count
 
with open('clientlist.html', 'w') as f:
    f.write('<html><head><title>Twitter Client list</title></head><body>')
    f.write(''.join(['<h1>Twitter Client Popularity</h1>',
                     '<div style="float: left; margin-right: 20px;">Out of <em>', str(total), 
                     '</em> Twitter messages, the following were the clients used.']))
    f.write('<table><thead><tr><th>Client</th><th>Tweet Count</th><th>% of total</th></thead><tbody>')
 
    for (link, count) in sorted(clientlist.iteritems(), key=lambda item: item[1], reverse=True):
        f.write(''.join(['<tr><td>', link, '</td><td>', str(count), '</td><td>', 
                         str(round(100.0 * count / total, 2)), '</td>']))
 
    f.write('</tbody></table></div>')
 
    f.write('</body></html>')

The script writes out a pickle file with the list of counts for each client, so multiple runs will generate a cumulative result (don’t run it more than once every 60 seconds, or you will add the same results in more than once).

It then writes out an .html file with a nice sorted list of the most common Twitter clients (well, all the twitter clients it’s seen, sorted by most common first).

I’ve set up a cron job on this server to update the file linked to above every minute, so as time goes on the list will become a more and more accurate representation of what people are using to tweet.