Archive for August, 2009

Walking Journal: 9 miles, thunderstorm, etc.

Thursday, August 27th, 2009

weight-2009-08-26-2

I walked at least 4 miles today. I’m not sure if I got too far over that, though, because a thunderstorm came through and killed the power before I could look at the exact amount on the treadmill. The last time I looked it was 4, so I’ll count it as that.

Thunderstorm

I was excited when I first saw the clouds rolling in, because we haven’t gotten a good rain for months now, and our lawn (as I mentioned in my last entry) is getting pretty brown.

However, when lightning started striking across the street from me, I started to get a little worried. That, and the fact that my wall-mounted monitor was swaying back and forth. Fun fact: speakers make a nice crackly noise when lighting strikes close by.

I don’t think I’d seen lightning strike before. It was quite a sight to see. A bright flash, then a sort of green glow and a halo of sparks that drifted slowly away from the strike point. And it happened in the same place, three times in a row, a few minutes in between. “Never strikes the same place twice” my ass!

After the storm had passed, we went outside to see how bad the damage was. Our South and street facing section of fence had been blown over into our yard, all of the fence posts simply snapped off. It takes a pretty strong wind to break a healthy 4×4, I think. We also lost the corrugated plastic roofing over half our back patio, and worst of all, the planter box with my ginger plants in it got blown over! Three out of four of the plants seem to be okay, but one of them got snapped off at the base of the stalk. We also found some roofing shingles in the back yard, so we’ll probably need to take a look at the roof when it’s light outside.

We were lucky compared to some neighbors. One guy down the street got a huge tree (somewhere around 3 feet in diameter) through the roof of his station wagon, and partway through his PT Cruiser (it hit the station wagon first, so that took the brunt of the damage). He seemed in relatively good spirits despite the damage, telling us that he was going to convert the station wagon into some sort of el Camino-type thing by cutting the damaged back half of the roof off. Perhaps all the beer he and his friend were drinking was helping him to stay in a good mood about it.

500 languages slowdown

I’ve been very busy with work and personal projects recently, and so I haven’t had much time for the 500 programming language project. I started learning about Nu, a lispy, interpreted, object-oriented language built in objective-c. I got the interpreter compiled and working on my 64-bit Ubuntu machine, which took more work than I would have liked due to the fact that most of the development of Nu takes place on Macs. Anyway, there’s a webserver for Nu, called Nunja and a Nu markdown processor, so I was thinking I’d write a simple blog engine as my Nu program. Nu is looking like a really interesting programming language so far, so it might be one I come back and spend some more time on.

But as I was saying, I haven’t had as much free time lately, so the posts have slowed down. Don’t let that fool you, though. I haven’t given up on the challenge, I’ve just accepted that it might be a while before I can be pumping out 5 languages a week. And it might, in fact, take me more than the 2 years I was originally thinking (dreaming) that it would take.

Walking Journal: 5 Miles

Wednesday, August 26th, 2009

weight-2009-08-26

I’m starting a walking journal, to keep track of how far I walk each day, and to give me something to post about more frequently. Despite the fact that most of my posts on here are Probably about Programming, this is still, in fact, a personal blog, so I think posting personal things on here pretty much makes sense to me.

I’ll also use the walking journal as an actual journal, to talk about things other than walking. I can justify this by noting that I will, most likely be writing a lot of the journal while walking, so the name still works. I wrote most of this one while walking, so it works!

Walking journal

I’ve walked 5 miles since deciding to start this morning, so that’s a good start, I suppose.

Perhaps I’ll do a 100200 miles to a chumby-style thing with this. Though I have an advantage with my walking desk, because I can walk and be working at the same time. If I walked during my entire workday, I’d be able to get 16 or so miles a day without a problem, and I’d have 100 miles in no time :) , so maybe I’ll make my goal about losing weight instead of a specific distance. 20 pounds to a Pandora?

I’ve redoubled my efforts at weight loss, and I’ve started both walking daily and tracking my weight again.

You can join in on the fun and track my progress on my Physics Diet profile

It looks like I’ve been slowly on an upward gain, but it’s just because I weighed myself before dinner, rather than before breakfast as I always did before, and it filled in the blanks since February, since I haven’t weighed myself since then. I’ve actually been holding pretty steady in weight, I think, so it’ll be good to break through this plateau and get my weight lower.

School’s started again in Round Rock

School is back in session. I guess no more sleeping in since the children gather outside our bedroom window in the morning to wait for the bus.

On the plus side, I can once again go out on my front lawn in a bathrobe and yell at those “damn kids” to “get off my lawn!”

I could have done that during the summer, but I would have looked a little silly when all that was out there was dead grass.

Teeth are expensive

I had a 9am (super early for me) dentist appointment today to get an impression for a crown taken. Good god these teeth are costing me a lot of money.

I’ve used up all my COBRA dental insurance from my last job, and from what I hear it’s nearly impossible to get private dental insurance, so it’s going to all be out-of-pocket from now on.

Since with my current job I’m technically a “private contractor,” because they have no offices in the US, they are just paying me what would cover insurance (though it might not cover personal insurance after my COBRA is over).

Perhaps the way to go is really personal insurance in the sense that I will put away that same money every month in case of medical emergency.

Or I could just have all my teeth removed and switch to an all-liquid diet. I’m sure Angie would love having a toothless 25-year-old husband. For sure.

Live List of the Most Popular Twitter Clients

Friday, August 21st, 2009

I just put up a live list of the most popular Twitter clients. The contents of that page are updated every 60 seconds (the length of time twitter caches their public timeline).

For some reason I woke up at 7 this morning, unable to get back to sleep, and I was randomly wondering how many Twitter clients were out there, and which were the most popular. So I decided to find out.

After a bit of hacking around, I came up with the following Python script:

from urllib import urlopen
from contextlib import closing
import pickle
import json
 
try:
    clientlist = pickle.load(open('clientlist.pickle'))
except:
    clientlist = {}
 
with closing(urlopen('http://twitter.com/statuses/public_timeline.json')) as f:
    json_str = f.read()
 
tweets = json.loads(json_str)
 
for tweet in tweets:
    source = tweet['source']
    clientlist[source] = clientlist.get(source, 0) + 1
 
with open('clientlist.pickle', 'w') as f:
    pickle.dump(clientlist, f)
 
total = 0
for count in clientlist.itervalues():
    total += count
 
with open('clientlist.html', 'w') as f:
    f.write('<html><head><title>Twitter Client list</title></head><body>')
    f.write(''.join(['<h1>Twitter Client Popularity</h1>',
                     '<div style="float: left; margin-right: 20px;">Out of <em>', str(total), 
                     '</em> Twitter messages, the following were the clients used.']))
    f.write('<table><thead><tr><th>Client</th><th>Tweet Count</th><th>% of total</th></thead><tbody>')
 
    for (link, count) in sorted(clientlist.iteritems(), key=lambda item: item[1], reverse=True):
        f.write(''.join(['<tr><td>', link, '</td><td>', str(count), '</td><td>', 
                         str(round(100.0 * count / total, 2)), '</td>']))
 
    f.write('</tbody></table></div>')
 
    f.write('</body></html>')

The script writes out a pickle file with the list of counts for each client, so multiple runs will generate a cumulative result (don’t run it more than once every 60 seconds, or you will add the same results in more than once).

It then writes out an .html file with a nice sorted list of the most common Twitter clients (well, all the twitter clients it’s seen, sorted by most common first).

I’ve set up a cron job on this server to update the file linked to above every minute, so as time goes on the list will become a more and more accurate representation of what people are using to tweet.

A brainfuck synthesizer

Thursday, August 6th, 2009

waves

For my latest in my series of 500 Programming Languages, I decided to write a (simple) synthesizer in brainfuck.

Introduction to brainfuck

brainfuck is an esoteric programming language based on the idea of making a language with the smallest possible compiler. Indeed, my Python brainfuck compiler is only 30 lines of Python code, and I’m sure there are some optimizations I could have done to make it shorter if that was my goal with the whole thing.

The only data structure you get is a single array (if it were infinite you’d have a good representation of a Turing machine), and all you can do is move around this array with a pointer, increment and decrement values, input and output values, and loop.

The entire language consists of 8 commands:

  • > move the pointer forward one cell
  • < move the pointer back one cell
  • + increment the current cell
  • - decrement the current cell
  • . output the byte in the current cell
  • , input a byte into the current cell
  • [ move to the matching ] if the current cell is zero
  • ] loop back to the matching [ if the current cell is non-zero

Anything other characters in a program are ignored, and thus comments can be inserted anywhere into the program. Just be careful not to type proper English sentences with commas and periods, or you might get some funny bugs until you realize what’s happening.

That’s really all there is to the language. There are a couple of areas of behavior that aren’t standardized between interpreters, relating to the size of the array, the cell size, and handling end-of-file when inputing. Those are covered in more detail on the brainfuck Wikipedia page.

So, since this is such a simple language (to comprehend, not to write in), and I already provided a working compiler in my previous post, I’m going to do this post a little differently, and just cover the process I went through in writing my first non-trivial brainfuck program.

The idea

I tried to come up with an idea for a program you don’t normally see implemented in brainfuck, and I’ve never seen anything done with brainfuck and audio, so I decided to write a simple (non-interactive) synthesizer.

My first brainfuck program

To start out with, I wrote a program to just generate one second of simple square wave, at 4kHz (the max for 8kHz sample-rate audio).

With 8000Hz audio, there are 8000 bytes per second, so if the wavelength is two bytes (one byte high, one byte low, makes a complete single wave), that makes the frequency 8000/2, or 4000kHz. If I doubled the wavelength, I’d get half the frequency, or 2000Hz.

set counter to four thousand
> ++++ four
[ > +++++ +++++ times ten
  [ > +++++ +++++ times ten
    [ > +++++ +++++ times ten
      [ <<<< + >>>> - ]
      < -
    ]
    < -
  ]
  < -
]
 
keep a zero value and generate a 255 value
>> +++++ five
[ > +++++ times 5
  [ > +++++ +++++ times ten
    [ <<< + >>> - ]
    < -
  ]
  < -
]
< +++++ plus five (255 total)
 
<<
[
  >> . output 255
  < .  output 0
  <
  - decrement counter
]

This demonstrates how even generating a large number in brainfuck is quite a chore. I had to use four nested loops and four different cells to calculate 41010*10. Still, it’s better than having 4000 plusses at the beginning of my program.

The way loops are done in brainfuck reminds me of doing loops in assembly language. In fact, brainfuck in general reminds me of assembly language, but without all those convenient opcodes you’re used to. :)

What’s your frequency, brainfuck?

Next, I wanted to generate a lower frequency, working my way toward taking user input to determine frequency.

I picked the A below middle C, which has a nice, even frequency of 220Hz. When you divide 8000 by 220, you come up with a wavelength of approximately 36. Half low and half high makes that 18 bytes per half, so I needed to update the previous program to output 18 bytes of each value instead of just one byte per value.

222 waves of 36 bytes each equals 8000 samples for 1 second of audio
> +++++ +++++ +++++ +++++ ++  22
[ > +++++ +++++ times ten
  [ << + >> - ]
  < -
]
++ plus 2 more
 
>> +++++ five
[ > +++++ times 5
  [ > +++++ +++++ times ten
    [ <<< + >>> - ]
    < -
  ]
  < -
]
< +++++ plus five (255 total)
 
<<
[
  >>>
  +++++ +++++ +++++ +++ eighteen up and down = 220Hz
  [
    < . output 255 byte
    > - subtract from counter
  ]
  +++++ +++++ +++++ +++ eighteen up and down = 220Hz
  [
    << . output 0 byte
    >> - subtract from counter
  ]
  <<<
  -
]

This version adds two extra loops and an extra cell to generate eighteen 255s and eighteen zeroes for each iteration of the loop.

Like MIDI, but crappier

So, the next and final step was to allow input of notes to play, and the lengths to play each of those notes.

I found a helpful reference on the frequencies of musical notes.

Due to the fact that brainfuck only allows you to read a single byte at a time, and because it made the program simpler over all, I decided to have the two values input for each note represent the length, in waves, and the length of each half-wave. This allowed me to use the basic structure of the previous program, and to support a wider range of frequencies without having to handle multiple-byte input for notes.

generate the 255 byte for output
>>> +++++ five
[ > +++++ times 5
  [ > +++++ +++++ times ten
    [ <<< + >>> - ]
    < -
  ]
  < -
]
< +++++ plus five (255 total)
<<
 
, read in the length of the first note (in waves)
> , read in the length of each half wave
[
  <
  [
    >
    [->>+>+>+<<<<] copy the value to two cells for use in the following loops plus an extra copy
    >>>>[-<<<<+>>>>] copy the value back to the original location
    <<
    [
      < . output 255 byte
      > - subtract from counter
    ]
    > goto next counter
    [
      < . output 0 byte (zeroed by the last loop)
      > - subtract from counter
    ]
    <<<<
    -
  ]
  , read in the length of the next note (in waves)
  > , read in the length of each half wave
]

Since the half-wave length is input from the user, I can’t simply re-generate the value each time through the main loop, so I had to make extra copies of the value, and then re-copy the value back to its original input location, since it’s zeroed out when it’s copied to the other location.

Like I said, it’s harder than assembly language.

Now all you have to do is fire up your favorite hex editor and create a binary file with a list of lengths and notes (for example mary.notes), and save the output from the program into a file (for example, mary.raw), and you will have your very own brainfuck-generated music!

In an 8-bit unsigned 8kHz mono raw PCM file. Linux users can use aplay to play this file. It just so happens the default format is the same as the output of this file. This also means you can pipe the output of this program directly into aplay, as well.

All you need now is some sort of front-end which will interactively take keyboard input and translate it into notes, and you could use this for live performances. (note: you probably don’t want to use this for live performances)

Learning more about brainfuck

I learned everything I know about brainfuck from the brainfuck Wikipedia page, and so can you!

Conclusion

This is probably not a language you want to use, like, ever.

Unless you’re a huge nerd, like myself.

Then you might want to, just for the fun of it.

If it doesn’t sound like fun to you, then you can safely say you’re less of a nerd than me.

500 Programming Languages: Python

Saturday, August 1st, 2009

sillywalk

Python is one of my favorite programming languages. It’s almost always the first one that I reach for when I have a programming task I’d like to try out, and often enough, it’s the final language of that task, too.

Introduction to Python

On the official Python website, Python is described as “a dynamic object-oriented programming languages that can be used for many kinds of software development.”

They also claim that “Many Python programmers report substantial productivity gains and feel the language encourages the development of higher quality, more maintainable code.”

I am inclined to agree.

Though Python is a true “everything is an object” OO language, you can do more than just Object-Oriented programming with it. You can do plain-ol’ procedural programming, functional programming, Object-Oriented programming, and probably some other types of programming if you wanted.

One of the nicest things about Python is the “batteries included” approach, which means that Python comes with a ton of modules that do a large amount of what you want to do, right out of the box (or tarball, or installer…I don’t know if you can actually get Python in a box). Does your app need to write a cgi app? Or even better, a wsgi app? Web client? Web server??

Yep, it has all those, and many, many more.

There are many, more thorough introductions to Python out there, so I won’t attempt to cover the language in too much more depth, but I will show you a couple of things that really capture the spirit of Python:

Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18) 
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from __future__ import braces
  File "<stdin>", line 1
SyntaxError: not a chance
>>> import this
The Zen of Python, by Tim Peters
 
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
>>> import __hello__
Hello world...
>>> 
</stdin>

My First Python Program

This is far from my first program in Python. I first discovered Python when I was in High School, so possibly as early as 1999-2000. I had discovered Blender, a free 3D editing application, and it had Python embedded within it, for use in creating extensions and scripting games for the built-in game engine. Back then, I still thought I was going to be a video game developer, so I made some effort to learn Python so I could write games in Blender.

Anyway, enough history. :)

For this program, I implemented a Brainfuck-to-Python compiler. I won’t go into Brainfuck right now because I will be doing a post on it next.

This program is nowhere near demonstrating all of, or even a large number of, Python’s features, but it does demonstrate how easy it is to write a quick-and-easy program.

I threw this compiler together in less than 5 minutes, and it functioned properly on my first try (the Brainfuck hello world from Wikipedia).

The only addition I had to make later was handling the case of end-of-file on the input.

def compile(program):
    indent = 4
    code = [
        'def compiled(input=None, output=None):',
        '    import sys',
        '    if not input: input = sys.stdin',
        '    if not output: output = sys.stdout',
        '    i = 0',
        '    a = [0]*30000'
        ]
    commands = {
        '>': 'i += 1',
        '< ': 'i -= 1',
        '+': 'a[i] += 1',
        '-': 'a[i] -= 1',
        '.': 'output.write(chr(a[i]))',
        ',': 'a[i] = ord(input.read(1) or "\0")',
        '[': 'while a[i]:',
        ']': ''
        }
    for command in program:
        line = commands.get(command, None)
        if line: code.append((' ' * indent) + line)
 
        if command == '[':   indent += 4
        elif command == ']': indent -= 4
    exec '\n'.join(code)
    return compiled

This function takes a string containing BF code as its one argument and returns a function which will run that code when called.

The function returned will have two optional arguments, input and output, which are expected to be file-like objects (at least implementing read and write, respectively), which allows for passing in specific input and/or handling the output from within a python program.

It’s kind of a dirty way to do it, building Python source, an then running it with exec, but it gets the job done. I could also save the function to a compiled .pyc file, so you could later import it directly, but that would require a bit more code and ruin the simplicity of it all. I could also have generated input for my Python assembler, and used that to generate a compiled .pyc file, but like I said, I was going for simplicity here, and encouraging people to learn Python, not necessarily Python bytecodes.

You can download the Brainfuck-to-Python Compiler if you’d like. You can download the Brainfuck-to-Python Compiler if you’d like.

continue on to learn more about Python

(more…)