We are huge fans of the Arduino stack which is hosted on GitHub, and there are dozens of Arduino boards running around the office. The beauty of writing code for the Arduino is that you rarely end up writing a lot of it and once you’ve written a sketch, it runs on a whole variety of compatible boards. Compared to how microcontroller prototyping was done 5 years ago, this is a serious breath of fresh air. There are some of us, however, who would rather continue to write code in vi (MacVim in my case), so I came up with a nice hack…

Enter vim-arduino.

This is my first attempt at getting Arduino IDE functionality into MacVim. Right now you can compile, deploy and open a serial port. The board for deployment always defaults to ‘uno’, but you can override that by putting the name of the board  on the first line of your sketch like so (any board found in boards.txt is valid):

// board: atmega328

Compiling code can be done like so: <Leader>ac

Compiling and the deploying is done like so: <Leader>ad

You can also open a serial port for debugging like this: <Leader>as

The goal here is to be 100% compatible with existing Arduino sketches and to take advantage of the rest of the Arduino environment. Your hardware hacking peers should never know you are secretly rocking vi.

Checkout the README to get things setup and bug me on Twitter if you find any of this useful.

Abstract

Ever wonder how software is designed and developed at GitHub? Are you curious about how new features are deployed to the site? (Hint: ask the robot.) Want to know why we don’t have any managers and don’t track vacation days?

This talk will explore running your company like an open source project and give some insight into how GitHub continues to leverage ruby and other open source tools to keep up with massive data loads and a growing community of users.

Slides

http://speakerdeck.com/u/tclem/p/code-first-ask-questions-later

Video

I know there was AV taken at the conference and I will post a link once it is available.

http://confreaks.net/videos/723-rockymtnruby2011-code-first-ask-questions-later

Credits

The font in my presentation is called Junction and can be found on the League of Movable Type. Many of the photographs are my own or from the GitHub Instagram stream. A couple of them are from flickr under the creative commons license: easy button, just do it, robot. The grainy robot attacking image is actually from SNL.

Part 2: GitHub

Okay, now we can move on to GitHub. Unlike writing a paper, software developers rarely write code all by themselves in a closet. Well, some do… but most of us collaborate on our software projects. GitHub provides a meeting place for software developers to collaborate around code and the different versions of that code. It stretches the analogy, but imagine that your paper with all the different snapshots you took along the way was available for other people not just to read, but to derive new papers from. You would always be credited as the original author, but everyone can create derivative works and if someone added a really nice abstract you could pull just that change back into your version of the paper. GitHub acts like a digital room where everyone hangs out to share their projects, ideas, and most importantly their code. The same way that Facebook provides a medium for people to hang out digitally and share status, photos, notes, etc –GitHub provides a medium for programmers to share code.

Code is a very abstract notion to people who don’t program for a living, and this is where I think the idea of writing a paper as an analogy for writing code is very helpful. Just as you could have written your paper in Spanish or French or English, code can be written in any number of languages. The primary purpose of code is to communicate something to a computer. You can think of it like writing an instruction manual. In 3rd grade, I remember, we did an activity where we had to explicitly write out the instructions for making a peanut butter and jelly sandwich. The teacher then took our written instructions and followed them to a tee. If you wrote “Take two pieces of bread”, she would literally tear off two very small pieces of bread to start making the sandwich. If you didn’t explain how to open the jelly jar, there would be no jelly on your sandwich. If you didn’t explicitly say that the jelly and peanut butter sides of the bread should be touching in the middle, you would get a sandwich with jelly and peanut butter on the outside. This is what it is like when you have to communicate with a computer. At the lowest level, computers are really dumb; they can only do what they are told, so your instructions (code) must tell the machine exactly what you want it to do.

As you can probably imagine, writing all this code can get very unwieldy. Not only because of the sheer amount of code that it takes to develop a complex system like the Android operating system, but because that code is written by a large number of programmers (think lots of people editing the same paper all at once). This is where GitHub really shines. GitHub provides the ability for developers to publish and share code, solicit collaboration from other programmers, gain reputation in the community and discover/contribute to other code projects of interest. In many ways it is a realization of the original intent of the internet. To throw out yet another metaphor, GitHub creates a framework for the evolutionary improvement of software in real time. Like the evolution of species, software is allowed to mutate and change over time creating new variants, symbiotic relationships (sometimes even parasitic ones!), and a dazzling array of constructs and programs. GitHub is like primordial soup in which all these software projects are interacting, growing and changing. And instead of eons, this all happens at the speed of the interwebs.

If you want to learn more about GitHub, you know where to go.

If things are still fuzzy, then all you need to know is that we run a web site…. And we sell these cute little creatures that are a cross between a cat and an octopus…

So here, presented in two parts, is the crash course for the layman:

Part 1: Git

We have to start with Git. Git and GitHub are two different things. Git is a version control system. You don’t need to know anything about it other than it is amazing.

Okay, maybe I’ll tell you a little more than that. If you have ever typed a paper into a computer (or written one by hand) you have probably used a low fidelity form of version control. Say you are working on a big paper and at certain points you want to save off a version or draft. You might do this for many reasons. Maybe it is an important paper and you want to put a copy somewhere else safe. Maybe you add a `01` to the filename and later a `02` and so on (or you might use today’s date `ImportantPaper_4_13_2011.txt`). Maybe you need to turn in a draft or pass a draft off for editing. These activities are all analogous to creating a snapshot of what your paper looks like at a certain point in time and saving it off. Another good example of why having snapshots is important is if you are going to completely re-write your conclusion. Having a saved version of your old conclusion lets you go back to it at any time to see what you wrote or even to revert back because your new conclusion turned out to be a bad idea.

Software developers are usually focused on code instead of papers, but they have the same problems. You need to be able to quickly and easily take snapshots of the current state of your code base (along with some comments about that revision) and have the ability to come back to that point in history at any time. You might also want to compare the current state of your code with any previous state. Programmers call the solution to this problem version control. Just as you could write your paper in any number of document editing systems (Word, Pages, text files, Quark, LaTeX, etc) there are many different version control systems. Git is one of those systems. Developers write programs in a variety of computer languages and these programs are sort of like big papers, but vastly more complex. Having the ability to take snapshots frequently and travel back in history to see how a code file changed over time is an essential activity.

If you are really interested in learning more about Git you can download and learn about it here. I would also highly recommend my friend Scott Chacon’s book Pro Git.

If you have stuck around just for the Octocats, then I suggest you head over to the octodex until part 2 comes out.

I took a new position as a developer at GitHub working out of their main San Francisco office and couldn’t be more excited about it. The culture is ideal, my co-workers are some of the smartest people I have worked with yet, I’m learning a ton, and I have big plans for all the amazing things this company is going to do in the next few years. Ramping up in a new job is always exciting, with all the new things to learn and people to meet. Finding your way around a new city just adds to the excitement: where to live, where to eat, where to buy crap you need. I cannot speak more highly of the SF GitHub peeps who have gone out of their way to make my family and I feel welcome. From driving us around all weekend to find an apartment, to offering rides, to supplying blankets & pillows & air mattresses for sleeping in newly rented apartments, to dinner invites and insights about living in SF –we have been enthusiastically ushered into the community.

As far as what I’m working on at GitHub, here is just a taste (more to come later, I promise)…

Tom and I are working hard to extend the reach of our campfire robot (hubot) into the physical world. It still needs some better packaging, but version 1 of “Door Me” is up and running; allowing GitHub scros to never leave their seats to answer the door buzzer again.

I’m also jamming on libgit2 –building out objective-c bindings, contributing to rugged and some of the other language bindings. If you aren’t familiar with the project, you should check it out. It is probably one of the best examples of TDD on a C project that I’ve seen recently.

Rock climbing (and especially bouldering which is what I do the most of) involves solving problems and this, I think, is at the root of the correlation. You might think that at the end of a day of writing code, the last thing you’d want to be faced with is another set of problems to solve. But there is something very refreshing about the mental and physical challenges associated with climbing that compliment a day job lived in the abstract. The rock is real, the goal is fixed. You need a combination of mental awareness, physical strength, dexterity, technique and lots of plain old perseverance to get up a rock face. There is also a individual/collaborate dichotomy to climbing similar to the world of math, science, and programming. At the end of the day, it is you the individual that must solve the problem and climb the route. No one is there to help you and it comes down to how well you execute the sequence of movements required to accomplish the climb. However, in the background there is a constant community of collaboration going on. Beta is exchanged. Technique is discussed. Methods of training or recovery debated. Different movements are tried over and over again with subtle variation and the results are communicated to the group.

If you get your thumb fully engaged and pinch while you are crimping off the top of that left hand you can gain enough pull to make that high move. But you have to keep that right toe on or it is all over.

Beta often doesn’t work for everyone though. Height, ape index and a variety of other physical factors determine how you go about climbing a particular problem and in many cases a friend’s method will be of no help to you. One of my peers is tall enough that I almost always have to come up with a different set of beta for climbing the same problems because I just can’t reach the holds and keep my feet where he does. Another friend is so much stronger than I that he doesn’t worry about his feet too much and is very adept at jumping for holds and then catching and stabilizing his swing. I’m a much more cerebral and technique based climber. I’m not overly strong, so I rely on body position, using my feet, maximum efficiency of movement and subtle leverage points to get up the rock. One of things I’m trying to learn now as I have progressed to harder climbs is actually how to move very dynamically on the rock by jumping and pogo-ing to catch holds while using my core to contain swing and stay put after I come back in contact with the rock again.

As a developer moving from the abstract world of 1′s and 0′s to the physical world of rock, sweat and sometimes blood it seems there is endless entertainment in the realm of conquering boulders and cliffs. The same way I sometimes wake up in the morning with the solution to a programming challenge appearing in my head, I will often dream about a particular bouldering problem I’m working on. Or I’ll be driving to work, visualizing the movements in my head of a particularly hard climb, my hands sweating on the steering wheel. When I come back to that problem the next day things click that weren’t working the day before and I’ll find myself  sending what previously felt impossible.

I’ve taken a few of my fellow developers climbing and have been fascinated by the reaction to the sport. For someone who has never climbing before (and especially if they are particularly strong or adept at another sport) climbing for the first time can be a very frustrating activity. Traditional strength does you nothing in climbing until you learn how to use it. And often it is forearm and contact strength that matters way more than your ability to bench press. You legs also matter much more than your arms do and some of the most common climbing advise heard is to climb with your legs, not your arms. One of these developers in particular impressed me. Not necessarily with his climbing ability, but with his persistence in solving the problems. I think he spent the entire night on a single bouldering problem. He worked every move. He listened intently to any guidance that I offered and then was back at it with renewed vigor. And by the end of the night he made it all the way up! What is really interesting is that over the years of working with this guy, I’ve noticed that this is the same way he tackles programming problems: with quiet persistence and a never die attitude. I learned it is equally important to check in on him and make sure he has what he needs to be productive or give some piece of guidance, but at the end of the day he delivers.

I think software engineering is much like this situation where you are constantly being thrown at problems that you don’t know enough about. You are continually learning new things and being asked to communicate where you are in solving the problem and when it will be done. You need a combination of a love for what you are doing, maybe some base talent, the ability to learn new skills on the fly, and a whole lot of persistence. Your attitude as you approach the inevitable road blocks and set backs matters more than anything else. That is why programmers love climbing. Because they thrive off those situations of solving hard problems, dealing with set backs and ultimately accomplishing great things. It isn’t burdensome to us, it is drives us forward and onto the next challenge. We thrive off it.

To give some brief background: Advanced Mobile 311 is a new product for Blue Dot that is providing the ability for citizens to easily report issues in their communities. 311 is a phone number set aside in the US for reporting non-emergency problems and our applications take this concept into the hands of smart phone users allowing them to mark the location of problems on a map (possibly using GPS), take a picture and submit a specific service request to their city or municipality. We certainly aren’t the only one doing this and some US cities like San Fransisco and New York have already implemented and been running crowd sourcing 311 solutions for a while now. City Sourced is working hard at this same problem with limited partnerships around the US. Much more can be said around the business models and players in the Gov 2.0 movement, but for now I’m more interested in the process of writing and delivering consumer applications which is something very new for me.

While it will take a week or so for the application updates I just submitted to make it through the review process in their respective store, you can check out the existing apps both in the iPhone App Store and the Windows Marketplace (requires Zune unfortunately). Both are free of charge so feel free to try them out! By in large it has been a positive experience developing both applications.

Prior to this, I had no real experience developing applications on a mac, using Xcode or writing Objective-c. Once I got over the C# to Objective-c transition and understood the basic paradigms of the development framework I was quite productive in creating the iPhone version of the application. I have been a mac user for a long time (really since I was a kid) so the OS is familiar to me and I intuitively understand the UX. The API and maturity of the iOS SDK is really very impressive and the 311 application fit very well into what the platform was designed to do. I actually created the iPhone app first (as Windows Phone 7 was just a rumor at this time) so it was a good exercise to have to think in another language. I think the one thing that I struggled with the most was memory management specifically around the didReceiveMemoryWarning and viewDidUnload paradigm. Debugging programs and understanding errors in Xcode is kinda like taking a step back in time when you are used to Visual Studio and .net development. The maturity of the iOS platform also means that you have to deal with supporting multiple devices and multiple versions of the operating system along with new devices like the iPad which have a different form factor and require a different set of graphics. All that said, it was quite pleasurable developing on my mac and the speed of the iPhone simulator is a breath of fresh air after using the WM emulator. The tools felt lightweight, high performing and easy to work with. I even dove in and did some of the graphic design for the internal menu icons and status bubbles.

Submitting to the App Store was really no big deal. Instructions to properly build and sign your application were straightforward. The only thing that bites me each time is that I chose to have a space in my application’s name “Mobile 311″ and this seems to be a continued source of  problems when uploading to the Store (there are workarounds – comment if you are interested).

On the Windows Phone side of things we were lucky enough to be on the white list for applications that came out with the release of the physical phone. The Microsoft stack is really where Blue Dot feels at home and the development experience on the Windows Phone was very impressive (especially since I was basically working on a pre-release tool set and a version 1.0 product with limited access to real devices and all the other interesting things that come with being an early adopter). I must say that Microsoft was very supportive in this process, even providing consulting resources when necessary (although most of my issues were resolved by waiting for the next release of the tools). Development for Windows Phone applications is done in Visual Studio 2010 and in C# & Silverlight. I had never done any WPF or Silverlight development before, but the ramp up was pretty easy and I must say that it is a huge improvement over WinForms development. Visual Studio is a truly first class development environment and VS 2010 only raises the bar. The new wp7 emulator is fantastic (MS obviously took a page from Apple’s book here) – it loads very quickly and makes it easy to develop and debug your application. I struggled a little bit with things like the camera functionality (which does not work on the emulator) and understanding how the Task system works. It is really nice not to have to worry about memory management and the debugging tools are far superior to those in Xcode. Overall, it was very fun to re-create this application in another tool set/environment. I tried to make the app feel like a windows phone app and not an iPhone port and was pretty happy with how things turned out.

Submitting to the Marketplace was a very similar process to App Store submittal. The only thing that annoys me here is that the Marketplace holds all sorts of other stuff (not just phone apps) and you can only view things in the store you have Zune installed.

We are considering creating apps for the other leading smartphone operating systems, but until that happens I also created a mobile optimized web site that is reachable by any browser for submitting service request. Obviously there is no way to use things like GPS, but you can still submit requests to the system.

Finally, all of the server components are now up and running in Windows Azure, which is Microsoft’s cloud OS. Our server is architect-ed in a way that it can run both in a traditional hosted model and on Azure. The difference between our 311 system and some of the other offerings right now is that we are pre-integrated into some of the major ERP systems that cities already use to mange service requests. And by having our infrastructure in Azure we can offer a data stream to anyone without them having to purchase and provision servers or worry about physically infrastructure.

Go check out the apps! iPhone, Windows Phone

Lutz Roeder recently passed ownership of this free Utility to Red Gate Software and the new Reflector under Red Gate’s direction sucks big time. The first annoyance is that the auto update feature forces you to go get Red Gate’s version. You run reflector and it says:

This version of .NET Reflector is out of date. Do you want to update automatically?

It used to be if you said ‘No’ you’d just get to run the version you already have. Now the program just quits. No Reflector for you! If you say ‘Yes’ you are alerted that Reflector cannot be automatically updated and that you need to go to the Red Gate site to download the new version. Brilliant, now I’ve completely lost the context of what i was using Reflector for in the first place, the auto update appears to be purposely broken, and I need to do a manual install of the new version to even be able to use Reflector again. And because I’m running reflector via TDD.NET’s VS plugin I have to go searching my hard drive for where the actually reflector exe is located (C:\Users\<username>\AppData\Roaming\Mutant Design\TestDriven.NET 2.0\Reflector if you are interested).

And that’s not the end of it. Once you do get the new version installed you get dragged into a trial of the ‘Pro’ version which has VS integration and all sorts of fancy features. The problem with this we realized a few days later and after a lot of hair pulling is that it massively slows down .NET CF builds. Once you open a visual designer in a .NET CF project the new ‘Pro Reflector’ is doing something crazy in the background that takes a long time (30-40 seconds) causing your build times to sky rocket and the hallway sword fights to commence. Thanks to my friend AB for figuring all that out after disabling each of his VS plugins one by one to sort out the issue.

I used to recommend this tool to everyone, but with Red Gate at the wheel, I’m not so sure anymore. Anyone know of some alternatives? I miss Lutz’s version.

Seems like I’m not the only one with this gripe:

http://wiert.wordpress.com/2010/04/06/net-reflector-catch-22/
http://apps.red-gate.com/messageboard/viewtopic.php?p=27756&sid=daab8e6756fbb5ce571073443d41a71c

I started writing the erlang program first because I wanted to try and solve the problem from an Erlang perspective and I though it would be interesting to try and think in a new language. Now, no laughing at my code, this is literally the first Erlang program I’ve ever written and I’m sure there are ways to improve this. Here is what the code looks like: (link to gist)

About 39 lines of code including white space and a couple of commented out lines. Next, I attempted to solve this same problem in C# and here is where I need to give some disclaimers and caveats. I knew going into this that there was no way that directly creating threads in C# was going to compete with the lightweight erlang processes. However, creating a solution that overcame this hurdle was a bit outside what I was trying to accomplish here so my end implementation in C# actually creates a thread for each node and uses and AutoResetEvent to provide a locking mechanism that allows the very basic passing of a message from one node to the next. I am sure there are issues with this, but here’s what I came up with. Actually the program is so much larger that it is hard to capture in a screen shot so here is a link to the gist if you want to see it in a better format. All in all it is about 140 lines including whitespace and my default C# formatting that I have setup with my Reshaper profile.

Now for some metrics! I’ll just present the basics here and you can run things on your own if you are curious. Just for the record, I took all these measurements on my Dell desktop (Windows 7 64bit, Xeon dual core 2.0 GHz, 8 GB RAM) although I did actually develop the erlang program on my mac (which was even faster than what I’m showing below).

Just a few words of explanation here. For the Erlang program, I wasn’t even able to measure a time in milliseconds until I got above 500 nodes in the process ring which is why there are no values there. The times for 10-100 nodes are in the microsecond range and aren’t worth showing on these graphs. For the C# program I started get OutOfMemory Exceptions when I got above 1000 nodes which is obviously due to the overhead of having 1 thread / node. Because of this there is no data beyond 1000 nodes. You can see that there is a massive difference between these two implementations even at the levels where the data does cross over. Creating 1000 processes in Erlang and passing a message around all 1000 processes 10 times took about 16 ms. Doing the same thing with threads in C# took almost 25 seconds.

If I were to spend more time with the C# program I could move to using the thread pool instead of discrete threads, and doing a better job of simulating message passing. However, the point is that Erlang has these ideas of concurrency built into the language constructs whereas in C# I’d have to create my own infrastructure to support similar features.

My last word on all this is about the difference between writing programs in the two languages. I uses Visual Studio and C# day in and day out, so writing that program wasn’t particularly difficult (plus I did it after solving the problem already once in Erlang). I do feel like programming in C# is often like writing a novel. The tools and intellisense are nice, but there is SO much typing that you have to do to get things done. Lots of files, lots of classes, many lines of codes. Even with intellisense, I often feel like my thoughts race ahead of what my fingers (and especially visual studio) are capable of doing. As much as I like VS, I do wait on it a lot.

I wrote my Erlang program in Emacs, which is a completely new editor for me (and fun to learn new key bindings after knowing vi, textmate and VS bindings) and it felt more like writing a piece of poetry than a novel. I though hard about the problem and the code I was writing, but did very little typing. Part of my slowness is obviously a lack of experience with the language, but in the end I felt like the language itself led me to solving the problem in a very koan like way.