Explaining GitHub Part 2

If you just tuned in, I’m attempting to explain Git and GitHub to the layman. The previous post was all about Git. This one focuses on what GitHub actually does.

Part 2: GitHub

Okay, now we can move on to GitHub. Unlike writing a paper, software developers rarely write code all by themselves in a closet. Well, some do… but most of us collaborate on our software projects. GitHub provides a meeting place for software developers to collaborate around code and the different versions of that code. It stretches the analogy, but imagine that your paper with all the different snapshots you took along the way was available for other people not just to read, but to derive new papers from. You would always be credited as the original author, but everyone can create derivative works and if someone added a really nice abstract you could pull just that change back into your version of the paper. GitHub acts like a digital room where everyone hangs out to share their projects, ideas, and most importantly their code. The same way that Facebook provides a medium for people to hang out digitally and share status, photos, notes, etc –GitHub provides a medium for programmers to share code.

Code is a very abstract notion to people who don’t program for a living, and this is where I think the idea of writing a paper as an analogy for writing code is very helpful. Just as you could have written your paper in Spanish or French or English, code can be written in any number of languages. The primary purpose of code is to communicate something to a computer. You can think of it like writing an instruction manual. In 3rd grade, I remember, we did an activity where we had to explicitly write out the instructions for making a peanut butter and jelly sandwich. The teacher then took our written instructions and followed them to a tee. If you wrote “Take two pieces of bread”, she would literally tear off two very small pieces of bread to start making the sandwich. If you didn’t explain how to open the jelly jar, there would be no jelly on your sandwich. If you didn’t explicitly say that the jelly and peanut butter sides of the bread should be touching in the middle, you would get a sandwich with jelly and peanut butter on the outside. This is what it is like when you have to communicate with a computer. At the lowest level, computers are really dumb; they can only do what they are told, so your instructions (code) must tell the machine exactly what you want it to do.

As you can probably imagine, writing all this code can get very unwieldy. Not only because of the sheer amount of code that it takes to develop a complex system like the Android operating system, but because that code is written by a large number of programmers (think lots of people editing the same paper all at once). This is where GitHub really shines. GitHub provides the ability for developers to publish and share code, solicit collaboration from other programmers, gain reputation in the community and discover/contribute to other code projects of interest. In many ways it is a realization of the original intent of the internet. To throw out yet another metaphor, GitHub creates a framework for the evolutionary improvement of software in real time. Like the evolution of species, software is allowed to mutate and change over time creating new variants, symbiotic relationships (sometimes even parasitic ones!), and a dazzling array of constructs and programs. GitHub is like primordial soup in which all these software projects are interacting, growing and changing. And instead of eons, this all happens at the speed of the interwebs.

If you want to learn more about GitHub, you know where to go.

If things are still fuzzy, then all you need to know is that we run a web site…. And we sell these cute little creatures that are a cross between a cat and an octopus…

Explaining GitHub

GitHub is well beyond being just another startup, but I still find myself explaining over and over again what it is that GitHub actually does. My brother posted this link on Facebook the other day with a comment that he still has no idea what we do, but has so far surmised that it has “something to do with a cat and an octopus”.

So here, presented in two parts, is the crash course for the layman:

Part 1: Git

We have to start with Git. Git and GitHub are two different things. Git is a version control system. You don’t need to know anything about it other than it is amazing.

Okay, maybe I’ll tell you a little more than that. If you have ever typed a paper into a computer (or written one by hand) you have probably used a low fidelity form of version control. Say you are working on a big paper and at certain points you want to save off a version or draft. You might do this for many reasons. Maybe it is an important paper and you want to put a copy somewhere else safe. Maybe you add a `01` to the filename and later a `02` and so on (or you might use today’s date `ImportantPaper_4_13_2011.txt`). Maybe you need to turn in a draft or pass a draft off for editing. These activities are all analogous to creating a snapshot of what your paper looks like at a certain point in time and saving it off. Another good example of why having snapshots is important is if you are going to completely re-write your conclusion. Having a saved version of your old conclusion lets you go back to it at any time to see what you wrote or even to revert back because your new conclusion turned out to be a bad idea.

Software developers are usually focused on code instead of papers, but they have the same problems. You need to be able to quickly and easily take snapshots of the current state of your code base (along with some comments about that revision) and have the ability to come back to that point in history at any time. You might also want to compare the current state of your code with any previous state. Programmers call the solution to this problem version control. Just as you could write your paper in any number of document editing systems (Word, Pages, text files, Quark, LaTeX, etc) there are many different version control systems. Git is one of those systems. Developers write programs in a variety of computer languages and these programs are sort of like big papers, but vastly more complex. Having the ability to take snapshots frequently and travel back in history to see how a code file changed over time is an essential activity.

If you are really interested in learning more about Git you can download and learn about it here. I would also highly recommend my friend Scott Chacon’s book Pro Git.

If you have stuck around just for the Octocats, then I suggest you head over to the octodex until part 2 comes out.

mobile 311

I just finished releasing updates to the mobile 311 application into the iPhone App Store and Windows Phone Marketplace and felt inclined to write a bit about my experiences.

To give some brief background: Advanced Mobile 311 is a new product for Blue Dot that is providing the ability for citizens to easily report issues in their communities. 311 is a phone number set aside in the US for reporting non-emergency problems and our applications take this concept into the hands of smart phone users allowing them to mark the location of problems on a map (possibly using GPS), take a picture and submit a specific service request to their city or municipality. We certainly aren’t the only one doing this and some US cities like San Fransisco and New York have already implemented and been running crowd sourcing 311 solutions for a while now. City Sourced is working hard at this same problem with limited partnerships around the US. Much more can be said around the business models and players in the Gov 2.0 movement, but for now I’m more interested in the process of writing and delivering consumer applications which is something very new for me.

While it will take a week or so for the application updates I just submitted to make it through the review process in their respective store, you can check out the existing apps both in the iPhone App Store and the Windows Marketplace (requires Zune unfortunately). Both are free of charge so feel free to try them out! By in large it has been a positive experience developing both applications.

Prior to this, I had no real experience developing applications on a mac, using Xcode or writing Objective-c. Once I got over the C# to Objective-c transition and understood the basic paradigms of the development framework I was quite productive in creating the iPhone version of the application. I have been a mac user for a long time (really since I was a kid) so the OS is familiar to me and I intuitively understand the UX. The API and maturity of the iOS SDK is really very impressive and the 311 application fit very well into what the platform was designed to do. I actually created the iPhone app first (as Windows Phone 7 was just a rumor at this time) so it was a good exercise to have to think in another language. I think the one thing that I struggled with the most was memory management specifically around the didReceiveMemoryWarning and viewDidUnload paradigm. Debugging programs and understanding errors in Xcode is kinda like taking a step back in time when you are used to Visual Studio and .net development. The maturity of the iOS platform also means that you have to deal with supporting multiple devices and multiple versions of the operating system along with new devices like the iPad which have a different form factor and require a different set of graphics. All that said, it was quite pleasurable developing on my mac and the speed of the iPhone simulator is a breath of fresh air after using the WM emulator. The tools felt lightweight, high performing and easy to work with. I even dove in and did some of the graphic design for the internal menu icons and status bubbles.

Submitting to the App Store was really no big deal. Instructions to properly build and sign your application were straightforward. The only thing that bites me each time is that I chose to have a space in my application’s name “Mobile 311″ and this seems to be a continued source of  problems when uploading to the Store (there are workarounds – comment if you are interested).

On the Windows Phone side of things we were lucky enough to be on the white list for applications that came out with the release of the physical phone. The Microsoft stack is really where Blue Dot feels at home and the development experience on the Windows Phone was very impressive (especially since I was basically working on a pre-release tool set and a version 1.0 product with limited access to real devices and all the other interesting things that come with being an early adopter). I must say that Microsoft was very supportive in this process, even providing consulting resources when necessary (although most of my issues were resolved by waiting for the next release of the tools). Development for Windows Phone applications is done in Visual Studio 2010 and in C# & Silverlight. I had never done any WPF or Silverlight development before, but the ramp up was pretty easy and I must say that it is a huge improvement over WinForms development. Visual Studio is a truly first class development environment and VS 2010 only raises the bar. The new wp7 emulator is fantastic (MS obviously took a page from Apple’s book here) – it loads very quickly and makes it easy to develop and debug your application. I struggled a little bit with things like the camera functionality (which does not work on the emulator) and understanding how the Task system works. It is really nice not to have to worry about memory management and the debugging tools are far superior to those in Xcode. Overall, it was very fun to re-create this application in another tool set/environment. I tried to make the app feel like a windows phone app and not an iPhone port and was pretty happy with how things turned out.

Submitting to the Marketplace was a very similar process to App Store submittal. The only thing that annoys me here is that the Marketplace holds all sorts of other stuff (not just phone apps) and you can only view things in the store you have Zune installed.

We are considering creating apps for the other leading smartphone operating systems, but until that happens I also created a mobile optimized web site that is reachable by any browser for submitting service request. Obviously there is no way to use things like GPS, but you can still submit requests to the system.

Finally, all of the server components are now up and running in Windows Azure, which is Microsoft’s cloud OS. Our server is architect-ed in a way that it can run both in a traditional hosted model and on Azure. The difference between our 311 system and some of the other offerings right now is that we are pre-integrated into some of the major ERP systems that cities already use to mange service requests. And by having our infrastructure in Azure we can offer a data stream to anyone without them having to purchase and provision servers or worry about physically infrastructure.

Go check out the apps! iPhone, Windows Phone

erlang: off and running

I’ve been reading through Armstrong’s book about Erlang as I mentioned last week, and decided to actually get my hands dirty and write some code. One the suggested exercises in the book is to write a ring benchmark program where you create a ring of N process and pass a message around that ring M times. The program should accept different values of N and M to vary both the number of processes in the ring and the number of times the message is passed around the ring. I decided I was going to do this both in Erlang and then in C# (which I’m much more experienced in) and compare both the code and some performance metrics.

I started writing the erlang program first because I wanted to try and solve the problem from an Erlang perspective and I though it would be interesting to try and think in a new language. Now, no laughing at my code, this is literally the first Erlang program I’ve ever written and I’m sure there are ways to improve this. Here is what the code looks like: (link to gist)

About 39 lines of code including white space and a couple of commented out lines. Next, I attempted to solve this same problem in C# and here is where I need to give some disclaimers and caveats. I knew going into this that there was no way that directly creating threads in C# was going to compete with the lightweight erlang processes. However, creating a solution that overcame this hurdle was a bit outside what I was trying to accomplish here so my end implementation in C# actually creates a thread for each node and uses and AutoResetEvent to provide a locking mechanism that allows the very basic passing of a message from one node to the next. I am sure there are issues with this, but here’s what I came up with. Actually the program is so much larger that it is hard to capture in a screen shot so here is a link to the gist if you want to see it in a better format. All in all it is about 140 lines including whitespace and my default C# formatting that I have setup with my Reshaper profile.

Now for some metrics! I’ll just present the basics here and you can run things on your own if you are curious. Just for the record, I took all these measurements on my Dell desktop (Windows 7 64bit, Xeon dual core 2.0 GHz, 8 GB RAM) although I did actually develop the erlang program on my mac (which was even faster than what I’m showing below).

Just a few words of explanation here. For the Erlang program, I wasn’t even able to measure a time in milliseconds until I got above 500 nodes in the process ring which is why there are no values there. The times for 10-100 nodes are in the microsecond range and aren’t worth showing on these graphs. For the C# program I started get OutOfMemory Exceptions when I got above 1000 nodes which is obviously due to the overhead of having 1 thread / node. Because of this there is no data beyond 1000 nodes. You can see that there is a massive difference between these two implementations even at the levels where the data does cross over. Creating 1000 processes in Erlang and passing a message around all 1000 processes 10 times took about 16 ms. Doing the same thing with threads in C# took almost 25 seconds.

If I were to spend more time with the C# program I could move to using the thread pool instead of discrete threads, and doing a better job of simulating message passing. However, the point is that Erlang has these ideas of concurrency built into the language constructs whereas in C# I’d have to create my own infrastructure to support similar features.

My last word on all this is about the difference between writing programs in the two languages. I uses Visual Studio and C# day in and day out, so writing that program wasn’t particularly difficult (plus I did it after solving the problem already once in Erlang). I do feel like programming in C# is often like writing a novel. The tools and intellisense are nice, but there is SO much typing that you have to do to get things done. Lots of files, lots of classes, many lines of codes. Even with intellisense, I often feel like my thoughts race ahead of what my fingers (and especially visual studio) are capable of doing. As much as I like VS, I do wait on it a lot.

I wrote my Erlang program in Emacs, which is a completely new editor for me (and fun to learn new key bindings after knowing vi, textmate and VS bindings) and it felt more like writing a piece of poetry than a novel. I though hard about the problem and the code I was writing, but did very little typing. Part of my slowness is obviously a lack of experience with the language, but in the end I felt like the language itself led me to solving the problem in a very koan like way.

7 series

Here is why I’m excited about Windows phone 7 series, which i’m just going to call 7 series from now on because it hurts me to type the entire name.

Finally something new. I wasn’t sure that MS would take the leap and literally throw away backwards compatibility with Windows Mobile. But they did and I’m fully on board with this decision. I’ve spent the past 4 years developing applications and frameworks for Windows Mobile and while Microsoft’s development tools are first class, the actual OS is one of the most frustrating pieces of software I have ever dealt with. It is always entertaining to watch first time mobile developers try to make sense of WM and do something simple like visit a web site. I think the smart and agile companies are going to quickly move on from the old Windows Mobile platform and see significant increases in the rate at which they can develop/deploy mobile solutions. I think the end users are going to require less training, and make fewer support calls. It is a rare day when Microsoft is seen as a leader in something like UI look-and-feel and everything I’ve seen so far about the 7 series interface has been fantastic in that arena.

Commercial grade devices in the enterprise. My prediction is that 7 series will bring on further movement of commercial grade devices into the enterprise. Right now, most of our customers use special ruggedized devices that are expensive, but very difficult to break and have embedded RFID/barcode readers, etc.  However, the decision makers in these companies are now coming up on year 2 or 3 of using things like the iPhone as their own personal computing device and expectations for mobile computing have grown considerably. From the durability perspective it is hard to rationalize a $2500 device when you can buy 5 or 6 commercial grade devices for the same price and just replace them when they break. In light of all this, we are starting to see more and more people request comercial grades devices for their enterprise applications and I think this trend is only going to accelerate. There will continue to be room for Windows Mobile, but the mobile companies that can deliver 7 series apps and phones are going to lead us into the next phase of mobile computing.

Development story. My guess as to the development story goes back to pre-PDC a year ago where there were small hints of something new in the mobile world at MS and then a complete blacklisting of everything mobile at PDC. At that time we were being told something new was in the pipeline but we weren’t privy to any details. Our own research brought us to the conclusion that the next platform would probably be Windows CE 6 or 7 based with an entirely new UI layer requiring XNA based development. XNA is managed code, but with a completely different model for rendering the UI that lends itself to game development much better than the existing WinForms message pump. This is what i would guess 7 series development is going to be like. I’m also hearing that there is a mobile version of silverlight as part of the development story. I think this will be a great option for many applications but it doesn’t solve the central issues in enterprise app development which usually entails large amounts of data, some of which must be cached locally on the device. As it stands right now, silver light doesn’t have an answer for this. And besides, the text rendering is awful like the rest of WPF. Every time I start down the WPF/silverlight route I end up frustrated with how unreadable text is and how unacceptable to any customer the migraine inducing application I might create would be.

mfLY! will live on. mfLY! is our internal MVC framework for developing rich mobile applications and because of the MVC pattern we will have only a small amount of work to do to create new views, but the rest of our applications should stay exactly the same. To be perfectly honest, this is a blessing, b/c the old Windows Mobile UI is painful at best. We made the decision 2 years ago to fully commit to the managed code story and while there are still some challenges we have come up with a highly  performant and compelling development platform that will easily move up to the 7 series platform. Good news all around here.

Sleep and dreams and developer double time

First of all, if you are not listening to radiolab, I am actively getting smarter than you. And yes, I know radio is an old technology – I listen via podcast.

I’ve been mulling over this post for the course of the long weekend after listening to a very old radiolab show on the topic of sleep. The entire show is fascinating but there are a couple parts in particular that resonated very clearly with all this talk lately about movement and loving your tools. Have you ever had the experience where you focus so intently on learning something to the point where you aren’t even really making progress anymore? And then, after a night of sleep you wake up and suddenly you can play that rift on the guitar or that problem you were trying to solve so hard is suddenly clear? This happens to me all the time and I’ve joked that I actually do some of my best problem solving in my sleep. Well Jad and Robert present some interesting theories around this behavior which is very widely recognized. It seems that the first couple hours of sleep are largely devoted to replaying the events of the day and a process not unlike erosion takes place where the master volume knob is turned down, lowering the noise of all your memories and experiences. What they found, however, is that things like practicing the guitar for hours and hours stick prominently above the rest of the background noise and although these erosive forces literally wash across your brain causing you to remember everything a little less clearly; at the end of the night the really important stuff now stands out clearly above everything else. That mountain of your focused guitar playing memory still got eroded away, but since it was so much higher than the rest of your memories from the day it actually ends up standing out clearer when  you wake up.  Not only do things that you focus on have this permanence in comparison to everything else, but basically anything that invokes some sort of emotional response. You emotions are like sticky notes to the brain signalling the importance of certain memories.

The next part of sleeping involves mixes the days experiences (after the erosive process has finished) with all your other memories. This is where things get weird and interesting and where in many cases people have claim to invent things in their sleep. It seems that you brain just kinda randomly starts combining things together – mixing and matching experiences with memories and senses.

What really stood out to me about all this was what it means for developers who love their jobs and their tools vs. developers that don’t. Between Joel and Brooks and many others, the general consensus is that all software developers are not created equal and that the good ones are better by orders of magnitude in terms of productivity and quality. I’m thinking a  lot of that comes down to passion and emotional response to what you do on a daily basis. Sure, raw talent probably has something to do with it too, but when developers are passionate about what they do and in an environment where they love the tools they use and the software they are creating then you are basically getting double time from them.

Do your developers dream in code? Are you aware how valuable the ones that do are to your business?

Some ideas about the how

So the first question I got from my friend Sean in response to my post Movement was ‘How?’ As in, how do you iterate quickly? How do you prevent the freeze frame?

Two ways: automation and love your tools.

In order to move with agility in software development, you must have any and every mundane and reoccurring task automated. This means understanding every step that needs to be taken to go from writing some code to seeing that particular piece of functionality running live in the real world. That path might include building source, running tests, interacting with version control, building installers, deploying the application, running more tests, rolling back the deployment and so on and so forth. All the little pieces it takes to connect your code to live functionality in the hands of the end user. Iterating quickly also means you must understand all the keyboard shortcuts and secret tricks of your tools. Those seconds wasted formatting your code in VS instead of pressing Ctrl-K-D matter. You must also have  a version control system that allows you to prototype and iterate quickly without fear or mal consequences. This is very much an evolutionary/natural selection-like process where you need to be able to rapidly try out ideas and designs, keeping the ones that work and leaving behind the ones that don’t.

Secondly, in order to move quickly in software development you absolutely have to love your tools. I’m talking about a real emotional connection here. When you love your tools you want to know everything about them. How do I do this faster? Is there an option for that? Wouldn’t it be great if it did this? How do I extend my tools to do this? Maybe you’ve built some of your own tools. Maybe you love Visual Studio. Maybe you love Emacs. I’ve seen developers who use nothing but a text editor and a shell (plus a whole array of scripts and custom utilities for making things happen) so the actual specifics of the tools doesn’t really matter. What matters is that you love them and learn the ins and outs of them.  Right now, I think ReSharper is brilliant; Dropbox makes me smile daily; and Git and Github are revolutionizing version control. Your tools are everything from the physical hardware of your PC to your OS to where you write your code, to where you application gets deployed and how you managed everything in between.

Finally, your ability is iterate is directly tied to the speed at which your feedback loop can communicate. The quality and speed of communication through the network of people involved determines how quickly you can put your tools to use to connect feedback, ideas, innovations and real working software.

Movement

Have you ever noticed how grotesque people look in videos when you press pause. The most beautiful move star looks odd and unattractive when frozen mid facial expression. To be alive requires that you change and move. Art, film, music, culture, fashion, ideas all rely on this flux. Yet change and growth are difficult – it is a constant fight against entropy. Sometimes it’s joyful, sometime it’s painful.

Great software requires change and growth as well. Sometimes that means more features. Sometimes that means less. Sometimes it means taking the same ideas and implementing them in a new technology or applying them to a different space. Sometimes that means starting from scratch. Sometimes that means evolving what you already have. As people and culture change, so must software change to keep pace. And to support this entropy fighting back and forth our software must be designed in a way that allows iteration. Rapid prototypical development creates a natural selection pipeline for the truly ingenious ideas to be brought forth. Interestingly, but not surprisingly this idea hints that along that line of evolutionary change are 10,000 ideas that failed for one reason or another. This is the reason why you must have a way to iterate quickly.

And if your software isn’t growing (or isn’t changing fast enough) then you become like those actors frozen in a single frame or like a poorly streamed video that starts and stops: showing a static image every few seconds. And unlike the eons afforded to natural selection your time is a little bit more limited.