Archive for the ‘Programming’ Category

Google App Engine Announcement

Posted on April 9th, 2009 in Programming, Technology | No Comments »

Recently Google made a particularly big announcement concerning the Google App Engine: it will support Java. This announcement comes shortly after announced support for Groovy. Google has been making a serious effort to “win” the cloud computing wars, but this is the first time I’ve really seen a path to victory.

If you have never heard of the Google App Engine, it is basically a way for developers to build web applications that will be hosted in a distributed “cloud computing” fashion by Google. There are quotas and application limits, but it is possible to try things out for free. If your application becomes a big hit, then you can scale using Google’s infrastructure. You can find out more here.

Google App Engine is a big deal to developers because the worst part about developing web applications is that deployment, maintenance, and installation are the job of the developers rather than the users. End users like web applications because they are available everywhere they can find a computer and Internet access, but also because they no longer have to do any installation or software maintenance like patching or upgrading. Google App Engine makes all of this dead simple for developers as well.

Perhaps the most interesting thing about this announcement are its impacts on “cloud computing.” Cloud computing is the worst-defined buzzword since network neutrality, so perhaps that’s why they are staying away from it. I’m using it to refer to any web application where the vast majority of the data needed to run the application is stored by the server. The infrastructure and hardware that store this data and run the web application are a complete unknown to the end users. Furthermore, this infrastructure can scale dynamically as it is needed and without end user knowledge. There are all kinds of things that would fall into this category: Gmail (or really any web-based email), Apple’s MobileMe, Dropbox, or Mozy. Truly, the list could go on and on.

The only “real” competition that Google has in this space is Amazon Web Services. (I’m sure that others will emerge, but developers can use the Google and Amazon offerings now.) Developers can use AWS API to manage data and payment services, but they still have to install, host, and maintain the hardware for their application on their own.  There are some platform-specific hosting providers built on top of Amazon Web Services that can help developers on that front.  For example, a Ruby on Rails developer could host an application on Heroku, which uses Amazon Web Services. By default though, there is no “platform” built into Amazon Web Services as there is with Google App Engine. Google App Engine is the only wholly-owned development platform native to the “cloud,” and it just got a lot better for developers.

And that is how Google could win the cloud computing wars. There’s actually precedence for this if you take a look at Java. Java “won” the language wars of the 1990s not because of its beautiful syntax (blech!) or because of its amazing GUI frameworks (hah!) but because of its universal JVM. This is why so many dynamic languages are being built or ported to the JVM. Look at JRuby or Groovy for examples. It’s really only a matter of time before Ruby, Python, Groovy, and JavaScript are all as fast or faster on the JVM than they are in their native environments. Other dynamic languages, like Scala, are being built specifically for the JVM. In short, the platform was everything in the language wars of the 1990s, so why would we expect it to be any different in the cloud?

Executing Your Ideas

Posted on March 31st, 2009 in Education, Life, Programming, Technology | No Comments »

Below is a (hilarious) video about executing ideas that I saw thanks to Merlin Mann’s posting of it at the beginning of the year. Warning: this video is possibly not safe for work watching due to some language.

[Side note: If you've never heard of Ze Frank before, then I would recommend Ze Frank's TED talk.]

One of the things I would like to focus on is a quote from Ze Frank that Merlin highlighted as well. This quote from the middle of the video:

And the longer they wait, the more they convince themselves of how perfectly that idea should executed…But the bummer is most ideas kind of suck when you do them.

I love this quote and really the whole section in the video where Ze talks about ideas. There’s something both true and subtle in what he says. Think about everything you’ve ever seen, read, heard, or come across that made you think, “Wow, that’s clever.” You would never have felt that way without someone else executing their idea. Here’s the subtle part: How many ideas are just as clever, but were not executed upon by their thinker?

Good economists recognize the possible value in unrealized potential. Bastiat may have been the first to write about what is seen and what is not seen. Essentially, his argument boils down to this: Fixing a broken window may appear to be productive, but if that were really the case, then we should all break every window we can find to help improve the economy. In reality, the money spent on fixing the window could have been spent on something else that would have improved the world before the window was broken.

Although Bastiat was talking about the allocation of resources generally across industries, I think his argument applies equally well at the personal level. We need to allocate our resources on things that are actually productive and not just on things that appear to be productive. We need to stop convincing ourselves that our ideas are inherently valuable when they are actually not. If you convince yourself that you should hold off in executing on your idea until you’ve completely thought it through, then you will never realize the potential of the idea. It’s not enough to stop being actively unproductive; we have to force ourselves to continually produce.

Paul Graham has an excellent essay on ideas for startups that also touches on the value of an idea without execution. The hardest part of founding a successful startup is not generating the idea, it is executing the idea. In other words, there’s no such thing as a million dollar idea. Google was not a million dollar idea. Facebook was not a million dollar idea. Graham’s proof of this is dead simple:

Actually, startup ideas are not million dollar ideas, and here’s an experiment you can try to prove it: just try to sell one. Nothing evolves faster than markets. The fact that there’s no market for startup ideas suggests there’s no demand. Which means, in the narrow sense of the word, that startup ideas are worthless.

In other words, Google and Facebook are examples of million dollar execution, and I believe this concept is just as important at a personal level. Executing ideas is much harder than not executing them. There are all kinds of blogs out there that are devoted exclusively to dispensing advice on how to be more productive. It is easy to feel productive by reading them. It is easy to feel like you’re working on stuff. We humans are extraordinarily good at distracting ourselves or, as Ze Frank pointed out, convincing ourselves not to act, which is probably why executing ideas is so valuable.

The Cult of Done is the only example I can find that might (maybe) take executing ideas a step too far. They take an extreme position on doing things rather than thinking of things to do. (Here’s a good analysis on the Cult of Done.) We certainly need to emphasize actual execution of ideas since most people fall so far on the side of thinking and not even close to the side of doing. Perhaps adopting the spirit of The Cult of Done wouldn’t be a bad thing. After all, Ze’s right: most ideas really do suck when you do them, and the only way to find out is through execution.

Martin Fowler’s Observed Requirements

Posted on October 16th, 2008 in Programming, Technology | No Comments »

Martin Fowler recently wrote something with which I found myself jarringly disagreeing. The post is about a concept called observed requirements.

Although I strongly disagree parts of his recent post, I did want to say upfront that I really like Martin Fowler’s work. Ever since I read Martin‘s book UML Distilled, I have been a fan. For something that is meant to simplify understanding, UML always appeared overly complicated to me and his book does a nice job of focusing on making it useful. Martin is also a big proponent of Agile Methods and Extreme Programming, both of which have improved software development practices by turning software development on its head.

Martin’s post starts with this quote from the book “Mastering the Requirements Process,” which I read last spring. (Note: I read the first edition of this book, which contains the quote Martin uses. I have not seen the second edition. Then again, I only have the second edition of UML Distilled. Such is the life of a grad student.) Here’s the questionable quote:

Requirements are the things that you should discover before starting to build your product. Discovering the requirements during construction, or worse, when you client starts using your product, is so expensive and so inefficient, that we will assume that no right-thinking person would do it, and will not mention it again.

Suzanne and James Robertson

When I first read this quote, I had pretty much the same initial, gut reaction that Martin had. It advocates an extreme position in a field where many different development methodologies have been successful. However, Martin’s post takes a position at the other extreme and is equally questionable.

Martin seems to think that the word “requirement” itself is a bad word, and that requirements are incompatible with agile methods. He claims that web sites developed using agile techniques violate the “waterfallish” requirements process and suggests four specific ways that such web sites can observe requirements throughout development:

  • Look at what people are trying to do with the site and provide easier ways for them to do it.
  • Look at where people are abandoning doing something, and look for ways to fix whatever was frustrating them.
  • Build a new feature and see if people use it.
  • Build an experimental feature and make it available to a subset of the user base. Not just can you see if they like it, you can also assess how much load it puts on your servers.

He goes on to say that web sites should monitor how their users actually use their site because what a user really does is much more accurate than what a user says they do. First, let’s get this out of the way. Suzanne and James Robertson dedicate a significant portion of their book (about half a chapter) to requirements elicitation through observing users! Let’s look at two more quotes from the first edition of their book.

First quote:

It is unlikely that many users can explain what they do in enough detail for the developer to completely understand the work, and thus capture all the requirements.

Second quote:

For example, one of our clients, [...], had 20 different products. [...] The way the users handled each of these products at first looked to be different. However, a common pattern emerged as we studied the structure of the work – we were looking for similarities, not differences. We observed that each product was in fact a different way of [...]. The end result was that we found a common set of requirements, and were able to make a single core implementation, and then dress it differently for each of the products.

I have edited the second quote to remove project-level details and focus on the “observed requirements” concept. Quite simply, I don’t understand how these quotes are irreconcilable with Martin’s four suggested approaches to observing requirements. In fact, they seem to be quite compatible. The jarringly disagreeable part of his post is that Martin paints requirements as a software artifact that can only be used in Waterfall development, which is completely untrue.

Second, let’s talk about requirements. Requirements can, and should, be a part of any agile method of development. They answer a critical question: “why?” In the case of an observed requirement, the answer is obvious: “Because that is, by definition, exactly what the user wants or needs!” In fact, requirements are even easier to integrate into the agile process than other software development processes. The regular meetings with customers provide a great opportunity for requirements-based techniques, but agile proponents typically eschew actually documenting any of the contextual information involved in these meetings in favor of self-documenting code and UML.

Self-documenting code has always seemed a bit silly to me. Just as an author can’t write to two audiences at once, a programmer can’t satisfy the compiler and provide complete documentation at the same time. Martin Fowler writes in UML Distilled (2nd Edition):

The fundamental reason to use the UML involves communication. I use the UML because it allows me to communicate certain concepts more clearly than the alternatives. Natural language is too imprecise and gets tangled when it comes to more complex concepts. Code is precise but too detailed. So I use the UML when I want a certain amount of precision but I don’t want to get lost in the details.

Many agile proponents like UML and in particular, use cases. Unfortunately, use cases, and other UML artifacts, typically don’t offer enough contextual information to answer “why” questions. (Though, they are excellent for answering “how” questions.) It is possible to augment use cases with contextual information. Later in UML Distilled, Martin says that developers should feel free to modify UML to meet their needs. Of course, once you have added the needed contextual information, you’re effectively just writing requirements.

Any agile proponent would find a lot of Suzanne and James Robertson’s book useful if they went into it with an open mind. Many of the techniques for discovering customer requirements could improve the efficiency of regular customer meetings in an agile development process. Conversely, many proponents of detailed requirements specs would find a lot of useful information in agile-based books like Martin Fowler’s Refactoring book and Kent Beck’s Extreme Programming Explained book.

Third, and finally, let’s talk about the current use of user interaction data as an “observed requirement.” The one thing in Martin’s post with which I completely agree is that observed requirements are extremely useful and haven’t been fully explored. As a privacy researcher, I think there are some unresolved issues for user data protection, but as a developer it is clear that this data can improve the product. The often cited example in this is’s book recommendation service, which I enjoy. For the moment, let’s set aside the privacy concerns because that could be a whole post in and of itself.

Martin mentioned at the end of his post that he hasn’t found much advice on leveraging customer website use for the express purpose of improving their systems. I don’t think the idea is being used as well as it could be, but it is out there. John Musa has been talking about Operational Profiles, which are effectively a set of observed requirements, for years as a way to improve software development. He’s even got his own book out there on the subject.

That’s the only work that I know that gives extensive, useful guidance on how to take user interactions with a software product and directly tie it back into the development process. I certainly don’t have all the answers here. If anyone knows another place where this concept has been studied, I would love to hear from you.

vim plugins..

Posted on June 5th, 2005 in Programming | No Comments »

I just thought that I would give some praise to two vim plugins that I absolutely adore. The first is one that I’ve been using since college called SuperTab. This was something that my Computer Architecture project group found very useful in modifying the VHDL files that were automatically generated by Cadence. There were many long, long variable names used in those that were a huge pain to have to type out. SuperTab made editing those files far easier than it could have been without. It’s also allowed me to use adequately descriptive variable names in my other programming projects where I would otherwise have been tempted to use something that was faster to type.

The next plugin is called taglist. I found this to be really useful in larger programming projects, usually when there were single files consisting of thousands of lines of code. It allows you to bring up a window with a list of the various tags in the file you’re editing. This requires that the exuberant ctags program is installed on your machine. You can jump to the various functions and variables by switching over to that window and pressing ‘enter’ on the one to which you want to jump. It’s quite nice when you have 15 functions and they are spread out over a 1500 line code file. I tend to find myself either searching repeatedly or slowly scrolling up and down to find the spot I want to go to when I don’t have taglist available. It really makes a difference. The taglist project also has its own webpage if you’re interested.

For those of you who use Gentoo Linux, like me, these plugins are both in portage and waiting for you to emerge them.

so much to do, so little time..

Posted on April 14th, 2005 in Life, Programming | No Comments »

I’m going out of town for my brother’s wedding this weekend. I’m not very good at packing. Actually, it’s not that I’m not good at packing. It’s really that I’m not good at packing so much as I used to always forget something. It’s rare that I do that now, but I never feel like I have everything I need as a result. Anyhow, the whole process of traveling gets me a little flustered as a result. Plus, it’s a strange week already since I’m enrolled in that training class. It just feels like I’m not getting to cross many things off my to do list.

So it’s been a little while since I last posted and I thought I would give an update on the RedHat training class that I’m taking. The instructor is very good. He’s down to earth and willing to research answers to questions on things that he’s not familiar with. The manual is probably even better. It’s a very direct and concise guide with clear examples that will prove to be a wonderful reference to have after the class is over.

I have to admit that I was skeptical when I first signed up for the class. So many people have learned the intricacies of the linux kernel on their own, that it may seem a little unorthodox to take a class to learn these things. I’m almost done now and I’m convinced that this impression was right and wrong. It was wrong in that I really have learned a lot about the kernel and device driver programming. It was right in that I’m sure that I could have done this on my own only it would have taken a great deal longer to get to the point that I’m at now. Also, I’m not sure I could put together a better reference manual than the one they have.

I recently created a wikipedia account and made my first post on Crown Hill Cemetery in Indianapolis. I was a little surprised to see that they didn’t already have an entry. There’s actually a lot of people who have bios on wikipedia that say they were buried in Crown Hill Cemetery, but there was surprisingly no entry for the cemetery itself.

All in all, I was pretty impressed with how easy the process was to create and update an article on wikipedia. I have used wiki style webs before, but I suppose I thought that there was a lot of anal moderation on wikipedia. Maybe there is, but I haven’t seen it yet and I’m enjoying it.