Why build a Widget?

So you’ve heard about Konfabulator, that fabulous little engine running at the heart of Yahoo! Widgets. The Widgets are cute, and it all looks pretty simple to do. But why build one? Well, the honest answer is that quite often, you don’t. There are lots of times when making a web site instead would be quicker, easier and simpler. Widgets are not a web replacement. But there are lots of things that Widgets provide that a web page doesn’t:

1. Persistence, persistence, persistence.
   1. Persistent presence: nobody keeps a web page open all day. No, really. Obviously, one use-case Widgets already have heavily covered is information that changes a lot: stock prices, the weather. And with the advent of the Dock in 4, you don’t even have to worry that your widget is going to take up too much space: your users can hide it and bring it back up whenever they feel like it, and you can build a mini-display for the dock to alert them when something interesting might be happening.
   2. Persistent use: are you building a messaging application, and you’ve considered building a little popup window that users keep open all the time to keep track of their messages? Then you were already considering building a Widget, just on the wrong platform.
   3. Persistent data: does your application play with a lot of data that it’s tedious to pull in from a network? Is it possible your users would want to use their data offline? Would a client-side cache improve the performance of your app? Widgets have access to the filesystem and their own, local SQL database. If that doesn’t make you giddy with possibilities, you’re not the developer I thought you were.
2. Desktop interaction
   Upload a file… click browse… blah. If you want to interact with your user’s files, web pages suck. Widgets can accept drag-and-dropped information, send data to and from the clipboard, and any number of capabilities that the browser won’t give you without significant effort and a couple of security dialogs. JavaScript freed from the chains of the browser is fun.
3. Performance
   Web applications are an amazing, wonderful hack: a medium that was designed to display static pages of information has been twisted into an application development platform. But despite all the joy that brings, it remains fundamentally a hack. Some applications are just too rich and too complicated to be made in a web context, or worse, when made in a web context their performance makes them unusable. Konfabulator on the other hand was built to make applications from the start, and it shows.

Desktop 2.0

Note what I said just now: applications, not toys or gimmicks. Widgets are not a web replacement. They’re not web 3.0. They are a completely new breed of desktop application: a term we somebody threw out in the office as a joke was Desktop 2.0, but that’s really what we’re talking about. Persistent, performant, web-connected, and a damn sight easier to build than Java or C++ or any other desktop-application development language you care to name.

Widgets are the natural result of the explosion in the number of developers with web-centric skills since the late 90s. Any development environment that attracts large numbers of developers produces a push to increase the capabilities of that environment into other areas: this is why there is command-line PHP, GUIs written in Python and web apps written in VBScript. So it was inevitable that JavaScript would migrate to the desktop, and taking a bunch of other web-like technologies like XmlHttpRequest and the DOM with it is good and important too, because Widgets aren’t just a different language for writing applications, they’re a new way of writing applications.

The thing about Widgets that endears them to me as a development platform is that they get software development the right way around: build the interface first, and hook the functionality into that. This is such a simple idea, and yet there’s no desktop development language that works this way: yes, there are visual ediitors for C++ and Java, but they are just working with a set toolkit: the code is actually central, and if you want to do something out of the ordinary with the interface, you have to go back to the code. Because the truth is that for the vast majority of applications, the processing that’s going on is minor, and all the value is in the interface: presenting the right information at the right time, in the right way. Widgets let you concentrate on that.

Not all roses

Of course the problems. JavaScript is a lot faster in Konfabulator than it is in any browser, but JavaScript is still an interpreted language, and that has performance implications. There are other problems, too: the documentation (a personal bugbear and current project of mine) still has some way to go before it’s up to the standards of developer.mozilla and PHP.net. You are still insulated from the operating system to a certain extent. And despite our best efforts, there will inevitably be some bugs.

Widgets are not the be all and end all. They do not replace desktop applications, nor do they replace web applications. Instead they complement both, and produce a useful middle ground, where a web dev like you can create something that looks and works like a desktop application.

Why use PHP frameworks?

Once upon a time, Perl was (and in fact remains) a perfectly capable language for writing web applications. But capable is not the same as suitable: it just wasn’t as good a choice for web applications as PHP, because even in version 2.0 of PHP you could do all the same things by using built-in functions, and people recognized that these things were faster and more reliable than building them themselves. The savings produced by not reinventing the wheel outweighed even the problems of switching languages.

Frameworks — like Ruby on Rails, and a raft of emerging PHP on Rails MVC frameworks, of which my favourite is CodeIgniter — are just the next generation of this principle. Where once we marvelled at how easy PHP makes it to query a database (only 5 lines of code!), now we can marvel at how easy a framework makes it (after you’ve set up your framework, only one line of code!).

The “I don’t need all that” trap

A common issue experienced coders run into when they look at frameworks is that they will say “this framework is too heavy, I don’t need any of that stuff”. This is particularly likely if what they’re building is supposed to be an experiment or a “prototype”. Being too heavy is of course a perfectly valid criticism of some frameworks, depending on the nature of your web application. But to use the same excuse to brush off frameworks in general is dangerous, and the “prototype” excuse even more so.

You don’t need the overhead of a framework to build your single-page blog software: often, doing it from scratch would be shorter and possibly even faster. This is, in fact, a problem with the overly simple demos the frameworks often promote via screencasts to demonstrate their capabilities. What you need a framework for is when your application becomes more than a single page, and when there’s more than one developer working on it.

This is why the “prototype” argument is also a false one: when’s the last time you actually threw away a prototype? You build the base, it works, so you tell everyone in the office about it. So you add a few extra features, tidy things up, eventually things snowball and the whole thing goes into production. And by the time you do that, you have to start maintaining it, and you should have used a framework.

The reason you need frameworks is because there’s no such thing as a small application. There’s just baby applications, which like all babies are small, simple and cute, and old applications, which are bigger, uglier, and frequently stink.

The real benefit of a framework is not in the screencast

There are two main branches of benefits of using frameworks:

1. First-write functionality: by not reinventing the wheel, you develop faster
2. Re-write functionality: having code that fits a standard pattern eases debugging, maintainability, and portability of code from one developer’s brain to the next.

The first benefit is the one the frameworks advertise in their screencasts, and often the area where they concentrate their further development efforts, building up complex AJAX and other functionality*. But it’s my strongly-held conviction that the second benefit is by far the greater: however, it’s achieved pretty much as soon as the framework is created, so it’s difficult for the developers working on the framework to remained excited about it. Fundamentally, significantly reduced maintenance cycles aren’t sexy, but they are useful as all hell.

Maintainability remains an enormous stumbling block in web development. It’s easy to write a monolothic, procedural script that handles all your data capture, validation, processing and output. Once you keep your internal model of that script in your head, it’s even relatively easy to maintain. The problem turns up six months later: you’ve forgotten how you wrote the script, or worse, you’ve had to pass the code on to another developer, and they can’t make head or tail of it.

A framework gives you built-in breakpoints for effective debugging: if the return statement in your model code is returning accurate data, you know for sure the problem is presentational, and vice versa. The nature of framework URLs makes tracing execution whole orders easier: you know automatically which function is being called in which controller, just from the URL. As your project begins to grow past the capabilities of a single developer, these features become essential, since your team members will be working with code they didn’t write. Frameworks by their nature provide your team-mates with a lot more information about function X than your average function name.

Speaking of which, some will say: what about naming conventions? Surely a framework is just a really elaborate naming convention, involving whole directories and files rather than just function names?

Not really. It’s a naming convention, but more importantly a coding convention: we don’t just specify what the function does, we specify where whole classes of related functions go, and the nature in which they interact. This provides those debugging benefits I mentioned earlier, and MVC also provides a structure that scales easily to applications with dozens of models and hundreds of views: modern web applications.

Maturity, at last

Frameworks provide something that has never before existed in the web development field: a convention that exists in more than one company. The more we as an industry use frameworks, the great the network effect: it means code and debugging can work effectively across companies, that new hires will be able to quickly understand the operation of your software and get productive faster.

Frameworks are a sign of a new maturity in the field of web development, a side-effect of the shift from writing “pages” to “sites” to “applications”. And it’s about time.

* I have yet to see a framework build AJAX that meets a high standard of web development. It doesn’t count as separating behaviour from content if your view has to make a bunch of ajax-specific calls and you end up with a bunch of inline onclick handlers.
** Unless you write consistent, comprehensive and up-to-the-minute documentation, of course. But nobody ever has. No, not even that guy.

Let me put you in the magic position

So it may strike some as odd to start a blog about web dev and Widget dev by naming it after a Patrick Wolf song, but that’s what I’ve done, so you’re all going to have to learn to live with it. I chose the name partially because web dev is all about position, but mainly because I really like the song. I could make up some bullshit about how Widgets are also related to position in some way, but I really think that would be reaching.

I’m writing the web dev stuff because that’s what I was and what I still do sometimes, and I’m writing the Widget dev stuff because that’s what I am now. So the web stuff will likely be relatively advanced, and the Widget stuff relatively simple. In particular I’m hoping to write a lot of the Widget stuff from the perspective of a complete beginner, before that feeling wears off for me.

I’m aiming for one post a week. Let’s see how we go, shall we?