Alistair Jones Blog

Size of your head

Not-very-small tablet PC, via Wired.

Finely-Sliced Operations

Since the previous post, I've reading up more on the devops movement and found a few nice pieces, particularly Graham Bleach on internal borders.

Thinking about borders within operations departments, I'm reminded of a recent client who operated with a mind-blowing number of siloed operations teams.  Coming at it from the development side, working through a problem with the operations staff was like exploring a never-ending labyrinth of teams.  Every time I probed a little deeper, my contact would defer responsibility to another mysterious team, usually located somewhere distant, that owned a slightly lower level of the stack.  Once I tracked down a real person in the mysterious team, the same thing would happen again, and I would be left searching for another distant team.  It seemed like I would never actually get to the real hardware.  An unforeseen consequence of virtualization is that it allows someone to declare themselves in charge of a Virtual Machine, but not in charge of the host or guest operating systems nor the hardware itself - staggeringly unhelpful.

The genius of these extremely-finely-sliced organisations is that they provide innumerable cracks down which responsibility, ownership and useful work can fall.  If a team has a sufficiently narrow focus, it is almost certain that no problem will ever occur that falls cleanly within its boundaries, so there is no responsibility at all - a manager's dream.

One thing that encourages me a little bit is a recent trend for counting the number "engineers" in an organisation.  I think all technical staff are counted in this metric, though I'm not sure whether people are including managers and other IT staff who might not be hands on.  In his QCon talk, Aditya Agarwal was very proud of Facebook's metric of 1.1 million users per engineer, while Andres Kütt was similarly proud of Skype's 600,000 users per employee.  Note that Facebook is very open about its number of engineers but secretive about its number of employees - I have no idea why.  I think counting the total number of technical staff is useful because helps people look at the broader cost of running software - not just development or just operations.  Also, I hope in the long term these kinds of metrics will highlight the painful inefficiency of silos and barriers.

Specialisation without Silos

I'm watching Michael Nygard's talk in the devops track at QCon.  This is my first taste of the devops movement and it's certainly something I like the look of.  I'm really struck by his assertion that we don't need a division between development and operations, but crucially he isn't saying that everyone needs the same skills - we will still have specialists.

I'm struck that this idea of bringing people together into a single team, while still maintaining specialisation, comes up a lot in the software world.  Some examples:

• We always try to have a single User Story implemented by a single team.  There may be different specialist skills required to implement a single feature, but all those skills should be in the same team and the people should work together; we don't want a "back end" team and a "front end" team.
• I'm frequently explaining to clients the value of having QA specialists integrated within a development team.
• More recently, with Pat Kua, I've been arguing for the integration of performance testing into the development team.

Having a single team has all sorts of implications.  I'm particularly keen on having a single backlog of work, even if not everybody has the skills to pull from that backlog.  Most importantly, a single team stands a better chance of having a single goal than do multiple teams.

Why are specialist teams so common in software?  My intuition is that it's a social phenomenon that people with similar skills and interests tend to congregate, and like working together.  Finding an effective mechanism to counteract this force, while still maintaining high-skill and specialisation, seems like a Social Technology (in Malcolm Gladwell's terminology) that organisations will need to acquire to be successful.

Agile 2009, Day 1

For the first session I went to Developing Agile Leaders and Teams: A Developmental & Transformational Path with Gilles Brouillette, because I've recently taken on a management-heavy role and my ideas on leadership are are, shall we say, unformed.  Gilles has a PhD in Transpersonal Psycology, and the session was heavy on what I can assume psychologists think about all day: questions about our perception of reality, and the models that we use to glean meaning from the world round us.  It was all rather deep.
His spent most of the 90 minutes building up a model of levels of physcological development.  The top three levels are apparently very rarely attained, but are where you need to be if you're going to be a successful leader.  So how do you get to these high levels of physiological development?  Apparently it's about changing the underlying way that you make sense of the world, from an "OR" model to an "AND" model.  It sounded to me like being able to hold more than possibility in your mind at the same time; a bit like understanding a smooth spectrum of probability rather than jumping to single binary outcome.

Compact syntax for Hypirinha tables

When you're designing a builder API, you have to decide what to return from each method call. In Hypirinha I made a choice early on that most methods should add a new child element, and return that new child element. This is optimal for trees that are deeply nested but where siblings are rare, and it's a reasonable compromise for most HTML. (If if I wanted to optimised for very wide trees, where nesting is rare, I would instead make methods return the parent node so that chained method calls create siblings.)

I had a dilemma about what the special text() method should return. It's slightly different from other methods on an element because in XML terms it creates a text node instead of an element node. I decided for the first release to have a Java type for text nodes and to return this type from the text method. This seemed logically consistent with all the other methods, which return the object they create. It would also allow me to add behaviour to the Text class if I found something useful for it to do.

I've been using Hypirinha in a real application for a few months now. Mostly it's used to produce simple status pages, for example showing how many messages are in a particular state. I found we were writing a lot of simple HTML tables, and they were more verbose than I would have liked. For example:

        Table table = table();
        Tr headerRow = table.tr();
        headerRow.th().text("State");
        headerRow.th().text("Number of messages");

        for (MessageState state : counts.keySet()) {
            Tr row = table.tr();
            row.td().text(state.name());
            row.td().text(valueOf(counts.get(state)));
        }

It seems a shame to have to store the Tr objects as variables. The problem was having to call text() on the Td and Th objects; because it returned Text I couldn't use the convenience contains() method that was designed for these occasional sibling element cases.

On reflection, I realised I still hadn't found anything useful for the Text type to do, so it was fair to drop it. I changed text() to return the parent element instead of the new text node, and the problem's solved:

        Table table = table();
        table.tr().contains(th().text("State"),
                th().text("Number of messages"));

        for (MessageState state : counts.keySet()) {
            table.tr().contains(td().text(state.name()),
                    td().text(valueOf(counts.get(state))));
        }

The change has found its way into Hypirinha 0.3. I'm sure there'll be a few more small tweaks as we get further along using the library in anger.

html2hypirinha Bookmarklet

I've been playing with the html2hypirinha app for a few weeks now, and it's been very convenient to have a bookmarklet. Try dragging this to your Bookmarks Toolbar: Convert to Hypirinha.

HTML to Hypirinha, via Google App Engine

Some weeks ago Felix sent me a small app he'd written to parse HTML and generate Java source code that would build the same HTML using Hypirinha.  I packaged it up so that it would run on the new Java flavour of Google App Engine and deployed it here:

http://hypirinha.appspot.com/html2hypirinha

The source code is here:

http://hypirinha.googlecode.com/svn/trunk/java-extras/html2hypirinha

I think it's a really nice facility because it quickly gives you a feel for what Hypirinha syntax looks like.  If you were actually writing an application using Hypirinha, it gives you an easy starting point - just point the app at some a page you like the look of, and then copy and paste the code into your IDE.  For example, check out the builder for the Guardian home page:

http://hypirinha.appspot.com/html2hypirinha?url=http%3A%2F%2Fwww.guardian.co.uk%2F

Once you've pasted the code, the real power begins - lean on your IDE's refactoring tools to remove duplication and introduce abstractions.

Hypirinha: HTML from Java

For the first time in my career I've been travelling to the office each day by train.  For some reason I find my time on the train extraordinarily productive, especially when I get my laptop out.  The first fruit of that productivity is a new open source project, hypirinha.  From the project home page:

Hypirinha is a Java library for generating HTML. Hypirinha follows the fluent interface style, whereby several method calls can be chained together as single statement to achieve the desired result.

I've found the exercise really fun, and I think the the library will actually be useful.  At my current client we're using it for some minor reports and find it works well.  Using it on a large-scale web application is still some way off, but I think it's useful to challenge the usual template-based approach we see if we can find something better.