Why call the blog 'One Less Cut in a Thousand' ?

Is there anything more dully self important than writing a post on why you called your blog a certain thing?

However, I was asked, so here's the answer.

You may have heard the expression 'death by a thousand cuts'; it derives from a barbaric method of execution used until 1905 in China but generally the phrase is used to describe 'creeping normalcy', or negative change which happens slowly in unnoticed increments.

My career so far has been spent on the inside of enterprises of all size. In my experience the most inspiring visions, innovative ideas and game changing technology fails to be adopted not because of a considered architectural or strategic decision to reject it but because of 'creeping normalcy'; the organisation fails to improve because there is inertia and apathy. Great ideas simply fail to find fertile ground - or, worse, we drift, apathetic, into creating terrible, unethical organisations.

Likewise, improvement projects fail not because of a considered decision to close them down - in fact, this is a successful scenario - but because lots of small negative actions chip away at the business case, or reduce uptake, or invalidate assumptions.

There are thousands of negative actions that reduce an organisations capacity to change, to improve. When I started the blog, my aim was simple; every post should have something positive a reader can take away that helps them to change their organisation; something positive to counteract the thousands of possible negatives.

Every post should be one less cut in a thousand. Perhaps one day we'll never make the first cut at all.

Real world enterprise social networks; why quality trumps quantity

I think the uptake of social media within the enterprise is really interesting. There are few independent studies (and lots of sponsored ones) so it was interesting to see the adoption of an experimental, completely unsupported and unpublicised enterprise Yammer implementation – it would be inappropriate to say when, where or how.

What happened was that following an initial burst of activity where the recruitment of new users went viral, it then went silent for about 9 months. There were very few messages, very few new users. Then the activity picked up again of its own accord; users would put up more messages and the join activity started to rapidly increase.

I must admit that having read about the Twitter new user ‘9 month bounce’ last year I was expecting it – basically, users go quiet after joining and then come back. This resonated because it’s exactly what I did with Twitter.

Of course, Twitter’s new user activity is obscured by the continuous near vertical (but linear, apparently) user number growth. In a limited environment any slow down in growth is much easier to see.

So why the dip? Why the pickup?

I think the dip was down to a few things;

1. Public nature of posting – people were a bit shy/wary.
2. Users unsure what to post, or why.
3. Insufficient followers – you post more when you have followers. Probably it's something to do with 9mths being the amount of time it takes to get enough followers as a new user in order for it to be worth tweeting.
4. Growth by spam; it was pointed out that on joining the platform spammed you to invite new users. These users might join but weren’t necessarily engaged.

Why the pickup?

Just my thoughts but:

1. It took 9 months to find the right users; a couple of users popped up out of nowhere and began posting on a regular basis, making the community active and therefore:
1. Useful
2. Accepted behaviour (cf.Gladwell’s ‘The Tipping Point’, particularly sections about ‘permission’.)
2. Personal social media penetration; users are relaxing about posting stuff. Without doubt enterprise users will get burned by posting inappropriate comments – and learn from it.
3. Viable network size; one of the reasons that Yammer wants to drive network growth (see 4 above) is that they’re aware that a big network is more likely to be robust and active, hence useful, hence endorsed by the enterprise.

Now, fascinatingly the relationship between the size of a network in the early stages of development and it’s activity was almost (could be?) mathematical: they correlated precisely. It was almost as if every single new post added a user, even though that user wasn’t addressed.

In fact, the number of messages was far below what we might expect; perhaps the quality of the network and network activity didn’t match the growth; perhaps quality drives quality, as growth drives growth.

The take away for me is that to build a quality network is more difficult and rewarding building a big one.

Where I've been going wrong with IT strategy

I love strategy. I love the idea. But it's only recently that I've come to realise where I've been going wrong all these years.

I've yet to see a company who has a non-IT core business define and execute a successful IT strategy.

By 'define' I mean clearly describe the vision, the goals/objectives,

By 'execute', I mean delivery of the strategic platforms and the solutions that sit on them; effective communication; mass adoption; proven benefit; everything defined delivered.
  

Sure, we see organisations deliver components of a successful strategy but never the whole hog.

Why?

Well, I don't think strategies are sufficiently agile. Certainly small, modern, agile enterprises seem to express themselves in terms that make big, mature, static organisations wince. Which is a bit strange seeing as they're both reliant on the same species to function.

Often, strategies don't actually mean anything. As soon as someone says the word, 'strategy', it seems to be the green light for academic techniques that don't actually resolve in anything a user would recognise. It's like even the communication of the strategy is FUD driven and scared of someone deconstructing the buzzwords.

Because they don't mean anything, they don't engage users. The guys on the ground floor don't care. The guys in the middle are busy being squeezed by the guys at the top and the guys on the ground floor. Vague, long range planning is your enemy. It doesn't translate into the real world.

Often, an the people within an organisation doesn't know what it cares about; what it stands for; what it's principles are. Think this is outlandish? Check out John Oliver, Grow Your Own Heroes.

So, strategists:
1. Make your strategy agile.
2. Eliminate buzzwords, be simple.
3. Engage users with tangible objectives.
4. Know what you care about.

As I said at the start, I love strategic thinking and know what a well chosen strategy can do. It's only now  I'm at Sabisu that the importance of it - and the way to make it really work - is becoming a little clearer to me. Here at Sabisu we're working up a bit of a guide on how we think strategy should be done and over the next few weeks we'll get round to putting up some ideas.

Why self-service BI is falling short

Having spent some time over the last couple of weeks introducing companies to Sabisu, it's clear that 'self-service' is seen as a big win - though the precise definition of what that means differs.

Every enterprise appears to have the same problems; complex business processes, a wide variety of often proprietary data sources, heavy use of IT expertise in integrating systems so that end users have the data where they want it. These problems result in duplication of data and a dependency on IT that destroys agility. How can you respond to incipient situations when you have to wait on an IT release schedule?

Everyone we've spoken to sees the answer in shifting capability out to the masses; empowering the end users by providing pre-configured reports or cubes where an end user can build report on-demand with recent data. Limited menu, often reheated data, but hot all the same - like fast food. This works to a degree, particularly in a slow moving environment, because you can schedule report generation or cube maintenance and the data will be 'recent enough'.

But it's not quite self-service. There's a long tail of requirements that the pre-built cubes aren't going to satisfy; all the IT department can do is invest more time, money and effort in building ever larger cubes as each new requirement is uncovered. Before you know it the costs associated with BI are spiralling, so the likelihood of tackling non-relational data or proprietary format manufacturing data is slight.

The end-user experience is often not great. User queries get invalidated as cubes are rebuilt (usually for IT reasons). Reports generated by different users don't tie up because different fields from different systems are confused - and implementing a data dictionary is often not viable, even if you can get cross-department agreement on a single version of the 'truth'. End users have to become proficient in what is, in effect, a development environment for building reports.

All this points to partial adoption at best on the grounds that the service just isn't great. It's got to be better.

I'm looking for:

• End user driven platforms, so no IT involvement - particularly none that could invalidate a trusted, end-user designed report
• Genuine end-user driven data access without needing to train everyone first - it's got to be built around modern UX principles
• Real-time, or as near as makes no difference
• Direct access to source data - if we're going to have a debate about the data, let's at least be clear on what we're looking at
• Some way to action BI; curate it for a community, make it actionable, collaborate on it
• Controllable expense - there's no way the enterprise should be penalised with increased expense or complexity for a user wanting to extract or share data

The perils of over-engineering

A couple of weeks ago, as referred to obliquely in our Sabisu blog we upgraded our development and QA environments. Our hand was forced because we lost the development environment irretrievably due to over-engineering in the earliest days of the company; something that couldn't possibly harm us in the future did.

The lessons learned can be summarised thus;

1. If you do experience an exponential increase in activity, you can probably find the funding for more capable infrastructure.

2. Advanced technology steals time. Use the simplest technology you can find that will do the job.

3. Only use technology you understand and have a track record in configuring successfully. If you're an infrastructure guy/company, fine - if not, stick to what you're good at and outsource the rest.

Anyway, here's the story.

When you set up a company, particularly a boot-strapping start-up, you are short of everything apart from ambition - that's what makes it fun. The two things you're critically short of are money and time but you can't help planning for the 'hockey-stick' eventuality; a graph that suddenly swings from gradual linear to exponential growth.

So we originally built two hardware platforms in-house on high grade consumer kit rather than industry standard hardware. (This is a problem in itself because high grade consumer kit often has the features of professional hardware but it's less expensive - and that cost saving appears as a time cost as, inevitably, the hardware is less reliable.)

Then we created three virtual environments on each machine; mirrored VMs for Development, QA and fileserver/DC. Each virtual environment was snapshotted in its entirety on a schedule. Data was backed up to a spare hard drive in one machine and the source code was backed up off-site to Amazon Web Services.

(Worth saying that the live system has always been hosted to the highest corporate standards by a third party infrastructure specialist.) 

In the worst case scenario, we'd lose a hardware platform but we'd never lose any data.

As a little start-up we didn't have the time or people to define and implement the kind of processes you need to manage this kind of environment. So, the VM paused mid-snapshot one day in something of a dither having ran out of room mid-snapshot.

We didn't expect this; we expected each snapshot to be a few GB at most, because we didn't understand snapshots fully or have the processes in place to monitor storage usage. We had no processes defined to allow us to bring back a failed VM. It was catastrophic.

As a backup, the snapshot was useless; it took several hours of crunching to get the most recent snapshot back and loaded to figure out that it was too old. So the VM was effectively bricked because the data was old - so the multiple platforms were irrelevant.

The VMs were a waste of time. We would have been better with any alternative. As it happens we reconstructed the source code from the online Amazon Web Services backups and each developer's work in progress version. We rebuilt the hardware platforms as decent standalone Development servers, which we understand.

How we set up Trac for agile development

Following this post, I had a couple of queries about how we set up Trac for managing Sabisu. Hope this helps.

Where does Trac live?
We implemented it onto our Dev server, so we have control over the environment. It could live anywhere but just seemed to make sense. We expose certain elements of our dev environment to the internet through the Sabisu platform, so that's how we get remote access.

How do you divide your product into Trac?
Initially we divided the platform up at the top level, so we had a separate instance of Trac for each major application; platform itself, Chronos time logging, Forms and so on. However, this made it difficult to see ticket allocations across the team so now everything is in a single project.

We then split the platform using into each Component, e.g., Chat, Chronos, Communities Functions, Communities View etc, through to Widget Editor. Every component is assigned to a different member of the team to make default work allocation easy.

Milestones
We use Milestones a lot. Every milestone corresponds to a release and we allocate each a codename because it's easier to say, "We're moving ticket 192 to the Nestor release" and have everyone know what you mean. We try to get a balance of about 20-30 tickets per release and we release new revisions on a weekly basis.

Priorities
Our priorities, running from highest to lowest; blocker, critical, major, minor, trivial, cosmetic. If part of the system is inaccessible, or we can't complete a test, that's the highest priority. At the other end of the scale a cosmetic indicates something that's genuinely cosmetic - if it affects UX in anyway it's major or minor.

Severities
Our severities, running from most to least; Multiple Customer Outage, Customer Outage, Customer Inconvenience, Irritant, Risky to leave, One for later.
For any severity lower than Customer Outage there's generally a work around available. 'Risky to leave' tends to be architectural or infrastructure work but no reason why it should be limited to that.
Also a ticket regarded an 'outage' mightn't be a Blocker; it could be that the functionality is accessible but fails.

Ticket Types
Couple of interesting categories: Defect, Enhancement, Live Incident, PoC.
Of course, Live Incident and Defect are both important categories. However, in conjunction with the Severity and Priority we can properly direct our efforts; a Live Incident could be relatively minor and addressed at a later date without significant impact.
The 'PoC' type is used to denote 'proof of concept' work. This is usually pure R&D work that needs productisation at a later date, usually through a series of Enhancement tickets.

The Priority, Severity and Ticket Types fields work together; the most serious ticket is a Live Incident causing Multiple Customer Outage preventing access to part of the system (a Blocker).

Resolutions
Very dull; Fixed, Invalid, Wontfix, Duplicate, Worksforme, Unable to Replicate.
I hate the Worksforme resolution because it's not a resolution of any kind…but tolerate it because sometimes you just can't reproduce a user reported defect.

We don't use Versions and don't link Trac to SVN, though it's perfectly feasible - it's just not something we've needed to do.

Comments or thoughts welcome.

Five rules for organising an agile, timeboxed product dev team

Over at the official Sabisu blog we outline some guidelines we use to manage the development of the product. I thought it might be good to expand why we established them , what they really mean in practice, and what they give us.

1. Work to the next release. It’s always next Monday.

The 'release early' philosophy is well established in agile software development; the sooner you can expose your work to your customers and react to their needs the better. Weekly releases allow us to turn around requests, incorporate customer feedback and incrementally improve the user experience.

Early in the lifecycle we tried to go to weekly releases but such a rapid release cycle caused a dip in quality as we tried to work in complex back end code too fast - now we have a mature platform and processes, the weekly releases are more logical. We take care not to expose users to complex functionality until it's usable, but the functionality is being gradually constructed behind the scenes as we go.

2. Incidents first. Defects second. Then enhancements. Always.

Many IT teams will hit incidents first; if your customers can't use your product for some reason, that needs to be sorted.

However, we only work on enhancements once we've got through defects; only defects waiting on a third party are put on hold.

This is in stark contrast to a lot of development teams where enhancements and defects are worked simultaneously. The problem with this approach is that (i) no one wants to work defects over enhancements and (ii) regression testing is tough.

Our approach does mean that in some releases there's little new functionality. We think that's a good thing; we concentrate on quality.


3. Every work item & every update goes into Trac.

Our defect/incident/enhancement/release management tracking tool is Trac. It's open source, simple but full featured tool that's our day to day management tool. Everything is logged, graded in terms of criticality and severity, assigned to a developer and allocated to a release. We tend to work about 25-30 Trac tickets into each release, with some developers taking only 2 or 3 effort intensive tickets.

Developer updates, testing notes, screenshots and anything else relevant goes into Trac. If we need a progress report, need to write release notes or are affected by a live incident, Trac will tell us what changes to commitments have to be made. Of course, it's all auditable if we need to trace the route of a change back through the process.

This means that we can forecast when a new feature or defect fix is going to be made available and if it should move, then all the relevant parties are informed.

4. The work plan gives the high level resource allocation.

Basically, the development team moves too fast for project plans to remain current, so we have a high level work plan that simply shows who's allocated to what customer (if they're off production) or release (if they're on production).


Having spent a lot of years as a project/programme manager trying to squeeze huge plans onto a small screen in Primavera/MS Project or whatever it's difficult for me to say this but...it's not something we do at Sabisu.

Basically there's no point. We work on such short timescales that a detailed project plan isn't much use beyond 3 weeks and all the detail is in TRAC anyway. All we'd be doing is shifting data from TRAC into a Gantt chart. By the time we've shifted the data the work's done.

So it's easier just to hit TRAC for a report of what's done and what's assigned to the next release. As long as we can get the 'big' bits of functionality into the main code trunk in a safe and sensible way we'll be ok.

You might legitimately ask how we plan the implementation of significant amounts of new functionality; there's a planning process implied in order to get the work into the build. The answer is simply that it happens offline, outside TRAC and the work is broken down into simple, achievable pieces of independent functionality prior to entry into TRAC. If a function is too big to be completed inside a release window, it gets broken down further into chunks that will fit.

Any bespoke customer work is dealt with the same way; we chunk it, tell the customer when they can expect each chunk and go for it.

(Now it's particularly interesting that back in my day at Motorola, we were expected to give the PMO a four week forecast for task completion, separate to the plan. I wonder if the data from the Primavera implementation didn't quite cut it?)

Beyond the high level work plan we have a long term roadmap which guides us in choosing the right features to bring into the product.

Whilst timesheets are done through our own Sabisu application, they're principally done for invoicing  customers as we do some bespoke work; we can be very accurate about how much time we've spent on each task.

5. Flex enhancements out to meet the release date (timeboxing).

Finally, we flex scope all the time. Making the Monday release with quality code is more important than shoehorning in new functionality. Generally, it means the new feature is delayed a week and we've never encountered functionality that's so time critical that it's worth endangering the quality of the code for.