Five rules for organising an agile, timeboxed product dev team

Over at the official Sabisu blog we outline some guidelines we use to manage the development of the product. I thought it might be good to expand why we established them , what they really mean in practice, and what they give us.

1. Work to the next release. It’s always next Monday.

The 'release early' philosophy is well established in agile software development; the sooner you can expose your work to your customers and react to their needs the better. Weekly releases allow us to turn around requests, incorporate customer feedback and incrementally improve the user experience.

Early in the lifecycle we tried to go to weekly releases but such a rapid release cycle caused a dip in quality as we tried to work in complex back end code too fast - now we have a mature platform and processes, the weekly releases are more logical. We take care not to expose users to complex functionality until it's usable, but the functionality is being gradually constructed behind the scenes as we go.

2. Incidents first. Defects second. Then enhancements. Always.

Many IT teams will hit incidents first; if your customers can't use your product for some reason, that needs to be sorted.

However, we only work on enhancements once we've got through defects; only defects waiting on a third party are put on hold.

This is in stark contrast to a lot of development teams where enhancements and defects are worked simultaneously. The problem with this approach is that (i) no one wants to work defects over enhancements and (ii) regression testing is tough.

Our approach does mean that in some releases there's little new functionality. We think that's a good thing; we concentrate on quality.


3. Every work item & every update goes into Trac.

Our defect/incident/enhancement/release management tracking tool is Trac. It's open source, simple but full featured tool that's our day to day management tool. Everything is logged, graded in terms of criticality and severity, assigned to a developer and allocated to a release. We tend to work about 25-30 Trac tickets into each release, with some developers taking only 2 or 3 effort intensive tickets.

Developer updates, testing notes, screenshots and anything else relevant goes into Trac. If we need a progress report, need to write release notes or are affected by a live incident, Trac will tell us what changes to commitments have to be made. Of course, it's all auditable if we need to trace the route of a change back through the process.

This means that we can forecast when a new feature or defect fix is going to be made available and if it should move, then all the relevant parties are informed.

4. The work plan gives the high level resource allocation.

Basically, the development team moves too fast for project plans to remain current, so we have a high level work plan that simply shows who's allocated to what customer (if they're off production) or release (if they're on production).


Having spent a lot of years as a project/programme manager trying to squeeze huge plans onto a small screen in Primavera/MS Project or whatever it's difficult for me to say this but...it's not something we do at Sabisu.

Basically there's no point. We work on such short timescales that a detailed project plan isn't much use beyond 3 weeks and all the detail is in TRAC anyway. All we'd be doing is shifting data from TRAC into a Gantt chart. By the time we've shifted the data the work's done.

So it's easier just to hit TRAC for a report of what's done and what's assigned to the next release. As long as we can get the 'big' bits of functionality into the main code trunk in a safe and sensible way we'll be ok.

You might legitimately ask how we plan the implementation of significant amounts of new functionality; there's a planning process implied in order to get the work into the build. The answer is simply that it happens offline, outside TRAC and the work is broken down into simple, achievable pieces of independent functionality prior to entry into TRAC. If a function is too big to be completed inside a release window, it gets broken down further into chunks that will fit.

Any bespoke customer work is dealt with the same way; we chunk it, tell the customer when they can expect each chunk and go for it.

(Now it's particularly interesting that back in my day at Motorola, we were expected to give the PMO a four week forecast for task completion, separate to the plan. I wonder if the data from the Primavera implementation didn't quite cut it?)

Beyond the high level work plan we have a long term roadmap which guides us in choosing the right features to bring into the product.

Whilst timesheets are done through our own Sabisu application, they're principally done for invoicing  customers as we do some bespoke work; we can be very accurate about how much time we've spent on each task.

5. Flex enhancements out to meet the release date (timeboxing).

Finally, we flex scope all the time. Making the Monday release with quality code is more important than shoehorning in new functionality. Generally, it means the new feature is delayed a week and we've never encountered functionality that's so time critical that it's worth endangering the quality of the code for.

The limits of metadata in the manufacturing enterprise

Moving on from the previous topic of curation being a better fit for manufacturing needs than 'conventional' BI, we should really look at the other data produced in large volumes within any enterprise: documents.

Documents tend to be produced by a fat client on an end-user OS, both of which imbue them with metadata and place them in a taxonomy.

Often both the metadata and taxonomy are of varying usefulness and accuracy as taxonomies are corrupted, folders duplicated and metadata rendered invalid by server processes. At least an end user can make a value judgement about the document and an enterprise search tool has something to index, meaning that from a list of apparently similar documents returned by a search query a user can make an educated guess as to the valuable item.

Once that valuable item has been located, the user might well share the location of the file with a distribution list...

...and that's curation picking up where enterprise search has failed.

When ERP data is considered, you'll find little metadata of value to the end user. Again, it would be a common scenario where an enterprise search returns possibilities and the end-user selects and publicises those of value to the wider community.

Manufacturing systems also generate very little metadata as they're designed around a single purpose, e.g., to log data in real-time. The metadata is limited to that which is necessary to make sense of the reading - you could argue it's not metadata at all. Clearly, in these instances enterprise search has nothing to offer here but expert end-users do; they can identify the key trends and highlight them to a wider community.

Of course, there's an network effect as more curation takes place; as more data is linked together by expert judgement, the value of the network increases exponentially with each link created.

Just as internet search engines are devalued by systematic metadata corruption (link stuffing, spamming, or any other 'black' SEO practice) so enterprise search is devalued by closed, proprietary or legacy systems producing unlinkable data.

And just as on the internet the value of curated content (usually) outweighs that of content returned by a search algorithm, so it will be in the enterprise, where the editors or curators are experts in the technical aspects of their business.

Why conventional BI fails manufacturing enterprises but curation succeeds

Back here I was describing what the terms democratisation, syndication and curation mean in the Enterprise 2.0 environment.

Of these, curation is particularly important to the process industries and perhaps to manufacturing as a whole. And here's why.

The data generated in a manufacturing environment can be thought of as broadly falling into the following categories; documents, ERP data and manufacturing data.

Whilst tempting to exclude documents from any BI discussion it's false to do so; whether in Lotus Notes, SQL or elsewhere, this is where day-to-day manufacturing decisions, events and instructions are stored. They represent a key data source for understanding trends yet are often ignored by BI solutions.

ERP data is typically proprietary and stored deep in an inaccessible database designed with system and process integrity rather than data reuse in mind, to be accessed only by a vendor specific MIS client.

Manufacturing systems data is generally generated with very little metadata by proprietary systems that are designed around a single purpose, e.g., to log data in real-time.

As any business intelligence vendor will tell you, the value of collecting such data is in the analysis of trends; identifying series of points that demand action. Yet the value of such analysis is exponentially increased by deriving relationships between trends, e.g., an interesting manufacturing trend may become a critical decision point when placed against an ERP trend. Causal relationships are what drive effective decisions - decisions which may require considering ERP data alongside manufacturing data alongside operational documentation.

This is precisely where conventional BI fails in the manufacturing environment; it's usually vendor aligned and incapable of dealing with proprietary data from multiple sources.

It's also difficult for end users to get to grips with, which means the enterprise can't leverage the expertise within the wider user base; conventional BI relies on users to be experts in the construction of queries, when their expertise is the construction of manufacturing processes.

It's end users, expert on the business process but inexpert on BI tools, who will spot these relationships and must be empowered to act.

This is curation; without meaningful metadata to make connections algorithmically, expert human filtering and nomination is the only way a community of users can be notified of a relevant trend. This is the real data that needs user collaboration, selected by a user that appreciates the nuances of the community's shared interests.

These users must have easy access to data from multiple proprietary sources; a level playing field that promotes mash-ups and comparisons. End users must be able to identify their own causal relationships and share their findings immediately with the wider community, driving quick decisions and developing knowledge that is in turn utilised in the future. There can be no reliance on IT to enable this process - it has to be in the hands of the end-users so they can act quickly.

In this way, data can be socialised; business intelligence can become social business intelligence; communities can benefit from shared expertise, expertly applied to their data.

Syndication, democratisation and curation of data within the enterprise; killer combination

Syndication

If we take syndication to mean making content available to other sites/subscribers (e.g., web syndication)  then although enterprises seem to be happy with the idea of syndicating public facing content such as press releases, articles or white papers via social media, the idea of syndicating enterprise data isn't yet mainstream. 

So the definition of data syndication would be that data is published simultaneously in a number of other channels - those channels could be outside, or inside the enterprise.

If you're inside the enterprise it makes it more likely that whatever channel is preferred by a user, the user gets the exposed to the data. Perhaps more importantly, it ensures that whatever the data source is, there's a common way to access it. By syndicating the data into multiple channels, the data can be event driven to the user, who can then take control it - it's there on demand as opposed to needing to be found. So the benefit is speed; awareness; engagement.

The case for syndicating data externally is potent. Firstly, it's transparent, so it builds partnership and trust; for instance, the near instant visibility of customer demand down the supply chain can reduce the 'bullwhip effect'. In addition, real-time situational awareness speeds up vendor response. Perhaps organisations will want to bring customers into the product/service development process even earlier than they already are. Or customers could take a complete, cradle to grave view of product quality.

Democratisation

Democratisation concerns the spread of knowledge amongst end-users as opposed to a hierarchical broadcast mechanism. Kevin Rose (@kevinrose) makes some interesting points about the sheer volume of data on the internet and how democratisation is the masses deciding what's important or relevant.

Users in the enterprise face the same challenges as users on the internet; the volume of data is huge and increasing. (This is why enterprise search is big business - HP are buying Autonomy, Google are pushing their search appliance and Microsoft are working their FAST acquisition/Bing into the enterprise where they can). The enterprise needs the same democratisation capability as the internet. Users need to be able to find and through sharing 'vote up' those key trends and issues that need attention, whether it's the trend of data from a manufacturing plant instrument or an unresolved safety audit finding.

Democratisation means the right attention can be given to the right issues at the right time.

Of course, this is a tough thing to do; the nature of data in the enterprise is different - the historical lack of enterprise-wide search means metadata is lacking or inaccurate and vendors have spent years implementing protected, proprietary, and closed systems. But with intelligent design we can overcome this.

Curation

And so to curation. Buzzword du jour. There's a definite move away from the academic definition to curation as a development of the democratisation of internet content, usually via social media. The key factor is that content is filtered and organised by a human for a community of people with common interests, rather than aggregated by an algorithm designed to respond to a single query.

This is a logical step; faced with huge quantities of available data, there's just got to be some way to make sense of it all; what's worth reading and what's worth keeping? An algorithm can't answer those questions because it doesn't appreciate the nuances of the community and those shared interests. You need a curator to take the step from the personalisation around their interests (dealt with in Web 2.0) to the relevance of content or data to the wider community built around shared interests. 

As I write this in August 2011, there are few content curation solutions out there - most are in private beta and all are aimed squarely at the internet and socialising content.

Just as it's the next logical step for the internet, it's the next logical step for the enterprise. The enterprise is full of communities, from those with a shared interest in cycling to those with a shared interest in energy and sustainability, or safety. Communities are reliable, robust and inclusive; they lead to better decisions and by their very nature engage users and ensure that the relevant data gets to interested parties. A well curated community kills a reliance on email, or any other form of serial information delivery. Developing the idea of curating syndicated data mentioned above opens the door for expert third parties to be consulted or offer services. 

Altogether now…

Simply put, I believe that democratising syndicated enterprise data through user communities is a good thing. Each enterprise and each employee that works within it can be more autonomous, more expert and better connected. It's a killer combination.

The job of your staff is to populate your to-do list

....not the other way around.

What the hell…? The job of my staff is to do the stuff I tell them to, surely?

Well, no. The problem with a CxO/manager telling the staff what to do is that the staff are closer to the customer. Or the systems. Or the problem.

Now, this isn't intended as a recipe for corporate anarchy, or some sort of democratic leadership. The CxO has a job in terms of putting in place the structures and processes which ensure that the work gets done. (cf. 5 Rules, 8 Commandments). From that point on, the CxO/manager's job is to respond to what the customer/problem facing team is telling them - or needs from them.

So, the CxO/manager having decided that they/you want, say, an agile development team where the focus is on frequent releases at a production level of quality with low risk, it's the CxO/manager's job to get in place:

The simplest tool you can find.

The simplest, clearest, fewest guidelines necessary.

The simplest, clearest, smallest organisation structure you can define.

Then, let the guys on the ground talk to the customer, do their job, work within the tools, guidelines and org structure that's been prepared for them. The CxO/manager's job is to support them.

(You don't need someone to draw really complex processes that look great and never get followed.)

(You don't need to invest in something expensive or complex.)

(You don't - and as a former Programme Manager this pains me - need MS Project.)

For instance, managing a small development team on an agile software product development project, we have:

• Tool: Defect/issue management software TRAC (an open source solution). 
• Guideline: We flex enhancements in favour of defects for a weekly release.
• Org structure:
• One person takes responsibility for the crucial production trigger points 
• Everyone has a specialism.

Our sales team approach is looser as the team's smaller:

• Tool: Salesforce.
• Guideline: Find new & interesting people to work with.
• Org structure: One person per sector.

The beauty of this approach is that it's scalable. Want to expand the sales team to talk to more sectors? Duplicate it. Again and again. Want to scale up development? Duplicate it.

Every time you duplicate you increase the value of the information the CxO/manager supplies their teams as it can be used by each duplicated unit. Therefore you increase the value of every question that's asked of the CxO/manager.

Therefore you increase the value of every item your team places on your to do list; you increase the value of your contribution to the company.

I'll let you know how we get on.

Four things I say a lot to my team and what they really mean

Those around Sabisu are used to hearing me say;

1. 'Tell me what you need'

Nothing's worse than screwing up when it could have been avoided. I hate the phrase, 'if only'.

If we're about to lose a sale because we don't have a one page summary describing how great Sabisu is for supply-chain collaboration, then it would be nuts not to ask for it. 

Equally, if a customer needs to know why we've gone with functionality A and not B, or any other difficult question, then put them straight through to someone who can help; clear the decks.

Delegate upwards so those at the sharp end can focus on what they need to do.

2. 'Show me what you've done'

I see this as pretty close to Terry Leahy shopping in his own Tesco stores, or a chef working the pass; it's all about checking the quality. It says in Rework: everything is marketing. So everything has to be high quality.

By constantly reviewing what the team is doing, you don't need to check what the team is doing as much in the future; they'll get used to the possibility of a review and up their game to meet a higher standard.

For me, this is nothing more than the 'test first' approach of, say Extreme Programming, or the 'go to production early' approach of any agile methodology extended backwards into the pre-production lifecycle. Get it in front of the customer early to prevent defects being found expensively late in the development cycle.

3. 'Good job.'

Or 'nice one'. Or 'thanks'. 

(Or 'good stuff'. Or 'rocking'. Or 'awesome'.)

Being enthusiastic. Saying thanks a lot. It's just the right thing to do.

4. 'Riiiiiight...?'

Never smack down an idea until you're absolutely sure that (i) it's been expressed fully, (ii) you understand it, and (iii) you understand its implications. 

I suppose it's about pre-judging an idea. It's easy to do - particularly when you're tired, the kids are playing up at home and the dog has developed a horrific bowel ailment or whatever, but I try to remember that the guys in the office are great at what they do and most of the time their ideas are good.

So, 'Riiiiight?' really means 'tell me more'. They get that.

Guest post for MMUCFE

Lead Us Not Into Evil: my thoughts on ethical leadership - guest blog post for MMU's leadership programme @mmucfe - http://bit.ly/e6kYbK