Tuesday 15 January 2013

Is the end in sight for bloated MI/BI systems?

There are 2 reasons you do not need a big, monolithic, singular, global datawarehouse.

1. Distributed processing

The techniques and architectures exist to move the aggregation process close to the data - in fact, right on top of it. This makes it quick, efficient and easy to modify without significant overhead - unlike cube -dependent data-warehouses.

Hybrid-cloud architectures make it possible to aggregate on a local basis while making the aggregates available over the cloud to the global enterprise, which is then dealing with relatively small datasets.

So there's no need to spend a fortune on the communications and storage involved with pulling data to a central location, and no need to take the risk of having all your data centralised onto a single point of failure.

So no need for a big datawarehouse.

2. Cloud power

At Sabisu we do much work in the process industry where the perennial question is: do we connect our essential production systems to the cloud?

Sure, you can take advantage of the virtualisation and outsourcing available for risk mitigation and cost reduction, but in fact you're just shifting the risk to the communications provider and you're unlikely to find multi-tenant cost benefits because you're going to want a very private cloud indeed for all your valuable process data.

Cloud computing is valuable to our customers because it gives unlimited, immediately available processing power. This means that all those clever data network modelling techniques that have been the preserve of those with entire datacentres at their disposal are now accessible by anyone with a bit of budget.

So what we have now is an opportunity to try new analysis techniques that do not need a local, on-premise, expensive data-warehouse. All you need is enough communications capability to get the dataset you want to analyse to the cloud, or as described in (1) above, get the right level of aggregate to the cloud.

In fact, you don't need to persist any data in the cloud; you can reconstruct the set of results later if required by supplying the raw/aggregated data.

So, distributed processing and cloud power; an antidote for bloated MIS perhaps?


No comments: