Monday, October 19, 2009

Petitio Principii, ”Assuming the Initial Point”, (or ”Begging the Question”)

Last time, two weeks ago, I examined what may be termed ”the cross-paradigm fallacy” when attempting to look into the future. The fallacy may be said to derive from assuming – falsely – that contemporary statistical relations remain valid in radically different circumstances. But those different circumstances need not be in the future. They can equally well have occurred in the past, and I will explore this possibility in the following.
This different, but related fallacy, is the well-known petitio principii, ”assuming the initial point”, or ”begging the question”. Let’s critically consider a well-known argument, viz. –
”Human ingenuity is without bounds; the divide that humanity in general and western civilization in particular is now facing and must soon cross with respect to energy supply is inconsequential. We have successfully spanned similar divides several times earlier on (human to animal power, on to wood/charcoal, on to coal, to oil/gas, with some hydro and nuclear power...). - We will do it again; there is positively no need for concern!”
Yes, we have succeeded several times before, but is that important? Not at all! It is irrelevant and misleading. The problem is, that if civilization had not succeeded some earlier challenge then we would not be around to consider the question. Our society would instead be found on history’s ”junkyard” of past civilizations together with those of Sumerians, Mayans, Easter Islanders, and countless others. The so-called Bayes’ theorem states it very succinctly:
P(AlB) = P(BlA)*P(A)/P(B)
- where
· P is Probability, typically expressed as a percentage, 0-100%
· ‘A’ denotes current [or future, soon-to-be] successful transformation of energy supply,
· ‘B’ denotes past successful transformation(s) of energy supply.
(Cf. Wikipedia):
· P(A) is the prior probability, or marginal probability, of A. It is "prior" in the sense that it does not take into account any information about B,
· P(AlB) is the conditional probability of A, given B. It is also called the posterior probability because it is derived from, or depends upon, the specified value of B,
· P(BlA) is the conditional probability of B given A,
· P(B) is the prior or marginal probability of B, and acts as a normalizing constant.
Now, obviously P(B) is 100% (since ”we know, we’ve made it so far”), and P(BlA) is also 100% (since ”if we make it again, then we can still be certain we made it before”). Therefore:
P(AlB) = P(A)
In so many words, the probability ”we can make it again, since we did it before” is equal to the probability that ”we can make it again” – in this case past successes are simply irrelevant to future possibility of success!

A similar line of false reasoning applies when one all too frequently hears a statement like; -
“It doesn’t matter people in third world countries have many children today. It was just like that here in the developed countries a 100 years ago. My great grandmother was out of a family of SOOooo many siblings... And we all have such ancestors – don’t we? – so it was really the same thing, - right?”
Dead wrong! But why is this anecdotal reasoning plain nonsense? Well, make some simplifying assumptions, notably that there are equally many boys and girls, and enter some simple combinatorial statistics...
1. Where a family in a third world country today averages 6 children, that means that each woman on average gives birth to 3 girls/women in the next generation,
2. Assume that in the developed world 100 years ago the pattern was [not too far off]
200 of a 1000 women had no children (0; of which 0 girls)
150 of a 1000 women had 1 child (150; of which 75 girls)
150 of a 1000 women had 2 children (300; of which 150 girls)
150 of a 1000 women had 3 children (450; of which 225 girls)
150 of a 1000 women had 4 children (600; of which 300 girls)
100 of a 1000 women had 6 children (600; of which 300 girls)
100 of a 1000 women had 9 children (900; of which 450 girls)
Now, with this distribution 1000 of each generation of women gave birth to 1500 girls/women in the next generation – just half of what has been and in many places still is prevalent in the third world today.
Also, it is easy to see why today we almost all have ancestors with many siblings: There is NO possibility that our great grandmother had ZERO children – or we wouldn’t be around to ponder the probability! - Conversely, there is (300+450)/1500*100% = 50% probability that our great grandmother was out of a family of 6, or more, children. The remainder 50% is the probability our great grandmother was out of a family of 1 to 5 children. Given the fact that we all have four great grandmothers, it is further possible to make a simple estimate that NONE of them was out of a family of at least six children:
½ * ½ * ½ * ½ * 100% = 6%
Conversely, there is (100-6)% = fully 94% probability each of us can point to a great grandmother out of a family of 6, or more, children, even though the total number of children in the developed countries at that time only amounted to half of what is still the case in many developing countries today.
Now, THAT is a real fact presented to you by simple application of basic statistics, and any denial of it is a damned lie. – Stay tuned here at morlin’s BI blog to see what follows next!

Monday, October 5, 2009

More than money and counting…

In BI monetary units and simple counts are ubiquitous. But there is much more than money and counting to a good BI system. In fact, it is dangerous and deceptive to rely only on economic indicators. To the credit of business executives in general, I have hardly encountered any who would question the wisdom of this – and there’s no problem of just paying lip service to academic concepts – they know, and they really mean it. But of course, the financial side must always be handled, and quite likely first… For me, with an engineering background for working in BI, this broader view on business is very satisfactory. There is a host of measures and units other than money that need controlling in business, and for me they are quite often the ones most “fun” to work with.
But change the focus – just a bit actually – to society as a whole, or to local bodies of government and administration. I believe responsible management entails almost the same within public administration and within business administration. Call it BI or GI (Government Intelligence!) - it is all decision support and conceptually very similar.
At top level, however, it is my impression that actually businesses has a better grasp of non-financial indicators than do most political executives. Perhaps it is because non-financial measures are typically more long-term than financial measures that many politicians seem to care less; perhaps it is because politicians tend to be more superficial and less focused in what they do. Anyway, the outcome is the same, and I am regularly worried about what seem to be closed-loop arguments between economists and politicians in purely monetary terms about our common future.
With these general considerations I wish to introduce a well-timed exception to this worrisome trend – and welcome David MacKay as new special advisor on energy issues to British Government as of 1 October. David MacKay is professor at Department of Physics at University of Cambridge, and - just as important - he is author of a splendid new book on [British] society’s options for future energy supply, “Sustainable Energy – without the hot air” - which he has also made freely available for download at http://www.withouthotair.com/.
The bad news is that there are no easy and convenient fixes. The good news is that there are coherent models available for society to choose from, which do not imply turmoil and upheaval – in fact MacKay presents five different models. Coherence is the essential feature, coherence in energy units such as PJ or kWh – not in pounds or Euros, as no one can really predict the economical regime, in which our future energy shall be produced and consumed.
Taking one step back, it is possible to describe quite generally and accurately when economists and economic planning should be side-stepped and natural scientists and engineering should be put to the fore. It should be done whenever a change of regime is investigated, where knowledge of the current economic regime is of little value, or is actually an impediment when the future regime must be envisaged.
With an example from mathematics it is a bit like the difference between normal, Euclidean geometry and non-Euclidean geometry. The rules are simply different in each case and someone ingrained with the rules of Euclidean geometry is hardly the right choice for exploring the options of non-Euclidean geometry.
A Dane and a[n] [engineering] statistician myself, but not primarily an economist, I am not happy with the actions and opinions of Björn Lomborg (“The Sceptical Environmentalist”, “Cool It”), but I am particularly concerned with his ingratiation with politicians, whose language he obviously knows well. The problem is he holds and voices strong opinions about environmental degradation, energy supply, climate change and everything else. He articulates his considerations and opinions only in economical terms and he derives his experiences and theoretical basis from an economic regime, which is hardly comparable and relevant for the economic regime of the future society likely to emerge.
...‘sorry for being fully five days late with this posting – particularly as it celebrates professor MacKay’s taking up his office with British government – but events do not always move along a straight line (my attempt at classic British understatement!) - stay tuned here at morlin’s BI blog to see and read what happens next.

Tuesday, September 15, 2009

Continuity in BI Solutions – the User Perspective Revisited

Ever so often in BI development one must go a fair distance from A to B and has to draw up a roadmap for doing so. That plan should be agreed upon between Business and IT – between users and developers. And there are several pitfalls in the process.
Here I will discuss the case of continuity as experienced from a user perspective. In some cases it is evident that continuity must be preserved when moving from the old BI solution, A, to the new BI solution, B: continuity of the collected data, historical and new, - and continuity of presentation in that calculated KPI’s remain intact with effectively identical business definitions.
In other cases disenchantment with the existing, old solution may be such that it may be tempting to opt for a “revolutionary” new solution without regard for real cross-solution continuity. From an IT perspective this may seem the more attractive and easier way to go, and Business may be persuaded to go along. Or so it may seem. According to my experience, things are rarely as simple as that.
The problems are that initially Business may not fully comprehend and understand the scope of a new BI solution, and that the Business representatives at the defining stage may not really be representative of all Business users.
As an IT person one should ask the right questions and those questions are not always just the simple and direct questions. In this case it is important to know what triggered the need to go from A to B. Are we looking at a native and immediate desire for a new BI solution, or are we really handling a spinoff project from a change of ERP system, a merger or acquisition, or a high level policy change in the company? In any of the latter cases, even if Business superficially agrees to a whole new BI solution one should be quite wary of possible complications. After all, the existing solution has been and still is supporting business needs in the short term. Any excitement about a new solution may quickly disappear when faced with the challenges of implementing it. How deeply rooted is dissatisfaction with the current solution? It may actually prove quite shallow!The solution then, I propose to be, that IT should really probe the level of user dissatisfaction with the current solution before settling for taking off with both legs to leap into the future and construct an entirely new BI solution to replace the existing one. And unless completely satisfied that the old solution will be missed be no-one, the prudent thing to do is to dissect the old solution and to document any changes, especially where features may be discontinued. It may seem tedious at times, but it actually has a number of advantages for IT too. Any such documentation and understanding is useful as part of the specification of the new solution. It also helps to clearly define and emphasize what is actually new features in the solution to be. And eventually, any difference in data presentation between the old and the new solution is just a potential error as long as it remains unexplained, - once understood and explained it becomes an improvement and an extra feature! - Last, but not least, a thorough understanding of the old solution is an immense help when constructing and implementing the new. Contrary to the leap into the future described above, it is like a long walk where one foot always remain in contact with the ground. And after all, when literally going somewhere, I – as most people – do tend to arrive faster and in better shape after a steady walk than after a series of leaps and jumps. Agree? - Stay tuned here at morlin’s BI blog for more deliberations on both technical and organisational aspects of BI!

Tuesday, September 1, 2009

A Simple Business Proposition (Motor Insurance)

Less than a week ago Kim Rud-Petersen, Fair Forsikring’s [Insurance] executive of the Business division blogged [in Danish] on Borsen.dk: Kim Rud-Petersen’s subject was the extreme conservatism and very slow - even to the point of laziness - business development within the insurance industry at large. I used to work with DW/BI for an insurance company for several years, and I strongly agree with Kim Rud-Petersen. Let me give a simple example of what I mean; a case of how insurance companies might very usefully activate some of their more or less dormant data to the immediate mutual benefit of their private motoring customers - and themselves!
If I want to trade my car, papers and data are important. With a comprehensive service history and knowledge of accident repairs, trading the car is easier and the price is usually significantly higher, see e.g. www.bilpriser.dk. Of course, some information – e.g. about serious prior damages – will depress the price, but the trend is clear-cut and important: With less uncertainty, the risk premium is reduced and a better deal is possible for both an honest buyer and an honest seller. But when you’re a private trader there’s a snag. You have really no access to solid information about repaired damages, that is privileged information for the auto trade with their proprietary data bases and professional experience. [At least in Denmark; in Sweden a comprehensive and public database is maintained. In the US a less comprehensive privately organised database exists.]
Now enter the insurance industry: Suppose I wanted to sell my car and advertised it at e.g. www.bilbasen.dk. Wouldn’t it be nice to have the option of going to the website of my auto insurer and simply release for public showing the damage and repair history of my car? I could even link directly from my advertisement to my insurance company, and thus make my car more attractive with a more comprehensive description immediately available to prospective buyers. Perhaps I might even be willing to pay some small amount for the privilege of attracting viewers and generating traffic to my insurance company’s website!
But that is only one side of the coin. Try taking a look from the side of the insurance company. As it is now, the insurer hears nothing before the car is sold and the business of insuring it is discontinued. With the described feature, the insurance company gets early warning when a customer releases damage and repair history of his car for public viewing. The company is then able to proactively take steps towards protecting the business; ideally business may actually be increased – doubled - when the company is able to hold on to both the owner/seller (with a new car) and the traded car (with a new owner)!!
This was only an appetizer. Rest assured that serious money is involved overall, making it worthwhile for a progressive and competent insurance company to establish such a feature. It is not even particularly difficult to develop and calculate the idea as a business case. I once did it over 10 pages, but that is a story too long for a BI blog. I am personally convinced the idea will be realised; only question is when and by whom. It would have to be a company genuinely concerned with helping and empowering its private customers, even at the risk of annoying somewhat the auto trade by slightly infringing upon their perceived privileges. I hope some time to be able to tell that story. Stay tuned here at morlin’s BI blog for any possible developments.

Saturday, August 15, 2009

Pitfalls of CRM – the Case of Honda (Cars)

This weekend I found myself at the thin – receiving – end of a CRM customer satisfaction programme for Honda Motor Co. Now, that was not exactly surprising since I acquired a new ‘Honda Civic 1.8 Sport I-Shift’ some two months ago. On such an occasion, however, as a BI consultant I cannot help taking a professional stance and consider whether the questionnaire adequately measures what [I think] it should be designed to measure.

First, let me sketch the situation. My wife’s car suddenly quit in mid-May, and then she wanted to take over my VW Golf Tiptronic 6-12 months before planned and anticipated. For that to be possible, I would need to find another car at short notice. Now, being led by events and acting reactively rather than proactively, I did not want to commit myself for a very long time to some new car, I could not be sure would be just right for me. We like to buy new, or virtually new, cars and keep them for 10+ years, plus I enjoy a youngtimer 1995 Mercedes E220 (W124C) cabrio for sunshine driving. As it happened, the problem solved itself as quickly as it had appeared. I really, really like the design of the current Honda Civic, and just at the time Honda Denmark advertised a batch of leftover 2008 Hondas available for two-year leasing [hire-purchase] at very reasonable cost, manual and I-Shift automatics even being offered at the same price. In an MC accident in 1994, I lost almost half my left leg and now cannot operate a normal [in Europe] clutch for a stick shift. To lease a Civic I-Shift was a solution which was at the same time cautious and affordable and experimental and fashionable, considering how I-Shift can [optionally] be operated by hand from the steering wheel, and how car leasing is being discussed in [Danish] media as both modern and – in the right circumstances – economical. Furthermore, it is so fortunate and practical that although Honda dealerships are quite sparse, one small Honda dealer is situated just a little over 1 km from where we live. All in all, I had to go elsewhere to locate and try out a Civic I-Shift, but having done that, the acquisition of the car was really a formality associated with only minimal contact with the actual local dealer.

Now for the subject matter: How did I experience Honda’s CRM questionnaire?
On one hand, the questionnaire was brief, just 4 pages not very densely printed. It could easily be completed within 5 minutes. Credits to Honda for that! Working professionally with CRM and statistics, I also tend to oblige and answer more questionnaires than an average citizen, I think. Still, it is far from rare for me to decline, or simply break off, not being polite at any cost. I do that whenever I feel some company is transgressing either what is their reasonable interest in the subject matter, or my patience. And by experience, what is more often the critical point is clearly my patience rather than my privacy!
On the other hand, questionnaire focus was obviously the dealership rather than the car. The local Honda dealer being small and anything but sophisticated, there were few points I could reasonably award it. Now I feel just a little bad and sad about the whole thing. Perhaps my situation is a little special, but the questionnaire did nothing to disclose that. I required little, expected little, and got little from the dealer in connection with the deal.
But what I needed, I got, including a local point-of-service in the time to come. And location and distance was not even the subject of a CRM question, I had to make a separate note of it! – I just hope, Honda’s questionnaire is not “designed with a purpose” to provide a rationale for consolidating dealerships into a select few distant “temples of motoring” the way certain other makes have already done.

And the Civic is great, by the way. It is no Super car, of course, but a superb value for money proposition with a highly distinctive design plus some interesting features. I just hope, probably in vain, that when another Civic is released in a couple of years, it will not be a totally different design but retain a certain familiarity with its predecessor, as is usual by European car manufacturers.

Wednesday, August 5, 2009

Applied Microsoft Analysis Services 2005, by Teo Lachev

There are BI books - and then there are BI Books. In this posting I will name one from the latter category: ‘Applied Microsoft Analysis Services 2005, and the Microsoft Business Intelligence Platform’, by Teo Lachev. It is no easy read, and it is hardly possible to read cover to cover, - but it is the type of book which provides background and understanding rather than “only” quick fixes to immediate problems. It is the type of book one can always delve into and find a section to broaden ones understanding of some topic. Let me give just two examples:
(1) I used to wonder, now and then, whether plain dimensions or parent-child dimensions should be preferred when both was an option. I could not really get it right, since different cases flipped the coin to either side and neither choice was always quite satisfactory. Given the desire for conformity and parsimony in model formulation, it did not occur to me that both might be required for an optimal solution. Then I chanced upon part of a sentence in TL’s book (p.77) explaining that a [single] Parent-Child dimension may not be enough for every need. If that is fine with TL then it is fine with me! Now I no longer worry about that question. If I can satisfy myself that different needs are best served with different – but not duplicate – views of the same dimension, then I do not hesitate long to produce them.
(2) As a newcomer to MDX, I stumbled upon a problem where results consistently appeared to be twice what I knew them to be. Somehow I solved the problem without really understanding what had happened. My various MDX sources at the time did not seem to provide an explanation. Only later did I read TL (p.309) detailing a case where [Reseller].[Reseller].Members is twice that of [Reseller].[Reseller].[Reseller].Members, because the former includes the [All] member for a doubling of the actual value! - Quite simple really, once you think of it, but nowhere else have I found such an explanation of the most likely reason for spurious double-counting in MDX.
I’m sorry for a few days delay with this posting. Stay tuned here at morlin’s BI blog for the August mid-month posting!

Sunday, August 2, 2009

What’s in a Word – the Catch ?!

If you try to google the phrase “Putting the I back in IT” you’ll get quite a few hits. In Denmark it has been employed extensively by major BI consultancy firm Platon A/S, and in 2005 they applied to have it as a trademark. Today, however, I do not see it on Platon’s homepage.
How can that be? Well, the I means, of course, Info[rmation], and that definitely should be part of IT. If it went missing, it would certainly need to be put back! But is that really where we want mission critical information to be? No, it is not! And should it be the goal for IT and BI to put it there? In all likelihood not!
Mission critical information derives from Business and should be collected and kept by IT, but that is not all. It should also be managed, made available and returned to Business as it is needed again later.
Can that mission for IT and BI be summed up as succinctly in a catchy phrase? I think it can! Try:
“IT - Serving the I back to Business to a T”!
(The phrase is absolutely open source, un-patented and un-trademarked. See also www.designedtoaT.com/.)
Stay tuned here at morlin’s BI blog for a longer and more technical posting tomorrow!