Somebody once asked Mahatma Gandhi what he thought of western democracy, and he said he thought it sounded like a good idea. We can probably say the same thing about Business Intelligence. Business Intelligence. Have you heard about it? Probably not just once, but too many times. Search engines will give you tens of millions of hits. It's bounced off the walls at countless board meetings in the past few years.

Why?

Executives are still struggling with the frustration of trying to harness the vast and expensive power of information technology for real help with strategic and tactical planning and decision-making.

"Business Intelligence" (BI) is an expression with a long pedigree in the world of business information technology. Like anything with so long a lifetime, it's had its ups and downs.

The term can be traced back to October of 1958 (no, that's not a misprint), when H.P. Luhn published an article in the IBM Journal describing what he called a "business intelligence system". In his struggle to define this concept, he defined "intelligence" as "the ability to apprehend the interrelationships of presented facts in such a way as to guide action toward a desired goal".

The idea of his system was to automate the process of abstracting (summarizing) the content of documents and then routing them to the functions or individuals in an organization who might find them interesting.

Much of his article takes up the now quaint issues of obtaining and storing images of documents. He talks a lot about translating microfilm (remember that?) to magnetic tape (remember that too?).

When it gets to the question of interpreting, summarizing, abstracting the document, he gets a bit vague. And why not? That ventures into territory we still don't have a handle on: how is a computer program supposed to figure out the meaning of the document? Obviously, it can't be routed to the right place without some understanding of what its actual semantic content – its meaning – is. Think about military intelligence today: it's supported by the most sophisticated technologies imaginable: GPS, satellite surveillance, CCTV, and listening devices that give you the creeps. Why is it so often faulty? Because of the lack of so-called HUMINT – Human Intelligence – feet on the ground and human brains to put together disparate bits of information into a meaningful form.

Since then, Business Intelligence has sometimes been the hot buzzword, overshadowed for a long period by "Decision Support Systems".

This often amounted to little more than a change of packaging of existing solutions to meet what the marketplace had been told it ought to be interested in. Every major vendor has developed or acquired a Business Intelligence "solution", but market satisfaction is still all too thin. Vendors show off their latest technology improvements in information handling – data warehouses, and clever approaches of adding keywords to documents to help the software to recognize content. Yet business users may find that behind all that technology, there are still some critical missing pieces to complete the puzzle.

What's missing? We think its HUMINT, or its software equivalent. Imagine a decision support system that tells the user "I know what kind of decision you're trying to make, and here's something I found that you might not have thought of". Now we're talking real business intelligence. In short, what the next generation of Business Inteligence has to do is make a quantum leap to systems that understand, learn from experience and act as real partners to the decision maker.

Are we any closer to it? The answer is a definite and unequivocal "maybe".

In recent research (particularly in Japan), some significant breakthroughs have been made in software for semantic analysis – that's the kind of software that understands what it's reading or hearing. This has been made possible through advances in computing technology, more sophisticated development, and, above all, advances in the linguistic science that underpins our understanding of how people acquire, use and extend language. One important breakthrough is in the area of using context to determine the sense of a word or phrase that is ambiguous. This alone is (dare I say it?) a great leap forward.

We're still probably some years from integration in real business intelligence systems. Even if the method were flawless, the real hurdle will be finding a way to integrate it functionally with the traditional data-mining and keyword-driven text-parsing tools that are in use today. It's a methodology that's lacking.

The application of these tools that really understand what it's all about will make business less dependent on issues like data-modeling which is endless, costly and never quite gets it right.

Suddenly we're looking at the missing piece from H.P. Luhn's puzzle: the component that can figure out, based on meaning, what information should go to whom. Hopefully we won't have to wait as long as we have waited since Luhn's 50-year old idea.

Business Intelligence: From Buzzword to Breakthrough? – by Jay Fogelman Academic Director of IT Management and Tibor Vörös Senior Lecturer of IT Management

November, 2008