Mainfram Reality

During my research on various management styles and techniques, it became apparent to me that entire management structure is not based on what is right or wrong, but opinions.  I’ve written about this before.  But, one thing bothered me about this, who’s opinion, and if such opinion is wrong, why do masses of people follow those bad decisions.  Practically, I see this all through my career and life.  What became apparent to me is that it is all a question of authority.  As long as someone with authority is willing to take the responsibility for a bad decision or action, most people will follow and perpetuate in the bad decision.

Stanley Milgram, performed the famous Milgram Experiment at Stanford University during the 60s that has shown how authority can influence people to perform actions outside their moral comfort zone.  Same thing applies to business and life within our culture  in general.

Notice that I do not believe this is human nature, but a behaviour our culture instills in us.  Something in our culture teaches us to respond to authority the way we do, maybe because for generations we have had authority and no true freedom.

One of the main reasons I’ve started thinking about the issue of authority is because I’ve been reading about risk management.  Risk management is something I am not very familiar with, but is one of those subjects that should be ridden with common sense and logic.  Risk management however is not.  One of the simple, most popular ways to manage risk is to employ “Risk Transference”.  Risk must be managed, as long as we push the risk to someone else, someone of higher authority then us, then the risk as far as we are concerned has been mitigated.  This struck me as universally stupid as risk has not been mitigated but transfered to another point of authority, so what is the point.

I’ve watched a documentary on Enron and the famous collapse.  What happened at Enron is a text book case of doing what authority figure asks, Milgram’s theory in action.

So to bring this closer to home, I’ve read about Milgram’s experiment years ago and always thought I would tell the researcher to f*** off.  However, now I see I am no different.  If I raise issues during my job, I am quite happy to except someone of higher authority hearing me out and then instructing me to proceed with the wrong action.  Is it because I am getting paid? Is money the issue? I do not know, but I know I do not like it, I don’t like that I do not have the freedom to say no.  I wonder what and how I would eat if I just did what I thought was right?

Our culture truly is sick and primitive.  Every day I become more determined to help the Venus Project.  Practical way to help alludes me, however I do feel a solution is close, I just need to persevere.

Ok, so this is a post I’ve been writing for a few weeks now and just could not get out.  It is one of those rant posts and I’ve decided to have less of those on my blog, but this rant needed to see the light of day.What made me finish these ideas was a post I saw of MSDN BI blog:
http://blogs.msdn.com/bi/archive/2009/03/22/history-of-business-intelligence.aspx
Now everything the blog entry (well video) talks about is correct. Currently BI is about people, about making smarter decisions, better accessibility of data, etc, etc.. So what about it is redundant stupidity?  It has to do with two ideas: root cause analysis (wikipedia) and scientific method (wikipedia).

Our businesses generally run on people’s opinions and not facts, this is evident for example in the financial crisis we have.  Business intelligence of today is designed to make people make decisions based on some magical data and their opinions about the data.  What is missing is a proper method of analyzing and interpreting the data.  Scientific method is probably the best tool we have to solve the problem of data analysis.  People propose a hypothesis, design a series of tests to validate the ideas and then perform the tests to verify the validity of the hypothesis.  If the tests fail a new hypothesis is created, new tests are designed until enough tests support the hypothesis so that decisions can be made using the hypothesis in question. There are two important parts to the process, one is that failure is a beneficial part of the process where unworkable hypotheses are removed. The second is continuous improvement of knowledge as new tests are always designed to constantly challenge the validity of the hypothesis.  What this means is that decisions made based on scientific method are made based on real test facts and not opinions. Business Intelligence tools of today offer nothing more then a pretty graph representing some data that is interpreted by people however they please.  No rigorous testing is created to validate the assumptions and no one in the business community would tell you that failure is acceptable and welcomed. I for one am glad that businesses employing business intelligence are commanded by super humans that never make mistakes.

Now this brings us to root cause analysis.  I’ve seen many businesses that have business intelligence tools coming out of their ears still fixing problems that are completely irrelevant simply because these are not the root causes of the problem.  There is no point creating a beautiful graph representing the crappiest, most complicated data.   Most systems that are great are actually simple. The world does not need more business intelligence tools, we need simpler systems to solve our problems.  We don’t need multiple data entry points, we need authoritative data stores.  We can have copies of data around the place for convenience or performance reasons, but why do we need to collect product description data from one system and account data from another that do not talk to each other at all other than through some business intelligence tool?  In a sane environment both the product system and account system should be fully aware of each other and provide user feedback in real time based on criteria (tests) configured.  There is no point for someone to make a bad product decision and find out about it a year later on some report. It must be real time, it must be part of the process, test, test, test, everything. It seems to me that we have given up on making sure our data entry is understood and coherent.  So, the root causes are: people making decisions based on opinions and not facts and data being stored in such a bad way we need an entire industry for business intelligence in order to make any sense of it.

Someone once said, “Automate an ineffective process and you make it officiently ineffective”.  That is exactly what business intelligence tools do. They are complicated and solve no root causes and encourage people to make decisions based on opinions and not facts.  We need data collection strategies that use standards, we need clear scientific method of understanding the data where it is not interpreted but tested, and we need to look at root causes and find solutions for those, not an endless parade of bandages used to mask away our problems.

Business Intelligence is 95% marketing, 3% pretty graphs, 2% effective.