Douglas Merrill is spot on in his assessment of the “button effect” with data analysis; the “button effect” being the instance when someone becomes so happy that they can “push a button” and receive an answer to a business questions, that they do not question whether the answer is real or accurate. I’m a big believer in the rise of easy-to-use software and the impacts that software-as-a-service (SaaS) can have on data analysis. There are a number of operational functions within businesses that are under-served by IT departments when it comes to business intelligence (BI) and data analysis projects. These business functions can benefit from easy-to-use data analysis that can help these business managers make smarter decisions. Still, having said this, Merrill’s cautions are important.
Data in general, and business transactions in particular, need to be analyzed across several dimensions. In analyzing travel and expense transactions, as an example, it is important to analyze not only the purchases made by travelers, but also the day of the week the purchase was made, the expense type that was used to classify the purchase, the frequency of purchases like this made by this traveler, the frequency of purchases like this made by all travelers, the frequency of expense classification by all travelers, and other information that can help determine whether the purchase was compliant with policy. In the absence of maximum context, the likelihood of both false positives (identified exceptions to policy that are actually compliant) and false negatives (non-compliant transactions that go unnoticed) is high. The “button effect” of false positives is that results are quickly ignored. The “button effect” of false negatives is that the results will be ignored once evidence of false negatives becomes known. In either case, lack of appropriate context will result in the analysis tools going unused.
Analysis not only needs to be accurate, it also needs to be actionable. Results that don’t prompt action are merely interesting news. As an example, last week I saw a graphic showing the highest paid coaches in college football. The link paid homage to the Southeastern Conference and had a beautiful bar chart showing the relative earnings of each SEC coach. Nick Saban of the University of Alabama was at the top of the list at $6.9 million per year followed by Kevin Sumlin of Texas A&M at $5 million per year and Les Miles of Louisiana State University at $4.3 million per year. The average annual salary for the SEC was listed at $3.65 million and included other commentary including “Saban far out earns his peers”. You can also move to other charts that show the salaries for the top 20 coaches. While these charts are interesting, they’re not actionable.
Also last week, The Wall Street Journal looked at college football coaches and how they fare against the Top-25 teams. By leveraging the concepts of applying context and making actionable results, one can take the salary information from the first chart, plus the record against Top-25 competition, and number of national championship wins to find the “valuation” of each well-paid SEC coach. Applied across the top three SEC coaches: Saban is 28 wins and 12 losses versus Top-25 teams with four national championships, Sumlin is 5-6 (wins to losses) versus the Top-25 with no championships, and Les Miles is 37-19 versus the Top-25 with one national championship.
By adding in additional information, it is clear why the University of Alabama values Saban at $6.9 million per year. If we were to evaluate the financial impact of wins and championships on the university budget, we would have an even clearer view of a coach’s “valuation” and potentially make a case for the coach as under-valued. Using our context valuation concept it is clear that Texas A&M values Sumlin for future results since the past results are not there. And it makes Miles look like a good value compared to Sumlin. *
The bottom line is that data analysis needs to be well constructed for delivering useful information that can result in action. BI tools often require exploring and manual correlation of results. Incomplete data analysis can lead to false or merely interesting conclusions. The bottom line is to beware the “button effect” and enjoy the ease of use these new data analysis software platforms provide, but never forget that what you do with the analysis is up to you.
*I will also add that my favorite college coach, David Shaw, is 14-4 versus the Top-25 and makes $3 million per year with no national championships, but two straight PAC-12 championships and three straight BCS bowl appearances including two straight Rose Bowls (Fear the Trees!).
