Xcelsius Present – Fast Track to Nowhere

Recently, Business Objects released a new product named Xcelsius Present 2008. They are promoting this new version of Xcelsius as an application that will “make it easy for non-technical users to create interactive data presentations.” This product comes with ten pre-built analytical templates; novices supposedly can just match their data to a template and they’re ready to analyze. For instance, if you need to analyze sales data, you can use the sales data template. For compensation analysis, you can use the human resources template. Perhaps you can see how this sort of cookie-cutter approach, despite its appeal, might fall short. Let’s take a look at one of these templates to see what can happen when a business intelligence software company that focuses on superficial glitz rather than analytical substance makes “one size fits all” templates.

Business Objects calls this template “Unemployment Trends.” This title is misleading, however, for trends are discerned through time, but the template only accommodates a snapshot of data from a single point in the year 2007. The employment data comes from a 53 page PDF file filled with tables of numbers, prepared by the Bureau of Labor Statistics. What’s sad is that the original PDF file, with its many tables (one per state) is actually much more useful for data analysis than the analytical application that Business Objects has built. Even though tables can be used for analysis only in limited ways, the Xcelsius application, which claims to apply powerful data visualization, actually limits what can be done. It does this primarily by reducing what can be seen to only a few pieces of information at a time. The act of comparison is the essential task of quantitative data analysis, but with this application you can compare fewer values than you could with the original tables. The benefits of data visualization have been missed entirely.

The application provides a pie chart for comparing the America’s non-institutional population to the civilian labor force and two gauges for comparing employment and unemployment rates. Neither pie charts nor gauges of this type support effective comparisons, and in this case, because of fundamental design problems, comparisons between slices of the pie or the two gauges actually mislead.

The Gauges

Gauges designed like these are never very useful. They wastefully spread their girth across an extravagant amount of space to report a single number. Because they omit quantitative scales, we can’t compare the employment or unemployment rates based on the positions of the pointer. We don’t even know if they share a common scale. They could start and end at different values—we have no way of telling. Take a moment to see if you can figure out the values that are associated with each of the tick marks. Impossible, isn’t it? We’re forced to read the values printed as text, which we could do more quickly using the original tables.

Besides these problems, there’s another that isn’t obvious unless you review the original data set. Although it seems logical to compare employment and unemployment rates, this isn’t terribly useful because the two measures have been calculated using different methods. In the original report, the employment rate was based on the number of people working compared to those who could be working, including those who aren’t looking for work, including retirees and stay-at-home parents. The unemployment rate, on the other hand, is based on the number of people who are unemployed and looking for work compared to all those who want to work (both those employed and those looking for work). If the employment rate in the above screenshot were calculated the same way as the unemployment rate, it would be 95.4% instead of 63%. While the rates shown in the gauges might be useful, the different ways that they’ve been calculated should be explained so we can compare them appropriately.

The Pie Chart

If you’re familiar with my work, you probably already know that I’m not a fan of pie charts. They require us to compare the areas or angles of slices, but visual perception supports neither well. This particular pie chart, however, fails in an even more fundamental way. I’m not referring to the distracting flag image in the background or the simulated reflection of light on the pie, which almost makes it look like there’s a third, light blue slice. Rather, the problem has to do with two slices are “Non-Institutional Population” and “Civilian Labor Force.” “Non-Institutional Population” represents everyone who could be working, whether they want a job or not, while “Civilian Labor Force” represents those people who either have a job or are actively looking for one. In other words, the “Civilian Labor Force” is a subset of the “Non-Institutional Population”, not a separate segment that combines with it to make up some whole. The 39.77% that appears when I hover my mouse over the red pie slice is meaningless; the correct percentage doesn’t appear anywhere. The following pie chart is an example of one that actually makes sense.

Filter Controls

So far, I’ve focused on problem with the graphs, but the filtering controls exhibit fundamental problems as well. They allow us to select subsets of data such as women between the ages of 55 and 64 years old. Most filtering is done through the following scrolling list box:

To select a particular set of data, we must click the up or down arrows to scroll through various categories until we find what we want. For instance, to select “Black or African American Women,” we must scroll down to the “Black or African American” section and then select “Women.” If we tried to search for the “Women” section first, we’d fail to find it, because it doesn’t have its own section. We can only make a single selection from available options that combine multiple variables (sex, ethnicity, and age groups). This means we must select from 34 filtering options. Filtering shouldn’t be this difficult or limited. Ideally, separate filters should be available for sex, ethnicity, and age group, perhaps as three groups of check boxes that could be easily turned on or off, independently from one another. For instance, to view Black and Hispanic women of all ages, we would uncheck “Men” in the Sex filter, uncheck all but “Black or African American” and “Hispanic or Latino Ethnicity” in the Ethnicity filter, and leave everything checked in the Age Group filter. Unfortunately, the original tables that the Bureau of Labor Statistics provided in the PDF file don’t support this level of flexibility, but they support much better filtering control than the scrolling list box provides.

In addition to the scrolling list box, this application also allows us to filter by state. To do this, we must go to a separate screen in the application, which looks like this:

By clicking one of the states on the map, we can see employment data for that state—at least that’s the plan. Can you find Rhode Island on the map? Let me see if I can make it easier for you. In the image below, I’ve greatly magnified the New England portion of the map.

Rhode Island is the little blue spot that the cursor is pointing at. Even at this level of magnification, it’s almost impossible to see. Imagine how hard it is to click! While the map works alright for large states like California and Texas, it’s worthless for selecting small states like Rhode Island or Delaware. Furthermore, is a map really the best way for people to select at state? It works well enough for selecting familiar states like Florida, but anyone who’s geographically challenged like I am might not be able to pick out Indiana or New Hampshire. The state names appear in tooltips as you hover your mouse around the map, but this forces us to search around for the states with unfamiliar locations. A simple alphabetized list of states would probably work much better.

There’s one more problem with the map that isn’t obvious at first. When we click on a state, it doesn’t refresh the data in the pie chart and gauges as expected. We must also make a selection in the list box filter control (for instance, switching from Men to Women) for the data to update. This is such an obvious bug in the application, it’s hard to imagine how the folks at Business Objects could have missed it. Perhaps they no longer test their software before releasing it.

Basically, this application took a series of Bureau of Labor Statistics tables and transformed them into an unwieldy mess that actually undermines our understanding of employment data.

And Wait…There’s More

Several other analytical applications are packaged with and used as demos for Xcelsius Present, including one shown below for Compensation Analysis.

Apparently compensation analysis should be performed using a single pie chart along with a table that reiterates the same values. I encourage you to visit Business Objects’ website to try out the demos of these applications for yourself. I think you’ll find it enlightening, perhaps entertaining, and very, very sad. Be sure to let them know how much you appreciate Xcelsius Present when you’re done looking.

How Should We Respond?

Business Objects is a leading business intelligence vendor (based on sales), but its products consistently demonstrate that they don’t understand analytics and haven’t a clue about data visualization. A vendor that claims to be the best, which Business Objects unabashedly claims (just like every other major BI vendor), should be ashamed of selling such moronic products. Don’t reward them for irresponsible work—products that assume their customers are halfwits—by wasting your money on them. I’m not suggesting that if you use their products, you should necessarily abandon them. I’m suggesting that that stand up and let them know that you deserve better and don’t sit down until they start listening. They dress products up with a thin veneer of flash and no substance and rely on misdirection to sell them to you.

Why? In part because, when it comes to analytics, they must not know what they’re doing, but also because they believe this is what you want. “It’s not our fault, we’re just giving them what they asked for”, they reason. It’s time to let them know that they (and many of their competitors as well) are dead wrong. 

Take care,

Read More