I had lunch with a technology consultant the other day. This fellow rakes in money by building information dashboards for executives. “The hardest cases are when the client doesn’t know what he wants the dashboard for. He just knows he has to have one,” he tells me in-between mouthfuls of pasta, “Then everybody in the organisation gets involved to figure out what to track on the thing.” I reply, “I bet that’s a long, drawn-out process with lots of back-tracking, especially if you can’t get all of the decision-makers in the room at the same time.” He rolls his eyes, “Oh, yah. For sure.” I sense he’s having some amusing flashbacks, then he adds, “We have a joke back at the office. We thank god for cocktail parties. Every executive wants an information dashboard because he’s gone to some cocktail party and met another executive who’s got one.” He smiles a big devilish smile. The smile tells me that there’s more than a grain of truth to the joke.

An information dashboard is a single screen of continually updated facts and figures that’s designed to help someone monitor an organisation, process, or market. The purpose is to alert a decision-maker to important developments so that she can intervene appropriately. In some cases, the purpose is to let a decision-maker know that nothing alarming is going on—no fires need to be put out—so that she can carry on with other duties without any nagging anxiety. Unfortunately, compared to the dashboard on a car or other machine, the information dashboard for an organisation is not so straightforward. It’s not clear what needs to be monitored. And if those things can actually be measured accurately. And if decision-making can be reduced to something akin to playing a video-game.

The last time I wrote about the topic, I reviewed Stephen Few’s book Information Dashboard Design (2006), which you can find here. I’d like to build upon that review because I’ve just taken Few’s workshop and have more to say about his advice. And a lot has happened in the information-dashboard business during the intervening years.

Dozens of dashboard companies have emerged, only to be swallowed up by one of the big enterprise-software firms, such as Oracle and IBM. These big firms have spent lots of money trying to get the acquired dashboard software to work seamlessly with their own products (with varying degrees of success). They’ve also spent money selling info-tech managers on the idea of dashboards, adding lots of bells and whistles to differentiate their software from competitors’ offerings. And so dashboards are getting more elaborate but not necessarily more useful. If anything, the upgrades are making matters worse because the software hasn’t become flexible, intuitive, and tasteful. It’s only getting more “feature rich”. That means software engineers have added lots of complexity and useless gimmicks. Many of these gimmicks have a superficial appeal for those shopping around for new software. And many have an appeal only for software engineers.

“The problem for these companies is, when you get down to the fundamentals, all of their dashboards do basically the same thing. Dashboards have become a commodity.” That’s my lunch companion again, who adds, “The real hard part isn’t in the front-end. It’s in the plumbing. Its hard to get all of the right data to flow into the dashboard in the right form and get it to continually update. Nothing just works out of the box.” The big enterprise-software companies are happy about that. They make more money selling services and tailored infrastructure to organisations. That’s how the technology consultant makes his money. I ask, “But what about design? Isn’t good design just as important?” “Nah, you can’t charge clients for design work. They have a hard time seeing the value in that. They don’t know what a good design looks like.”

Thus, just as many executives don’t know what they want dashboards for exactly, many don’t know what a good dashboard even looks like. They rely on the software vendors to tell them. And the vendors are peddling gimmicky bells and whistles; they’re peddling razzmatazz. The result is predictable: lots of initial buzz but, ultimately, dashboards disappoint and fall into disuse. Either that or another round of dashboard development happens. The technology consultant also rakes in money by replacing botched dashboards. Business is brisk.

I’ve met a lot of people implementing dashboards, mostly the variant known as the executive dashboard. Much can be learned from their travails; which is to say, their failures.

• First, many executives want dashboards for the wrong reasons. Or they have only a vague idea about how a dashboard can be useful. Thus, the developer and the executive (with entourage) have to figure out what purpose can be served. The challenge is that executives have complicated jobs full of unstandardised tasks. And many organisations have operations that can’t be easily measured, especially in the public sector where I ply my trade. Thus, only a fraction of the information an executive wants can be shoehorned into a dashboard. That information tends to be the easiest to measure, not the most important. I call this the scavenger-hunt problem: the boss wants “hard” numbers showing “key” results, and so the underlings gather a hodgepodge of whatever measures are at hand (or are easy to cook up). Quality-control guru W. Edwards Deming argued that organisations should only measure the three percent that matters. Most dashboards end up measuring the one percent that’s handy. Smart executives don’t find that very enlightening.
• Dashboard developers never give any thought to the workflows of executives. Executive dashboards are usually designed to be viewed while sitting at a desk behind a computer. That’s fine for most office workers. But executives, especially senior-ranking ones, don’t spend that much time fiddling with desktop computers. Some actually have their dashboards printed out on paper and handed to them at meetings. That’s changing slowly. But even tech-savvy, laptop-toting executives have little patience for fiddly dashboards. Most importantly, once an executive sees something of interest on the dashboard, it’s usually something she can’t deal with right away even if it’s urgent. I call this the parking problem, as in “park that thought”. Executives with busy schedules can’t use the dashboard information for the task at hand and so they set it aside for later use. A quick Blackberry message gets sent. Meetings and phone calls are scheduled for some future date. But the executive often can’t explore the finding right away to get more information. So it gets forgotten. Or, by the time a meeting or call takes place, the finding is stale. Or the finding wasn’t such a big deal in the first place. In any case, there isn’t much opportunity for serious elaboration at the point of discovery. Executives prefer workflows that place the elaboration very close to the point of discovery, which is why they prefer to learn about new developments in meetings and business lunches. They can ask probative questions … ask for clarification and elaboration … ask for advice. Dashboards don’t provide the interaction they’re used to and don’t complement the way they work. Mobile technology might change that but executives tend to be tech-laggards, not early adopters.
• Many dashboards monitor information that gets updated very infrequently. Sometimes the data is annual. In such cases, the dashboard information isn’t really used for active, on-going monitoring. It’s used for an occasional checkup. The result is the report problem: the slow updating of data turns the dashboard into just another report. And it’s not always clear to the executive why this particular report deserves so much prominence compared to all of the other reports that land on her desk. Moreover, sometimes the executive learns about the information from other reports first if the dashboard isn’t updated immediately. Once an executive learns that a piece of information on the dashboard isn’t the freshest information available, they distrust the timeliness of the dashboard. Timeliness is usually the dashboard’s raison d’être.
• Dashboards are instruments of scrutiny. Where there’s scrutiny, there’s political manoeuvring by the people being scrutinised. That creates the office-politics problem. Dashboard developers aren’t senior-ranking people with power. Many are outside consultants. A sphere of autonomy doesn’t exist, as you’d (ideally) get with an internal auditor. Consequently, the choice of measures becomes the subject of political negotiations. That’s a big problem if the intention is to use the dashboard to reveal thorny problems yet only the “safest” (most sanitised or reliably favourable) information is presented. Politics is also a big problem when those with a stake in the information find devious ways to kill (or break) the dashboard development process. Given how long it takes to build a dashboard, stakeholders have plenty of time to moan and scheme.
• Finally, dashboards tend to be designed poorly. Developers may have lots of technical skills but information design is usually not one of them. The software vendors instil bad practices. Thus, even if the information is on-topic, timely, and of good quality, the dashboards are garish and overly ornate screens that report the information in a meaningless way. That’s the information-design problem that I will talk about here.

Sadly, I know of several dashboard project-managers who stepped on all of these landmines, not just one or two of them. When the dust settles, there isn’t anything lasting to show for all of the effort except, perhaps, the big crater where the project manager’s office used to be.

Information Dashboard Design Course, Visual Business Intelligence Workshop, by Stephen Few, San Francisco, September 28-30, 2010.

Stephen Few has given the most thought to the information-design problem. Office software (such as Microsoft Excel) as made it easy for people to create tables and graphs. But most people don’t know that there are rules for such things, just as there are rules of grammar for writing prose. Moreover, writing is a joy to read when it is crafted with style and purpose, as well as designed visually to make the reading experience more engaging (by using good typography and document layout, for example). So too with tables and graphs. There are things you can do —“affordances” you can add, distractions you can subtract—to make tables and graphs more purposeful and easier to use. Unfortunately, the software doesn’t encourage good usage. Indeed, the various pre-sets and templates encourage bad habits; habits which, sadly, we’ve grown accustomed to. Thus, Few wrote a book called Show Me The Numbers (2004) to help remedy the situation.

When information dashboards arrived on the scene, much of the content came in the form of tables and graphs. Many bad practices were transplanted from office documents. What’s worse, dashboard developers incorporated all sorts of visual elements from other types of monitoring equipment. I’m talking about imitation gauges, dials, warning lights, and so forth. Some of these elements might look cool but they’re imprecise and difficult to interpret. Many are completely obsolete. Developers also use too many different kinds in one display, resulting in meaningless variety that makes data comparison extremely difficult.

Mr Few tells the story of a software developer who gave users the option of adding imitation gauges to their dashboards. The gauge was animated. When the values changed, the pointer on the gauge would act like an old-fashioned, spring-loaded gauge: the pointer would wobble back-and-forth until it settled on the right value.

The software developer was impressed with his achievement. He thought it was a great feature. What he didn’t realise is the wobble of the pointer on a real gauge is a defect. The pointer wobbles because the underlying mechanism was designed poorly. As Few explains, the developer thought it was a feature because it’s a difficult thing to pull-off technically with software. As with all software, just because you can add a feature, it doesn’t mean that you should. And some features are useless and distracting. Most importantly, a gauge is a poor choice for dashboards because it takes too much space, isn’t conducive to quick comparisons, and is very imprecise.

Mr Few wrote Information Dashboard Design (2006) to help correct these bad practices. I’m not going to rehash my review of that book. I’ll only point out that, according to Mr Few, certain visual elements are suitable for information dashboards and certain elements are not. The suitable elements tend to be space-efficient, precise, easy to compare, and elegant in their simplicity. They also allow meaningful context to be presented. Unsuitable visual elements don’t meet those criteria.

That list hasn’t changed much since the book was published. Mr Few is just as resolute in his opposition to circular charts (such as pie and bubble charts) because humans don’t have the cognitive capacity to compare the areas of circles with any accuracy. Maps and scatterplots that require lots of analysis are discouraged. And there are some cases when a picture, such as a photograph, can prove useful. Here’s an example that’s similar to the one Few presents in class.

Here, the photographs are a way of grouping the data. The bullet graphs and sparklines to the right of each photo relate to that particular person. There’s no need to draw borders nor colour the backgrounds in order to differentiate each set of data. Using photos in this way is also potentially useful because of our ability to easily recognise and remember faces, as well as use images to associate information with a particular person. This usage, however, does introduce a bias when interpreting. We’re more likely to judge attractive people differently than unattractive people, a phenomenon called lookism or physical attractiveness phenomenon (which I’ve discussed at length here). Employee-ID mugshots usually don’t help the non-photogenic either. As with many visual methods, there are tradeoffs to consider.

The example above is for monitoring salespeople. There isn’t any strategic information being shown, which is what most executives expect of their dashboards. But the example strikes me as one of the more useful applications of a dashboard. A particular operation is being monitored. Notice the order of the salespeople. Many of the bad cases Few showed in class were arranged in alphabetical order, which is useless for anything but looking things up as a reference. Here, the salespeople are ranked according to their sales. A simple icon (a red circle) highlights an alarm-worthy finding. The contrast caught your eye without being overwhelming. There’s plenty of contextual data that tells a deeper story. The top salesperson is usually an under-performer, whereas the bottom one is a great performer having a rough spell. Both findings might be temporary blips. But without this context, a supervisor might draw the wrong conclusions. That’s the danger of a purposeless data-dump.

As an aside, Mr Few is quite insistent that the dashboard be a single screen of information without scroll bars. That enables direct comparisons. Indirect comparisons are undermined by the limits of working memory, which can only hold three pieces of information at a time. Thus, multiple and scrolling screens shouldn’t be used, so the logic goes. Here, however, it seems perfectly alright to having a scrolling dashboard so that the viewer can look over a large salesforce. Indeed, the dashboard I use on a regular basis operates like this and the scrolling doesn’t detract from my ability to monitor what I need to. This may be the only option for this example of using photographs to group data, otherwise the dashboard could only monitor a few people. Another option might be to add interactivity, something which Few recommends.

This example allows easy comparison of the most important indicator along a single column, with contextual details appearing when the mouse-cursor hovers over the data.

The slogan that Few uses to sum up his design approach is: "eloquence of communication through simplicity of design." A pixel on the screen should be blank unless it's communicating data or guiding the eye purposefully. Many dashboard designs are needlessly ornate and visually cluttered. As Few puts it, "If everything shouts, nothing gets heard."

There is an austerity to his preferred design. Objects on the screen should be crisp and clean. Command buttons shouldn't be used except, perhaps, in a de-emphasised corner of the screen (such as for a help screen). If elaboration is required, the data itself can be made interactive, with information popping up when the cursor hovers over an item or a user clicks an item. That reduces clutter. Blacks and greys are the prominent colours, with colours used selectively to highlight or differentiate. Given that ten percent of men and one percent of women are colour-blind, certain colours aren't juxtaposed (such as red and green of the same level of saturation). Yet, "anyone can distinguish between shades of grey." If vibrant colours dominate the screen, Few explains, they are difficult to look at for long periods and raise the viewer's stress levels, which is precisely what you don't want from a display used for careful decision-making. Thus, desaturated (pale, less intense) colours are used unless you're trying to highlight something.

There are a number of examples of bad design in this whitepaper (PDF), plus an example of Few’s preferred style.

Some students in the workshop weren’t comfortable with how simple Few’s preferred dashboards look. Some worry that colour-spare dashboards look a bit boring. One student argued that his boss wouldn’t accept a design that looked so drab. Mr Few had some insightful come-backs. First, he claimed that there isn’t necessarily a tension between having a functional dashboard and a visual pleasing one. But a different type of beauty is required, the beauty of elegant and minimalist consumer products. I would add that, in a culture dominated by colourful commercial messaging and art, a simple and elegant look is precisely what stands-out. The problem with plain office documents is that they are usually hacked together and not designed properly. Second, Few notes that designers who know a lot about colour tend to use colour moderately. They know how to create visual appeal without resorting to bombastic colour pallets and they use that appeal to great affect. Third, developers shouldn’t just present a design to the boss and ask, “Do you like it?” Instead, they should get the person to use the dashboard in order to see the logic and experience the functionality that comes with careful colour choices. Ornate designs might impress on first inspection, but will often fall into disuse because flashiness can get tiring very quickly and the distraction gets frustrating.

Then there’s the design of the data itself. Mr Few reminds us that the most important data should be emphasized. Sophisticated calculations may be required. The contextual data necessary for meaningful interpretation should be present. And the arrangement of data should facilitate useful comparisons. As Few suggests, the dashboard designer should be asking herself certain questions: How can the data be expressed in performance terms? What is the goal? Is the dashboard clear about what is a good and a bad state of affairs? What context is required for the user to make good judgements? For example, is it necessary to place a time-series of historical data alongside a number in order to understand the implications? It’s amazing how often dashboards are constructed in a way that simply dumps a miscellany of data into an executive’s lap and expects that to be useful.

I stress some distinctions along those lines when advising. Differentiate between (a.) nice-to-know and need-to-know information. The constraints of the dashboard format provide no room for nice-to-know information. Even if there is room, leave that data out and report it in another form, a form that isn’t so front-and-centre. Furthermore, there’s a difference between (b.) need-to-know and need-to-attend-to. Not all important information needs to be highlighted. Only the useful findings should stand-out. Ideally, these are findings that require action. Finally, (c.) need-to-attend-to is different than need-to-attend-to-urgently. The world of work is full of false urgencies. Executives often get into the bad habit of insisting that things get done right away, regardless of whether the time pressure is appropriate. The dashboard shouldn’t create unnecessary panic. Only the need-to-attend-to-urgently items should sound alarm bells. And with a dashboard, these items can literally sound alarm bells. As Few correctly points out, if you’re sounding too many alarms, the user becomes inured to them … or decides to turn them off because of the irritation.

My advice to project managers has always been to mock-up a functional dashboard first. Shortcomings can be fixed with little fuss. If the dashboard is fundamentally flawed, then the process can start over. I use the building of Columbia University’s main campus as an analogy. After all of the buildings were built, developers didn’t lay down sidewalks right away. They laid down grass. After a year, developers could see where people tended to walk because they could see that trails had formed in the grass. Those are the places where sidewalks were placed. When you’re creating a dashboard, it’s important to remember that it has an inertia; choices made today get locked in and modification can be costly. So it’s important to get the choice of content right. It’s equally important to test it within a real decision-making context. And test-subjects should be the people who will eventually use the dashboard. Actively involving users in the development process also leads users to make a bigger effort to use the dashboard when finished. Functional mock-ups are the way to do all of that.

I would also stress there is a dangerous cognitive bias at play. Many development processes are subject to the commitment bias, or the tendency of people to continue on with a flawed initiative even if they know it to be flawed. That’s because they’ve sunk so much money and effort into it already. Backtracking or course-correction is thought of as a defeat. Awkward questions are raised by others. Plus, interests build up. Various parties have their pet features that they try to protect. By creating a mock-up and thoroughly testing it, those battles can take place before flaws get locked into the dashboard.

Mr Few stresses the importance of mock-ups too. As he points out, it is important that the mental model implicit in the dashboard match the mental model of the user. That can be tested with a mock-up. And much can be done with off-the-shelf office software. That also gets around the problem of info-tech gatekeepers. Mr Few tries to do all of his examples in Microsoft Excel. That requires some finagling with certain graph types. Software plug-ins may be required to add extra functionality. Yet the large feature-set of Excel offers some options for dashboard developers. The difficulty is finding out what those features are. Mr Few points to the camera tool as particularly important for dashboards (instructions here). The various visual elements can be arranged on a screen and a snapshot can be taken. The best part is that the data can change and the snapshot gets updated. Thus, the feature allows the spreadsheet programme to emulate the behaviour of a dashboard. There’s no longer any excuse not to create and thoroughly test a mockup.

Also, some software (such as Tableau) can grab data from various sources and formats on your computer with relatively little fuss. Thus, real numbers can be used to test the various visual elements before they’re added to the mock-up. That further reduces the risk of wasteful back-tracking.

At several points during the workshop, Few mentioned devices by Apple as examples of functional and beautiful design. Good choice. But I’m wondering why more of us don’t use that company’s approach to organisational monitoring instead of resorting to dashboards and burdensome paper-pushing.

At Apple, the top executives meet in a big meeting every Monday. All important bits of data are reviewed, product-line by product-line. If the top boss has questions, then the Vice President responsible for a product can answer detailed questions about what’s going on. And Vice Presidents are expected to know what’s going on in their areas in great detail, not just using numerical proxies. Some have speculated that one of the Vice Presidents (Mark Papermaster) was told to leave the company because he couldn’t maintain the situational awareness within his area of responsibility (Kane & Sherr, 2010). He came from IBM, a large bureaucracy where knowing the numbers is viewed as an appropriate substitute. Not so at Apple. And so senior-ranking executives have to get actively involved in the activities within their purview. They have to know the details … the qualitative details too. Of course, there is a role for formal reporting at Apple. The laws governing accounting in public companies require it. But in such a fast-moving industry, the company resists the rigidities caused by too much paper-pushing. And it chooses to not lock-in those rigidities within an electronic report that’s expensive and time-consuming to change. If a good dashboard reinforces the mental model of the user, as Few argues, then executive dashboards can lock-in a way of seeing. That should be the rigidity that worries you the most.

Electronic dashboards can serve an important purpose, just as mechanical ones do. Many processes, such as those related to manufacturing and quality control, are well served by good dashboards. That’s why I maintain an interest in the field. If you are developing such a dashboard, I highly recommend taking Stephen Few’s workshop. It’s an excellent investment of time, even if you’ve read his books. Mr Few is an engaging teacher. I also applaud his efforts to make information design more of a discipline (although some of the finger-wagging in his online newsletter is a bit too perfunctory and snide).

All of that said, I maintain my doubts about the usefulness of executive dashboards, even for such activities as financial reporting. Better alternatives exist. Thus, the big omission in the workshop, as in the book, is a lack of guidance about the appropriateness of a dashboard in the first place. That’s important, even when focusing on interface design, because every design has to heed the organisational context. That’s why designing screens of information is no longer called “interface design” but “user-experience design”. I’ve seen many executive dashboards. I have yet to see one that doesn’t create more problems than it solves. And I haven’t seen one that generates an enlightening experience.

By Peter Stoyko

REFERENCES