Many people have tried to define a bad chart, but the simplest approach is to consider a chart’s core purpose. As Edward Tufte said, charts should “above all else show the data” (Edward Tufte, The Visual Display of Quantitative Information, 2nd Ed., Graphics Press, 1983). Showing the data requires that graphical elements have a consistent relationship with the underlying numbers.
This article will show that most companies break Tufte’s rule. The most common bad charts were bar charts, with many lacking a consistent ratio between each bar’s height and value. These charts often also hid the vertical axis, which conceals the error. Pie charts regularly had slices not matching their displayed values. Flawed area charts were also found, with some matching values to an element’s diameter instead of its area. Other problems included inaccurate labels, bad captions, or misleading 3-D effects. The charts presented in this article have been reproduced as faithfully as possible in the current format from the original sources surveyed by the author.
This article provides guidance to avoid these same mistakes. It can serve as a checklist for reviewing charts in financial reports, especially when reviewing work outsourced to a design consultant.
Background
Prior work has found distorted charts in company reports. For example, a UK study found that 49% of key financial graphs were distorted by at least 10% (Vivien Beattie and Michael John Jones, “Corporate Reporting Using Graphs: A Review and Synthesis,” Journal of Accounting Literature, vol. 27, pp. 71–110, June 2008). A 2022 review of US 10-K reports found that 26% of charts had at least one problem (Nathan Garrett, “An Evaluation of SEC 10-K Charts,” Journal of Emerging Technologies in Accounting, 2022, https://tinyurl.com/xum739av).
Charts are pervasive in companies’ annual reports. The author examined investor presentations, CSR/ESG documents, annual reports, and 10-K filings from 50 S&P 500 companies (Exhibit 1). Charts were in 85% of these documents. They were mostly heavily used in investor presentations and CSR/ESG reports. They were more rarely used in 10-K filings; many filings contained a single chart showing stock investment returns. Annual reports used the fewest charts, as they were often a brief 2–3-page letter to company shareholders. Overall, the author reviewed 1,584 charts in 162 documents. The discussion below will categorize the common flaws and distortions found in these charts and then highlight best practices that should be followed instead.
EXHIBIT 1
Companies’ Use of Charts
Proportionality Flaws
A chart has a proportionality problem when its elements do not consistently correspond to the underlying data. Of the 1,584 charts found in this sample, 192 (12%) had a fatal proportionality flaw. Most of these errors were in bar charts, but errors were also found in pie and area charts.
Bar height does not match data.
The most common proportionality problem was when a bar’s height did not match its value. This error obscures the underlying data (S. Bresciani and M. J. Eppler, “The Pitfalls of Visual Representations: A Review and Classification of Common Errors Made While Designing and Interpreting Visualizations,” SAGE Open, vol. 5, 2015, https://tinyurl.com/bd9npkkf). An example of this error is a chart from Altria’s investor presentation (Exhibit 2).
EXHIBIT 2
Altria Oral Tobacco Retail Share
The “on!” bar is too tall in the original chart. This effect can be recreated in Excel by creating the chart and then setting the vertical axis to start at 30. This effectively hides the bottom half of the bars, encouraging the reader to overestimate the retail volume of the “on!” products.
This error can be measured with the Relative Graph Distortion measure (Dineli R. Mather, Paul R. Mather, and Alan L. Ramsay, “Is the Graph Discrepancy Index (GDI) a Robust Measure?” SSRN Electronic Journal, 2003, https://tinyurl.com/mwac9xb7). RGD calculates a data-to-height ratio for one data point and then compares that ratio for other data points. In Exhibit 3, the chart below is skewed because the vertical axis starts at $5 instead of $0. It has an error of 100%, showing that the taller bar’s ratio is 100% greater than its shorter neighbor.
EXHIBIT 3
RGD Error Example
Charting the errors across the entire sample shows a range of distortions (Exhibit 4). This study ignores errors between –10% and +10% (because they may have been introduced during the coding process). This leaves 109 charts with errors between –50% and +50%, 23 charts with errors between ±50% and ±100%, and 20 charts with errors over 100% or under –100%. The distribution is slightly skewed towards positive distortions.
EXHIBIT 4
Bar Chart Distortions
How can accountants detect and correct these errors? The simplest approach is to ensure that all bar graphs have a visible vertical axis starting at zero.
If a chart has been created without an axis, there are two possible ways to review it:
- ▪ Count the size of the graph in pixels.
- Take a screenshot of the chart. This can be done in Windows with the Snipping Tool.
- Paste the image in an image editing software. Windows has a free Paint tool available.
- Measure the size (in pixels) of each bar. Most image editors show the position of the cursor on the bottom-left corner of the screen.
- Calculate the RGD distortion.
Anything over 5% is not acceptable.
- ▪ Overlay a transparent graph.
- Take a screenshot of the chart.
- Paste the image in an Excel file.
- Create a new bar chart with the correct values.
- Make the chart fill transparent.
- Place the correct chart on top of the test chart to compare.
These review methods are described in the “Bad Charts Detection Tool” Excel file that can be downloaded with the online version of this article. It also contains several transparent charts that can be overlaid on top of a chart, as well as instructional videos.
Size does not match data.
Other kinds of charts can also have proportionality problems. The charts from Broadcom’s investor presentation (Exhibit 5) illustrate this error. These errors can be detected by overlaying a transparent version of the chart, similar to the process described for bar charts above. Another type of proportionality problem occurs in area charts. These charts should encode values as the area of a circle, rather than its diameter. An example of this error can be found in Avalon Bay’s 2012 Sustainability report (Exhibit 6). These errors can be detected by creating a bubble chart in Excel. While setting the position of a circle involves some trial and error, the downloadable Excel sheet provides a template.
EXHIBIT 5
Charts from Broadcom’s Investor Presentation
EXHIBIT 6
Charts from Avalon Bay Sustainability Report
Other Flaws
Beyond fatal proportionality flaws, there were other kinds of flaws violating best visual practices found in this sample. Prior research has established guidance for chart designs (e.g., Wan Ying Hill and Margaret M. Milner, “Guidelines for Graphical Displays in Financial Reporting,” Accounting Education, vol. 12, no. 2, pp. 135–57, 2023). Synthesizing these guidelines yields several categories of best practice violations that are found in this sample.
First, do all axes start at zero? All axes should start at zero, as non-zero axes affect user perceptions of the data (Anshul Vikram Pandey, Katharina Rall, Margaret L Satterthwaite, Oded Nov, and Enrico Bertini, “How Deceptive Are Deceptive Visualizations? An Empirical Analysis of Common Distortion Techniques,” Proceedings of the ACM Conference on Human Factors in Computing Systems 2015, Forthcoming, NYU School of Law, Public Law Research Paper No. 15-03, https://ssrn.com/abstract=2566968). For example, a chart from Textron’s 2021 CSR report (Exhibit 7) exaggerates promotions to diverse employees by starting the axis at 40 instead of 0.
EXHIBIT 7
Charts from Textron’s CSR Report
Second, do all axes read left-to-right? A reversed axis breaks chart conventions (S. Bresciani and M. J. Eppler, “The Pitfalls of Visual Representations: A Review and Classification of Common Errors Made While Designing and Interpreting Visualizations,” SAGE Open, Oct. 14, 2015, https://tinyurl.com/bd9npkkf). Similar to reading a document, visualizations should progress from left-to-right and top-to-bottom. But Public Storage reverses time in their charts (Exhibit 8), with the most recent year on the left.
EXHIBIT 8
Chart from Public Storage
Third, do all labels match their values? For example, GLW’s sustainability report (Exhibit 9) shows a label for 2019 of 2.6, but plots the bar higher than the prior year’s value of 2.7.
EXHIBIT 9
Chart from GLW’s Sustainability Report
Fourth, are any 3D effects used? These distort users’ perceptions, causing them to overstate the size of slices oriented to the left or right (Robert Kosara, “Evidence for Area as the Primary Visual Cue in Pie Charts,” 2019 IEEE Visualization Conference, https://tinyurl.com/45yw7a5m; Michael Siegrist, “The Use or Misuse of Three-Dimensional Graphs to Represent Lower-Dimensional Data,” Behaviour & Information Technology, vol. 15, no. 2, pp. 96–100, 1996, https://tinyurl.com/4ndtdp6u). A chart from Union Pacific’s 10-K (Exhibit 10) illustrates this error.
EXHIBIT 10
Chart from Union Pacific 10-K
Patterns in the Data
Are these errors isolated incidents, or do companies tend to consistently make mistakes? Within this sample, error rates in one document (such as the 10-K) were not correlated with errors in another document (such as a sustainability report). This suggests that different teams may be involved in each document, and that errors introduced by specific people are limited to individual reports.
Some companies are heavier users of charts than others. The number of charts in ESG/CSR reports is correlated with the number of investor presentations. But neither of these measures are correlated with annual reports or 10-K report chart use. Furthermore, a series of t-tests on the data found that a company’s total assets, net income, revenue, and EPS were uncorrelated with chart distortions.
This study found that 85% of companies use at least one “broken” chart in their yearly CSR, sustainability, investor presentation, 10-K, or annual reports. Of the 1,584 charts analyzed, 12% broke Tufte’s principle of “above all else, show the data.”
The poor quality of graphs in corporate annual reports is not new. Penrose’s 2008 synthesis found a “rampant disregard” for guidelines on accurate and non-misleading presentations (J. M. Penrose, “Annual Report Graphic Use: A Review of the Literature,” Journal of Business Communication, vol. 45, no. 2, pp. 158–180, 2008, https://tinyurl.com/4yazxdpd). Unfortunately, with the rapid expansion of CSR and financial analyst reports, there is even more room for users to be swayed by the misleading presentation of data. How can accountants respond to these issues?
First, accountants should be wary of handing off their data to graphic designers. Creating charts in a specialized tool (such as Excel or Tableau) can prevent many of these issues. While graphic designers can jazz up a chart, they might also introduce errors.
Second, accountants should review all charts included in their
reports and check them for the common flaws discussed above. This includes ensuring that the vertical axis is displayed on bar charts, and that it starts at zero, has a consistent interval, and reads from left-to-right or top-to-bottom. It also includes making sure that the widths of pie charts match their values and the values in area charts match area, not diameter. Company leadership should mandate that such reviews are performed and highlight the importance of communicating financial information clearly and accurately.
