Leaders of the profession have conflicting views on the state of audit quality today. According to current SEC Chair Mary Jo White, “investor confidence in audited financial statements and independent auditors is high … attributable, at least in part, to improvements in audit quality” (Remarks at the AICPA annual national conference, Dec. 12, 2015). On the other hand, PCAOB member Steven Harris has said that “investors and regulators alike do not believe that audit quality is where it should be” (Remarks at PCAOB open board meeting, Dec. 15, 2015). Among those more likely to agree with Harris are the devastated investors raising the usual cry (“Where were the auditors?”) over examples of business failures that stretch from the crisis of 2007/08—Bear Stearns, Lehman Brothers, AIG, and Fannie Mae, as a few examples—down to such recent headlines as Valeant Pharmaceuticals and Martin Shkreli.
The cliché—“When I’m right, nobody remembers, when I’m wrong, nobody forgets”—applies to auditors: cases of differentiated success are elusive, hard to measure, and underappreciated, whereas the rare, difficult, and inevitable breakdowns are vilified and punished. Put another way, the “expectations gap”—between the professed desires of financial statement users and the confessed limitations of the current audit model—remains in full view. Rather than continue to rely on the episodic and sampling-based techniques used since the Victorian era, it is time to consider a radically different approach to the search for audit quality.
Measurement Models
Although investors had long displayed general satisfaction with the traditional audit report and its standardized, commoditized language, it has more recently become recognized that the “pass/fail” auditor’s report delivers only limited assurance—saying, in effect, that a company’s financial information is more or less okay, in general, most of the time, so far as the auditors can tell—except with the sudden exposure of large-scale financial impropriety, when the reporting model breaks down and provides no comfort at all! The current model’s shortcomings are receiving attention from standards setters around the globe: the PCAOB in the United States, the Financial Reporting Council in the United Kingdom, and the International Auditing and Assurance Board internationally—all of whom are laboring to construct more informative auditor communications.
But beyond the report’s too-narrow scope, there is an additional key reason, similar to the inadequacies of the risk models that failed all tests of usefulness during the 2007/08 financial crisis, for the perception of diminished utility of the auditors’ work. Namely, the fundamental assumptions that underlie the historical audit process—just as with the risk models—are those of the comfortable and familiar “bell curve,” with its attention to so-called tail risks and its reliance on the regressive principles of the law of large numbers.
The current model’s shortcomings are receiving attention from standards setters around the globe—all of whom are laboring to construct moreinformative auditor communications.
That model, embedded in Western culture since the days of Isaac Newton, does have broad and practical applicability, where “good-enough” results are averaged across large and diverse bodies of data. But much of the world of financial information, as elsewhere, falls entirely outside these rules, instead following a “power law”—when graphed, a distribution shaped like a hockey stick. Many phenomena follow this model, including personal wealth, the sales of best-selling books, the number of Facebook friends or Twitter followers, and the size of companies. (Exhibit 1 illustrates the two curves. No formal training in mathematics is needed to appreciate that their dramatically different shapes reflect populations that follow very different organizing principles.)
EXHIBIT 1
Bell Curve versus Power Law Curve
Under power law distributions, most members of a population occupy the long, low part of the curve at the right (i.e., the handle of the hockey stick), while very few members make up the tip at the high part in the upper left-hand corner. Consider the huge number of routine individual doctor visits, at relatively modest cost, compared to the small number of hugely expensive cardiac surgeries or organ transplants. In the accounting profession, most CPA practices consist of one to five members, while the large-company audit market is completely dominated by the Big Four. Exhibit 2replicates the generic power law curve—graphing the 23 international accounting networks with global annual revenue for 2015 above $100 million. (Note the spike at the left representing the Big Four, with the dramatic fall-off down the scale to the right.)
EXHIBIT 2
Power Law Curve: Annual Revenue of Large International Accounting Firms
Analyzing Audit Quality
With the contrasts between the two models discussed above, we may move to consider audit quality or audit failure. To start, in general, much noncompliance with laws and regulations by offenders also follows a power law distribution. Examples include the small number of cars and trucks that spew most vehicular pollution, the concentration of police misconduct in the tiny number of “rogues,” and the demands on social services by a relative handful of the chronically homeless.
Specifically here, cases of financial statement malfeasance do not regress; there is no “average” dollar amount among cases of financial misstatements. Instead, minor cases of misappropriation, employee theft, or manipulation are numerous, and deemed by auditors and users alike to be immaterial—that is, they fall onto the long, flat, and low-consequence end of the power law curve. At the other end of the curve, there are several outbreaks in each business cycle that are rare, huge, and devastating in their impact. (Anecdotally supported among the large accounting firms’ lawyers and risk managers, this proposition has not to my knowledge been researched, but is testable.)
There is both good and bad news for auditors and financial statement users. First, the bad news: to the extent the traditional sampling—based audit approach is designed and expected to satisfy the underlying bell-curve assumptions of a pass/fail report, it is predictable that the effort will continue to be both inefficient and ineffective. In this author’s opinion, that approach will continue to fail to address the rare but consequential occurrences that really matter to investors and other participants in the capital markets. (Picture the two curves as graphically overlaid. The obvious mismatch would illustrate the unsatisfactory design of one to serve the other.)
The good news, on the other hand, is that audits could be redirected to identify and act effectively on the indicators of potentially large-scale breakdowns. The ability to shed needed light on the presently ignored or underappreciated symptoms of impending failure is real. Achieving success, however, will require a recalibration of the tools of audit execution, because the effort will be subtle and demanding, pushing the effort away from the inconsequential and toward the upper end of the curve, where the few large-scale audit failures reside.
Perfection is never achievable: the possibility of “black swan” events can never be either completely excluded or infallibly detected in any complex system.
Next Steps
The road to financial statement disaster has been paved with unresolved warnings and signals that later morphed into catastrophe—“close calls” that too often were either ignored, rationalized, or missed altogether. Instead of auditing to the “good-enough” approach of bell-curve, materiality-based sampling (not to mention the nitpicking of PCAOB inspectors), a partial list of issues for redesigned audit scrutiny would include the following:
- Partner performance quality, and the readiness to recognize that the risk of recurring substandard behavior is too high to be tolerable.
- Partner override, where the informed perceptions and judgments of staff or managers in the field with their hands on a problem are ignored, rationalized, or suppressed.
- Elevation of “client service” above compliance with standards, especially where firm policies and central expertise are in tension with “judgment calls” made at the margins.
- The impact of revenue, profitability, or economic pressure—where an office, a geographic region, or an industry sector of a firm’s practice makes compromises to meet targets that are in tension with firm-wide policies and quality of work.
Perfection is never achievable: the possibility of “black swan” events can never be either completely excluded or infallibly detected in any complex system designed and operated by fallible human beings. But the expectations gap between audit performance and user desires requires adjustment and recalibration, and it is time for the profession to begin a dialogue about these issues. The role within the capital markets of a robust and sustainable audit function is too important to allow it to languish. Concerted and cooperative efforts by all parties will be required to create an audit model fit for the 21st century.
