Some Thoughts on Rational Decision-making

Adrian Vermeule has a new paper, Rationally Arbitrary Decisions (in Administrative Law). Here is the abstract:

How should administrative law cope with genuine uncertainty, in which probabilities cannot be attached to outcomes? I argue that there is an important category of agency decisions under uncertainty is which it is rational to be arbitrary. Rational arbitrariness arises when no first-order reason can be given for the agency’s choice one way or another within a certain domain, yet the agency has valid second-order reasons to make some choice or other. When these conditions obtain, even coin-flipping may be a perfectly rational strategy of decisionmaking for agencies.

Courts should defer to rationally arbitrary decisions by agencies. There is a proper role for courts in ensuring that agencies have adequately invested resources in information-gathering, which may dispel uncertainty. Yet in some cases the value of further investments in information-gathering will itself be genuinely uncertain. If so, courts should defer to agencies’ second-order choices about informational investments on the same grounds that justify deference to agencies’ first-order choices under uncertainty.

The idea that coin flipping might be a rational decision-making process is provocative, but Vermeule is referring to a limited (though important) class of decisions. To fall into this class, a decision must exhibit "first-order uncertainty" -- genuinely unquantifiable uncertainty due to absence of information about facts or individuals' future actions -- and also "second-order uncertainty" -- impossible to gather further information to resolve the first-order uncertainty.

One of his examples involves an agency that must decide whether or not to list a species as endangered. The evidence is not conclusive either way. Further information-gathering will not assist, such that the following situation could be said to arise:

[A] hallmark of problems calling for a rationally arbitrary decision is a kind of mirror-image reversibility. If the agency chooses A over B, and the court overturns that decision as arbitrary, it will also be the case that the agency’s choice of B over A could be overturned on exactly the same ground. Indeed, any choice the agency makes could be overturned for lack of first-order reasons. Recognizing this, the reviewing court should realize that the agency may be facing a situation in which no non-arbitrary choice is feasible.

What is the agency to do in such circumstances? Sometimes it can extrapolate from past experience, default to the status quo or use conventional wisdom. But in some circumstances, Vermeule argues, these options will not avail the agency any more than a coin toss or other means of randomization.

This arresting logic is not to be pushed too far. Vermeule notes that courts have a role "in ensuring that agencies have adequately invested resources in information-gathering". This seems quite correct. Agencies should explain the existence of first- and second-order uncertainty. If they do not provide cogent reasons or evidence to support a claim that a decision was reached under uncertainty, the courts should intervene. A 'reasoned decision-making' requirement is as appropriate here as it is elsewhere.

A recent Canadian example comes to mind. Tervita Corporation v. Commissioner of Competition, 2013 FCA 28 involved a situation in which the Competition Tribunal had to put its mind to the economic effects of a proposed merger of landfill operators. Both quantitative and qualitative effects had to be considered. Evidently, not all of these could be measured with precision. In part, the analysis in Tervita depended on the answer to a question about the future actions of individuals and companies: how would others react to a decision to allow or forbid the merger? I think this is a modest example of what Vermeule styles as "strategic uncertainty" (uncertainty "arising from the strategic interdependence of actors’ choices").

Nonetheless, the Federal Court of Appeal quite rightly emphasized the importance of rational decision-making:

[148]      An objective offset analysis means that the quantification of both gains in efficiency and anti-competitive effects must be carried out whenever it is reasonably possible to do so. When precise quantification is not reasonably possible for a given element, a rough estimate is to be preferred to a subjective judgement call. When neither a precise quantification nor a rough estimate is reasonably possible for a given element, then of course there will be a certain degree of discretion in attributing weight to any remaining qualitative gain in efficiency or effect, but this discretion must be curtailed and limited by the principles of reasonableness. In other words, any weight given to the remaining unquantifiable qualitative effects must be reasonable, i.e., it must be supported by the evidence, and the reasoning behind the Tribunal’s weighting must be clearly articulated or otherwise discernable.

A coin toss is very much a last resort! And if a coin toss is chosen, every step up to the choice should be explained in an objective fashion.

A more difficult question arising from Vermeule's analysis is in what circumstances limitations of time and resources can be a basis for refusing to resolve first-order uncertainty. In particular, there are areas of administrative law where the need for speedy and cost-effective decisions is quite pressing and might be thought to outweigh the need for perfect rationality.

An example of this may be a case currently under reserve at the Supreme Court of Canada: Syndicat canadien des communications, de l’énergie et du papier, section locale 30 c. Les Pâtes et Papier Irving, Limitée, 2011 NBCA 58. At issue here was the imposition by the employer of mandatory random alcohol testing for employees working in safety-sensitive areas of a paper mill located in a major city. A test of reasonableness applies in Canadian labour law to unilateral decisions of this nature.

An arbitrator found in favour of the union. A categorical approach was adopted. In "dangerous" environments, evidence of an existing alcohol problem is needed before proceeding to random tests which compromise employee privacy. In "ultra-dangerous" environments (nuclear facilities, airplanes, etc), no evidence is needed. Here, the arbitrator found, the workplace was not ultra-dangerous, even though there was a potential for catastrophic accidents. Accordingly, evidence was needed. In the arbitrator's view, the available evidence of alcohol-related incidents did not justify the imposition of random testing. The employer's decision was unreasonable.

On judicial review, both the first-instance judge and the New Brunswick Court of Appeal struck down the arbitrator's decision: once the potential for catastrophe was evident, imposing random alcohol testing had to be reasonable. As Robertson J.A. concluded:

[57]          In summary, it is not difficult to support the contention that Irving’s kraft paper mill qualifies as an inherently dangerous workplace as would a chemical plant. This is why evidence of an existing alcohol problem in the workplace was not required to support its policy of random alcohol testing. This is why the arbitration board’s decision cannot stand and the application judge was correct in determining that its decision should be set aside and the grievance dismissed. In the circumstances, it is unnecessary to delve into issues dealing with the board’s perception that evidence of near disasters is required to justify the imposition of alcohol or drug testing policies.

But the reasoning of both the arbitrator and the reviewing courts took place in a significant factual vacuum. The possibility of catastrophic accident was noted, but it was not quantified. Surely it could have been. The possibility that alcohol impairment would lead to catastrophic accidents is also, in principle, a risk that can be quantified. It was not. This is really an example of risk rather than what Vermeule styles as "brute uncertainty" ("epistemically unattainable facts that in some sense lie out there"), but in the absence of analysis, risk seems to shade into uncertainty.

The interesting question is whether resources should have been invested in calculating the risks. My tentative answer is no: as long as the reasoning process of the arbitrator was clear and supported by evidence in the record, there was ample material for the reviewing courts to assess the rationality of the reasoning process and ultimate decision. In turn, the reviewing courts were able to explain where the arbitrator's logic fell short. A much more satisfying decision could have been reached if time and money had been expended on gathering more information. But whether that would have satisfied the interests of the parties is doubtful. (Perhaps those living and working near the paper mill should also have a say, but the structure of the arbitration process did not allow for that.)

At a minimum, however, a reviewing court should require alertness on the part of the arbitrator to the fact that risks had not been adequately calculated and sensitivity to the possibility that some mechanism for calculating the risks (perhaps expert evidence) could be used. If the arbitrator decided not to calculate the risks (because of limitations of time and resources), he should have explained why. The objective reasonableness standard emphasized in Tervita requires, I think, alertness and sensitivity to this problem.

Vermeule's paper can be downloaded here.