ChatGPT on Picking Evidence in Statutory (and Constitutional) Interpretation (9/13/26)

Today, I was reworking the part of my 2027 Working Draft for my Federal Tax Procedure Editions on the problem of cherry-picking or selecting evidence used in constitutional and statutory interpretation. The issue is presented anecdotally by conservative judges trotting out Judge Leventhal’s famous remark that using legislative history is like looking over the crowd and picking your friends. There is a similar phenomenon with so-called “law office history” deployed to support arguments made by advocates and judges in their opinions. Even broader, the same phenomenon is presented for the entire body of law that advocates and judges deploy—such as cases, scholarly treatises, law review articles, etc. The risk is always present that we as lawyers (or judges) pick our friends rather than make a neutral assessment of the evidence.

Today, continuing my experimentation with AI, I asked ChatGPT Plus about the issue and got what I think are pretty good responses. I presented the issue in two prompts (the second asking for more than the first) noted below in red. The bold face was from the original responses. I checked the citations to the two Supreme Court cases and they are correct (i.e., no hallucinations). I will use the responses in re-working my discussion in the Federal Tax Procedure Editions (not verbatim but to inspire how I state the concepts in my own words which I hope are at least as good as ChatGPT's).

As an introduction, I use the term "law office history." That term commonly describes the phenomenon of lawyers and judges evaluating history as a process of cherry-picking to support arguments. Of course, selecting the evidence is what we all do when making arguments or taking positions; lawyers do that in briefs and oral arguments; judges do that in opinions; we all do that in order discourse. Even when a person--say a judge--sees his or her duty to make an objective or fair assessment of the evidence, personal prejudices or preferences may influence the evidence used. I wanted to explore what that may mean.

One caveat: The same phenomenon occurs in factfinding where a judge selects from a body of factual evidence the evidence that he or she finds persuasive to determine facts relevant to the outcome of the case. I don't think factfinding is materially different from lawfinding--selecting among the relevant evidence to reach the outcome.

Prompt #1

Please write a short discussion of the features of "law office history" and "legislative history" that permit a judge or lawyer to cherry pick (or as one judge called it to look over a crowd and pick your friends).

Response to Prompt #1

Here is a short version in a law-review style that emphasizes the structural similarity between the two problems:

Law-Office History and Legislative History

Both “law-office history” and legislative history present a similar danger: they often supply the interpreter with a large and heterogeneous body of material from which supporting evidence can be selected without any obvious neutral rule for determining what evidence should receive the greatest weight. Alfred H. Kelly coined “law-office history” to describe the lawyerly practice of selecting historical data favorable to a position while failing adequately to confront contrary evidence or evaluate its significance in historical context. The problem is partly inherent in advocacy. The historical record is usually extensive, incomplete, and capable of supporting competing narratives; choices must be made about the relevant period, the appropriate level of generality, which practices are representative, and which sources deserve weight. Those choices can permit the advocate—or judge—to begin with a desired conclusion and assemble a plausible historical narrative supporting it.

Legislative history presents much the same problem, perhaps in an even more concentrated form. A statute may generate committee reports, sponsor statements, floor debates, hearing testimony, rejected amendments, conference materials, and statements by legislators who supported or opposed the final bill. These materials may point in different directions, and statements by particular legislators do not necessarily represent the understanding of the legislative body that enacted the statute. Moreover, participants aware that courts consult legislative history may have incentives to place favorable statements in the record. The resulting abundance of potentially conflicting material makes selective quotation unusually easy. The Supreme Court has accordingly repeated Judge Harold Leventhal's famous description of using legislative history as akin to “looking over a crowd and picking out your friends.” Exxon Mobil Corp. v. Allapattah Services, Inc., 545 U.S. 546, 568 (2005).

The common problem, therefore, is not simply that historical or legislative materials are unreliable. Much of the material may be quite reliable for what it actually establishes. The difficulty is selection: when the universe of potentially relevant sources is sufficiently large and internally diverse, an interpreter may emphasize the evidence supporting the desired result, minimize contrary evidence, choose a favorable level of generality, and still produce an argument that appears well documented. Justice Sotomayor recently made essentially the same point about judicial reliance on “history and tradition,” comparing the process to entering a crowded cocktail party and looking over everyone's heads to find one's friends. Vidal v. Elster, 602 U.S. 286, 327–28 (2024) (Sotomayor, J., concurring in the judgment).

Thus, law-office history and legislative history share a central methodological vulnerability: the more numerous, varied, and contestable the sources, the greater the opportunity for an interpreter to make source selection do much of the work of interpretation. The appearance of objective historical support can therefore conceal a considerable element of judgment—and, at its worst, cherry-picking.

I think the last paragraph captures particularly well the point you are making: the problem is less the existence of bad evidence than the discretion involved in selecting among abundant evidence.

Prompt #2

Please compare that phenomenon with the body of resources such as cases and commentators that judges and lawyers use for constitutional and statutory interpretation.

Response to Prompt #2

The comparison is close, but there is an important difference: cases and legal commentary are themselves part of a structured legal system, whereas historical and legislative materials generally are not. That structure constrains cherry-picking, but it does not eliminate it.

Cherry-Picking Among Legal Authorities

The danger of selective use is not confined to law-office history or legislative history. Constitutional and statutory interpretation themselves draw upon large bodies of potentially relevant legal materials—cases, canons of construction, treatises, law-review articles, dictionaries, administrative interpretations, historical practice, and competing theories of interpretation. Here too, the abundance and diversity of available materials can permit a lawyer or judge to “look over the crowd and pick out [his or her] friends.”

There is, however, an important difference. Legal authorities ordinarily come with rules of hierarchy and relevance that historical materials lack. A controlling Supreme Court decision cannot legitimately be disregarded in favor of a congenial law-review article, and binding circuit precedent normally outranks contrary decisions from other circuits. Holdings generally carry more authority than dicta; majority opinions more than dissents; enacted statutory text more than commentary about it. These conventions place meaningful limits on the interpreter’s freedom of selection.

Even within those limits, however, considerable discretion remains. Cases rarely speak with one voice. A lawyer may characterize one precedent broadly and another narrowly, emphasize language in one line of cases while treating contrary language as dictum, select the favorable formulation of a legal test, or argue that an inconvenient precedent is distinguishable. Constitutional cases are especially susceptible because doctrines often develop through numerous opinions written over decades, containing different rationales and competing descriptions of the governing principle. A judge may therefore assemble a perfectly respectable string of precedents supporting a proposition while an opponent constructs an equally respectable string pointing the other way.

The same phenomenon appears with the canons and other tools of statutory interpretation. One canon may counsel reading language broadly while another counsels restraint; one precedent may emphasize ordinary meaning while another invokes statutory purpose, context, or avoidance of absurd results. Karl Llewellyn famously illustrated the problem by pairing traditional canons with counter-canons pointing in the opposite direction. The difficulty is therefore not necessarily that any particular interpretive tool is illegitimate, but that the interpreter often possesses some discretion over which tool to invoke and how much weight to give it.

Commentary increases the available range still further. Treatises and scholarly articles can illuminate difficult questions, but they also provide an enormous reservoir from which advocates can select authorities compatible with their preferred conclusions. Unlike precedent, scholarly commentary has no formal hierarchy, so judgments about which scholars, methodologies, or empirical studies deserve weight inevitably introduce additional discretion.

Thus, there is a continuum rather than a sharp distinction. Law-office history and legislative history make cherry-picking particularly visible because the source materials are numerous, heterogeneous, and weakly governed by rules of authoritative weight. Legal precedent is more constrained because the law supplies rules about hierarchy, precedent, holdings, and jurisdiction. But once the interpreter moves beyond a clearly controlling text or holding into competing precedents, canons, analogies, and commentary, much the same selection problem reappears. In all three settings, the methodological safeguard is therefore not merely citation of supporting authority, but a candid account of the significant contrary authority and an explanation of why the selected sources deserve greater weight.

One distinction may be worth emphasizing even more strongly: cherry-picking cases is generally easier to expose than cherry-picking history, because the legal system provides relatively objective rules for identifying controlling and contrary authority. That makes the obligation to confront adverse legal authority particularly important.