The Scope of Situated Judgment - An Ontology of the Judged Subject in a Generative Relational Framework

Transcript

Abstract

Judgments of persons are routinely formed inside one relation and used inside another. A supervisor’s account of a researcher is read by an admissions committee; a clinician’s formulation follows a patient into a different service; an electorate that has never met a candidate decides on the strength of endorsements by those who claim to know him. This paper asks what such a judgment warrants once it leaves the relation in which it was formed. On the framework used here, what a relation generates is generated within it and is available only from the positions its parties occupied, so an attribute exhibited in one relation is not thereby a property carried into others. The paper distinguishes two claims about assessment that are usually run together: that a judgment is a noisy estimate of a real underlying quality, for which the remedies are structure and de-biasing, and that there is no such quality to estimate, for which those remedies are misdirected. It argues that both hold, in different domains, and states where the boundary falls. A limiting case is then examined in which a referee signs an account written by its subject: no shared experience exists and the instrument functions regardless, which indicates that such instruments do not carry an account of a relation at all. What they transmit is a field of relational conditions that reconfigures what the person can subsequently generate; the account is the justification offered for opening that field rather than the thing conveyed. This explains why an empty instrument works, why such instruments are exchanged for things other than accounts, and why the same property that lets an unassessable person enter a system is what makes the instrument tradeable. Two cases are compared, political selection and academic recommendation, and the comparison is drawn not through the motives of those who judge but through the regime under which judgments are issued: who may issue one, at what weight, with what observational access, with what transferability, and under what provision for revision. The paper’s positive proposal follows from the political case. Political theory did not resolve the problem of judging a candidate by making judgment more accurate; it made the mandate revisable. That remedy is institutional and temporal rather than epistemic, and it generalises. Where the paper’s claims are already made by professional assessment standards, by the conferralist account of social properties, or by the selection model of representation, they are conceded rather than asserted.

Keywords: situated judgment; validity; social ontology; representation; assessment of persons; relational conditions.

Discussion Paper Note

This paper is a preliminary discussion paper intended to share an evolving idea and invite further dialogue, criticism, revision, and independent development. Its definitions, distinctions, and constructions remain provisional. Circulation across scholarly and practical communities is part of the purpose of releasing the manuscript at this stage.

The author treats the viewpoints, concepts, and lines of reasoning presented here as contributions to a shared field of inquiry. Similar or related ideas may have appeared in other intellectual, cultural, and disciplinary traditions. The manuscript therefore states its known antecedents, separates the researcher-origin proposal from later formal reconstruction, and leaves historical priority open pending a systematic originality review.

The arguments should be understood as provisional and historically situated. Readers are encouraged to question, test, revise, extend, reinterpret, or independently develop the ideas presented here. Where appropriate, acknowledgment of this paper as one point of encounter in the development of a related idea is appreciated. Such acknowledgment records an intellectual route; the ideas themselves remain available for criticism, revision, and independent development.

Responsible Use and Rights Reservation

This section separates requested scholarly conduct from the legal permissions stated on the following page. It records an ethical request for responsible use and then defines the narrower scope of retained legal rights.

The author encourages good-faith discussion, criticism, independent inquiry, and responsible use of the material in this work. Separately from the licence’s terms, the author asks users to consider foreseeable harms when adapting or applying the arguments made here. This paper argues that judgments of persons warrant less than they are ordinarily taken to warrant. An argument of that shape can be used to dismiss any judgment whatever, including judgments that are well founded and that protect people from harm, and the paper accordingly states the conditions under which a judgment retains its force rather than leaving the matter open. The author asks that the argument and those conditions be applied together. This ethical request leaves the licence’s permissions and legally authorized uses unchanged.

The author retains the rights preserved under CC BY-NC 4.0 and may pursue remedies to which the author is legally entitled for breach of the licence or violation of the author’s independently applicable rights. Reuse remains independent from authorial endorsement. Third-party rights require authorization from their respective holders where applicable. Copyright exceptions and limitations, including applicable forms of fair use or fair dealing, remain fully available.

Notices

This page consolidates the manuscript’s publication status, licence, development disclosure, research-programme relation, declared interest, and suggested citation.

Publication status of this draft.

This working draft records an evolving stage of the author’s position and is circulated for discussion. Definitions, section structure, statements, and numbering remain subject to revision. Several literatures the argument bears on are represented only in part, as the accompanying literature audit records. Empirical work on the claims advanced here, specialist review of the political-theory and measurement material, and a systematic originality audit remain future research stages.

Licence.

Except where otherwise indicated, copyright 2026 Wanhong Huang. This work is made available under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). Subject to its terms, the licence permits sharing and adaptation for noncommercial purposes with appropriate attribution, a link to the licence, an indication of changes, and attribution that preserves the licensor’s independence from the reuse. Reuse is governed solely by that licence; the responsible-use request on the preceding page remains separate from its terms. The licence deed and legal-code link are available at https://creativecommons.org/licenses/by-nc/4.0/. The licence governs in case of conflict with this summary. Third-party material remains subject to the rights held by its respective rights holders.

Statement on the use of language models.

The exploratory discussions and preparation of this paper involved Anthropic’s Claude. The model supported exploratory dialogue, source discovery followed by verification against publisher, journal, governmental, and institutional pages, argumentative criticism, and drafting in LaTeX. Every source cited here was verified before it was written into the manuscript rather than after. The author selected the research question, directed and approved the theoretical commitments and the epistemic status of the claims, and bears sole responsibility for the manuscript, including its definitions, constructions, taxonomy, arguments, conclusions, and errors. Authorship credit remains with the human author. The access level and claim limit for every cited source are recorded in the accompanying literature audit.

Declared interest.

The author is subject to procedures of the kind examined in Section 11 and has an interest in the conclusions reached about them. The paper is written as a general analysis of judgments of persons, of which academic recommendation is one case among several, no individual process or institution is described, and the arguments are assessed against criteria stated in advance. The interest is declared because a paper arguing that a class of judgments warrants less than it is taken to warrant should say where its author stands with respect to that class.

Related research programme.

This paper is project P004 and the fourth in a series on trust, neutrality, and the transmission of shared experience. Project P001 takes responsibility for the account of neutrality as the governance of a field of relational conditions; P002 for the individual-scale credibility problem and the commitment trilemma; P003 for the procedure of reference-based selection and for the reading of multiple reference as reconstruction. The present paper takes responsibility for the ontology of the judged subject, for the account of what such judgments transmit, for the comparison of issuance regimes, and for the scope conditions proposed. Later papers take the duties of the party who issues a judgment, the distribution of the conditions under which judgments are generated, and the authority to interpret a shared experience; those questions are marked where they arise and are not argued here.

Suggested citation.

Huang, Wanhong. “The Scope of Situated Judgment: An Ontology of the Judged Subject in a Generative Relational Framework.” Working discussion paper, 2026.

Introduction

An electorate decides among candidates it has never met. Its members have read about them, watched them speak, and heard from people who claim to know them. Almost nobody voting has stood in any relation to the person voted for, and the decision is made anyway, at enormous stakes, and has been made this way for as long as there have been elections.

The situation recurs far beyond politics. An admissions committee decides on a candidate it will meet for twenty minutes, on the strength of accounts written by people who worked with the candidate for years. A clinician’s formulation follows a patient into a service that did not produce it. A performance rating made by one manager governs a promotion decided by another. In each case a judgment of a person is formed inside one relation and then used inside a different one, by parties who did not stand in the first.

This paper asks what such a judgment warrants once it has left the relation in which it was formed.

The question is old, and the paper claims no novelty in asking it. It is one of the oldest questions in political theory, where the problem of how an electorate can judge a candidate it cannot know has been argued since antiquity, and where the received answers include filtration through better-placed intermediaries, the deliberate selection of persons of distinction, and the substitution of competitive selection for direct assessment. The paper concedes that tradition in Section 3.1 before advancing anything of its own.

What the paper adds is narrower. The tradition asks who should judge, on what grounds, and how often the judgment should be renewed. It does not ask what the judgment is a judgment of. This paper takes that question, and takes it under a specific commitment: that what a relation generates is generated within it, and is available only from the positions its parties occupied. On that commitment, an attribute exhibited in one relation is not thereby a property that the person carries into others, and a judgment formed in a relation is a judgment about what that relation produced rather than about a quality residing in the person judged.

Three consequences organise the paper.

The first concerns what is wrong with such judgments when they go wrong. Section 7 distinguishes two claims that are usually run together. On the first, a judgment is a noisy or biased estimate of a real underlying quality, and the remedies are structure, calibration, and the removal of bias. On the second, there is no such quality to estimate, and those remedies produce a more reliable estimate of nothing. The paper holds that both claims are true in different domains, and its contribution here is to say where the boundary falls rather than to assert the second everywhere. The strongest opposing position, that validities generalise across settings and that apparent situational specificity is an artifact of sampling error, is stated at length in Section 3.4 and answered in Section 7.3.

The second concerns what such judgments transmit, and here the paper’s answer departs from what is usually assumed. Consider a case that is neither rare nor scandalous: a referee tells a candidate to draft the account and undertakes to sign it, or asks for a curriculum vitae and writes from that. No shared experience is being reported, and the instrument works exactly as it would otherwise. Section 6 takes this as decisive. What such an instrument transmits is not an account of a relation but a field of relational conditions: it alters the set of relations available to the person in the receiving system, and what the person subsequently generates is produced in that new configuration. The account of the relation is the justification offered for opening the field, not the thing conveyed. Three features of practice follow from this and are otherwise puzzling: that an empty instrument functions; that such instruments are exchanged for things which are not accounts, including undertakings of future cooperation; and that the property which allows a person no institution can assess to enter one is the same property that makes the instrument tradeable. The good and the bad here are one property and not two.

The third concerns what follows for practice, and the paper’s proposal is borrowed. Two cases are examined. In political selection, judgment is issued by many parties with weak but public observational access, and the judgment is revisable on a fixed schedule. In academic recommendation, judgment is issued by few parties with private access that may in the limiting case be empty, and it is not revisable at all. The comparison in Section 12 is drawn through the regime under which judgments are issued rather than through the motives of those who issue them, since votes are cast for patronage and inheritance as much as for programmes, and motive purity distinguishes nothing.

What that comparison shows is that political theory solved its version of this problem without solving the epistemic problem at all. It did not make the electorate better at judging. It made the mandate expire. Fixed terms, periodic re-election, and removal are institutional and temporal devices, and they render a judgment that was never reliable tolerable by ensuring it does not stand indefinitely. Section 14 proposes that this is generalisable, and that judgments of persons in other domains lack it.

Where the paper’s claims have prior owners, they are conceded at the point they arise. Professional assessment standards already locate validity in the interpretation of a result for a specified use rather than in the instrument, which anticipates the scope argument of Section 8. The conferralist account of social properties already holds that such properties are conferred in context rather than possessed, which anticipates much of Section 5. The selection model of political representation already reframes electoral choice as the identification of a type rather than the measurement of a quality, which anticipates part of Section 10. And the account of interactive kinds already holds that classifications change the people classified, which anticipates Section 13. The paper’s contribution lies in what remains after these concessions, and Section 3.8 states what that is.

One further caution belongs at the outset. Several literatures that a paper of this kind might lean on report effects that are small, conditional, or unreplicated. Teacher-expectancy effects are on the best synthesis typically small and more likely to dissipate than to accumulate; stereotype-threat findings have a contested replication record; and labelling theory was itself modified by its principal proponents in response to weak evidence for its strongest form. Section 13 accordingly rests on the mechanism those literatures do support, which is the alteration of the opportunities available to a person, rather than on the mechanism they do not, which is the alteration of the person.

Section 2 supplies the practice and the vocabulary. Section 3 locates the account. Section 4 states the method and the conditions of disconfirmation. Section 5 develops the ontology and Section 6 the account of transmission. Section 7 distinguishes the two claims about assessment, Section 8 states the scope condition, and Section 9 examines the professional rules that already apply one. Section 10 and Section 11 present the two cases and Section 12 compares them. Section 13 treats the fixation of an interpretation and its return upon the person, Section 14 states what may be claimed of a person, Section 15 states what the analysis returns to the wider framework, Section 16 records the limits, and Section 17 consolidates the position.

Background and Preliminaries

This section describes the practices the paper examines, states the vocabulary in which professional assessment already discusses the scope of a judgment, and defines the terms carried from the preceding papers. Analysis is reserved for Section 5 onward.

Judgments of Persons in Political and Institutional Selection

The practices at issue share a structure and differ in almost everything else. The structure is this: a judgment of a person is formed by a party standing in some relation to them, and is then used by a party standing in no such relation, to decide something that affects the person judged.

Political selection.

An electorate chooses among candidates. Almost no voter has stood in any working relation to the person voted for. What is available is public performance, reported record, and the endorsement of parties, colleagues, and associations that claim closer acquaintance. The decision is consequential, it is taken at scale, and the arrangement is very old.

Academic and professional recommendation.

A committee selects among candidates on the strength of accounts written by those who worked with them. The accounts are ordinarily confidential to the candidate, free in form, and written by parties the candidate nominated. The committee has no independent access to what the accounts report. This case was examined as a procedure in the preceding paper in this series; here it is one of two cases and is examined for what the judgment warrants rather than for how the procedure is run.

Clinical formulation and its transfer between services.

A clinician forms an account of a patient in the course of a therapeutic relation and records it. The record follows the patient into services that did not produce it and whose relation to the patient differs. Professional standards treat the resulting transfer as requiring care, and Section 9 examines those standards as the paper’s principal precedent.

Performance appraisal and risk assessment.

A rating made by one manager governs decisions taken by another; a risk instrument scored in one setting informs a decision in a second. Both are instances of the same structure, and both are treated here through the professional standards that govern them rather than as separate cases.

Two features common to all four are recorded because the argument uses them. The judging party in the second stage cannot inspect what the first-stage party observed, and has access only to the account. And the person judged ordinarily stands outside the second stage altogether.

Validity Vocabulary in Professional Assessment Standards

The professions that assess persons formally have developed a vocabulary for exactly the question this paper asks, and it is set out here so that Section 8 can concede it before extending it.

The governing statement holds that validity is the degree to which evidence and theory support the interpretations of test scores for proposed uses, and that validation is the accumulation of evidence supporting those proposed score interpretations (American Educational Research Association, American Psychological Association, and National Council on Measurement in Education 2014). Two consequences of that formulation matter here. Validity attaches to an interpretation for a use rather than to an instrument, so that validity is a property a test holds only under a specified interpretation. And a validation argument is therefore bounded by the use it was constructed for, so that employing a result for a purpose outside that use is not a lesser degree of the same warrant but a different question, on which the accumulated evidence is silent.

The unified conception behind that formulation treats validity as a single judgment supported by several kinds of evidence, and includes the social consequences of test use within the validation argument rather than outside it (Messick 1989, 1995). The paper uses this in Section 13, where the consequences of a fixed interpretation for the person interpreted are treated as bearing on the interpretation’s warrant rather than as a separate ethical matter.

The vocabulary is a professional consensus about instruments with scores. What this paper takes from it is the form of the claim rather than its domain: that a judgment carries warrant for a specified use and not beyond it. Section 8.2 states what has to be added when the judgment is not a score and the specification is not a use but a relation.

Vocabulary Carried from the Preceding Papers

The account uses a small vocabulary established across this series, restated here at the length the argument requires.

A relation is an ongoing process between parties rather than a state obtaining at a moment, and is described by what it produces.

The generativity of a relation is its capacity to continue producing such outcomes, including outcomes no party can specify in advance. Generativity is a property of the relation and not of either party.

A relational condition is an arrangement whose presence or absence changes which relations can be formed or continued, without determining which of them are. The field of a set of parties is the set of arrangements consistent with the conditions then in place. These two terms are the preceding papers’ central objects and they carry the weight of Section 6.

Co-experience is what a relation generates between its parties: the undertakings, difficulties, judgments and understandings that arose in it and belong to neither party alone. A relational position is the standpoint from which a party stands in a relation, which fixes what of the relation is available to that party.

Revisability is the requirement that an interpretation remain open to being reopened, and that no interpretation be placed beyond the reach of further interpretation. Section 10 finds this requirement already institutionalised in one of the two cases examined.

Two terms are introduced by this paper. A situated judgment is a judgment of a person formed within a relation, on the basis of what that relation made available from the position the judging party occupied. And the issuance regime of a domain is the set of arrangements governing who may issue such a judgment, at what weight, with what observational access, with what transferability, and under what provision for revision. Section 12 compares the two cases along exactly these five dimensions.

The framework within which the paper works may be stated in one sentence: it is a theory of how subject, meaning, value, creation, and normativity co-emerge through generative relational processes. Section 5 develops the part of it the argument requires and imports no more.

Literature Review

This section locates the account among the literatures it depends on. It opens with the tradition that has argued the paper’s question longest and concedes it, then turns to the ontology of social properties, the evidence on whether attributes travel across settings, and the position that most directly opposes the paper’s ontology.

Judgment of Candidates in Political Theory

The problem of how a body that lacks acquaintance with a person can nonetheless judge them is the constitutive problem of representative government, and it has been argued for as long as representative institutions have existed. The paper concedes that tradition before advancing anything.

The form of the concession should be stated exactly, since the paper’s claim to have added something depends on it. This subsection engages three treatments: a historical account of how representative institutions were established and what the choice of election produced (Manin 1997); a conceptual analysis of what representation consists in (Pitkin 1967); and a contemporary account of what an electorate is doing when it chooses (Mansbridge 2009). Two primary statements of positions the paper relies on are cited directly, and the remainder of the tradition is cited through these treatments, which makes this subsection a positioning of the paper’s question within the tradition rather than a survey of it. Section 16.3 records the consequence.

Election as a device of distinction rather than of resemblance.

The most developed historical treatment holds that representative government should be understood as a combination of democratic and undemocratic, aristocratic elements, and that when representative institutions were first established in Europe and America they were designed in opposition to democracy proper (Manin 1997). The argument proceeds by recovering the alternatives that were available and forsaken, from classical Athens to Renaissance Florence, where selection by lot was in use; and it brings out what the choice of election produced, including the elitist aspect of elections and the non-binding character of campaign promises (Manin 1997). The relevant result for this paper is that election operates as a device of distinction rather than as a neutral instrument for identifying the competent. It systematically selects those whom an electorate perceives as distinguished, which is a property of the selection device rather than of the persons selected.

Representation as several distinct relations.

The standard conceptual treatment distinguishes formalistic, descriptive, symbolic and substantive senses of representation, and holds that no one of them exhausts the concept (Pitkin 1967). Two features of that treatment bear on the present argument. It establishes that what an electorate is choosing is underdetermined by the act of choosing, since the same election can be read as authorising, as resembling, as standing for, or as acting for. And its governing image is one this paper’s framework would recognise: the concept is compared to a complicated three-dimensional structure in a dark enclosure, of which theorists supply photographs taken from different angles (Pitkin 1967, 10). The image is offered there about a concept rather than about a person, and the resemblance to the account of standpoints developed in Section 5.1 is noted without being claimed as continuity.

Selection of a type rather than measurement of a quality.

The distinction that treatment develops has a canonical earlier statement. A programme examining whether the mechanisms of accountability characteristic of democracies suffice to induce representatives to act in the interest of the represented separates two conceptions of what elections do: on the mandate conception voters choose a programme and the election authorises its pursuit, and on the accountability conception elections hold governments responsible for the results of past action, so that governments anticipating the voters’ judgment choose accordingly (Manin, Przeworski, and Stokes 1999; Przeworski, Stokes, and Manin 1999). The same volume poses the alternative in the terms the present paper needs, distinguishing the selection of good types from the sanctioning of poor performance (Fearon 1999).

The contemporary treatment nearest to this paper’s concern reframes what an electorate is doing. Where a representative has self-motivated reasons for acting as the represented would wish, it is more efficient to invest resources in advance in selecting the required type than to invest afterwards in monitoring and sanction (Mansbridge 2009). Accountability on that model takes narrative and deliberative forms rather than the form of reward and punishment. The same author’s earlier survey distinguishes promissory, anticipatory, gyroscopic and surrogate forms of representation (Mansbridge 2003).

This substantially anticipates part of what Section 10 says about the political case, and the concession is made here. The selection model already holds that electoral choice is the identification of a kind of person rather than the measurement of a quality. What the present paper adds is a question the model does not ask: what a judgment that someone is of the required type is a judgment of, given that the evidence for it was generated in relations the electorate did not participate in.

Collective judgment by parties individually poorly placed.

Whether many weakly informed judges can collectively judge well is disputed and has been since antiquity. The classical passage at Politics III.11 has been read as an argument that the many, taken together, may judge better than the few (Waldron 1995), and as an epistemic argument about the aggregation of dispersed information (Ober 2013); it has also been read as making a claim about the virtue of the multitude rather than about its collective epistemic quality (Cammack 2013). This paper takes no position in that dispute and records it because Section 12 relies on a contrast between many weak observers and few strong claimants, and a reader should know that the favourable reading of the first term is contested.

What voters can use in place of direct knowledge.

Empirical work establishes that voters substitute cues for knowledge, and that this sometimes works. Poorly informed voters who correctly perceive the endorsements attached to a measure can emulate the voting behaviour of the well informed (Lupia 1994), and the general account of low-information rationality holds that voters reason with shortcuts drawn from daily life rather than from encyclopaedic knowledge (Popkin 1991). The finding is double-edged and the countervailing work states why: heuristics can also mislead, and their advantages are unevenly distributed across levels of political sophistication (Lau and Redlawsk 2001).

Endorsement is therefore the political case’s version of this paper’s object. A party claiming relational access to a candidate issues a judgment, and an electorate lacking such access uses it. Section 10 takes that as the structure to be analysed.

Conferral of Social Properties and the Production of Subjects

The ontology this paper uses has prior owners in social ontology and in the history of the subject, and they are conceded here rather than in a closing note.

Status conferred rather than possessed.

The account nearest to the paper’s position holds that a range of social properties are conferred: a status is imposed by others adopting an attitude toward a person, ordinarily on the basis of some further property, and the person need not in fact possess that further property but need only be taken to possess it (Ásta 2018). Statuses so conferred consist in constraints and enablements, are sensitive to context, and are revocable. This anticipates a great deal of Section 5, including the claim that a judgment may function without a corresponding quality in the person judged, and it is conceded as such. A critical response develops the difficulties the account faces in handling intersecting categories (Jenkins 2019).

The wider apparatus is older. Social facts of this kind rest on the collective assignment of functions that exceed what the physical properties of objects and persons alone would support, under the general form in which something counts as something else in a context (Searle 1995). What the present paper adds to that apparatus is developed in Section 5.2 and stated as an addition rather than a discovery.

Production of the subject by the structure that judges it.

The claim that a subject is produced through relations of power and knowledge rather than pre-existing them belongs to a body of work whose stated project was a history of the modes by which human beings are made subjects (Foucault 1982). On that account power operates by structuring the field of possible actions of others, which is close enough to the present paper’s vocabulary that the debt must be stated plainly. Section 5.2 asks a question in that lineage and claims no independence from it.

A more recent analytic treatment supplies the distinction the paper needs to state its own question precisely: between what makes a social fact obtain in a particular case and what sets up the frame principles under which facts of that kind are possible at all (Epstein 2015). The paper’s second-order question concerns the second.

Priority of relations over the parties related.

Relational sociology holds that the social world consists in unfolding relations rather than in entities that subsequently enter them, and that the terms of a relation derive their meaning from it (Emirbayer 1997); parallel programmes develop the same commitment (Donati 2011; Crossley 2011). The philosophical statement that individuation precedes the individual, so that the individual is a provisional result of a process rather than its starting point, belongs to the same family (Simondon 2020).

One terminological collision requires statement. A constructionist account of selfhood published under a title one word from the name of the framework used here treats the self as constituted in and through relationship (Gergen 2009). The objects differ: that account theorises selfhood, and the framework used here concerns the conditions under which relations continue to be generated. The resemblance of terms is stated so that a reader reads it as coincidence rather than as derivation.

Cross-Situational Consistency of Behaviour

The paper’s ontology entails that an attribute exhibited in one relation need not be exhibited in another. Whether that is true is an empirical question with a long and now largely settled literature, and the settlement is less favourable to the paper than the paper’s ontology alone would suggest.

The situationist challenge held that correlations between measured traits and observed behaviour rarely exceed a low ceiling when behaviour is sampled independently of the questionnaire (Mischel 1968). The response held that correlations of that size are substantial, that predictions from situations are no larger, and that aggregation across occasions recovers substantial consistency (Kenrick and Funder 1988). The synthesis now generally accepted holds that behaviour reflects person, situation, and a large person-by-situation interaction (Fleeson and Noftle 2009).

Two contemporary formulations matter here and they pull in different directions.

The first supports the paper. Personality is expressed as stable patterns of situational variability, so that what is consistent about a person is a profile of if-then contingencies rather than a level of a trait (Mischel and Shoda 1995). On that account an attribute is exhibited in a relation because that relation supplies the conditions under which it is exhibited, which is the paper’s claim in psychological vocabulary.

The second cuts against it. Experience-sampling work finds that within-person variability across situations is very large, so that a person manifests nearly all levels of a trait in the course of ordinary life, and yet the central tendencies of these within-person distributions are almost perfectly stable (Fleeson 2001). On that account the person carries a stable distribution across relations, which is a property of the person and not of any relation. The paper accepts the finding and states in Section 7.3 what it takes to follow from it.

A further result bears on the paper’s scope condition rather than on its ontology. The accuracy of a judgment of a person depends on whether the relevant cues were available, detected, and used, so that accuracy is a function of the judge’s access as well as of the judge’s skill (Funder 1995). Section 12.2 makes observational access one of the five dimensions along which the two cases are compared.

Generalization of Validity across Settings

The position that most directly opposes the paper’s ontology is set out here at length, because a reader who accepts it will reject Section 7 and should be able to see exactly where the disagreement lies.

The programme holds that the observed variation in validity coefficients across settings is largely an artifact of sampling error, unreliability in the measures, and range restriction in the samples, rather than evidence that validity genuinely differs by setting (Schmidt and Hunter 1977). Refinements of the method concluded that after these artifacts are corrected the residual variability is close to zero, which was taken as further evidence against the hypothesis that validity is situationally specific (Schmidt et al. 1993). The synthesis of the programme reported general mental ability as the most valid single predictor across jobs, with an operational validity of approximately .51 overall, higher for complex work and lower for simple work (Schmidt and Hunter 1998).

If that programme is correct in the strong form, then attributes measured in one setting do predict performance in others, the apparent situation-dependence of behaviour is measurement noise, and the paper’s ontology is false for the domains the programme covers. Section 7.3 answers it directly.

Two more recent results bear on how strongly the programme should be read, and both are recorded here rather than saved for the reply. A re-examination of how range-restriction corrections have been constructed and applied concludes that the common procedures systematically overcorrect, that the validity of many selection procedures has therefore been substantially overestimated, and that revised estimates place general mental ability considerably lower than the earlier figure (Sackett et al. 2022). And within assessment centres, which are among the instruments the programme cites, variance associated with the exercise routinely dominates variance associated with the dimension the exercise was designed to measure (Lance 2008), which is situational specificity appearing inside the apparatus rather than outside it.

Classification and Its Effect on the Classified

A judgment recorded about a person may alter the person or their circumstances, and a literature on that possibility must be reported carefully, because its strongest versions are not supported by its own evidence.

Interactive kinds and looping.

The foundational statement holds that classifications of people and the people classified emerge together and interact, so that naming a kind of person alters the space of possibilities for being a person of that kind (Hacking 1986). The mechanism is developed as a looping effect: those classified become aware of the classification, respond to it, and thereby change what the classification picks out (Hacking 1995), so that kinds of people are moving targets (Hacking 2007). This anticipates a substantial part of Section 13 and is conceded there.

Labelling and its subsequent modification.

Sociological labelling theory holds that deviance is a consequence of the application of rules and sanctions rather than a quality of an act (Becker 1963), and distinguishes an initial act from the conduct produced by the social response to it (Lemert 1951); the associated account of spoiled identity describes how a discrediting attribute reorganises a person’s dealings with others (Goffman 1963).

The subsequent history of this literature is the part that matters for calibration. The strong reading, on which a label produces the behaviour it names, was not supported, and the theory was revised by its own proponents into a modified form in which the operative mechanisms are status loss, altered relationships, exclusion, and reduced opportunity rather than the production of labelled behaviour (Link et al. 1989). The consolidated statement treats stigma as the co-occurrence of labelling, stereotyping, separation, status loss, and discrimination, conditional on the exercise of power (Link and Phelan 2001).

Section 13.3 accordingly rests on the mechanism this literature supports, which concerns what becomes available to a person, and does not rest on the mechanism it does not support, which concerns what the person becomes.

Expectancy effects, at the size the evidence supports.

The general form of the claim is old (Merton 1948), and its best-known demonstration reported that induced teacher expectations produced measurable gains in pupils (Rosenthal and Jacobson 1968). That demonstration has a critical record which must be reported with it. A reanalysis found the original claims largely unsupported (Elashoff and Snow 1971). A synthesis of eighteen experiments found a mean effect on pupil measured ability of approximately one tenth of a standard deviation, and found that the effect approached zero once teachers had known their pupils for more than a short period before the expectancy was induced (Raudenbush 1984). The most careful assessment concludes that classroom self-fulfilling prophecies do occur, that they are typically small, that they do not accumulate greatly across perceivers or over time and may be more likely to dissipate than to accumulate, and that teacher expectations predict outcomes more because those expectations are accurate than because they are self-fulfilling (Jussim and Harber 2005).

The interpersonal demonstration of behavioural confirmation is a distinct and narrower finding, in which a perceiver’s expectation shapes the interaction so that the target’s behaviour comes to fit it (Snyder, Tanke, and Berscheid 1977). A related literature on performance under the salience of a group stereotype (Steele and Aronson 1995) has a contested replication record: a meta-analysis in one applied domain reported a small effect together with several indications of publication bias (Flore and Wicherts 2015), and a review of replication attempts found that a minority of unconfounded experiments reproduced the original result (Stoet and Geary 2012).

The paper uses none of these findings as load-bearing. They are reported because a paper on this subject that omitted them would be assumed to be relying on their strong versions.

Performativity of Descriptions in Social Life

A description of a social situation may contribute to bringing about what it describes, and a literature has developed the conditions under which this occurs.

The developed case concerns financial theory, where a model’s adoption altered market practice so that prices came to conform more closely to it, a process distinguished from the case in which a theory undermines its own conditions (MacKenzie 2006). The collection that consolidated the programme includes both its development and vigorous objections to it (MacKenzie, Muniesa, and Siu 2007), and the programme’s origin lies in work on the constitution of markets by the disciplines that describe them (Callon 1998).

The discipline this literature imposes on the present paper is worth stating. Performativity asserts something narrower than that an authoritative statement makes itself true. In the developed case the model’s adoption changed prices only because using it did not generate arbitrage that would have destroyed it; a mistaken formula would not have held (MacKenzie 2006). Any claim in Section 13 about a recorded judgment contributing to its own confirmation is bounded by the same requirement, and the paper states the mechanism rather than asserting the effect.

Evaluation of Selection Rules under Unobserved Outcomes

A decision that selects some and rejects others generates outcome data only for those selected, and a formal literature addresses what can and cannot be inferred from such data.

The problem has been stated precisely for algorithmic and human decisions alike: outcomes are observed only for the cases a decision-maker passed through, so a candidate decision rule cannot be evaluated against the outcomes of the cases it would have treated differently, and the unobserved cases may differ systematically from the observed on characteristics the analyst does not record (Lakkaraju et al. 2017). Applied to judicial release decisions, the same programme notes that outcomes are observed only for those released and not for those detained, which makes evaluation of a counterfactual rule difficult and requires explicit construction of unbiased decision counterfactuals (Kleinberg et al. 2018).

Section 13.4 applies this directly. Where a judgment of a person governs whether that person is admitted to a setting, the judgment cannot be tested against what those excluded would have done, because they were not admitted. The literature supplies the formal statement and the paper supplies the consequence for the warrant of the judgment.

Boundary of the Present Contribution

Table [tab:antecedents4] records what each literature licenses and where this paper’s contribution begins.

@P0.22YY@ Literature & Licensed role & Boundary of the present contribution
Political theory of representation & Election as a device of distinction (Manin 1997); the several senses of representation (Pitkin 1967); selection of a type rather than measurement of a quality (Mansbridge 2009) & Asks who should judge and how often the judgment is renewed. It does not ask what the judgment is a judgment of.
Social ontology & Conferred statuses that the person need not possess (Ásta 2018); status functions in a context (Searle 1995); grounding against anchoring (Epstein 2015) & Supplies the ontology, which is conceded. The transfer of a judgment between relational systems lies outside its treatment.
Production of the subject & The modes by which human beings are made subjects (Foucault 1982); individuation before the individual (Simondon 2020) & The second-order question belongs to this lineage and is claimed as an application, not as a discovery.
Personality and situation & Person, situation, and their interaction (Fleeson and Noftle 2009); if-then signatures (Mischel and Shoda 1995); stable within-person distributions (Fleeson 2001) & Establishes the empirical terrain and constrains the ontology; the scope consequence for judgments is drawn here.
Validity generalization & Situational specificity as an artifact (Schmidt and Hunter 1977; Schmidt et al. 1993); operational validities across jobs (Schmidt and Hunter 1998) & The opposing position. The paper concedes its domain and disputes its extension.
Classification effects & Interactive kinds and looping (Hacking 1986, 2007); modified labelling (Link et al. 1989) & Supplies the mechanism for the return of a record upon a person; the paper restricts itself to the opportunity mechanism the evidence supports.
Selective labels & Outcomes observed only for the selected (Lakkaraju et al. 2017; Kleinberg et al. 2018) & Supplies the formal result; the consequence for the warrant of a judgment is drawn here.
Opportunity and access & Access through weak ties (Granovetter 1973, 1974); credentials as entry licences (Collins 1979) & Nearest neighbours to the account of transmission in Section 6, and each stops short of it.

Four positions are left unoccupied by the literatures surveyed.

No treatment located here holds that an instrument of this kind transmits a field of relational conditions rather than an account, and none draws the consequence that such an instrument functions when the account is empty.

None unifies the case of the account written by its own subject with the case of the account exchanged for something other than an account, by identifying the single property that produces both.

None compares domains of person-judgment along the five dimensions of an issuance regime, and none draws the resulting contrast between many weakly placed issuers under guaranteed revision and few strongly placed issuers under none.

And none proposes that the institutional and temporal remedy developed in political selection, in which a judgment expires and must be renewed, generalises to judgments of persons in other domains.

The claim is that these four are unoccupied, not that their components are unprecedented; the components are conceded above.

A bounded search for prior use of the paper’s own formulations, covering the transmission of a field of relational conditions by an instrument of judgment, the comparison of domains by their regime of issuance, and the generalisation of an expiring mandate to non-political judgments, returned no related scholarly use. A bounded search establishes that a formulation was not found rather than that it does not exist, and a systematic originality audit remains outstanding and is recorded in Section 16.

Three questions arising in this material belong to later papers in this series and are marked where they arise rather than argued here: what a party who issues a judgment of a person owes, which is the subject of the ethics paper; how the conditions under which judgments are generated are distributed, which is the subject of the injustice paper; and who holds authority to interpret a shared experience, which is the subject of the jurisprudence paper.

Method and Selection of Cases

This section states the form of argument the paper attempts, why these two cases were chosen, and what would count against the account.

Conceptual Analysis with a Stated Ontological Commitment

The paper is a work of conceptual analysis conducted under a stated ontological commitment, and both halves of that description carry obligations.

The commitment is that what a relation generates is generated within it and is available only from the positions its parties occupied. It is neither established here nor treated as self-evident. It has prior owners in social ontology (Ásta 2018; Searle 1995) and in the history of the subject (Foucault 1982), it is contested by a substantial empirical programme (Schmidt and Hunter 1998), and it is constrained by findings about the stability of within-person distributions across situations (Fleeson 2001). The paper states it, concedes its owners, reports what opposes it, and argues about its scope.

The analysis then proceeds by asking what follows for the warrant of a judgment if the commitment holds. Its results are conditional in form: given the commitment, a judgment carries warrant of a certain extent and not beyond. A reader who rejects the commitment can identify precisely which results fall with it, and Section 7.3 states which do not.

Two obligations follow from the paper’s use of empirical literatures. Where a finding is contested or small, the paper reports it at the size the evidence supports and leaves it out of the argument’s foundations, as Section 3.5 has already done. And where a formal result is borrowed from another domain, the paper states the conditions under which the result holds and claims none of them for itself.

Selection of the Two Cases

Two cases are examined and they were chosen to differ maximally along the dimensions the paper’s argument identifies, while sharing the structure the paper’s argument concerns.

The shared structure is the one set out in Section 2.1: a judgment formed by a party standing in a relation to the person judged, used by a party standing in no such relation, to decide something affecting the person judged.

Political selection is examined because it is the case in which the judging party’s relational access is weakest and the consequences are largest, because the arrangement is very old and has been argued about continuously, and because it has developed an institutional response that the other case lacks. It is examined as a case rather than as the paper’s frame; the paper stands outside political theory and makes no claim about the justification of representative government.

Academic recommendation is examined because it is the case in which the issuing party’s relational access is strongest and most private, because the judgment is not revisable, and because it contains a limiting case, examined in Section 6.1, in which the relation the account reports does not exist.

The pairing is therefore chosen to hold the structure constant and vary the issuance regime, and Section 12 states the five dimensions along which the variation runs.

Three absences are recorded. Clinical formulation and actuarial risk assessment are treated in Section 9 as sources of professional rules rather than as cases, because what the paper needs from them is the rule and not the practice. Performance appraisal is set aside, since the literature it would require overlaps almost entirely with the material already surveyed. And algorithmic assessment of persons is omitted, which is a substantial gap recorded in Section 16.

Conditions of Disconfirmation

The account should be narrowed or withdrawn under any of the following conditions.

First, if attributes judged in one relation are shown to be exhibited at similar levels across relations that differ in the respects the account identifies as relevant, then the ontology is false for those attributes and the scope condition of Section 8 has no application to them. The validity-generalization programme asserts something close to this for a range of work-related attributes (Schmidt and Hunter 1998), and Section 7.3 concedes the domain in which it holds.

Second, if an instrument of the kind examined in Section 6 is shown to function only where an account of a genuine relation underlies it, then the transmission claim is false and the instrument does carry what it purports to carry.

Third, if the five dimensions of an issuance regime do not distinguish the two cases, or distinguish them in a way that does not track any difference in outcome, the comparison in Section 12 is idle.

Fourth, if judgments that expire and must be renewed are shown to perform worse than judgments that stand, the proposal of Section 14.2 is wrong. The paper notes that this condition is testable in domains where both arrangements exist and leaves the test to further work.

Fifth, if the scope condition of Section 8 cannot be stated in a form that permits a judgment to be assessed as within or beyond its scope, then the condition is a slogan and should be withdrawn. Section 9 treats professional rules that state such conditions operationally, and the paper’s condition should be held to the same standard.

The Generative Relational Account of the Judged Subject

This section states the ontology the argument uses. It is developed in four steps, of which the first two are conceded to prior owners and the second two are where the paper’s own work begins.

Co-Emergence of the Subject within Relational Systems

The framework may be stated in one sentence: it is a theory of how subject, meaning, value, creation, and normativity co-emerge through generative relational processes. Three consequences are used here.

The first is that what a relation produces is held by neither party. A working relation generates undertakings, difficulties, judgments, and understandings that neither participant would have produced alone and that neither possesses afterwards as a person possesses a memory of a fact.

The second is that access to what a relation generated is positional. Each party stands in the relation from somewhere, and that standpoint fixes what of the relation is available to them. The differences between standpoints run along several dimensions rather than along a single scale of completeness.

The third is that a person’s attributes, as they appear in a relation, are exhibited under the conditions that relation supplies. What a supervisor observes is what the supervisory relation afforded, under the resources, constraints, and demands that obtained.

None of this is original to the framework and the concession is made here rather than later. The account of conferred social properties holds that a status is imposed by others adopting an attitude toward a person, on the basis of a property the person need not in fact possess, and that such statuses consist in constraints and enablements, vary with context, and can be revoked (Ásta 2018). The general apparatus of status functions assigned collectively and holding in a context is older still (Searle 1995). And the priority of relations over the parties they relate is the founding commitment of relational sociology (Emirbayer 1997). What follows in this paper is an application of these positions to a particular problem, and the problem rather than the ontology is where the contribution lies.

Generation of the Relational Structure That Produces a Subject-Position

The framework asks a further question, and it is second-order with respect to the preceding subsection.

To say that a subject is constituted within a relational structure is to describe a synchronic relation between a person and a structure. The further question is by what process the structure that produces subjects of that kind was itself generated: what made available the positions that a person can now occupy, and what closed off others.

The question belongs to an established lineage and the paper claims no independence from it. The project of writing a history of the modes by which human beings are made subjects is stated in exactly those terms in the work that most influentially posed it, and the associated account holds that power operates by structuring the field of possible actions available to others (Foucault 1982). The claim that individuation precedes the individual, so that the individual is a provisional outcome of a process rather than its starting point, belongs to the same family (Simondon 2020).

An analytic distinction states the question more precisely than the paper could otherwise. What makes a social fact obtain in a particular case is one question; what sets up the frame principles under which facts of that kind are possible at all is another (Epstein 2015). The paper’s second-order question concerns the second, and the vocabulary is borrowed.

Two uses are made of this and both appear later. In Section 13.2 the institution that records a judgment is treated as participating in the generation of the position the person subsequently occupies, rather than as reporting on a position already held. And in Section 6.2 the instrument is treated as altering which positions are available, which is a claim about frame principles rather than about the person.

Dependence of an Attribute on the Relation That Exhibits It

The preceding subsections support a claim about attributes, and it is stated with the qualification the empirical literature requires.

Claim 1 (Relational exhibition). An attribute of a person, as reported in a judgment, is exhibited under the conditions the relation supplied. A report of it is therefore a report about what that relation produced, and its extension to relations supplying different conditions is an inference rather than an observation.

Claim 1 is weaker than the ontology alone would license, and the weakening is deliberate. It leaves open what the person carries across relations. The evidence indicates that they do: within-person distributions of behaviour across situations have central tendencies that are stable, even though the variability around them is very large (Fleeson 2001), and consistency recovered by aggregating across occasions is substantial (Kenrick and Funder 1988). What the claim asserts is that a single report from a single relation does not distinguish the stable central tendency from the variation, and that treating the report as though it did is the inference at issue.

The formulation that fits the claim most closely holds that what is consistent about a person is a profile of if-then contingencies rather than a level of a trait, so that an attribute appears when the conditions eliciting it are present (Mischel and Shoda 1995). On that account a report from one relation is a report of one arm of such a profile.

A development of that formulation states the claim in relational terms directly. On an interpersonal social-cognitive account of the self, knowledge about oneself is linked to knowledge about significant others, each linkage embodying a particular self-other relationship; variability across these relational selves depends on interpersonal contextual cues while their stability derives from the chronic accessibility of the corresponding representations; and the whole functions in the if-then terms of the account just cited, with relations supplying the conditions and relational selves the consequents (Andersen and Chen 2002). The underlying apparatus of relational schemas, in which representations of self, of other, and of the expected pattern of interaction between them are stored together and govern the processing of social information, is older (Baldwin 1992).

Two features of this literature bear on the paper and they run in the same direction. It locates the variation in the relation rather than in noise, which is what Claim 1 requires. And it locates the stability in the accessibility of representations carried by the person, which is consistent with the concession made above: something is carried, and a single report does not distinguish it from what the relation elicited.

A related result bears on the judging party rather than on the person judged. The accuracy of a judgment depends on whether the relevant cues were available, detected, and used, so that a judge’s access is a component of accuracy and not merely of confidence (Funder 1995). Section 12.2 uses this.

Criteria the Account Imposes on a Judgment of a Person

The account yields four criteria. They are stated here so that Section 8 through Section 14 assess judgments against a standard fixed in advance rather than against observations gathered afterwards.

1. Statement of the conditions of exhibition. A judgment should state the conditions under which what it reports was exhibited: the position the judging party occupied, the period, and the resources and constraints that obtained.

2. Restriction to the scope of access. A judgment should assert no more than the judging party’s position afforded, and should distinguish what was observed from what is inferred about relations the judging party did not enter.

3. Availability of the person’s own position. The person judged occupies a position no other party occupies, and a procedure that excludes their account excludes a standpoint rather than a partisan submission.

4. Provision for expiry and reopening. A judgment given durable institutional force should be renewable and revisable, so that it does not stand indefinitely on evidence generated at one time in one relation.

Two remarks about the criteria. They are requirements on the instrument and on the procedure, and none of them requires the judging party to be more diligent or more honest. And the fourth is not this paper’s invention: it is institutionalised in one of the two cases examined, and Section 10.3 describes it there.

What a Judgment of a Person Transmits

This section states the paper’s central ontological result. It proceeds from a case in which the usual account of these instruments cannot be correct, states what they transmit instead, and draws three consequences that the usual account leaves puzzling.

Absence of a Transmissible Object in the Signed Blank

Consider a practice that is neither rare nor regarded as scandalous by those who engage in it. A candidate asks a senior figure for an account. The reply is that the candidate should draft it and the senior figure will sign. Or the reply is that the candidate should send a curriculum vitae and a record of results, from which the account will be written.

Four features of the case matter.

The instrument functions. It is accepted by the receiving institution, weighed alongside others, and produces the same effects on the outcome as an account written from long acquaintance would.

No shared experience is being reported. In the second variant the signatory has observed nothing of the candidate at work; in the first the signatory has not even composed the sentences. Whatever the instrument conveys, it is not an account of what a relation between these two parties generated, because either no such relation exists or nothing of it entered the document.

The parties regard the practice as legitimate. It is common enough to have conventions, and those who use it describe it as a reasonable accommodation to the demands on senior people’s time.

And the receiving institution cannot distinguish this instrument from any other. Nothing in the document records how it was produced. That a practice of this kind persists inside institutions which require the instrument is what should be expected of a formal structure adopted for the legitimacy it confers and decoupled from the activity it purports to govern (Meyer and Rowan 1977).

The consequence is decisive for the account usually given of these instruments. If what they transmit were the content of a relation, then an instrument with no relation behind it would transmit nothing and would therefore do nothing. It does something. So what it transmits is not the content of a relation.

This bears on the preceding paper in this series, which read the requirement of several references as an attempt to reconstruct a co-experience from presentations taken at different standpoints. That reading holds where accounts exist to be reconstructed from. The present case shows that the instrument functions where they do not, so the reading describes what such instruments can do at their best rather than what they do as such.

Transmission of a Field of Relational Conditions

What such an instrument transmits can now be stated.

Claim 2 (Transmission of a field). An instrument of this kind transmits a field of relational conditions. It alters the set of relations available to the person in the receiving system, and it does so whether or not any account of a prior relation underlies it. The account, where one exists, is the justification offered for altering the set, and not the thing transmitted.

Claim 2 uses the vocabulary established across this series. A relational condition is an arrangement whose presence or absence changes which relations can be formed or continued without determining which of them are, and the field is the set of arrangements consistent with the conditions then in place. An admission, an appointment, or a place on a shortlist is a change in conditions of exactly this kind: it makes available relations with colleagues, supervisors, resources, and institutions that were previously unavailable, and it does so without determining which of them the person will form.

The claim is stated as a claim about what the instrument does rather than about what its users intend. Those who write and read such instruments ordinarily believe they are transmitting an assessment, and Section 6.4 explains why the belief is natural and why it survives.

Two neighbouring literatures come close to this and each stops short. Access to opportunities is shown to flow through ties that carry no detailed information about the person, which locates the mechanism in connection rather than in content (Granovetter 1973, 1974). And credentials are analysed as licences for entry to positions rather than as measures of the productivity they are taken to certify (Collins 1979). A third and closer neighbour distinguishes mentorship, which supplies guidance, from sponsorship, which supplies advocacy for advancement, and reports that only the second is associated with promotion, so that those who are well mentored and poorly sponsored advance less than those who are sponsored (Ibarra, Carter, and Silva 2010). That distinction is the closest existing statement of the difference between conveying an assessment and opening a position, and the present claim is an extension of it. Three differences remain: the source is a practitioner report rather than a study of the instrument’s mechanism; it treats sponsorship as advocacy by a party who does know the person; and it does not consider the case in which the advocacy proceeds without any observation at all. All three are about opportunity rather than information, and none states that the instrument functions when its informational content is null.

Reconfiguration of Generativity in the Receiving System

The consequence of a change in conditions is not itself an outcome. It is a change in what outcomes become possible.

A person admitted to a setting enters relations that did not exist before, and what those relations generate is produced within them, under the conditions that setting supplies. On the account of Section 5, the generativity at issue is a property of the relations and not of the person carried into them. The instrument therefore alters the configuration within which capacities are exercised, rather than transferring a capacity, and what follows is generated there.

Three implications are recorded and used later.

The receiving system’s own conditions determine much of what follows, so that the same person admitted to two settings may generate very differently in each, and neither result is evidence about a property the person carried. This is the ontological form of the observation in Section 3.3 that behaviour reflects a large person-by-situation interaction.

The instrument’s effect is therefore not well described as prediction. A judgment that a person will do well, followed by their doing well, is compatible with the judgment having been accurate and with the judgment having opened conditions under which doing well became possible. The outcome leaves the two indistinguishable, and Section 13.4 states why they cannot be distinguished by the available data either.

And an instrument that opens conditions for a person whom no institution could assess is doing something that no assessment could do. This is developed in Section 6.4 and is the reason the paper treats these instruments as a structure to be governed rather than as a pathology to be eliminated.

Separation of the Account from the Field It Opens

Claim 2 separates two things that ordinarily travel together: the account of a relation, and the alteration of conditions the account is offered in support of. Three features of practice follow from that separation, and each is otherwise puzzling.

An instrument with no account behind it functions.

This is the case of Section 6.1, and under Claim 2 it is expected rather than anomalous. What the receiving institution acts on is the alteration of conditions that a recognised signature authorises. The account is the stated ground, and the ground may be thin, formulaic, or absent without the authorisation failing.

The instrument is exchanged for things that are not accounts.

Where an instrument can be issued without content, it can be issued for consideration. The consideration may be an undertaking of future cooperation, access to resources the person will acquire once the conditions are opened, authorship, or continued association. Exchanges of this kind are recognisable to anyone who has been asked for one, and they follow from the instrument’s nature. They follow from it: an instrument whose force is separable from its content is an instrument whose force can be traded.

This paper records the exchange and leaves its analysis to the political-economy paper. The circulation of such instruments, the terms on which they are traded, and what accumulates to those who issue them are the subject of the political-economy paper in this series.

The instrument admits those whom no assessment could admit.

The same separation produces a consequence that counts in the instrument’s favour. A person whose work no receiving institution is placed to evaluate, whose field is unfamiliar, whose record is unconventional, or who comes from a setting the institution cannot read, cannot be admitted on the strength of an assessment, because no assessment is available. They can be admitted on the strength of an authorisation. An instrument that opens conditions without requiring assessable content is the mechanism by which the illegible enter systems that cannot read them.

A parallel case is instructive without being decisive. Lending against collateral admits only those who have collateral; lending on a group’s undertaking admits those who have none, and thereby moves capital to people no assessment reaches. The rigorous evaluations of that arrangement are sobering about what follows: borrowing rises and some business investment rises, while average consumption and several broader measures show no significant effect (Banerjee et al. 2015; Banerjee, Karlan, and Zinman 2015). The lesson the present paper takes concerns something other than microcredit. It is that opening conditions is a real and distinct thing to do, and that opening them does not by itself determine what is generated afterwards, which is precisely what Section 6.3 claims.

Claim 3 (Single property, two consequences). The separability of the instrument’s force from its content is one property. It is what allows an instrument to be traded for consideration other than an account, and it is what allows a person no institution can assess to be admitted. The two consequences are one feature of the instrument seen from two sides, rather than a defect and a benefit to be balanced.

Claim 3 constrains what a reform can achieve, and Section 14.4 states the constraint. A reform that ties the instrument’s force tightly to verified content removes the exchange and removes, by the same operation, the entry of those whom no verification reaches.

Measurement and Reification as Rival Diagnoses of Assessment

Criticism of the assessment of persons ordinarily proceeds from one of two claims, and the two are usually run together although they support different remedies and are true in different places. This section separates them and states where the boundary between them falls.

Assessment Treated as Measurement with Error

On the first claim, a judgment of a person is an estimate of a real quality that the person possesses, and the estimate is degraded by noise and by bias. The judging party observes imperfectly, brings expectations that colour what is observed, and reports in a register that inflates or that varies by group.

The remedies follow directly and are familiar. Structure the instrument so that every judging party answers the same questions. Train those who judge. Remove or suppress the cues that carry bias. Aggregate across judges so that independent errors cancel. Calibrate the judges against outcomes.

The claim is a serious position and a great deal of it is correct. The measurement literature has established the reliability properties of such judgments and the improvements that structure produces, and the professional standards governing assessment are largely constructed on this claim. A paper that treated it as an error would be wrong.

Assessment Treated as Reification of a Relational Product

On the second claim, the difficulty is not that the estimate is noisy but that there is no quantity of the assumed kind to estimate. What was observed was produced in a relation, under conditions that relation supplied, and the report converts it into an attribute the person is said to possess.

The remedies of Section 7.1 leave this untouched and can worsen it. A structured instrument obtains the same relationally produced observation from every judge in a comparable form, which makes the resulting figure more reliable without making it a measurement of anything the person carries. A consistent reading of a measurement model requires that the unobserved variable exist and stand in a causal relation to its indicators (Borsboom, Mellenbergh, and Heerden 2003); the reification claim is that in these cases it does not. Aggregation across judges who occupied similar positions compounds the same conditions rather than cancelling error. Calibration against outcomes calibrates against outcomes that were themselves produced under conditions the judgment helped to open, which Section 13.4 shows cannot be corrected from the available data.

The distinctive prediction of the second claim is therefore that improvement on the first claim’s terms will produce instruments that are more consistent, more defensible, and no better founded.

Domains in Which Each Claim Holds

Both claims are true, in different places, and the paper’s contribution here is the boundary rather than either claim.

The condition under which the first claim holds.

The measurement claim holds where three conditions obtain together: a construct exists that the person carries across settings; the assessments are genuinely parallel, in that each judging party is asked the same question about the same construct; and the conditions of exhibition are comparable across the occasions assessed. Where these hold, disagreement among judges is error, aggregation recovers a true score, and the remedies of Section 7.1 work.

That such conditions obtain for a range of attributes is the position of a substantial programme, which holds that apparent variation in validity across settings is an artifact of sampling error, unreliability, and range restriction, and which reports general mental ability as generalising across jobs (Schmidt and Hunter 1977, 1998; Schmidt et al. 1993). The paper concedes the programme’s domain. Where a construct is measured directly, with a standardised instrument, against a criterion recorded independently of the judgment, the reification claim has no application.

The condition under which the second claim holds.

The reification claim holds where the reported attribute was exhibited under conditions that the relation supplied and that are not reproduced elsewhere; where the judging parties occupied different positions and were therefore not answering the same question; and where the attribute reported is of the kind that appears when the eliciting conditions are present rather than at a level the person carries (Mischel and Shoda 1995).

The attributes that recommendation, appraisal, and endorsement most often report are of this kind. Collegiality, initiative, judgment under pressure, reliability under a particular supervisor, and fit lack standardised instruments and independently recorded criteria. They are reported by a party who stood somewhere, about conduct that occurred under conditions that party partly constituted.

Results that narrow the reach of the measurement claim.

Two findings recorded in Section 3.4 bear here and are used rather than merely noted. (1) The corrections that produced the strongest generalization estimates have been shown to overcorrect systematically, so that the validity of many selection procedures has been substantially overestimated and revised figures are materially lower (Sackett et al. 2022). (2) Situational specificity appears inside the instruments the programme cites, since in assessment centres the variance associated with the exercise routinely exceeds the variance associated with the dimension the exercise was built to measure (Lance 2008).

Neither result overturns the programme and the paper claims only that they narrow it. What they establish is that the domain in which the first claim holds is smaller than its strongest statements suggest, which is what the boundary argument requires.

What the empirical literature concedes in the other direction.

The paper owes a symmetrical concession. Within-person distributions of behaviour across situations have stable central tendencies (Fleeson 2001), and aggregation across occasions recovers substantial consistency (Kenrick and Funder 1988). Something is carried across relations. Claim 1 was formulated to accommodate this: it holds that a single report from a single relation does not distinguish what is carried from what the conditions elicited, not that nothing is carried.

The boundary can therefore be stated in one sentence. Where the construct is carried and the assessments are parallel, disagreement is error and should be aggregated away; where the attribute is elicited and the positions differ, disagreement is information about conditions and aggregating it away destroys what there was to know.

Scope of a Judgment and the Standards That Already Index It

This section states the paper’s central normative result. It begins from a principle that professional assessment has already established, states what must be added when the judgment is not a score, and answers the objection that the ontology destroys the criticism it was meant to support.

Validity of an Interpretation for a Specified Use

The principle the paper needs is already the professional consensus, and it is stated here before anything is claimed.

The governing standards hold that validity is the degree to which evidence and theory support the interpretations of results for proposed uses, and that validation consists in accumulating evidence for those proposed interpretations (American Educational Research Association, American Psychological Association, and National Council on Measurement in Education 2014). Two consequences are explicit in that formulation. Validity is a property of an interpretation for a use and not of an instrument, so that no instrument is valid as such. And the validation argument is bounded by the use for which it was built, so that using a result for a purpose outside that use is not a weaker version of the same warrant but a question on which the accumulated evidence is silent.

The unified conception behind the standards treats validity as one judgment supported by several kinds of evidence, and locates the consequences of use inside the validation argument rather than beside it (Messick 1989, 1995).

The paper therefore claims no novelty for the proposition that a judgment carries warrant for a specified use and not beyond it. That proposition is the settled position of the professions that assess persons formally. What the paper adds is stated next.

Relational Indexing of an Observation

The standards specify a scope by naming a use. The judgments this paper examines are neither scores nor administered for a specified use; they are accounts of what a relation produced, offered to whoever will read them. For such judgments the scope must be specified differently.

Claim 4 (Relational indexing). A situated judgment carries warrant with respect to the relation in which what it reports was exhibited. Its scope index is that relation: the position the judging party occupied, the period, and the conditions under which the conduct reported occurred. Warrant stops at relations supplying different conditions, and its extension beyond them is an inference requiring its own support.

Claim 4 is the standards’ principle with the relation substituted for the use, and three features distinguish it from a straightforward transposition.

A use is specified in advance by the party constructing the instrument, whereas a relation arises without any such specification. Nobody designs a supervisory relation as an assessment context, so the scope index must be recovered after the fact from the circumstances that obtained.

A use is a category and a relation is a particular. Two candidates assessed for the same use are assessed under the same specification; two candidates described by two supervisors were described from two positions, and the descriptions are not commensurable in the way two scores from one instrument are.

And a use has an associated body of validation evidence, while a relation has none. There is no accumulated evidence about what supervisory relations of a given kind warrant, and Section 16 records the construction of such evidence as the extension the account most needs.

Extrapolation beyond the Relation as a Scoping Error

Claim 4 identifies what goes wrong when a situated judgment is misused, and the identification matters because it selects the remedy.

The error is not that the judging party observed inaccurately. It may have observed with complete accuracy. The error is that a warrant holding with respect to one relation is exercised with respect to another, which is a mistake about scope rather than about accuracy.

Two consequences follow and they run in opposite directions.

The first restricts what may be inferred. A supervisor’s accurate report that a candidate worked poorly under close direction warrants nothing about how the candidate works under autonomy, and the inference to the second is unsupported however accurate the first is.

The second protects what may be inferred, and it is the reason the paper insists on the scoping formulation. A report within its scope retains its full force. The account does not hold that judgments of persons are worthless, that all reports are equally good, or that criticism of a report is impossible. It holds that a report warrants what its relation supports, which is a determinate quantity that can be exceeded, met, or fallen short of.

Answer to the Objection That the Account Disarms Criticism

The objection is familiar in form. If attributes are relative to relations, and no relation is privileged, then no report is better founded than another; the complaint against a bad report loses its ground; and any proposed reform lacks a criterion by which to prefer the reformed instrument.

The objection is a general one against constructionist accounts, which are regularly charged with removing the ground on which the criticism they motivate would stand (Hacking 1999). It would succeed against an account on which validity is absent. It does not succeed against an account on which validity is indexed.

Under Claim 4 a report can be assessed on three grounds that survive the ontology intact. It can be assessed for accuracy within its scope: whether what it reports of the relation is what occurred. It can be assessed for whether its scope is stated, since a report that conceals the conditions of exhibition prevents the reader from establishing its scope at all. And it can be assessed for whether it is used within its scope, which is a charge against the reader rather than the writer and is the principal charge this paper makes.

The three grounds also supply the criterion a reform requires. A reformed instrument is better if it states its conditions of exhibition, restricts its assertions to what the judging party’s position afforded, and is used within the resulting scope. These are the first two criteria of Section 5.4, and they are neither vacuous nor unmeasurable.

One residue survives the answer and is recorded rather than resolved. The account supplies no way to compare two reports from different relations when a decision must choose between the persons they concern. Selection requires comparison, comparison requires a common scale, and the account holds that the reports are not commensurable. Section 14.4 states what the account can and cannot offer a party who must nonetheless decide.

Professional Rules Governing Judgment without Examination

The proposal of Section 8 is not new to practice. Professions that judge persons formally have codified rules of exactly this shape, and this section examines them. Their existence is the paper’s principal evidence that a scope condition can be stated operationally rather than as an aspiration.

Limitation of Conclusions to the Access That Supports Them

The governing ethical standard for psychological assessment states the condition in three parts, and each part corresponds to something the present paper has argued for on other grounds.

The first requires that opinions contained in recommendations, reports, and diagnostic or evaluative statements rest on information and techniques sufficient to substantiate the findings (American Psychological Association, n.d.). This is a requirement about the relation between what is asserted and what supports it, and it is the professional form of the second criterion of Section 5.4.

The second is the one the paper most needs. Opinions about the psychological characteristics of individuals are to be provided only after an examination adequate to support the statements or conclusions; and where such an examination is not practical despite reasonable efforts, the psychologist is to document the efforts made and their result, to clarify the probable impact of the limited information on the reliability and validity of the opinion, and to limit appropriately the nature and extent of the conclusions or recommendations (American Psychological Association, n.d.). Three obligations are stated there and they map exactly onto this paper’s argument. The conditions of access are to be recorded; the consequence of those conditions for what the opinion is worth is to be stated; and the conclusions are to be cut back to what the access supports. That is Claim 4 written as a professional obligation.

The third governs the case in which no examination is warranted. Where a psychologist conducts a record review or provides consultation and an individual examination is not necessary for the opinion, the psychologist is to explain this and to state the sources of information on which the conclusions rest (American Psychological Association, n.d.). The rule therefore permits judgment at a distance. It requires that the distance be declared and that the sources be named, which is a scope statement rather than a prohibition.

Related guidance for forensic practice develops the same requirements for a setting in which opinions are given about persons who may not have been examined and in which the consequences are severe (American Psychological Association 2013).

Opinions about Persons Not Examined

A stricter rule exists in the neighbouring profession, and it applies to precisely the situation of the political case.

Psychiatric ethics provide that it is unethical for a psychiatrist to offer a professional opinion about an individual unless an examination has been conducted and proper authorization for the statement has been granted (American Psychiatric Association, n.d.). The rule was adopted after a magazine solicited psychiatric opinions about a presidential candidate whom the respondents had never met, and its scope was subsequently clarified to cover professional opinions generally rather than diagnoses alone (American Psychiatric Association, n.d.).

Two features of the rule bear on this paper.

The rule identifies the same defect the paper identifies, and identifies it as a defect of warrant rather than of accuracy. The objection concerns something other than whether opinions offered about unexamined public figures will be wrong. It is that nothing supports them, whatever their content, because the relation that would supply support was never entered.

And the rule is contested in a way that illuminates the paper’s own problem. Forensic practitioners routinely offer opinions from records without personal examination, so a rule requiring examination in all cases sits awkwardly with established practice, and the boundary between licensed and unlicensed judgment at a distance is disputed (Kroll and Pouncey 2016). The dispute is central to this paper; it is the paper’s question posed inside a profession that has had to answer it operationally. What the disputants disagree about is exactly where the scope of a judgment ends.

Features Common to the Rules Examined

Three features recur across the rules and each is used later.

(1) The rules regulate the scope of an assertion rather than the accuracy of an observation. None of them tells the professional what to conclude. They require that the conclusion not exceed what the access supports, which is the distinction of Section 8.3 in codified form.

(2) The rules require the conditions of access to be declared. The psychologist who cannot examine must document the efforts made and state the probable impact of the limitation (American Psychological Association, n.d.); the psychologist reviewing records must name the sources (American Psychological Association, n.d.). Declaration is treated as a substitute for access in respect of the reader’s ability to weigh the opinion, though not in respect of the opinion’s warrant.

(3) The rules bind the party who issues the judgment and not the party who uses it. This is their principal limitation for present purposes. A psychologist who correctly limits a conclusion has no control over how the limited conclusion is subsequently read, and nothing in these codes prevents a reader from treating a properly scoped opinion as though it were unscoped. Section 14.1 takes this as the gap the paper’s own proposal must address.

The three features together establish what the paper needed from this section. A scope condition of the kind Claim 4 states belongs to practice rather than to philosophical aspiration. It is written into the ethical codes of professions that assess persons under adversarial scrutiny, it is stated operationally enough to be enforced, and its disputed applications concern where the boundary falls rather than whether there is one.

Two absences are recorded. Conditions of applicability in actuarial risk instruments, which specify the reference populations for which a score is interpretable, would strengthen this section and are not cited, because the manuals were not verified for this draft. And the equivalent rules in other assessing professions, including auditing and educational measurement, await survey.

Case Study: Selection of Political Candidates

This section examines the case in which the judging party’s relational access is weakest and the consequences are largest. It is treated as a case and not as the paper’s frame; nothing here bears on the justification of representative government.

Judgment by an Electorate Standing in No Relation to the Candidate

The structural facts are these. Those who decide have not stood in any working relation to the person decided about. What is available to them is public performance, reported record, and the accounts of parties who claim closer acquaintance. The decision is consequential and is taken at scale.

Under Claim 1 the difficulty lies elsewhere than in the quality of the electorate’s judging. It is that most of what an electorate would need to know concerns conduct in relations it cannot enter: how the person behaves toward colleagues under pressure, whether commitments are kept when keeping them is costly, whether advice is taken. These are exhibited in relations, and the electorate occupies none of them.

The tradition has proposed three responses and each is a way of managing the absence of access rather than of supplying it.

The first is filtration through better-placed intermediaries, in which those with closer acquaintance select or screen, and the wider body chooses among what they present. The founding American statement of the case is explicit about its purpose: the sense of the people should operate in the choice, and that end is to be answered by committing the choice not to any pre-established body but to persons chosen for the special purpose, who will be most capable of analysing the qualities adapted to the station and best placed to possess the information and discernment such an investigation requires (Hamilton 1788). The argument is therefore not that the wider body judges well, but that a smaller body chosen for the occasion judges better, which is a claim about relational access rather than about legitimacy. The historical treatment shows that arrangements of this kind were adopted in preference to selection by lot and were understood at the time to introduce an aristocratic element, since election systematically selects those an electorate perceives as distinguished (Manin 1997).

The second is the substitution of a different question. Rather than assess qualities, an electorate may be understood to be identifying a type of person whose own motivations align with what the electorate wants, so that resources are invested in advance in selecting the type rather than afterwards in monitoring conduct (Mansbridge 2009). On that account the electorate attempts something other than the assessment the paper says it cannot perform, which is a partial answer to the difficulty rather than a solution to it, since identifying a type still requires evidence about the person.

The classical statement of why that response is available holds that a representative owes the constituents not industry alone but judgment, and betrays rather than serves them by sacrificing that judgment to their opinion (Burke 1774). If what is selected is a person whose judgment will thereafter be exercised rather than a programme to be executed, then the electorate’s task is the identification of a person rather than the assessment of a policy, and the difficulty this paper identifies applies to it with full force rather than being avoided by it.

The third is aggregation. Whether many individually poorly placed judges collectively judge well is disputed, and the classical passage on the question supports more than one reading (Waldron 1995; Ober 2013; Cammack 2013). The paper takes no position and notes that the empirical work is likewise mixed: poorly informed voters who read endorsement cues correctly can approximate the choices of the well informed (Lupia 1994; Popkin 1991), and heuristics can also mislead, unevenly across levels of sophistication (Lau and Redlawsk 2001).

Endorsement by Parties Claiming Relational Access

Endorsement is the political case’s instance of this paper’s object. A party that claims to have stood in a relation to the candidate issues a judgment, and parties that have not stood in any use it.

Three features distinguish endorsement from the academic instrument examined in Section 11, and the comparison in Section 12 develops them.

Endorsements are public. The endorser’s identity, the fact of the endorsement, and often its wording are available to everyone, including to the candidate and to those who would contest it.

Endorsers are numerous and heterogeneous. Colleagues, opponents, associations, former employers, and journalists all issue judgments about the same candidate, from positions that differ in the respects that would alter what each is placed to observe.

And endorsement is one source among others. Public performance supplies direct observation of a kind: how a person answers a hostile question, whether their account changes between audiences, how they conduct themselves under sustained scrutiny. This is thin evidence about conduct in working relations, and it is evidence the electorate gathers itself rather than receiving at second hand.

Revisability of the Mandate as the Historical Remedy

The feature of this case that the paper takes as its principal result is that the tradition solved a problem it could not solve.

It did not make electorates better judges of persons. The difficulty set out in Section 10.1 is unchanged: those who decide still lack access to the relations in which what matters is exhibited, and the responses catalogued there mitigate the absence without removing it.

What was built instead was temporal. The judgment expires. A mandate runs for a fixed period and must then be renewed; the person judged is subject to removal in the interval under specified conditions; and the judgment made at one time on inadequate evidence does not stand indefinitely on that evidence. Standard treatments of representation record the non-binding character of what is promised at election and the consequent centrality of the recurring decision (Manin 1997), and the conceptual literature distinguishes the several senses in which a representative may be said to stand for or act for those represented, each of which is renewed rather than settled once (Pitkin 1967).

The remedy is institutional and temporal rather than epistemic, and this is what makes it generalisable. It leaves the judging party’s knowledge as it stands. It requires that the judgment be provisional, that its renewal be scheduled rather than discretionary, and that the interval be short enough for the accumulated evidence of conduct in office to bear on the renewal.

Two limits of the remedy are recorded and the paper claims for it only what they permit. The renewal is made by the same body with the same lack of access, so what improves is the number of occasions rather than the quality of any single judgment, on which a judgment can be corrected. And the evidence available at renewal is evidence of conduct in the office, which is a relation the electorate still does not occupy but whose products are at least public.

Case Study: Recommendation in Academic Selection

This section examines the case in which the issuing party’s relational access is strongest and most private. It is examined here for what the judgment warrants; the procedure by which such judgments are collected and read was examined in the preceding paper in this series.

Judgment by Few Issuers with Private Access

The structural facts differ from the political case at almost every point.

The issuers are few, ordinarily two or three, and are nominated by the person judged. Their access is genuine and often extensive: a supervisor may have observed the candidate over years, in conditions no other party witnessed.

That access is private. What the referee observed is available to no one else, including to the receiving institution, which cannot inspect the relation and must take the account on the referee’s authority.

The account is ordinarily confidential to the person judged, so that the party whose trajectory it governs remains ignorant of its contents.

The instrument’s measured performance is correspondingly weak: accounts of this kind relate positively but weakly to later outcomes (Kuncel, Kochevar, and Ones 2014), and agreement between two referees describing one candidate is lower than agreement between two accounts written by one referee about different candidates (Aamodt 2006). And the weight the account carries tracks the referee’s standing in a field rather than the extent of the referee’s access to the candidate, for the reasons developed in the preceding paper: standing and access are generated in different relations and nothing correlates them.

Under Claim 4 the difficulty here is the mirror image of the political one. In the political case the judging party lacks access and knows it. Here the issuing party has access, and the receiving party cannot establish what that access was, because the conditions of exhibition are not recorded and the account is read as though it reported a property rather than a relation.

Issuance without an Underlying Relation

The limiting case examined in Section 6.1 belongs to this case and is restated here for what it shows about the regime rather than about the ontology.

Where an account is drafted by its own subject and signed, or written from a curriculum vitae by a signatory who observed nothing, the instrument still functions. Nothing in the receiving institution’s procedure distinguishes such an instrument from one written from long acquaintance, because nothing in the document records how it was produced.

Two features of the issuance regime make this possible and neither is present in the political case. The account is private, so no party who could contradict it sees it. And there is no record of the issuer’s past accounts against which the present one might be read, so an issuer whose accounts are uniformly favourable, or uniformly produced without observation, is indistinguishable from one whose accounts discriminate.

Absence of Revision after the Judgment Is Acted On

The feature that most sharply distinguishes this case is what happens after the decision.

Nothing expires. An account written at one time, about a relation obtaining at that time, under conditions then in force, is acted on and then remains in a file. There is no scheduled occasion on which it is renewed, no procedure by which the person judged may seek its reconsideration, and no mechanism by which subsequent conduct bears on it. The judgment that governed an admission is not revisited when the admitted person’s work is observed directly, and the judgment that governed a rejection is not revisited at all.

This is the fourth criterion of Section 5.4 entirely unmet, in a case where the first three are also unmet, and it stands in the sharpest possible contrast with the case examined in Section 10.3, where the same criterion is the constitutive feature of the arrangement.

Comparison of the Two Issuance Regimes

The two cases share the structure this paper examines and differ in the regime under which judgments are issued. This section states the difference along five dimensions and draws the consequence.

One preliminary. The contrast is drawn through the regime rather than through the motives of those who judge. Votes are cast for reasons of patronage, inheritance, identity, and habit as much as for any assessment of a candidate, and accounts of candidates are written for reasons of obligation, convenience, and expected return as much as for any assessment. Motive leaves the cases indistinguishable, and a comparison resting on it would be sentimental about one of them.

Distribution of Standing to Issue a Judgment

In the political case the standing to issue a judgment is distributed as widely as the franchise, and each issuer’s judgment carries the same weight as any other’s. Endorsers who claim relational access are numerous, arise without appointment, and compete with one another.

In the academic case the standing to issue is held by few, arises without any distributing rule, and carries weight in proportion to the issuer’s position in a field. The candidate selects among those who hold it, which distributes access to issuers unequally in a way the preceding paper examined and this one does not. Selection on the receiving side compounds this, since employers have been found to weigh cultural similarity heavily and at times above productivity (Rivera 2012).

Observational Access Available to the Issuer

The dimension on which the cases are most opposed is access, and they are opposed in both directions at once.

The political issuer has weak access and it is public. What an endorser knows of a candidate is often slight, and what any voter can observe directly is thinner still; but the observations are available to everyone, can be checked against one another, and can be contested by parties who observed the same events.

The academic issuer has strong access and it is private. A supervisor may know a great deal that no one else knows, and precisely because no one else knows it, no one can establish what was known or whether the account reports it.

A related consideration bears on the weight attaching to prominent issuers. Across a large body of recorded predictions, expert forecasters were often only slightly more accurate than chance, and the most visible and most confident were among the worst calibrated (Tetlock 2005). Since the accuracy of a judgment depends on whether the relevant cues were available, detected, and used (Funder 1995), the political issuer is worse placed to judge and better placed to be checked, and the academic issuer is better placed to judge and cannot be checked at all.

Transferability of a Judgment between Holders

A vote resists transfer. It cannot be sold, lent, or exchanged for a future undertaking; the prohibition is enforced; and an attempt to transfer it is a recognised offence rather than an accommodation.

An account is effectively transferable, in the sense developed in Section 6.4. Its force is separable from its content, which is what allows it to be given in exchange for consideration other than an account, and what allows it to be issued where no account exists.

This is the dimension on which the difference is starkest and on which the academic case has no protection whatever, since the exchange leaves no trace in the document and the document is not seen by anyone who could object.

Provision for Revision

In the political case revision is scheduled, guaranteed, and constitutive: the judgment expires and must be renewed, and provision exists for removal in the interval.

In the academic case there is no provision at all. The judgment is acted on once and is neither renewed nor reopened.

Table [tab:regimes] records the four dimensions together with a fifth, the weight attaching to a single issuer’s judgment.

@P0.24YY@ Dimension of the regime & Political selection & Academic recommendation
Standing to issue a judgment & Distributed as widely as the franchise; endorsers numerous and competing & Held by few; not distributed by rule; issuers nominated by the person judged
Weight attaching to one issuer & Equal across issuers, and fixed & Proportional to the issuer’s standing in a field
Observational access of the issuer & Weak, and public: checkable by others who observed the same events & Strong, and private: not establishable by any other party
Transferability of the judgment & Prohibited and enforced & Separable from its content, and exchangeable without trace
Provision for revision & Scheduled, guaranteed, and constitutive of the arrangement & Absent

Findings of the Comparison

Three findings follow and they are the section’s purpose.

(1) The two regimes trade access against checkability, and they trade it in opposite directions. Political selection accepts very weak access in exchange for observations that many parties can check against one another. Academic recommendation obtains strong access at the cost of observations that no party can check. The paper leaves open which trade is the better one; what the comparison establishes is that the trade is being made and is not acknowledged in either case.

(2) The dimension on which one case has a protection and the other has none is revision, and it is the dimension least connected to the quality of any single judgment. Political selection did not obtain better judgments. It obtained the guarantee that a judgment would not stand indefinitely, and it obtained that guarantee by institutional means available to any domain that records judgments of persons.

(3) The academic case has no protection on any dimension where the political case has one. Its issuers are few and unequally weighted, its access is unverifiable, its judgments are exchangeable, and its judgments do not expire. The paper draws no conclusion about which arrangement is better overall, since the cases differ in purpose, scale, and consequence; it draws the narrower conclusion that a remedy exists in one case, that it is institutional rather than epistemic, and that its absence from the other is not explained by any feature of the other.

Fixation of an Interpretation and Its Return upon the Person

A judgment formed in a relation does not remain a judgment. It is written down, entered into a record, and acted on, and what follows is the subject of this section. The section is written under a restriction stated in advance: the literatures bearing on it report effects that are small, conditional, or unreplicated, and the argument accordingly rests on the mechanism they support rather than on the mechanism they do not.

Selection of One Interpretation from Several Available

What a relation generated ordinarily admits of more than one account, and the account written is one of them.

A period of difficult work may be reported as evidence of a candidate’s limitations under pressure, as evidence of an unreasonable set of demands, or as evidence of a productive struggle from which something was learned. Each may be an accurate report of what occurred; they differ in what they take the occurrence to have been an instance of. Under Claim 1 this is what should be expected, since the conduct was exhibited under conditions the relation supplied and the account must say something about how those conditions bore on it.

Two features of the selection matter. It is made by one party, since the person judged stands outside it. And it is made once, at the moment the account is written, under whatever understanding of the relation then obtained.

Persistence of a Fixed Interpretation in an Institutional Record

Once written and acted on, the selected interpretation acquires properties the underlying judgment did not have.

It persists. The relation may end, the conditions may change, and the parties may revise their understanding of what occurred; the record stands unchanged.

It becomes a fact of a different kind. What was one party’s account of a relation becomes, in the receiving institution’s file, a datum about a person, detached from the standpoint that produced it and from the conditions under which the conduct occurred.

And it is inherited. Subsequent parties encounter the record rather than the relation, and their own judgments are formed with it already in place. The record is in this respect an inscription that travels while holding its shape, so that what circulates is the document and not the circumstances that produced it (Latour 1986); where such records are made comparable on a common scale, what the scale does not carry becomes invisible rather than acknowledged as missing (Espeland and Stevens 1998); and the resulting ordinal positions shape the life-chances of those ranked (Fourcade and Healy 2013).

A further consequence attaches only where the record is public. Parties who are measured alter their conduct in response to being measured, through self-fulfilling prophecy and through commensuration, so that a public measure changes the world it reports (Espeland and Sauder 2007). The records examined here are ordinarily confidential, which removes this consequence and removes with it any occasion on which the measure might be contested.

The literature on classification supplies the general form of what follows. Classifications of people and the people classified emerge together and interact, so that a classification alters the space of possibilities for being a person of the kind classified (Hacking 1986); those classified become aware of the classification and respond to it, so that the kinds are moving targets (Hacking 1995, 2007). This anticipates the present subsection and is conceded.

Alteration of the Relational Conditions Available to the Person

The mechanism by which a record returns upon the person it describes is the one this paper’s own account predicts, and it is also the one the empirical literature supports.

A record governs admission. Admission alters which relations the person can form, as Section 6.2 argued of the instrument generally. What the person subsequently generates is produced in the configuration the record helped to establish, and would have been produced differently in the configuration the record foreclosed.

The sociological literature reached the same mechanism by revision rather than by construction, and its history is the reason the paper relies on it. The strong version of labelling theory, on which a label produces the behaviour it names, was not supported by the evidence, and the theory was modified by its own proponents into a form whose operative mechanisms are status loss, altered relationships, exclusion, and reduced opportunity (Link et al. 1989); the consolidated statement treats stigma as the co-occurrence of labelling, stereotyping, separation, status loss, and discrimination under conditions of power (Link and Phelan 2001).

The mechanism the paper does not rely on should be named as plainly. It stops short of claiming that a recorded judgment alters the person’s dispositions through the expectations of those who read it. Effects of that kind are found, and they are typically small, do not accumulate greatly across perceivers or over time, may be more likely to dissipate than to accumulate, and are accompanied by the finding that expectations predict outcomes more because they are accurate than because they are self-fulfilling (Jussim and Harber 2005; Raudenbush 1984). A related literature on performance under stereotype salience has a contested replication record (Flore and Wicherts 2015; Stoet and Geary 2012). The paper’s argument requires none of this, and a reader should not supply it.

A discipline from the performativity literature applies here and is adopted. A description contributes to bringing about what it describes only under conditions, and the case in which this is best documented turned on the fact that acting on the description did not generate forces that would have destroyed it (MacKenzie 2006). The present claim is correspondingly bounded: a record alters what a person can do by altering what is available to them, and the paper asserts no further mechanism.

Unfalsifiability of Outcome Data under Selective Observation

The final consequence concerns what can be learned about the judgment afterwards, and it is the strongest result in this section because it is formal.

Where a judgment governs whether a person is admitted, outcomes are generated only for those admitted. The performance of those excluded goes unobserved, since they were never admitted, and the excluded may differ systematically from the included on characteristics no one recorded.

The formal statement is established. Outcomes are observed only for the cases a decision-maker passed through, so a candidate decision rule cannot be evaluated against the outcomes of cases it would have treated differently, and the unobserved cases may differ on unrecorded characteristics (Lakkaraju et al. 2017); applied to release decisions, outcomes exist only for those released and not for those detained, which makes the evaluation of a counterfactual rule require explicit construction rather than direct observation (Kleinberg et al. 2018).

Three consequences follow for the warrant of a situated judgment.

(1) Favourable outcomes among the admitted do not establish that the judgments were sound. They are compatible with the judgments having been accurate and with the judgments having opened conditions under which the outcome became attainable, and Section 6.3 argued that the second is what the instrument does.

(2) The absence of adverse outcomes among the excluded is not evidence, because there are no outcomes among the excluded. An institution that has never admitted a candidate of a certain description has no basis for its belief about how such candidates perform, and its confidence in that belief grows with the length of time it has held it.

(3) Calibration of the judging parties against outcomes inherits the problem. A proposal to score referees or endorsers against how those they described subsequently performed can score them only on the admitted, which is the sample the judgments themselves selected. The proposal retains some value, and Section 14.4 states what it can and cannot deliver.

Consequences for What May Be Claimed of a Person

This section states what follows for practice. Three proposals are made and the third is borrowed. A fourth subsection states what the account cannot supply.

Relation-Indexed Statement of an Observation

The first proposal follows from Claim 4 and from the first two criteria of Section 5.4.

A judgment of a person should state the relation with respect to which it carries warrant: the position the judging party occupied, the period, the conditions under which the conduct reported occurred, and the respects in which the judging party’s view was partial. It should distinguish what was observed from what is inferred about relations the judging party did not enter.

Section 9 established that requirements of this shape are already codified. The psychologist who cannot examine must document the efforts made, state the probable impact of the limitation on the reliability and validity of the opinion, and limit the conclusions accordingly (American Psychological Association, n.d.). What the present proposal adds is the substitution of the relation for the examination as the thing to be declared.

The gap identified in Section 9.3 bears directly on what such a proposal can achieve. These rules bind the party who issues a judgment and not the party who uses it, and a properly scoped statement can be read as though it were unscoped. The proposal therefore has a second limb which is the one that matters: the receiving party is obliged to establish the scope before relying on the judgment, and to treat a judgment whose conditions of exhibition are not stated as one whose scope cannot be established. Under Section 8.3 the principal error is the reader’s, and the principal obligation is correspondingly the reader’s.

Expiry and Reopening of a Fixed Interpretation

The second proposal is a generalisation of established practice rather than a novelty, and the paper states it that way.

A judgment of a person that is given durable institutional force should expire. It should be renewable on a scheduled occasion rather than at the discretion of those who hold it, and the person judged should have a route by which its reconsideration may be sought in the interval.

That is what political selection built, as Section 10.3 described: fixed terms, scheduled renewal, and provision for removal. The arrangement did not improve the electorate’s access to the relations in which what matters is exhibited. It ensured that a judgment made on inadequate evidence would not stand indefinitely on that evidence.

Three features of the generalisation are worth stating.

It addresses the difficulty the account identifies rather than the difficulty the measurement claim identifies. It leaves the accuracy of judgments where it found it, and attempts nothing there. It bounds the period over which a judgment formed in one relation governs a person’s trajectory.

It is available to any domain that records judgments of persons, since every capacity it requires is one that recording a judgment already requires.

And it is the only one of the paper’s proposals that survives Section 13.4. A proposal to improve judgments must be evaluated against outcomes, and the outcomes are selectively observed. A proposal that judgments expire requires no such evaluation, since its justification is that the evidence supporting a judgment ages rather than that the judgment was poor.

Availability of the Position of the Person Judged

The third proposal follows from the third criterion of Section 5.4 and is stated briefly because the preceding paper in this series develops it.

The person judged occupied a position in the relation that no other party occupied, and holds an account of the conditions under which the conduct reported occurred. A procedure that excludes it excludes a standpoint. Where the account is collected, it should be collected as a source and read against the others rather than as a submission answering them, and it should be collected before the other accounts arrive so that it cannot be composed in response to them.

The proposal bears on the ontology as well as on fairness. Under Claim 4 what a receiving party most needs, and least possesses, is the conditions of exhibition; and the person judged is one of the two parties in a position to state them.

Limits of the Remedies Proposed

Four limits are recorded and none is answered.

Absence of a procedure for comparison.

Selection requires choosing among persons, choosing requires comparison, and comparison requires a common scale. The account holds that accounts from different relations resist commensuration, and it supplies no procedure for comparing them. What it supplies to a party who must nonetheless decide is narrower: a reason to treat the comparison as an act performed by the deciding party on incommensurable materials, rather than as a reading of a quantity the materials contain, and therefore a reason to record how the comparison was made.

Tying force to content removes what the instrument does best.

Claim 3 holds that the separability of an instrument’s force from its content is one property with two consequences. A reform that tied force tightly to verified content would suppress the exchange of instruments for consideration, and would by the same operation close the route by which persons no institution can assess are admitted. The paper declines to propose such a reform and records that any proposal in that direction incurs this cost.

Calibration inherits the selection problem.

Scoring the parties who issue judgments against subsequent outcomes can score them only on those admitted, for the reasons given in Section 13.4. Such a record would establish whether an issuer discriminates among the candidates who were admitted, which is worth knowing and is less than it appears.

Declaration as a substitute for access.

Every proposal in this section improves what a receiving party knows about the conditions under which a judgment was formed. None of them gives the receiving party access to the relation. The account holds that such access is unobtainable in principle, since the relation is over and the receiving party was not in it, and the proposals accordingly improve the handling of a limitation rather than removing it.

Implications for the Generative Relational Framework

Four results return to the framework and two of them are corrections.

Fields as Objects of Transmission

The framework has treated a field of relational conditions as something a party maintains, governs, or damages. Claim 2 adds that a field can be transmitted: an instrument issued in one system can alter the conditions obtaining in another, and can do so without conveying any content about the system it was issued in.

Two consequences follow for the framework’s own vocabulary. The governance of a field, which the first paper in this series treated as a matter of what a custodian does within a system, must also cover what enters a system from outside it. And the instruments by which fields are altered across system boundaries are a class the framework has not identified, of which recommendation, credentialling, licensing, and admission are members.

Separability of an Authorisation from Its Justification

Claim 3 states a structural feature that the framework should carry beyond this paper. Where an instrument’s force is separable from the account offered in its support, the instrument can be issued without the account and can therefore be exchanged for other things; and by the same property it can admit those for whom no account could be constructed.

The framework’s normative vocabulary has tended to treat the alienation of a relational product as a defect. This case indicates that the same structural property produces the defect and the benefit, and that a framework which condemns the first without noticing the second will recommend reforms that close entry to those who most need it. That is a general caution and not a feature of recommendation.

Revisability as an Institutional Rather Than an Epistemic Remedy

The framework treats revisability as a requirement on interpretations. This paper finds it institutionalised, in one domain, as a remedy for a problem that resisted epistemic solution, and finds that its justification there does not depend on the judgment having been poor.

Two things follow. The framework’s revisability requirement acquires a precedent of the first rank, in constitutional practice, which is stronger support than a philosophical argument would supply. And the framework acquires a distinction it has not drawn: between revisability sought because a judgment may be wrong, and revisability sought because the evidence supporting a judgment ages. The second justification is available even where the judgment was as good as it could have been, and it is the one Section 14.2 relies on.

Limits of the Framework’s Reach into Assessment

The framework’s ontology entails a claim about attributes that the empirical literature only partly supports, and the correction should be carried back.

Behaviour reflects person, situation, and a large interaction between them; within-person distributions across situations have stable central tendencies; and aggregation across occasions recovers substantial consistency (Fleeson 2001; Kenrick and Funder 1988; Fleeson and Noftle 2009). A framework holding that attributes are generated in relations must therefore be stated in the weaker form of Claim 1, which concerns what a single report from a single relation can distinguish, rather than in a form that denies that anything is carried.

The general lesson is that the framework’s ontological commitments have empirical consequences in domains where those consequences have been measured, and that it should be stated in forms that survive the measurement.

Limits of the Account

Conditions of Falsification

Section 4.3 stated five conditions and their status is as follows.

Whether attributes are exhibited at similar levels across relations differing in the respects the account identifies is settled in favour of the account for some attributes and against it for others, and the boundary of Section 7.3 is argued rather than demonstrated. The paper concedes the domain of the validity-generalization programme (Schmidt and Hunter 1998) and relies on two results that narrow it (Sackett et al. 2022; Lance 2008). No study on the attributes the paper is most concerned with is available.

Whether an instrument of the kind examined in Section 6 functions where no account underlies it is asserted from a practice the paper describes and does not document. No systematic evidence about the prevalence or the effects of instruments issued without observation was located, and the argument would be strengthened considerably by any.

Whether the five dimensions of an issuance regime track differences in outcome is untested. Section 12 establishes that the two cases differ along them and leaves open whether the differences matter.

Whether judgments that expire perform better than judgments that stand is untested, and the paper notes that domains exist in which both arrangements are found.

And whether the scope condition can be stated operationally is answered affirmatively by Section 9, which finds it stated operationally in professional codes (American Psychological Association, n.d.). This is the condition the paper meets most securely.

Claims Advanced Without Support

Three claims rest on argument alone.

Claim 2, that such instruments transmit a field of relational conditions, is argued from the case of the instrument issued without observation and from the vocabulary of the preceding papers. Evidence about what receiving institutions do with such instruments would be required to establish it.

Claim 3, that one property produces both the exchange and the admission of the unassessable, is a structural argument. The parallel case cited in support concerns lending rather than judgment (Banerjee et al. 2015).

And the account of what political selection achieved, in Section 10.3, is a reading of an institutional arrangement rather than a finding about it. Others read the same arrangement differently, and the treatment relied on here itself emphasises the aristocratic element in election (Manin 1997).

Sources Not Yet Verified

This draft cites only sources verified against a publisher, journal, index, or institutional page before the section using them was written, and several literatures are consequently represented thinly or not at all.

The political-theory section rests on three treatments (Manin 1997; Pitkin 1967; Mansbridge 2009) together with two primary statements (Hamilton 1788; Burke 1774) and the accountability programme (Przeworski, Stokes, and Manin 1999; Manin, Przeworski, and Stokes 1999; Fearon 1999). The nineteenth-century arguments about the qualifications of electors and representatives are absent, as is any citable edition of the classical text whose disputed reading Section 3.1 records. The section remains a positioning of the paper’s question rather than a survey of the tradition.

Conditions of applicability in actuarial risk instruments are absent from Section 9, which weakens the precedent that section supplies and is now the principal omission of this kind.

And the literature on sponsorship as distinct from mentorship, which is the closest existing treatment of an instrument that opens opportunity rather than conveying information, is absent from Section 6.2, which is the paper’s most novel claim and therefore the place where an unacknowledged neighbour would matter most.

Extensions

Four extensions are identified and none is attempted.

The construction of validation evidence for classes of relation would supply what Section 8.2 identified as missing: a use has an associated body of evidence about what it warrants, and a relation has none. Evidence about what supervisory, collaborative, or adversarial relations typically afford would make relational indexing operable rather than merely statable.

An empirical study of the two-claims boundary would settle Section 7.3, and is feasible, since the prediction is that structuring and aggregating judgments of elicited attributes will improve consistency without improving foundation.

A treatment of algorithmic assessment of persons is absent from this paper and is the most consequential omission, since such systems fix interpretations at scale, are unusually resistant to expiry, and are evaluated against outcomes that are selectively observed in exactly the way Section 13.4 describes.

And a design for scheduled expiry in a non-political domain, specifying who initiates renewal, on what evidence, and with what effect on the intervening period, is what Section 14.2 proposes and does not supply.

Conclusion

A judgment of a person is formed inside a relation and used outside it. That is the situation of a letter, a rating, a formulation, and a vote, and this paper has asked what such a judgment warrants once it has left the relation that produced it.

The question is old and the paper claims no novelty in asking it. What it adds is that the tradition asked who should judge and how often the judgment should be renewed, and did not ask what the judgment is a judgment of. Under the commitment used here, an attribute reported in such a judgment was exhibited under conditions the relation supplied, so that a report of it is a report about what that relation produced. The claim was stated in a form the empirical literature permits: not that nothing is carried across relations, but that a single report from a single relation does not distinguish what is carried from what the conditions elicited.

Two claims about the assessment of persons were separated. On one, a judgment is a noisy estimate of a real quality and the remedies are structure, aggregation, and calibration. On the other, there is no quantity of the assumed kind, and those remedies produce more reliable estimates of nothing. Both hold, in different places, and the boundary was stated: where the construct is carried and the assessments are parallel, disagreement is error and should be aggregated away; where the attribute is elicited and the positions differ, disagreement is information about conditions and aggregating it away destroys what there was to know.

A limiting case then established something the usual account of these instruments cannot accommodate. A referee signs an account drafted by its subject, or writes from a curriculum vitae, and the instrument functions exactly as it would otherwise. If what such instruments transmitted were the content of a relation, an instrument with no relation behind it would do nothing. It does something. What it transmits is a field of relational conditions: it alters the relations available to the person in the receiving system, and what follows is generated there. The account is the justification offered for altering them, not the thing conveyed. From this one property come both the exchange of such instruments for undertakings that are not accounts, and the admission of persons for whom no account could be constructed. These are one feature seen from two sides rather than a defect and a benefit to be balanced, and a reform that removed the first would remove the second.

The scope condition the paper proposes is already familiar in practice. Professional codes already require that an opinion about a person rest on adequate examination, that the impact of limited access be stated, and that conclusions be limited to what the access supports. What the paper adds is the substitution of the relation for the examination as the thing to be declared, and the observation that these codes bind the party who issues a judgment and leave the party who uses it unconstrained, although the principal error is the user’s.

Two cases were compared through the regime under which judgments are issued rather than through the motives of those who issue them. They are mirror images: political selection accepts very weak access in exchange for observations many parties can check, and academic recommendation obtains strong access at the cost of observations no party can check. On four of five dimensions the academic case has no protection where the political case has one, and the dimension on which the difference is starkest is the one least connected to the quality of any single judgment.

That dimension supplies the paper’s proposal, and the proposal is borrowed. Political selection did not solve the problem of judging a person one cannot know. It made the judgment expire. Fixed terms, scheduled renewal, and provision for removal are institutional and temporal devices that leave the epistemic difficulty untouched and prevent a judgment formed at one time on inadequate evidence from governing indefinitely. This is the only proposal in the paper that survives the finding that outcomes are observed only for those selected, because its justification is that evidence ages rather than that a judgment was poor.

What the account withholds is recorded. It offers no way to compare accounts from different relations when a decision must choose between the persons they concern. It cannot recommend tying an instrument’s force to verified content without closing the entry it also enables. It cannot rescue calibration from the selection problem. And none of its proposals gives a receiving party access to the relation, which is unobtainable in principle, since the relation is over and the receiving party was not in it. The proposals improve the handling of a limitation and leave the limitation in place.

Acknowledgments

The present definitions, constructions, arguments, conclusions, and errors remain the author’s responsibility. The interest arising from the author’s own position with respect to the procedures examined in Section 11 is declared in the front matter.

Aamodt, Michael G. 2006. “Validity of Recommendations and References.” Assessment Council News, February, 4–6.

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. 2014. Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.

American Psychiatric Association. n.d. “The Principles of Medical Ethics with Annotations Especially Applicable to Psychiatry, Section 7.3.” American Psychiatric Association.

American Psychological Association. 2013. “Specialty Guidelines for Forensic Psychology.” American Psychologist 68 (1): 7–19.

———. n.d. “Ethical Principles of Psychologists and Code of Conduct, Standard 9.01: Bases for Assessments.” American Psychological Association.

Andersen, Susan M., and Serena Chen. 2002. “The Relational Self: An Interpersonal Social-Cognitive Theory.” Psychological Review 109 (4): 619–45. https://doi.org/10.1037/0033-295X.109.4.619.

Ásta. 2018. Categories We Live by: The Construction of Sex, Gender, Race, and Other Social Categories. Studies in Feminist Philosophy. New York: Oxford University Press.

Baldwin, Mark W. 1992. “Relational Schemas and the Processing of Social Information.” Psychological Bulletin 112 (3): 461–84.

Banerjee, Abhijit, Esther Duflo, Rachel Glennerster, and Cynthia Kinnan. 2015. “The Miracle of Microfinance? Evidence from a Randomized Evaluation.” American Economic Journal: Applied Economics 7 (1): 22–53.

Banerjee, Abhijit, Dean Karlan, and Jonathan Zinman. 2015. “Six Randomized Evaluations of Microcredit: Introduction and Further Steps.” American Economic Journal: Applied Economics 7 (1): 1–21.

Becker, Howard S. 1963. Outsiders: Studies in the Sociology of Deviance. New York: The Free Press.

Borsboom, Denny, Gideon J. Mellenbergh, and Jaap van Heerden. 2003. “The Theoretical Status of Latent Variables.” Psychological Review 110 (2): 203–19. https://doi.org/10.1037/0033-295X.110.2.203.

Burke, Edmund. 1774. “Speech to the Electors of Bristol, 3 November 1774.” In The Founders’ Constitution, 1:391. University of Chicago Press.

Callon, Michel, ed. 1998. The Laws of the Markets. Oxford: Blackwell.

Cammack, Daniela. 2013. “Aristotle on the Virtue of the Multitude.” Political Theory 41 (2): 175–202.

Collins, Randall. 1979. The Credential Society: An Historical Sociology of Education and Stratification. New York: Academic Press.

Crossley, Nick. 2011. Towards Relational Sociology. London: Routledge.

Donati, Pierpaolo. 2011. Relational Sociology: A New Paradigm for the Social Sciences. London: Routledge.

Elashoff, Janet D., and Richard E. Snow. 1971. Pygmalion Reconsidered. Worthington, OH: Charles A. Jones.

Emirbayer, Mustafa. 1997. “Manifesto for a Relational Sociology.” American Journal of Sociology 103 (2): 281–317. https://doi.org/10.1086/231209.

Epstein, Brian. 2015. The Ant Trap: Rebuilding the Foundations of the Social Sciences. New York: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199381104.001.0001.

Espeland, Wendy Nelson, and Michael Sauder. 2007. “Rankings and Reactivity: How Public Measures Recreate Social Worlds.” American Journal of Sociology 113 (1): 1–40. https://doi.org/10.1086/517897.

Espeland, Wendy Nelson, and Mitchell L. Stevens. 1998. “Commensuration as a Social Process.” Annual Review of Sociology 24: 313–43. https://doi.org/10.1146/annurev.soc.24.1.313.

Fearon, James D. 1999. “Electoral Accountability and the Control of Politicians: Selecting Good Types Versus Sanctioning Poor Performance.” In Democracy, Accountability, and Representation, edited by Adam Przeworski, Susan C. Stokes, and Bernard Manin. Cambridge University Press.

Fleeson, William. 2001. “Toward a Structure- and Process-Integrated View of Personality: Traits as Density Distributions of States.” Journal of Personality and Social Psychology 80 (6): 1011–27.

Fleeson, William, and Erik E. Noftle. 2009. “In Favor of the Synthetic Resolution to the Person-Situation Debate.” Journal of Research in Personality 43: 150–54.

Flore, Paulette C., and Jelte M. Wicherts. 2015. “Does Stereotype Threat Influence Performance of Girls in Stereotyped Domains? A Meta-Analysis.” Journal of School Psychology 53 (1): 25–44.

Foucault, Michel. 1982. “The Subject and Power.” Critical Inquiry 8 (4): 777–95. https://doi.org/10.1086/448181.

Fourcade, Marion, and Kieran Healy. 2013. “Classification Situations: Life-Chances in the Neoliberal Era.” Accounting, Organizations and Society 38 (8): 559–72. https://doi.org/10.1016/j.aos.2013.11.002.

Funder, David C. 1995. “On the Accuracy of Personality Judgment: A Realistic Approach.” Psychological Review 102 (4): 652–70.

Gergen, Kenneth J. 2009. Relational Being: Beyond Self and Community. Oxford University Press.

Goffman, Erving. 1963. Stigma: Notes on the Management of Spoiled Identity. Prentice-Hall.

Granovetter, Mark S. 1973. “The Strength of Weak Ties.” American Journal of Sociology 78 (6): 1360–80.

———. 1974. Getting a Job: A Study of Contacts and Careers. Harvard University Press.

Hacking, Ian. 1986. “Making up People.” In Reconstructing Individualism: Autonomy, Individuality, and the Self in Western Thought, edited by Thomas C. Heller, Morton Sosna, and David E. Wellbery. Stanford University Press.

———. 1995. “The Looping Effects of Human Kinds.” In Causal Cognition: A Multidisciplinary Debate, edited by Dan Sperber, David Premack, and Ann James Premack. Oxford: Clarendon Press.

———. 1999. The Social Construction of What? Harvard University Press.

———. 2007. “Kinds of People: Moving Targets.” Proceedings of the British Academy 151: 285–318.

Hamilton, Alexander. 1788. “The Mode of Electing the President (Federalist No. 68).” In The Federalist.

Ibarra, Herminia, Nancy M. Carter, and Christine Silva. 2010. “Why Men Still Get More Promotions Than Women.” Harvard Business Review 88 (9): 80–85.

Jenkins, Katharine. 2019. “Conferralism and Intersectionality: A Response to Ásta’s Categories We Live By.” Journal of Social Ontology 5 (2): 261–72. https://doi.org/10.1515/jso-2020-2004.

Jussim, Lee, and Kent D. Harber. 2005. “Teacher Expectations and Self-Fulfilling Prophecies: Knowns and Unknowns, Resolved and Unresolved Controversies.” Personality and Social Psychology Review 9 (2): 131–55. https://doi.org/10.1207/s15327957pspr0902_3.

Kenrick, Douglas T., and David C. Funder. 1988. “Profiting from Controversy: Lessons from the Person-Situation Debate.” American Psychologist 43 (1): 23–34.

Kleinberg, Jon, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. 2018. “Human Decisions and Machine Predictions.” The Quarterly Journal of Economics 133 (1): 237–93.

Kroll, Jerome, and Claire Pouncey. 2016. “The Ethics of APA’s Goldwater Rule.” Journal of the American Academy of Psychiatry and the Law 44 (2): 226–35.

Kuncel, Nathan R., Ross J. Kochevar, and Deniz S. Ones. 2014. “A Meta-Analysis of Letters of Recommendation in College and Graduate Admissions: Reasons for Hope.” International Journal of Selection and Assessment 22 (1): 101–7. https://doi.org/10.1111/ijsa.12060.

Lakkaraju, Himabindu, Jon Kleinberg, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. 2017. “The Selective Labels Problem: Evaluating Algorithmic Predictions in the Presence of Unobservables.” In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 275–84. https://doi.org/10.1145/3097983.3098066.

Lance, Charles E. 2008. “Why Assessment Centers Do Not Work the Way They Are Supposed To.” Industrial and Organizational Psychology 1: 84–97.

Latour, Bruno. 1986. “Visualisation and Cognition: Thinking with Eyes and Hands.” In Knowledge and Society: Studies in the Sociology of Culture Past and Present, edited by Henrika Kuklick, 6:1–40. JAI Press.

Lau, Richard R., and David P. Redlawsk. 2001. “Advantages and Disadvantages of Cognitive Heuristics in Political Decision Making.” American Journal of Political Science 45 (4): 951–71.

Lemert, Edwin M. 1951. Social Pathology: A Systematic Approach to the Theory of Sociopathic Behavior. New York: McGraw-Hill.

Link, Bruce G., Francis T. Cullen, Elmer Struening, Patrick E. Shrout, and Bruce P. Dohrenwend. 1989. “A Modified Labeling Theory Approach to Mental Disorders: An Empirical Assessment.” American Sociological Review 54 (3): 400–423.

Link, Bruce G., and Jo C. Phelan. 2001. “Conceptualizing Stigma.” Annual Review of Sociology 27: 363–85.

Lupia, Arthur. 1994. “Shortcuts Versus Encyclopedias: Information and Voting Behavior in California Insurance Reform Elections.” American Political Science Review 88 (1): 63–76.

MacKenzie, Donald. 2006. An Engine, Not a Camera: How Financial Models Shape Markets. Cambridge, MA: MIT Press.

MacKenzie, Donald, Fabian Muniesa, and Lucia Siu, eds. 2007. Do Economists Make Markets? On the Performativity of Economics. Princeton University Press.

Manin, Bernard. 1997. The Principles of Representative Government. Cambridge University Press.

Manin, Bernard, Adam Przeworski, and Susan C. Stokes. 1999. “Elections and Representation.” In Democracy, Accountability, and Representation, edited by Adam Przeworski, Susan C. Stokes, and Bernard Manin. Cambridge University Press.

Mansbridge, Jane. 2003. “Rethinking Representation.” American Political Science Review 97 (4): 515–28.

———. 2009. “A ‘Selection Model’ of Political Representation.” Journal of Political Philosophy 17 (4): 369–98.

Merton, Robert K. 1948. “The Self-Fulfilling Prophecy.” The Antioch Review 8 (2): 193–210.

Messick, Samuel. 1989. “Validity.” In Educational Measurement, edited by Robert L. Linn, 3rd ed., 13–103. Macmillan; the American Council on Education.

———. 1995. “Validity of Psychological Assessment: Validation of Inferences from Persons’ Responses and Performances as Scientific Inquiry into Score Meaning.” American Psychologist 50 (9): 741–49.

Meyer, John W., and Brian Rowan. 1977. “Institutionalized Organizations: Formal Structure as Myth and Ceremony.” American Journal of Sociology 83 (2): 340–63.

Mischel, Walter. 1968. Personality and Assessment. New York: Wiley.

Mischel, Walter, and Yuichi Shoda. 1995. “A Cognitive-Affective System Theory of Personality: Reconceptualizing Situations, Dispositions, Dynamics, and Invariance in Personality Structure.” Psychological Review 102 (2): 246–68.

Ober, Josiah. 2013. “Democracy’s Wisdom: An Aristotelian Middle Way for Collective Judgment.” American Political Science Review 107 (1): 104–22.

Pitkin, Hanna Fenichel. 1967. The Concept of Representation. Berkeley: University of California Press.

Popkin, Samuel L. 1991. The Reasoning Voter: Communication and Persuasion in Presidential Campaigns. University of Chicago Press.

Przeworski, Adam, Susan C. Stokes, and Bernard Manin, eds. 1999. Democracy, Accountability, and Representation. Cambridge Studies in the Theory of Democracy 2. Cambridge University Press.

Raudenbush, Stephen W. 1984. “Magnitude of Teacher Expectancy Effects on Pupil IQ as a Function of the Credibility of Expectancy Induction: A Synthesis of Findings from 18 Experiments.” Journal of Educational Psychology 76 (1): 85–97.

Rivera, Lauren A. 2012. “Hiring as Cultural Matching: The Case of Elite Professional Service Firms.” American Sociological Review 77 (6): 999–1022. https://doi.org/10.1177/0003122412463213.

Rosenthal, Robert, and Lenore Jacobson. 1968. Pygmalion in the Classroom: Teacher Expectation and Pupils’ Intellectual Development. New York: Holt, Rinehart; Winston.

Sackett, Paul R., Charlene Zhang, Christopher M. Berry, and Filip Lievens. 2022. “Revisiting Meta-Analytic Estimates of Validity in Personnel Selection: Addressing Systematic Overcorrection for Restriction of Range.” Journal of Applied Psychology 107 (11): 2040–68. https://doi.org/10.1037/apl0000994.

Schmidt, Frank L., and John E. Hunter. 1977. “Development of a General Solution to the Problem of Validity Generalization.” Journal of Applied Psychology 62 (5): 529–40.

———. 1998. “The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings.” Psychological Bulletin 124 (2): 262–74. https://doi.org/10.1037/0033-2909.124.2.262.

Schmidt, Frank L., Kenneth Law, John E. Hunter, Hannah R. Rothstein, Kenneth Pearlman, and Michael McDaniel. 1993. “Refinements in Validity Generalization Methods: Implications for the Situational Specificity Hypothesis.” Journal of Applied Psychology 78 (1): 3–12.

Searle, John R. 1995. The Construction of Social Reality. New York: The Free Press.

Simondon, Gilbert. 2020. Individuation in Light of Notions of Form and Information. Translated by Taylor Adkins. Minneapolis: University of Minnesota Press.

Snyder, Mark, Elizabeth Decker Tanke, and Ellen Berscheid. 1977. “Social Perception and Interpersonal Behavior: On the Self-Fulfilling Nature of Social Stereotypes.” Journal of Personality and Social Psychology 35 (9): 656–66.

Steele, Claude M., and Joshua Aronson. 1995. “Stereotype Threat and the Intellectual Test Performance of African Americans.” Journal of Personality and Social Psychology 69 (5): 797–811.

Stoet, Gijsbert, and David C. Geary. 2012. “Can Stereotype Threat Explain the Gender Gap in Mathematics Performance and Achievement?” Review of General Psychology 16 (1): 93–102.

Tetlock, Philip E. 2005. Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press.

Waldron, Jeremy. 1995. “The Wisdom of the Multitude: Some Reflections on Book 3, Chapter 11 of Aristotle’s Politics.” Political Theory 23 (4): 563–84.