Foundations of Quantitative Social Science Research - Research Design Chapter (First Drafr)

Posted on Edited on In , Views: Disqus: Word count in article: 8.7k Reading time ≈ 31 mins.

Citation: HUANG, W. (2026, January 9). [Book] Foundations of Quantitative Social Science Research. https://doi.org/10.17605/OSF.IO/EVR46

Abstract

This paper proposes a systematic framework for analyzing research designs, conceptualizing quantitative social science research designs as specific configurations across a finite set of design dimensions. Design dimensions describe the fundamental nature of a study, distinct from specific statistical methods or data collection techniques. By identifying ten core dimensions, including intervention structure, assignment mechanism, temporal structure, unit tracking, directionality of inquiry, comparison logic, data provenance, sampling strategy, unit of analysis, and confounding control strategy. This framework attempts to describe research designs and reveal the essence of canonical research schemes (such as randomized controlled trials, cohort studies, regression discontinuity designs, etc.) as particular dimensional configurations. This framework is expected to help researchers understand the underlying logic of existing designs and guide design choices and innovations under practical constraints.

1. Introduction: From Research Schemes to Design Dimensions

1.1 Problem Statement

In quantitative social science research, the choice of research design directly determines what questions can be answered, what inferences are supported, and what validity threats are faced. Traditional research methods teaching typically introduces specific research schemes directly—randomized controlled trials (RCTs), quasi-experimental designs, cohort studies, cross-sectional studies, etc.—but rarely systematically articulates the common structure underlying these schemes.

This leads to two problems:

  1. Fragmented understanding: Researchers view different designs as independent “recipes,” making it difficult to grasp the structural connections and differences between designs.
  2. Constrained innovation: When facing specific research contexts, researchers tend to “select” from limited canonical schemes rather than “construct” optimal designs based on actual constraints.

1.2 The Dimensional Analysis Framework

The core argument of this paper is: all quantitative social science research designs can be understood as specific configurations across a finite set of “design dimensions.”

A design dimension is a basic, conceptually independent property that describes research. Dimensions determine:

  • How data are generated
  • How comparisons are constructed
  • What inferences are demonstrably valid

Design dimensions are not statistical methods, estimators, or data collection techniques, but rather more fundamental primitive building blocks. Every empirical study can be represented as a specific combination of values across these dimensions.

While the Cartesian space of all possible combinations is large, only some combinations are logically coherent, practically feasible, and inferentially meaningful. Stable and recurrent dimensional configurations form what we recognize as canonical research schemes.

1.3 Paper Structure

  • Section 2 systematically articulates ten core design dimensions, explaining each dimension’s definition, possible values, and inferential implications.
  • Section 3 demonstrates the dimensional configurations of major research schemes, revealing the essence of canonical designs as specific configurational patterns.
  • Section 4 discusses interdependencies among dimensions and how to make design choices under constraints.
  • Section 5 summarizes the framework’s application value and discusses implications for research design teaching and practice.

2. Ten Core Dimensions of Research Design

2.1 Intervention Structure

The intervention dimension characterizes whether and how researchers actively manipulate exposure. This is perhaps the most fundamental design dimension, as it determines the degree of researcher control over the treatment assignment process and fundamentally shapes what type of causal inference is possible.

2.1.1 Experimental Intervention

The researcher directly assigns treatment or exposure. Experimental designs, when properly implemented, provide the strongest claims to internal validity because researchers control the assignment mechanism and can isolate treatment effects from confounding influences through randomization or other design features. The ability to manipulate treatment enables researchers to establish clear temporal precedence and couple assignment with measurement protocols optimized for the research question.

2.1.2 Quasi-experimental Intervention

Treatment assignment is determined externally, but structured in ways that enable causal comparison. Researchers do not control assignment but exploit systematic variation—such as policy changes, institutional rules, or natural discontinuities—that approximates experimental conditions under specific assumptions. Quasi-experimental designs bridge experimental and observational methods, leveraging naturally occurring variation while maintaining some structural features that support credible causal inference.

2.1.3 Observational Design

No manipulation of exposure occurs. Researchers observe naturally occurring variation in treatment or exposure without intervening in the assignment process. Causal inference in observational settings requires stronger assumptions about the absence of confounding, typically operationalized through statistical adjustment, matching, or other methods conditioning on measured covariates.

Inferential implications: Intervention structure strongly constrains the scope of causal inference. Experimental designs provide the strongest evidence for causal claims when properly implemented, but face ethical and practical limitations; observational designs offer greater external validity and feasibility but at the cost of stronger untestable assumptions.

2.2 Assignment Mechanism

The assignment mechanism specifies the process by which units come to receive treatment or exposure. While closely related to intervention structure, assignment mechanism is analytically distinct: it describes how units are exposed, not whether the researcher controls exposure.

This dimension is crucial for causal inference because the assignment mechanism determines what counterfactual comparisons are credible and what assumptions are required to validly identify causal effects.

2.2.1 Randomized Assignment

Assignment is determined by a random process controlled by the researcher. Randomization balances observed and unobserved confounders in expectation, providing a foundation for unconfounded treatment effect estimation. Assignment probabilities may be equal across units (simple randomization), vary by strata (stratified randomization), or depend on covariates (covariate-adaptive randomization).

Key property: Treatment assignment is statistically independent of potential outcomes.

2.2.2 Rule-based Assignment

Assignment follows deterministic rules or thresholds, such that treatment is assigned to all units meeting specific criteria (e.g., all units scoring above a cutoff, all individuals in certain geographic areas, or all applicants meeting eligibility requirements).

Rule-based assignment enables identification strategies like regression discontinuity designs when units near the threshold are comparable. The deterministic nature of assignment means treatment status is a known function of observed characteristics, which can be exploited for causal inference under appropriate continuity assumptions.

2.2.3 Natural Assignment

Assignment is driven by external forces outside the researcher’s control, such as policy implementation, geographic factors, historical events, or institutional processes. Natural assignment may approximate randomization if the assignment mechanism appears exogenous to potential outcomes—i.e., the factors determining treatment are unrelated to outcomes except through their effect on treatment receipt.

The credibility of causal claims depends critically on substantive arguments about the assignment process and empirical assessment of observed covariate balance.

2.2.4 Self-selection

Units select into exposure based on their own choices, preferences, or characteristics. Self-selection creates endogeneity problems because factors influencing treatment choice (such as motivation, risk preference, or expected benefits) may also directly affect outcomes.

This creates confounding that cannot be eliminated by design and must be addressed through modeling assumptions. Controlling for selection-driven confounding requires observing and adjusting for all relevant selection factors—an assumption (conditional independence given observed variables, or selection on observables) that is inherently untestable.

Inferential implications: Different assignment mechanisms produce different patterns of confounding and selection, which in turn require different identification strategies. Understanding the assignment mechanism is critical for assessing whether treatment and control groups are comparable and for determining what statistical or design-driven methods can make them comparable.

2.3 Temporal Structure

Temporal structure captures how observations are distributed over time. This dimension is crucial for quantitative social science because many theoretical questions involve change, dynamics, and causal processes unfolding over time.

The temporal dimension determines whether the design can:

  • Establish temporal precedence (a necessary condition for causal inference)
  • Track within-unit change
  • Detect time-varying effects
  • Distinguish short-term from long-term impacts

2.3.1 Cross-sectional

Units are observed at a single time point. Cross-sectional designs are efficient and avoid attrition, but cannot establish temporal precedence between exposure and outcome or distinguish cohort effects from age effects. All variation is between-unit variation, precluding within-unit comparisons.

2.3.2 Longitudinal

Units are observed at multiple discrete time points. Longitudinal designs enable analysis of change, assessment of temporal sequence, and estimation of within-unit and between-unit effects. The interval between observations may range from days to years depending on the process under study.

2.3.3 Time-series

Units are observed at high frequency or continuously over time. Time-series designs capture fine-grained temporal dynamics, can detect short-term fluctuations and long-term trends, and support methods like interrupted time series analysis that exploit detailed pre-intervention baselines.

Inferential implications: Different temporal structures support different analytical strategies. Cross-sectional designs efficiently estimate associations at a single point; longitudinal designs track change and enable within-unit comparisons; time-series designs capture high-frequency dynamics and support interrupted time series analysis. Temporal structure choice involves trade-offs between resource requirements, attrition, time-varying confounding, and ability to address temporally-grounded research questions.

2.4 Unit Tracking

Unit tracking describes whether the same observational units are followed over time. This dimension is distinct from temporal structure: a study may be longitudinal (multiple time points) but use repeated cross-sections (different units each time) or track the same panel units.

Unit tracking determines what sources of variation are available for analysis and what types of unobserved heterogeneity can be controlled.

2.4.1 Repeated Cross-sections

Different units are observed at each time point. Repeated cross-sectional designs can track population-level change and cohort dynamics without the costs and attrition challenges of panel data, but cannot identify within-unit change or control for unit-specific unobserved heterogeneity.

2.4.2 Panel Designs

The same units are repeatedly observed at all measurement occasions. Panel designs enable within-unit comparisons that control for stable unobserved characteristics, but require strategies to address selective attrition and time-varying confounding.

2.4.3 Cohort Designs

Units defined by shared entry conditions (e.g., birth year, program entry date, common event exposure) are tracked over time. Cohort designs may involve complete panel tracking or accept some attrition, naturally control for cohort-level confounders while allowing analysis of within-cohort heterogeneity.

Inferential implications: Panel designs track the same units, enabling within-unit comparisons that eliminate time-invariant unobserved confounders through fixed effects methods but face attrition and time-varying confounding challenges. Repeated cross-sections avoid attrition but cannot exploit within-unit variation. Cohort designs balance some advantages of both.

2.5 Directionality of Inquiry

Directionality refers to the temporal relationship between exposure measurement and outcome observation. This dimension is crucial for inference because it determines vulnerability to specific types of bias—particularly recall bias, social desirability bias, and reverse causation—and influences the credibility of causal inference.

2.5.1 Prospective

Exposure precedes outcome observation both in the causal process and in the data collection sequence. Prospective designs establish clear temporal precedence, minimize recall bias in exposure measurement, and enable real-time data collection. However, they may require long follow-up periods and are inefficient for studying rare outcomes.

2.5.2 Retrospective

Outcome status determines exposure assessment: researchers identify individuals with the outcome of interest and then ascertain their prior exposure history. Retrospective designs are efficient for rare outcomes and can be implemented quickly, but are vulnerable to recall bias (differential memory of exposure by outcome status), selection bias (if outcome occurrence affects study inclusion), and survival bias (if individuals must survive to outcome occurrence to be included).

2.5.3 Simultaneous

Exposure and outcome are measured concurrently, typically in cross-sectional surveys or assessments. Simultaneous measurement cannot establish temporal precedence from the design itself, making causal inference dependent on strong assumptions about causal direction, auxiliary information from theory or external studies, or structural modeling assumptions.

Inferential implications: Prospective designs minimize certain biases but may be time-consuming; retrospective designs are efficient but vulnerable to recall and selection bias; simultaneous measurement precludes establishing temporal precedence from design. Directionality choice involves trade-offs between bias, cost, and feasibility.

2.6 Comparison Logic

Comparison logic defines the source of the counterfactual—i.e., how the design constructs an answer to “compared to what?” In causal inference, we seek to compare observed outcomes under treatment with counterfactual outcomes under control. Since counterfactuals are never directly observed, research designs use various comparison strategies to approximate them.

Comparison logic determines what types of confounding the design can address and what assumptions are required.

2.6.1 Between-unit Comparisons

Different units serve as comparisons for each other. Between-unit designs compare individuals, groups, or other entities who received treatment with those who did not. Validity requires that units be exchangeable—i.e., comparable in terms of potential outcomes. This can be achieved through randomization, matching, statistical adjustment, or design-driven strategies that make units comparable.

2.6.2 Within-unit Comparisons

Units serve as their own controls, comparing outcomes for the same unit at different times or under different conditions. Within-unit comparisons control for all time-invariant characteristics of units (observed and unobserved) but require assumptions about temporal stability and are vulnerable to time-varying confounding, carryover effects, and secular trends.

2.6.3 Historical Comparisons

Past observations serve as counterfactuals for current or future outcomes. Historical comparisons are common in interrupted time series and difference-in-differences designs, assuming that pre-intervention trends would have continued absent intervention. Validity requires correctly extrapolating historical patterns to the counterfactual future.

2.6.4 External Comparisons

Other populations, contexts, or settings provide the comparison benchmark (e.g., comparing one country to another, or study population outcomes to national norms). External comparisons rely on assumptions about the transportability of causal effects across contexts or the comparability of populations, which can be difficult to justify without strong theory or auxiliary evidence.

Inferential implications: Different comparison logics support different identification strategies with characteristic vulnerabilities. Between-unit comparisons assume different units are comparable (exchangeability); within-unit comparisons assume units are comparable to themselves at different times (temporal stability); historical comparisons assume the past predicts the counterfactual future (trend continuity); external comparisons assume other contexts are informative (transportability).

2.7 Data Provenance

Data provenance specifies the origin of data used in analysis. This dimension affects data quality, construct validity, measurement error, completeness, and alignment between measurement and theoretical constructs.

2.7.1 Primary Data

Collected specifically for the study. Primary data collection allows researchers to optimize measurement instruments for theoretical constructs, timing of measurement, and implementation of quality control procedures. However, primary data collection is resource-intensive and may face challenges of recruitment, response rates, and attrition in longitudinal designs.

2.7.2 Secondary Data

Originally collected for other purposes, such as administrative operations, previous research, or commercial activities. Secondary data provide cost-efficiency and often offer access to large samples or populations, but may involve construct validity concerns (available measures may not align with theoretical constructs), missing data on key variables, and limited control over data quality.

2.7.3 Mixed Sources

A combination of primary and secondary data, such as linking survey responses with administrative records, augmenting administrative data with targeted primary data collection, or merging multiple secondary data sources. Mixed-source designs can leverage the strengths of each data type while potentially addressing their respective limitations, though linkage itself introduces challenges of matching accuracy and consent.

Inferential implications: Understanding data provenance is crucial for assessing measurement validity, anticipating missing data patterns, and evaluating whether available variables adequately operationalize theoretical constructs. Primary data allow optimized measurement but are costly; secondary data are efficient but may involve construct validity compromises; mixed sources can balance both advantages.

2.8 Sampling Strategy

Sampling strategy determines how units are selected from the target population into the study sample. This dimension governs external validity (the ability to generalize findings beyond the study sample) and the justification for statistical inference to broader populations.

2.8.1 Probability Sampling

Units are selected with known, non-zero probabilities. Probability sampling methods include simple random sampling, stratified sampling (random sampling within strata defined by covariates), cluster sampling (random sampling of clusters, then observation of all or some units within clusters), and multi-stage designs.

Probability sampling supports design-based inference, allows calculation of sampling weights to adjust for unequal selection probabilities, and provides a basis for standard error estimation accounting for sampling design.

2.8.2 Non-probability Sampling

Units are selected without a probability basis. Non-probability methods include convenience sampling (units easy to access), purposive sampling (intentional selection of informative cases), quota sampling (non-random selection to fill predetermined quotas), and snowball sampling (chain-referral sampling).

Non-probability samples may be appropriate for exploratory research, hypothesis generation, or studies focused on specific subgroups, but do not support formal statistical inference to defined populations without strong assumptions.

2.8.3 Census

All units in the target population are included. Census designs eliminate sampling error and maximize statistical power, but are typically feasible only for small, well-defined populations (e.g., all students in a school, all firms in an industry sector). Census data may still face coverage error if population enumeration is incomplete.

2.8.4 Selective Enrollment

Units meeting specific eligibility criteria are recruited, often combining elements of purposive selection with probability or convenience sampling among eligible units. Selective enrollment is common in clinical trials (eligibility criteria ensure safety and relevance), evaluation studies (targeting program-eligible populations), and research requiring specific characteristics for theoretical reasons.

Selective enrollment affects generalizability to broader populations.

Inferential implications: Probability sampling provides a basis for external validity and formal inference but may be costly or infeasible; non-probability sampling is more flexible but limits generalizability; census eliminates sampling error but is feasible only for small populations; selective enrollment affects generalizability. The sampling dimension interacts with other design features: a population-representative sample enhances external validity but may complicate causal inference; a convenience sample may achieve clearer causal estimates within a homogeneous subgroup but raises generalizability concerns.

2.9 Unit of Analysis and Observation

This dimension distinguishes the level at which data are collected (unit of observation) from the level at which inference is made (unit of analysis). Misalignment between these two levels creates aggregation or disaggregation challenges and potential logical fallacies.

This dimension is particularly important in quantitative social science because social phenomena are inherently multilevel: individuals are nested in families, students in classrooms, workers in firms, citizens in nations.

2.9.1 Individual-level

Individuals are both the unit of observation and analysis. Individual-level designs directly model individual outcomes and characteristics, providing maximum resolution for treatment effects and outcome heterogeneity. Standard regression methods and most causal inference techniques are designed for individual-level data.

2.9.2 Aggregate-level

Groups, organizations, or geographic units (e.g., schools, firms, communities, countries) are the units of observation and analysis. Aggregate-level designs are appropriate when theoretical questions concern collective entities or when individual-level data are unavailable. Aggregate analysis may be necessary for policy-relevant questions about organizational or jurisdictional effects.

2.9.3 Multilevel

Observations are nested within higher-level units, requiring hierarchical or multilevel modeling that explicitly represents this structure. Multilevel designs allow decomposition of variance across levels, estimation of cross-level interactions, and appropriate quantification of uncertainty accounting for clustering.

Multilevel analysis addresses questions about contextual effects and compositional effects while avoiding aggregation bias.

2.9.4 Ecological

Aggregate data are used to make inferences about aggregate-level processes or relationships. Ecological designs analyze associations between variables measured at the group level (e.g., relationships between county-level characteristics and county-level outcomes). Ecological inference is valid for understanding aggregate patterns but requires strong assumptions if used to infer individual-level relationships.

Inferential implications: The choice of analytical level affects what questions can be addressed, what confounders must be considered, and what statistical methods are appropriate. Aggregate analysis may mask individual-level heterogeneity; individual analysis may fail to capture contextual or compositional effects. When observation and analysis units are misaligned, the ecological fallacy (inferring individual-level relationships from aggregate data) and atomistic fallacy (ignoring higher-level structure) are key threats.

2.10 Measurement Modality

The measurement modality dimension characterizes how key variables are observed and recorded. This dimension has profound implications for measurement validity (whether measurement captures the intended construct), reliability (measurement consistency), and bias (systematic error in measurement).

Different modalities involve distinctive trade-offs among objectivity, construct validity, cost, participant burden, and ethical considerations. In quantitative social science, many constructs of interest (attitudes, behaviors, social processes) are not directly observable, and measurement modality choice fundamentally shapes what can be studied and how credibly.

2.10.1 Direct Observation

Researchers or trained observers record behaviors, events, or conditions in real-time through systematic observation protocols. Examples include classroom observation schedules, workplace time-motion studies, structured field observations, and behavioral coding in laboratory settings.

Direct observation minimizes recall bias and provides objective behavioral measurements, but is labor-intensive, may involve observer effects (reactivity), requires high inter-rater reliability, and is limited to observable behaviors rather than internal states or subjective experiences.

2.10.2 Self-report

Respondents provide information about themselves through surveys, interviews, or questionnaires. This includes standardized instruments (validated scales), open-ended interviews, ecological momentary assessment (repeated sampling of current states), and retrospective reports of past events or behaviors.

Self-report provides access to internal experiences, subjective states, and private behaviors inaccessible through observation, but is subject to recall bias, social desirability bias, acquiescence bias, satisficing, and question wording effects. Self-report measures vary substantially in structure and quality, from rigorously validated psychometric instruments to ad hoc single-item questions.

2.10.3 Proxy Report

Information is obtained from knowledgeable informants rather than directly from target individuals. Examples include teacher reports of student behavior, parent reports of child development, or family member reports of patient health status.

Proxy reports may provide information when self-report is infeasible (young children, cognitively impaired individuals) or may triangulate with self-report to improve measurement. However, proxy reports may differ from self-reports due to limited observability of internal states or private behaviors, differential interpretation of questions, or different perspectives and incentives.

2.10.4 Administrative Records

Data are extracted from institutional or governmental databases created for operational purposes. Examples include health records (diagnoses, procedures, prescriptions), educational transcripts (grades, test scores, attendance), tax records (income, employment), criminal justice data (arrests, convictions, incarceration), and vital statistics (births, deaths, marriages).

Administrative data provide objectivity, large scale, population coverage, and longitudinal tracking, but may suffer from recording errors, systematic missingness related to system contact, construct validity challenges when operational categories don’t align with research constructs, and changes in recording practices over time.

2.10.5 Biomarkers and Physiological Measures

Biological samples or physiological sensors provide objective measures of physical states or biological processes. Examples include cortisol levels (stress), blood pressure, neuroimaging (brain structure and function), genetic markers, and cardiovascular or metabolic indicators.

Biomarkers reduce self-report bias, provide objective indicators of physiological processes, and may capture information inaccessible through self-report. However, they require specialized equipment and expertise, involve participant burden and potential health risks, may raise privacy concerns (especially genetic data), and often serve as imperfect proxies for theoretically interesting constructs (e.g., cortisol as a measure of stress).

2.10.6 Digital Trace Data

Behavioral data passively collected from digital platforms, sensors, or transaction systems. Examples include social media activity (posts, likes, network connections), GPS traces (mobility patterns), online purchases (consumer behavior), web browsing (information seeking), sensor networks (environmental exposures), and smartphone data (app use, communication patterns).

Digital traces provide high temporal resolution, ecological validity (naturalistic behavior), large scale, and reduced participant burden (passive collection). However, they raise construct validity questions (does observed behavior map to theoretical constructs?), privacy and ethical concerns, selection bias (platform participation), algorithmic confounding (platform algorithms affect observed behavior), and measurement heterogeneity across user populations.

2.10.7 Performance Tests

Standardized assessments or experimental tasks measure latent constructs through behavioral performance. Examples include cognitive ability tests (IQ, working memory), achievement tests (math, reading), physical fitness assessments, experimental economic games (risk preference, cooperation), and implicit association tests (implicit attitudes).

Performance tests provide objective behavioral measures of constructs not directly observable, often with established psychometric properties. However, they may be sensitive to testing conditions, require careful validation, and may not generalize beyond test contexts.

2.10.8 Mixed Measurement

Multiple measurement modalities are combined to triangulate constructs or validate measurements. Examples include self-reported health combined with biomarkers and medical records, survey responses linked with administrative employment data, or direct observation combined with self-report interviews.

Mixed measurement can leverage the strengths of different modalities, provide convergent validation, and enable assessment of measurement error through cross-method comparison. However, mixed-mode designs introduce complexity of data linkage, require managing discrepancies across measurements, and may face consent challenges for linking data sources.

Inferential implications: Measurement modality choice involves trade-offs among validity, reliability, cost, participant burden, and ethical considerations. Objective measures (biomarkers, administrative records) reduce certain biases but may not capture subjective states or meaning. Self-reports provide access to internal experiences but introduce response biases. Digital traces provide scale and naturalism but complicate construct interpretation.

Measurement error structure varies systematically by modality: self-reports tend toward systematic bias, administrative records toward random missingness, digital traces toward measurement heterogeneity across subgroups.

2.11 Confounding Control Strategy

The confounding control dimension specifies how the design addresses potential confounding—the presence of common causes of treatment and outcome that threaten causal identification. Confounding is perhaps the central challenge in causal inference: if treatment and outcome share common causes, observed associations may reflect these shared causes rather than treatment’s causal effect.

Control strategies determine what assumptions must hold for valid causal inference and fundamentally shape the credibility of causal claims.

2.11.1 Randomization-based Control

Random assignment of treatment eliminates confounding by design, ensuring treatment is statistically independent of all potential confounders (observed and unobserved). Randomization balances confounders in expectation, making treatment and control groups exchangeable and allowing estimation of causal effects through simple mean differences (or regression adjustment for precision improvement).

This is the gold standard for internal validity because it requires no assumptions about which variables confound the relationship or how they operate. However, randomization is not always feasible (cannot randomize structural characteristics like race or gender), ethical (cannot randomize harmful exposures), or practical (cannot randomize many policy interventions). Even when feasible, randomization addresses only confounding, not other validity threats like attrition, measurement error, or spillover effects.

2.11.2 Design-based Identification

Research designs exploit structural features that approximate randomization or render confounding implausible under articulated assumptions. Design-based methods embed identification assumptions in the design rather than statistical models, often providing more credible causal inference than pure statistical adjustment.

Each design-based approach relies on specific identification assumptions:

Regression Discontinuity (RD): Assumes units just above and below a threshold are comparable, isolating treatment effects at the cutoff. Identification requires continuity of potential outcomes at the threshold: individuals on either side differ only in treatment status, not in other characteristics affecting outcomes. RDD provides credible local causal effects but limited generalizability beyond the threshold region.

Instrumental Variables (IV): An instrument affects the outcome only through its effect on treatment, breaking confounded pathways. Identification requires instrument relevance (strong first stage), exclusion restriction (instrument affects outcome only through treatment), and independence (instrument uncorrelated with potential outcomes). IV estimates the local average treatment effect for compliers—units whose treatment status is affected by the instrument.

Difference-in-Differences (DiD): Treatment and control groups share parallel counterfactual trends, allowing differencing to eliminate time-invariant confounding. Identification requires that absent treatment, treatment and control groups would have followed parallel trends. DiD controls for time-invariant confounding and common shocks but is vulnerable to time-varying confounding and parallel trends violations.

Interrupted Time Series (ITS): Pre-intervention trends establish the counterfactual trajectory absent intervention. Identification requires that absent intervention, pre-intervention trends would have continued and no concurrent events affect the outcome. ITS provides strong internal validity when long pre-intervention baselines establish stable trends but is vulnerable to concurrent events and structural breaks.

Natural Experiments: Exogenous shocks or policy changes create quasi-random variation in exposure. Identification requires that the natural assignment mechanism appears exogenous—i.e., unrelated to potential outcomes except through treatment. Credibility depends on substantive arguments about the assignment process and empirical assessment of balance.

Each design-based method relies on specific identification assumptions that must be empirically assessed (through balance tests, placebo tests, or sensitivity analyses) and theoretically justified.

2.11.3 Restriction and Stratification

Analysis is limited to units with similar values of known confounders, or conducted separately within strata defined by confounders. Restriction eliminates confounding by restricted variables by design: if analysis includes only units of a given age, age cannot confound the relationship. Similarly, stratified analysis ensures comparisons are made only within homogeneous strata.

Restriction controls for observed confounders without requiring functional form assumptions but does not address unmeasured confounders, reduces sample size and generalizability, and may introduce selection bias if restriction is based on post-treatment variables.

2.11.4 Matching

Treatment and control units are paired or weighted to achieve balance on observed covariates. Matching methods include exact matching (pairing on identical covariate values), propensity score matching (pairing on predicted probability of treatment), coarsened exact matching (matching on coarsened covariates), and genetic matching (optimizing balance metrics).

Matching controls for observed confounders by constructing a comparison group similar to the treatment group on measured characteristics. However, matching assumes no unobserved confounding (conditional independence given matched variables) and requires sufficient overlap (common support): for each treated unit there must exist comparable control units.

Matching is often combined with regression adjustment (doubly robust estimation) to improve robustness.

2.11.5 Statistical Adjustment

Confounders are controlled through regression modeling, inverse probability weighting, or doubly robust estimation. Statistical adjustment conditions on observed confounders, estimates treatment effects within confounder levels, then aggregates across levels.

Validity of statistical adjustment depends on three assumptions: (1) conditional independence—all confounders are observed and included in the model; (2) correct functional form—relationships among confounders, treatment, and outcome are correctly specified; (3) positivity—all covariate combinations have non-zero probability of receiving each treatment level.

Statistical adjustment is flexible and can accommodate continuous confounders and complex covariate patterns but relies on strong untestable assumptions (especially conditional independence) and may be sensitive to model misspecification.

2.11.6 Fixed Effects

Within-unit or within-group variation is exploited to eliminate time-invariant or unit-invariant confounders. Fixed effects models include unit-specific intercepts that absorb all stable characteristics of units (observed and unobserved). By comparing units to themselves over time (or groups to themselves across observations), fixed effects control for all time-invariant confounding.

Fixed effects are particularly powerful when important confounders are stable characteristics (ability, personality, community factors). However, fixed effects cannot identify effects of time-invariant treatments, are vulnerable to time-varying confounding, may exacerbate measurement error, and require correct specification of time-varying dynamics.

2.11.7 Sensitivity Analysis

Rather than eliminating confounding, sensitivity analysis quantifies how strong unmeasured confounding would need to be to overturn observed associations. Sensitivity analysis methods include bounding approaches (computing bounds on treatment effects under assumed confounding), tipping point analysis (determining confounding strength needed to nullify results), IV-based sensitivity tests, and simulation of confounding scenarios.

Sensitivity analysis does not address confounding but provides transparency about robustness of causal claims to violations of identification assumptions. Strong sensitivity results (effects robust to substantial confounding) strengthen causal inference; weak results (effects eliminated by moderate confounding) highlight fragility of causal claims.

2.11.8 Negative Controls

Negative control outcomes (outcomes that should not be affected by treatment) or negative control exposures (exposures that should not affect the outcome) are used to detect residual confounding. If negative controls show associations with treatment or outcome, this indicates the presence of confounding: the association cannot be causal (by definition of the negative control), so must reflect confounding or bias.

Negative controls provide empirical evidence about the adequacy of confounding control without requiring explicit identification of confounders. However, specifying valid negative controls requires strong substantive assumptions about what should and should not be causally related.

2.11.9 No Explicit Control

Some observational studies do not systematically attempt to control confounding, either because the research question is purely descriptive (non-causal) or because causal interpretation relies on conditional independence assumptions implicit in the research question.

Studies without explicit confounding control may still provide valuable descriptive evidence, generate hypotheses, or inform causal inference when combined with theory or evidence from other sources. However, causal claims from such studies are typically weak and require strong justification.

Inferential implications: Choice of confounding control strategy is central to causal inference. Randomization-based and design-based strategies embed identification assumptions in the research design itself, while statistical adjustment and matching embed assumptions in analytical models. Transparent articulation of the chosen strategy and its underlying assumptions is critical for evaluating the credibility of causal claims.

Multiple strategies can be combined in a single study. For example, a difference-in-differences design might additionally employ matching to improve covariate balance or regression adjustment to control for time-varying confounders. Combined strategies can provide robustness checks and strengthen causal inference if results are consistent across methods.

2.12 Dimension Summary Table

Dimension Possible Values
Intervention Structure Experimental / Quasi-experimental / Observational
Assignment Mechanism Randomized / Rule-based / Natural / Self-selection
Temporal Structure Cross-sectional / Longitudinal / Time-series
Unit Tracking Repeated cross-sections / Panel / Cohort
Directionality of Inquiry Prospective / Retrospective / Simultaneous
Comparison Logic Between-unit / Within-unit / Historical / External
Data Provenance Primary / Secondary / Mixed
Sampling Strategy Probability / Non-probability / Census / Selective enrollment
Unit of Analysis Individual-level / Aggregate-level / Multilevel / Ecological
Measurement Modality Direct observation / Self-report / Proxy report / Administrative records / Biomarkers / Digital traces / Performance tests / Mixed measurement
Confounding Control Randomization / Design-based identification / Restriction/stratification / Matching / Statistical adjustment / Fixed effects / Sensitivity analysis / Negative controls / No explicit control

3. From Dimensions to Research Schemes: Dimensional Configurations of Canonical Designs

3.1 The Nature of Research Schemes

Research schemes are coherent, recognizable configurations of design dimensions that support specific inferential goals. Unlike dimensions, which are abstract descriptors, research schemes function as inferential templates. Each scheme is associated with characteristic assumptions, strengths, and limitations.

While the Cartesian product space of dimensions is theoretically large, only certain configurations recur in practice because they represent stable trade-offs between inferential power and feasibility under particular constraints. These stable configurations are what we recognize as canonical research schemes.

3.2 Major Research Schemes and Their Dimensional Configurations

3.2.1 Randomized Controlled Trials (RCT)

Dimensional Configuration:

  • Intervention Structure: Experimental
  • Assignment Mechanism: Randomized
  • Temporal Structure: Longitudinal
  • Unit Tracking: Panel (or cohort)
  • Directionality of Inquiry: Prospective
  • Comparison Logic: Between-unit (or within-unit, as in crossover designs)
  • Data Provenance: Primary
  • Measurement Modality: Direct observation, performance tests, biomarkers (common)
  • Confounding Control: Randomization-based

Key Identification Assumption: Random assignment ensures treatment is independent of potential outcomes (unconfoundedness by design).

Strengths: Provides strongest internal validity when properly implemented; controls for observed and unobserved confounders.

Limitations: May face ethical constraints, feasibility limitations, external validity concerns; cannot address certain types of treatments (e.g., structural characteristics).

Common Variants: Parallel group designs, cluster randomized trials, factorial designs, adaptive randomization schemes.

3.2.2 Quasi-Experimental Designs

Quasi-experimental designs exploit structured variation in exposure without full randomization. Common examples include difference-in-differences, regression discontinuity, instrumental variables, and interrupted time series analysis.

General Dimensional Configuration:

  • Intervention Structure: Quasi-experimental
  • Assignment Mechanism: Rule-based or natural
  • Temporal Structure: Longitudinal or cross-sectional (depends on specific design)
  • Comparison Logic: Between-unit and/or within-unit
  • Data Provenance: Primary or secondary
  • Confounding Control: Design-based identification

Key Identification Assumptions: Vary by specific design (e.g., parallel trends for DiD, threshold continuity for RDD, exclusion restriction for IV).

Strengths: Provide credible causal inference when randomization is impossible; leverage naturally occurring variation.

Limitations: Rely on design-specific identification assumptions that may be untestable; typically estimate local effects.

Typical Applications: Policy evaluation, natural experiments, studies exploiting institutional features.

3.2.3 Cohort Studies

Dimensional Configuration:

  • Intervention Structure: Observational
  • Assignment Mechanism: Natural or self-selection
  • Temporal Structure: Longitudinal
  • Unit Tracking: Cohort
  • Directionality of Inquiry: Prospective
  • Comparison Logic: Between-unit
  • Data Provenance: Primary (common)
  • Measurement Modality: Self-report, biomarkers, administrative records
  • Confounding Control: Statistical adjustment, restriction, matching

Key Identification Assumption: Conditional independence given measured covariates (no unmeasured confounding).

Strengths: Establish temporal precedence; can study multiple outcomes; suitable for rare exposures.

Limitations: Vulnerable to unmeasured confounding; require long-term follow-up; attrition may introduce bias.

Typical Applications: Epidemiological research, long-term risk factor studies, developmental research.

3.2.4 Case-Control Studies

Dimensional Configuration:

  • Intervention Structure: Observational
  • Assignment Mechanism: Outcome-based selective enrollment
  • Temporal Structure: Cross-sectional or retrospective
  • Directionality of Inquiry: Retrospective
  • Comparison Logic: Between-unit
  • Data Provenance: Primary or secondary
  • Measurement Modality: Self-report, administrative records, proxy report (common)
  • Confounding Control: Matching + statistical adjustment

Key Identification Assumptions: Controls represent the source population generating cases; exposure measurement is unbiased by outcome status; conditional independence given covariates.

Strengths: Efficient for rare outcomes; can be implemented quickly.

Limitations: Vulnerable to selection bias and recall bias; difficult to establish temporal precedence; cannot directly estimate incidence.

Typical Applications: Etiological research on rare diseases, rapid outbreak investigations.

3.2.5 Cross-Sectional Studies

Dimensional Configuration:

  • Intervention Structure: Observational
  • Assignment Mechanism: Self-selection or natural
  • Temporal Structure: Cross-sectional
  • Directionality of Inquiry: Simultaneous
  • Comparison Logic: Between-unit
  • Data Provenance: Primary or secondary
  • Measurement Modality: Self-report, administrative records (common)
  • Confounding Control: Statistical adjustment (if attempting causal inference); IV (if available)

Key Limitation: Cannot establish temporal precedence; vulnerable to reverse causation.

Strengths: Cost-effective; rapid implementation; suitable for prevalence estimation and association studies.

Limitations: Causal interpretation limited; cannot distinguish causal direction.

Typical Applications: Population health surveys, prevalence studies, exploratory association analysis.

3.2.6 Panel Studies

Dimensional Configuration:

  • Intervention Structure: Observational
  • Assignment Mechanism: Self-selection or natural
  • Temporal Structure: Longitudinal
  • Unit Tracking: Panel
  • Directionality of Inquiry: Prospective
  • Comparison Logic: Within-unit and between-unit
  • Data Provenance: Primary or secondary
  • Measurement Modality: Self-report, administrative records, mixed (common)
  • Confounding Control: Fixed effects + statistical adjustment

Key Identification Assumptions: No time-varying unmeasured confounding (for fixed effects); correct dynamic specification.

Strengths: Control for time-invariant unobserved heterogeneity; can analyze change and dynamic relationships.

Limitations: Attrition may introduce bias; cannot estimate effects of time-invariant variables; must address time-varying confounding.

Typical Applications: Labor economics, organizational research, developmental trajectory analysis.

3.2.7 Regression Discontinuity Designs (RDD)

Dimensional Configuration:

  • Intervention Structure: Quasi-experimental
  • Assignment Mechanism: Rule-based (threshold)
  • Temporal Structure: Cross-sectional or longitudinal
  • Comparison Logic: Between-unit (narrow bandwidth near threshold)
  • Data Provenance: Primary or secondary
  • Confounding Control: Design-based identification (exploiting discontinuity)
  • Measurement Modality: Administrative records, self-report, mixed

Key Identification Assumptions: Continuity of potential outcomes at threshold; no manipulation of assignment variable; correct functional form specification.

Strengths: Provides credible local causal estimates; identification assumptions partially testable (e.g., covariate balance).

Limitations: Only identifies effects at the threshold; limited external validity beyond threshold region; requires large samples.

Typical Applications: Program eligibility evaluation, policy change assessment, score-based interventions.

3.2.8 Instrumental Variables Designs (IV)

Dimensional Configuration:

  • Intervention Structure: Quasi-experimental or observational
  • Assignment Mechanism: Natural or rule-based (instrument)
  • Temporal Structure: Varies (cross-sectional or longitudinal)
  • Comparison Logic: Between-unit
  • Data Provenance: Primary or secondary
  • Confounding Control: Design-based identification (via instrumental variable)
  • Measurement Modality: Administrative records, self-report, mixed

Key Identification Assumptions: Instrument relevance (strong first stage); exclusion restriction (instrument affects outcome only through treatment); monotonicity (no defiers); independence of instrument and potential outcomes.

Strengths: Can identify causal effects in presence of unmeasured confounding; estimates local average treatment effect for compliers.

Limitations: Requires strong substantive justification for exclusion restriction; sensitive to weak instruments; estimates local effect (LATE), may not generalize.

Typical Applications: Returns to education, medical effectiveness, policy impacts (exploiting policy variation as instruments).

3.3 Research Scheme Dimensional Configuration Summary Table

Scheme Intervention Assignment Temporal Directionality Comparison Confounding Control
RCT Experimental Randomized Longitudinal Prospective Between/within Randomization
Quasi-experimental Quasi-experimental Rule/natural Varies Prospective Between/within Design-based
Cohort Study Observational Natural/self-selection Longitudinal Prospective Between Statistical adjustment
Case-Control Observational Outcome-based Cross-sectional Retrospective Between Matching+adjustment
Cross-sectional Observational Natural/self-selection Cross-sectional Simultaneous Between Statistical adjustment
Panel Study Observational Natural/self-selection Longitudinal Prospective Within/between Fixed effects+adjustment
RDD Quasi-experimental Rule-based Cross/longitudinal Prospective Between (local) Design-based (discontinuity)
IV Design Quasi/observational Natural/rule (IV) Varies Varies Between Design-based (IV)

4. Interdependencies Among Dimensions and Design Choice

4.1 Structural Dependencies Among Dimensions

Design dimensions are not completely independent. Certain dimensional values constrain or necessarily imply other values, creating structural dependencies that shape the feasible design space.

4.1.1 Logical Dependencies

Certain combinations are logically impossible:

  • Randomized assignment requires experimental or quasi-experimental intervention structure
  • Retrospective directionality is incompatible with randomization
  • Within-unit comparison requires longitudinal temporal structure
  • Fixed effects control requires panel data

4.1.2 Practical Dependencies

Certain combinations, while logically possible, are rarely implemented due to practical constraints:

  • Randomized experiments typically use primary data collection
  • Meta-analyses necessarily use secondary data
  • Digital trace research typically adopts observational designs

4.1.3 Inferential Dependencies

Choices on one dimension affect which values on other dimensions are appropriate for valid inference:

  • High-frequency time-series data may enable design-based identification strategies (e.g., interrupted time series) unavailable with cross-sectional data
  • Panel data enable fixed effects control that cross-sectional designs cannot support
  • Rich covariate measurement makes matching and statistical adjustment more credible

4.1.4 Measurement-Design Dependencies

Certain measurement modalities naturally align with specific design structures:

  • Biomarkers and performance tests are common in experimental designs where precise measurement justifies cost
  • Digital trace data typically appear in observational designs with natural assignment
  • Administrative records often enable large-scale quasi-experimental designs exploiting policy variation

Understanding these interdependencies helps researchers navigate the design space efficiently and avoid internally inconsistent configurations.

4.2 Workflow from Research Question to Design Choice

Research design choice is not a mechanical process. Researchers do not simply “apply” a predetermined scheme to a given question. Rather, design choice emerges from an iterative process of conceptualization, operationalization, and constraint mapping.

Stage 1: Conceptual Clarification

  1. Articulate the substantive question: What real-world phenomenon or relationship are you trying to understand?

  2. Identify inference type:

    • Causal inference: Estimating effects of interventions or exposures on outcomes
    • Descriptive inference: Characterizing populations, distributions, or relationships
    • Predictive inference: Forecasting future outcomes or states
    • Exploratory inquiry: Generating hypotheses or identifying patterns

  3. Define target population and estimand: For whom or what should the inference hold?

Stage 2: Operationalization

  1. Define units of observation and analysis: What entities are studied?

  2. Operationalize key constructs: How will abstract concepts be measured? Available measurement modalities constrain design options.

  3. Identify exposure or treatment of interest: What is the intervention, exposure, or predictor whose effect or association is of interest? Can it be manipulated?

  4. Specify outcome: What is affected or predicted? When can it be observed? How long after exposure does it occur?

At this stage, researchers often discover their initial question cannot be operationalized with available measurements. This may require reconceptualizing the question, seeking new measurement tools, or accepting that some aspects of the question cannot be empirically addressed.

Stage 3: Constraint Assessment

  1. Ethical constraints: Can exposure be manipulated? Is random assignment ethically acceptable?

  2. Feasibility constraints: Can researchers control assignment? Do necessary data exist? Can subjects be tracked over time?

  3. Resource constraints: What is affordable in terms of time, funding, and personnel?

  4. Temporal constraints: How quickly are results needed?

  5. Data availability: What data already exist?

Constraint assessment often reveals that the ideal design for the question is infeasible. This forces researchers to choose among imperfect alternatives, balancing inferential power against practical realities.

Stage 4: Dimensional Configuration

Given the clarified question and assessed constraints, researchers now configure design dimensions to construct a feasible scheme:

  1. Intervention structure: If manipulation is ethical and feasible, experimental design is possible. If manipulation is impossible but structured natural variation exists, quasi-experimental design may be feasible. Otherwise, observational design is necessary.

  2. Assignment mechanism: If intervention is possible, can randomization be implemented? If not, what determines assignment?

  3. Temporal structure and unit tracking: Can units be tracked over time? Are repeated observations feasible considering attrition and cost?

  4. Comparison logic: What source of counterfactual comparison is most credible?

  5. Confounding control strategy: Given assignment mechanism, temporal structure, and available covariates, what confounding control is possible?

  6. Measurement modality and data provenance: What measurement methods are both valid and feasible for key constructs?

This configuration process is iterative: choices on one dimension constrain options on others. Researchers navigate this constrained design space seeking coherent configurations that support the intended inference.

Stage 5: Scheme Identification and Adaptation

Once dimensional values are configured, researchers often identify that their design corresponds to a canonical research scheme. This identification is valuable because schemes carry established inferential frameworks, known assumptions, and standard methods.

However, many practical designs do not perfectly match canonical schemes. Researchers may combine features from multiple schemes or create hybrid designs tailored to specific contexts. The dimensional framework makes precise communication of such designs possible even when they don’t fit standard categories.

4.3 Design Justification

Design justification articulates the logic connecting research question, constraints, dimensional configuration, and inferential assumptions. A complete justification should address:

  1. Why this inferential goal? What substantive question motivates the study?

  2. Why these operational definitions? How do chosen measurements capture constructs of interest?

  3. Why this dimensional configuration? For each dimension, explain:

    • What alternatives were considered
    • What constraints ruled them out
    • How the chosen value supports the inferential goal

  4. What assumptions does this design require? Explicitly state identification assumptions on which valid inference depends.

  5. How plausible are these assumptions? Provide substantive arguments or empirical evidence supporting key assumptions.

  6. What are the key limitations? Acknowledge what the design cannot do. What validity threats remain unaddressed?

  7. What sensitivity analyses are planned? How will you assess robustness of findings to assumption violations?

4.4 Inferential Trade-offs

Different dimensional configurations embody different inferential trade-offs:

Internal validity vs. External validity: Experimental intervention and randomized assignment maximize internal validity but may compromise external validity, feasibility, or ethical acceptability. Observational designs offer greater external validity and practical feasibility but require stronger modeling assumptions to support causal claims.

Precision vs. Bias: Large samples improve precision but may increase bias if non-representative. Representative samples enhance external validity but may complicate causal inference if they include substantial heterogeneity.

Cost vs. Quality: Primary data collection allows optimized measurement but is expensive. Secondary data are efficient but may involve construct validity compromises.

Objectivity vs. Construct validity: Objective measures (biomarkers, administrative records) reduce certain biases but may not capture theoretically interesting constructs. Self-reports provide access to internal experiences but introduce response biases.

Understanding these trade-offs requires recognizing that no single design dominates on all criteria. Optimal design depends on the research question, population and setting, available resources, and the relative importance of different inferential goals.

5. Conclusion and Future Directions

5.1 Core Contributions of the Framework

The dimensional analysis framework proposed in this paper makes three core contributions to understanding quantitative social science research design:

5.1.1 Conceptual Unification

By identifying a finite set of design dimensions as fundamental building blocks, the framework conceptually unifies seemingly distinct research schemes. This reveals that:

  • Canonical schemes (RCT, cohort studies, RDD, etc.) are not independent entities but specific configurations in dimensional space
  • Superficially different designs may share configurations on some dimensions while differing on others
  • “New” designs are often novel combinations of dimensions rather than fundamental innovations

5.1.2 Precise Communication

The dimensional language enables precise description and communication of research designs:

  • Researchers can explicitly specify choices on each dimension and their inferential implications
  • Reviewers and readers can systematically evaluate whether designs align with research questions
  • Cross-disciplinary communication is facilitated through common vocabulary

5.1.3 Systematic Innovation

The framework supports systematic design innovation under constraints:

  • When canonical schemes are infeasible, researchers can identify the closest feasible alternative
  • Hybrid designs can be constructed by combining dimensional features from different canonical schemes
  • New contexts (such as digital trace data, large-scale administrative records) can be systematically characterized by their dimensional configurations

5.2 Implications for Research Practice

5.2.1 Design Teaching

The dimensional framework has important implications for research methods pedagogy:

From principles to applications: Rather than teaching canonical schemes one by one, start with fundamental dimensions, then show how these dimensions combine to form recognizable schemes. This enables students to understand the underlying logic of design rather than merely memorizing scheme characteristics.

Critical evaluation: The dimensional framework provides tools for systematically evaluating others’ research. Students can ask: What choices were made on each dimension? How do these choices constrain each other? What are the key assumptions?

Creative problem-solving: By understanding design as a constrained optimization problem, students learn to construct feasible schemes under practical limitations rather than mechanically applying templates.

5.2.2 Design Planning

For researchers planning new studies:

Systematic constraint mapping: Explicitly identify constraints on each dimension (ethical, feasibility, resource, data availability) before attempting to choose a scheme.

Transparent trade-offs: Explicitly articulate trade-offs between different dimensional configurations and justify why the chosen configuration is optimal given constraints.

Sensitivity and robustness: Identify dimensions on which key assumptions depend (typically confounding control strategy) and plan sensitivity analyses to assess robustness.

5.2.3 Design Reporting

In writing methods sections:

Structured description: Systematically describe choices on each relevant dimension rather than providing narrative methods descriptions.

Explicit assumptions: Clearly state inferential assumptions implied by each dimensional choice.

Transparent limitations: Acknowledge limitations imposed by dimensional configuration and explain why alternative configurations were infeasible.

5.3 Future Directions

5.3.1 Dimensional Extensions

While this paper identified ten core dimensions, future work may identify additional dimensions or refine existing ones:

  • Compliance and adherence: Relationship between assignment and actual exposure
  • Spillover structure: How treatment spills over between units
  • Contextual stability: Temporal stability of research environment
  • Measurement frequency: Detailed specification of observation density

5.3.2 Formalization

The dimensional framework may benefit from formalization:

  • Formal language establishing logical dependencies among dimensions
  • Algorithmic tools to identify feasible configurations given constraints
  • Optimization frameworks to balance inferential goals against practical constraints

5.3.3 Methodological Development

The dimensional perspective may inspire methodological innovations:

  • New estimation methods optimized for specific dimensional configurations
  • Meta-analytic frameworks for combining evidence across designs
  • Sensitivity analysis tools tailored to key assumptions of specific dimensional configurations

5.4 Concluding Remarks

Quantitative social science research design is not a matter of selecting pre-packaged schemes from a menu, but rather a process of navigating constrained design space, configuring dimensions to balance inferential goals against practical realities, and justifying the resulting configuration through transparent articulation of assumptions and limitations.

The dimensional analysis framework provides systematic language to describe this process. By understanding research design as configuration across fundamental dimensions rather than application of fixed schemes, researchers can:

  • More deeply understand the structure of existing designs
  • Make informed design choices under constraints
  • Communicate design logic and assumptions clearly
  • Systematically innovate beyond canonical schemes

This is not merely a terminological contribution but an epistemological shift: from viewing design as recipes to be learned, to viewing design as a system of principles to be understood and applied. This shift enables researchers to become more sophisticated consumers of design and more effective creators of design.

In an increasingly complex data environment—digital traces, large-scale administrative records, sensor networks, genetic data—and with increasingly diverse substantive questions, this systematic understanding of design principles is more important than ever. The dimensional framework provides a language and way of thinking that enables researchers to navigate this evolving landscape while maintaining inferential rigor.

Appendix: Dimensional Configuration Checklist

To facilitate application of this framework in design planning, we provide a simplified checklist:

Design Planning Checklist

1. Research Question

  • Clearly articulate substantive research question
  • Identify inference type (causal/descriptive/predictive/exploratory)
  • Define target population and estimand

2. Operationalization

  • Define units of observation and analysis
  • Operationalize exposure/treatment
  • Operationalize outcome
  • Identify key covariates

3. Constraint Assessment

  • Assess ethical constraints
  • Assess feasibility constraints
  • Assess resource constraints
  • Assess temporal constraints
  • Assess data availability

4. Dimensional Configuration

  • Intervention structure: Experimental / Quasi-experimental / Observational
  • Assignment mechanism: Randomized / Rule-based / Natural / Self-selection
  • Temporal structure: Cross-sectional / Longitudinal / Time-series
  • Unit tracking: Repeated cross-sections / Panel / Cohort
  • Directionality: Prospective / Retrospective / Simultaneous
  • Comparison logic: Between-unit / Within-unit / Historical / External
  • Data provenance: Primary / Secondary / Mixed
  • Sampling strategy: Probability / Non-probability / Census / Selective
  • Unit of analysis: Individual / Aggregate / Multilevel / Ecological
  • Measurement modality: [Select applicable modalities]
  • Confounding control: [Select applicable strategies]

5. Identification Assumptions

  • Explicitly list key identification assumptions
  • Provide theoretical or empirical support for each assumption
  • Identify testable implications of assumptions

6. Limitations and Sensitivity

  • Explicitly articulate key limitations
  • Plan sensitivity analyses
  • Consider robustness checks

7. Alternative Designs

  • Document alternative dimensional configurations considered
  • Explain why they were rejected
  • Justify advantages of chosen design relative to alternatives

This checklist can serve as a tool for systematic design planning and justification.

This post is a draft for academic discussion. Comments and critiques are welcome.