Principles of Clinical Studies

OPBSResearch

Must Know

• A research question follows PICO. Participants, Intervention or exposure, Comparison, Outcome.
• One primary question gives one primary hypothesis, one primary outcome and the sample size calculation.
• Experimental studies are always prospective. The investigator controls the exposure.
• Systematic reviews and meta-analyses of level I studies sit at the top of the evidence hierarchy, then RCTs, then observational designs.
• Randomisation should spread unmeasured variables evenly between groups if the sample is large enough.
• Clinical equipoise means no reasonable basis exists for preferring one intervention for a given patient.
• Allocation concealment means the person enrolling patients cannot foresee the next allocation.
• Bias is systematic deviation from the truth and can enter at any stage.
• Risk of bias describes internal validity. Generalisability describes external validity.
• Efficacy is measured under ideal conditions (explanatory trials). Effectiveness is measured in real-world practice (pragmatic trials).

Study Designs

Design	How it works	Strength	Weakness
Cohort	Self-selected exposure groups followed to an outcome	Real-world, generalisable. Rare exposures	Groups may differ at baseline
Case-control	Start with cases who have the outcome and controls who do not, then look back for exposure	Rare outcomes and long latency	Recall bias. Exposure must precede outcome
Cross-sectional	Exposure and outcome measured at the same time	Quick, descriptive	Cannot establish cause, timing unknown
RCT	Random allocation, concurrent control, investigator controls exposure	Best for causation	Costly, slow, large samples. Poor for rare or distant events. Unethical for harmful exposures

• Effect modification means the exposure effect differs between subgroups, also called heterogeneity of treatment effect.
• A confounder distorts the apparent link between exposure and outcome.
• Subgroup and regression analyses explore effect modifiers. Regression can adjust for confounders that cannot be excluded.
• Correlation is one form of association. Neither proves causation.

Bias

Bias	Systematic difference in	Main protection
Selection	Baseline characteristics of groups	Randomisation and allocation concealment
Performance	Care or cointerventions received	Blinding patients and staff
Detection	How outcomes are measured	Blinded outcome assessors
Attrition	Withdrawals and missing data	ITT analysis, imputation, sensitivity analysis
Reporting	Reported and unreported findings	Keep outcome priority as in the protocol

• Outcome switching usually favours statistically significant results (selective reporting bias).

Sample Size and Error

• Every clinical study needs a sample size calculation, except case series and systematic reviews.
• Continuous outcomes need alpha, beta, the expected difference (delta) and its variance (SD or SE).
• Type I error (alpha) is rejecting a true null hypothesis.
• Type II error (beta) is failing to reject a false null hypothesis.
• Power is 1 minus beta.

Outcome Measures

• A good measure is valid, reliable, responsive and interpretable.
• Validity means it measures what it claims to. Types are content, construct and criterion validity.
• Reliability is repeatability. Forms are internal consistency, test-retest (intra-rater) and inter-rater.
• Responsiveness is the ability to detect small change over time (longitudinal validity).
• Precision is getting the same value on repeat measurement. Accuracy is closeness to the true value.

Diagnostic Tests

Term	Meaning
Sensitivity	Positive test in those with disease. Good for screening. SnOut
Specificity	Negative test in those without disease. SpIn
PPV	Chance a positive result truly has disease
NPV	Chance a negative result truly has no disease
ROC curve	Sensitivity plotted against 1 minus specificity. Area under curve reflects accuracy

• PPV and NPV depend on disease prevalence. Sensitivity and specificity do not.
• Prevalence is all cases at one time point. Incidence is new cases over a period.

Data and Statistics

Data type	Features	Example
Nominal	Unordered labels	Sex, implant brand
Ordinal	Ordered, unequal intervals	Mild, moderate, severe
Discrete	Whole numbers with order and magnitude	Heart rate in bpm
Continuous	Any value including fractions	Temperature, range of motion

Comparison	Parametric	Nonparametric
One sample	One sample t-test	Sign test, Wilcoxon signed rank
Two paired	Paired t-test	Wilcoxon signed rank
Two unpaired	Unpaired (Student) t-test	Mann-Whitney
Three or more	ANOVA	Kruskal-Wallis, Friedman

• About 68 percent of normally distributed data lie within 1 SD of the mean.
• Positive skew has a long tail to the right. Negative skew has a long tail to the left.
• A p value is the probability of a result at least this extreme if the null hypothesis is true.
• A 95 percent CI means about 95 of 100 such intervals from repeated samples contain the true value.
• ITT analyses patients in their allocated group and preserves randomisation. Per protocol analyses only compliant patients.
• Last observation carried forward is single imputation. Multiple imputation is stronger.
• A data monitoring committee is independent and exists mainly to protect patient safety. Stopping rules are preplanned.

More on statistics theory

• Frequentist inference treats probability as long-run frequency over infinite repeats of an experiment.
• Bayesian inference updates a prior distribution with observed data.
• Continuous data can be grouped into categories, at the cost of granularity.
• Normality can be tested with Kolmogorov-Smirnov or Shapiro-Wilk tests.
• CIs show magnitude, direction and precision of an effect.
• Cluster trials must account for grouping such as site. Crossover trials must check carry-over effects.
• Dichotomous outcomes need event numbers and totals in each group. Inflate the sample for expected drop-outs.
• Patient incentives, simple interventions and frequent contact reduce missing data.

Exam Traps

• A p value is not the probability that the null hypothesis is true or that the result is due to chance.
• Cross-sectional studies are descriptive and cannot show cause and effect.
• A study can have only one primary outcome.

Related pages. Level of Evidence, Grades of Recommendation, Landmark Trials

Recent Review. Bayesian Trials in Trauma and Orthopaedics

• Bayesian analysis rests on Bayes theorem. P(A|B) equals P(B|A) multiplied by P(A), divided by P(B).
• A prior is the probability held before the trial. Combining it with new data gives the posterior probability.
• Reference or minimally informative priors let the trial data drive the result.
• Data driven priors draw on earlier RCTs or meta-analyses and can be downweighted to test their influence.
• Priors must be set and declared before the trial starts.
• Results are reported as a posterior probability with a 95 percent credible interval (CrI), not a P value.
• Frequentist testing gives a yes or no verdict at P 0.05 and ignores what was known beforehand.
• Bayesian methods suit small, underpowered or low incidence trials, where a type II error is likely.
• They also help long term trials where loss to follow up shrinks the sample below the power calculation.
• A post hoc Bayesian reanalysis does not replace proper powering and a prespecified MCID at the design stage.

More on Bayesian trials

• In the linked WHiTE 8 reanalysis, the posterior probability that standard cement carried more 90 day infection was 92 percent.
• That analysis gave a posterior median relative risk of 1.42 with a 95 percent CrI of 0.87 to 2.36.
• Bayesian designs allow a flexible sample size and adaptive multi arm trials.
• UK-REBOA used a Bayesian design to recruit 90 rare major trauma patients over four and a half years.
• Many large orthopaedic RCTs risk type II error from overestimated control event rates, high loss to follow up or under recruitment.

Farrow L, Hudson J, George A, Reed MR, Campbell MK. Clinical trials in Trauma and Orthopaedics. Time to think Bayesian? Bone Joint J. 2025;107-B(9):862-4. PubMed

Recent Review. The MCID and PROMIS

• The MCID is a threshold of score change a patient would perceive as a real change in health state.
• It suits judging whether one patient improved and what proportion of a cohort reached the threshold.
• Comparing a cohort mean change with the MCID is less useful, as large responders skew the mean.
• Anchor based MCID ties score change to an external anchor, often a global rating of change from much worse to much better.
• Anchor methods use mean change in improved patients or an ROC cut point balancing sensitivity and specificity.
• Anchor weaknesses include recall bias and post hoc redefinition of the meaningful change threshold.
• Distribution based MCID uses cohort statistics such as half or one third SD, or the minimal detectable change, with no anchor needed.
• MDC95 equals 1.96 × √2 × SD × √(1 minus Cronbach α). MDC90 uses 1.65 and MDC99 uses 2.58.
• Distribution values are cohort specific and may not reflect clinical relevance.
• Patients with a baseline score within one MCID of the ceiling can never reach the MCID.

More on MCID variability

• PROMIS was an NIH initiative from 2004, built with classical test theory and item response theory and normed to the US population.
• Of 433 orthopaedic papers citing an MCID, only 8.8 percent calculated their own value.
• Spratt defines the MCID as 30 percent of possible improvement, maximum minus baseline score.
• Asher uses a 30 percent fall from baseline disability.
• In 564 lumbar decompressions, PROMIS PF MCID ranged from 2.8 (one third SD) to 22.2 (MDC99).
• Attainment in that cohort ranged from 5.7 to 64.5 percent depending on the method.
• Up to 18.4 percent of patients were excluded by ceiling effect at the MDC99 threshold.
• Patients with fast recovery were less likely to return for follow up, so the follow up window changes attainment rates.
• Devji credibility criteria ask whether the anchor is relevant, the estimate precise and the anchor well correlated with the score.

Karhade AV, Bono CM, Schwab JH, Tobert DG. Minimum clinically important difference. A metric that matters in the age of patient-reported outcomes. J Bone Joint Surg Am. 2021;103(24):2331-7. PubMed

Recent Review. Decision-Analytic Economic Evaluations

• A decision-analytic model combines inputs such as event probabilities, costs and utilities from several sources using mathematical software.
• Models can project costs and outcomes over a longer time horizon than an RCT follows patients.
• The ICER is the difference in mean costs divided by the difference in mean effects between comparators.
• An ICER alone cannot show cost-effectiveness. It must be judged against an external willingness to pay threshold.
• NICE accepts about £20000 to £30000 per QALY gained in England and Wales.
• Decision trees suit short problems but branch rapidly. Markov models suit recurring events over repeated cycles.
• Markov models place each patient in one of a finite set of health states, with transitions between states each cycle.
• Probabilistic sensitivity analysis varies all parameters at once using probability distributions. Deterministic analysis varies one at a time.
• Cost-effectiveness acceptability curves show the probability of cost-effectiveness across willingness to pay values.

More on appraising economic models

• Health utilities are the preference weights that give the quality adjustment in a QALY.
• A health economic analysis plan should be deposited before analysis to limit selective reporting.
• Patient-level and discrete event simulations track individuals whose accumulated history drives their outcomes.
• Ignoring parameter uncertainty gives cost-effectiveness results a false precision.
• Internal validation replicates the model independently. External validation compares predictions against trial or other outside data.
• Value of information analysis can use a model to judge whether further research, such as a larger RCT, is worth funding.

Petrou S, Png ME, Metcalfe D. Interpreting decision-analytic modelling-based economic evaluations in orthopaedics. Bone Joint J. 2023;105-B(1):17-20. PubMed

Recent Review. Machine Learning Uses and Limitations

• Artificial intelligence is the umbrella term. Machine learning is a subset, and deep learning uses neural networks beyond 3 layers.
• Data are split into training (fit), validation (tune hyperparameters) and test (final evaluation) sets, often 80 to 20.
• Supervised learning uses labelled data for classification or regression. Examples are support vector machines, decision trees and random forests.
• Unsupervised learning finds patterns in unlabelled data, for example k-means clustering and principal component analysis.
• Discrimination is the ability to separate classes, measured by the ROC curve and C-statistic (0.5 chance to 1.0 perfect).
• Calibration is agreement between predicted and observed risk. A model can discriminate well yet calibrate poorly.
• Brier score reflects both discrimination and calibration (0 perfect, 1 worst) but depends on outcome prevalence.
• Overfitting is learning noise in the training data, so accuracy falls on new data. More data and early stopping help.
• Dataset shift is a mismatch between training and deployment data. A hip fracture model fell from AUC 0.78 to 0.52.
• Only about 6 percent of medical AI studies report external validation.

More on bias and explainability

• Garbage in, garbage out. No algorithm can rescue poor quality input data.
• NSQIP data are limited to 30 day follow-up, skewed to large teaching hospitals and prone to coding errors.
• Algorithmic bias arises from model bias, variance from sparse minority data, and outcome noise.
• Black box models trade interpretability for complexity. Multiple regression is easy to interpret but handles complex data poorly.
• Saliency maps mark the pixels that drive a classification. Activation maximisation shows what features a network learned.
• The AI chasm is the gap between model development and clinical deployment.
• About 41 percent of orthopaedic prediction studies had high risk of bias from incomplete TRIPOD reporting.
• A neural network identified implants on radiographs with 98.9 percent accuracy.

Alsoof D, McDonald CL, Kuris EO, Daniels AH. Machine learning for the orthopaedic surgeon. Uses and limitations. J Bone Joint Surg Am. 2022;104(17):1586-94. PubMed

Recent Review. Interpreting Machine Learning Studies

• Machine learning is a subset of artificial intelligence in which algorithms improve their performance through experience.
• Data preprocessing is the most demanding step, taking more than three quarters of the work.
• Internal validation tests the model on a separate part of the same dataset. External validation uses new data from the target population.
• AUC (C-statistic) of 0.5 equals chance and 1 is perfect discrimination. Report it with 95 percent confidence intervals.
• AUC bands are 0.6 to 0.7 moderate, 0.7 to 0.8 good, 0.8 to 0.9 excellent and above 0.9 outstanding.
• Calibration is agreement between predicted and observed risk. A perfect model has intercept 0 and slope 1.
• Brier score assesses probabilistic models, with 0 the best and 1 the worst. Lower log loss also means better prediction.
• Cohen kappa measures agreement against a reference standard, with 0.8 the usual minimum acceptable in healthcare.
• TRIPOD covers prediction models. CONSORT-AI and SPIRIT-AI cover interventional AI trials and protocols.

More on machine learning pitfalls

• Probabilistic output gives the likelihood of belonging to each class. Deterministic output assigns one class.
• Supervised learning predefines the possible outcomes, so the control group and reference standard must be appropriate.
• A fracture model built on young controls and osteopenic cases may detect osteopenia instead of fracture.
• Models may learn artefacts, such as a radiographer marker on the image, instead of the pathology.
• A model trained on one implant type may not apply to other designs without external validation.
• A confusion matrix tabulates predicted against actual classes.
• Interpretable machine learning, such as variable importance plots or heat maps, opens the black box.

Farrow L, Zhong M, Ashcroft GP, Anderson L, Meek RMD. Interpretation and reporting of predictive or diagnostic machine-learning research in Trauma & Orthopaedics. Bone Joint J. 2021;103-B(12):1754-8. PubMed

Source

Gagnier JJ. Principles of Clinical Studies. In. Aaron RK, editor. Orthopaedic Basic Science. Foundations of Clinical Practice. 5th ed. Philadelphia, Wolters Kluwer, 2021.

Practise Type X questions on this topic

Reviewed by Professor Phong Tran, Head of Orthopaedic Surgery, Western Health. Last updated 17 October 2026.