A defensible method begins before data collection.
First define the decision, unit of analysis, population, constructs and plausible comparison. Then align measurement, sampling and analysis. Advanced statistics cannot repair an unclear question, invalid instrument, inappropriate sample or fabricated dataset.
1. FrameDecision, question and claim
2. DesignUnit, comparison and data source
3. MeasureConstructs, instrument and validity
4. AnalyseMethod, assumptions and uncertainty
Flagship reference entries
Does a reporting checklist choose the design?Separate design decisions from reporting guidance.Does reliability establish validity?Evaluate measurement properties as distinct evidence domains.Is there one sufficient SEM sample size?Use model-specific assumptions rather than universal thresholds.Can common-method bias be diagnosed by one test?Use mechanism-based design and sensitivity tests.Missing-data planning before collectionPlan mechanism assumptions before applying deletion or imputation.Power analysis inputs beyond sample sizeSpecify effect size, variance, design effect and alpha.Which design can answer the question?Review the full methodological standard.
Common failure points
- Starting with a preferred technique and reverse-engineering the question.
- Using convenience data while making population-level claims.
- Treating scale reliability as proof of construct validity.
- Collecting data before piloting language, response options and administration.
- Reporting significance without effect size, uncertainty, assumptions or alternative explanations.