Sampling Methods in Research: Probability and Non-Probability, with Real Data

Home › Research Methodology › Sampling Methods in Research

Below, we explain the main sampling methods in plain language and use our own live dataset, which is itself a non-random sample, to show what sampling bias looks like in practice.

Sampling methods at a glance

Example: the sample behind ScholarsColab’s data

To explain sampling, we look at how the data on these pages was sampled. Our data: 41,792 papers, mostly published between 1998 and 2026, by 967 ScholarsColab mentors (data updated 15 October 2026).

Step 1
Name the population and the sample. The population we might want to describe is all researchers. Our sample is the PhD researchers who mentor on ScholarsColab.

Step 2
Ask how the sample was chosen. Nobody picked our mentors at random; they chose to join. That makes it a non-probability, self-selected sample.

Step 3
Check what the sample looks like. Here is how our mentors’ papers are spread across fields:

Step 4
See the effect of sampling bias. If we used this sample to estimate how often researchers use questionnaires, the answer would be far too low, because fields that rely on questionnaires, such as psychology and education, are under-represented. 68 of the 41,792 papers in our data (0.2%) show this in their title: questionnaire / opinion survey. It is most common in Social sciences & education, where 2% of papers show it.

Step 5
Fix it with a better sampling method. A stratified sample, drawing a fair share of researchers from each field, would give a much more accurate picture.

Real questionnaire studies by our mentors

Each of these studies had to decide whom to survey and how to reach them:

How to choose a sampling method

1. Define your population precisely: who, where and when.
2. Check whether a complete list (sampling frame) exists. Probability sampling needs one.
3. Decide whether you need to generalize. If yes, prefer probability sampling; if you are exploring experiences in depth, purposive sampling may suit better.
4. Calculate the sample size needed for your analysis, using a power calculation for quantitative studies.
5. Plan for non-response, and record how many people you approached and how many took part.
6. Describe your sample honestly and discuss how it may differ from the population.

Common mistakes reviewers point out

• Calling a convenience sample “random”.
• Generalizing to a whole population from a small or self-selected sample.
• No sample-size justification.
• Not reporting the response rate, or ignoring who did not respond.

Frequently asked questions

Use this data in your own work

More research methodology guides

• Independent vs Dependent Variables
• Control Group vs Experimental Group
• Mediator vs Moderator Variables
• Validity vs Reliability (and Generalizability) in Research, with Real Data
• Retrospective vs Prospective Studies
• Data Collection Methods in Research
• Types of Study Design
• Research Hypothesis
• Literature Review vs Systematic Review vs Meta-Analysis, with Real Data
• Experimental vs Quasi-Experimental Design
• Correlation vs Causation
• Deductive vs Inductive Reasoning in Research, with Real Examples
• Incidence vs Prevalence
• Descriptive vs Inferential Statistics, Explained with Real Data
• P-Value and Statistical Significance
• Operationalization
• Surveys and Questionnaires in Research
• Qualitative Research Methods
• Bias in Research
• Research Gap, Problem Statement and Research Question, Explained with an Example
• Pearson Correlation Coefficient (Pearson’s r)
• Chi-Square Test Explained
• ANOVA and the Kruskal-Wallis Test, Explained with Real Data
• Regression Analysis Explained
• Standard Deviation and Variance
• Post Hoc Tests Explained
• Confidence Intervals Explained
• t-Test vs Mann-Whitney U Test
• Effect Size Explained
• Heteroskedasticity Explained