t-Test vs Mann-Whitney U Test: Which to Use, with Real Data

Home › Research Methodology › t-Test vs Mann-Whitney U Test

Quick answerBoth tests compare two groups. The t-test compares their means and works best with roughly normal data. The Mann-Whitney U test compares their ranks, so it works with skewed data, outliers and ordinal scales. If your data are heavily skewed, the Mann-Whitney U test is usually the safer choice.

Below, we compare the two tests and run both live on the same real data from ScholarsColab.

t-test vs Mann-Whitney U test at a glance

Independent t-test Mann-Whitney U test
Compares Means Ranks (whether one group tends to have higher values)
Data Numeric, roughly normal Numeric or ordinal, any shape
Sensitive to outliers? Yes Much less
Also called Student’s t-test (Welch’s version for unequal spreads) Wilcoxon rank-sum test
Paired version Paired t-test Wilcoxon signed-rank test

Worked example: are journal papers cited more than conference papers?

We compare two groups with both tests, notebook-style, on real publication data. Every result is calculated live, and you can run the same code yourself.

The data: 41,792 papers, mostly published between 1998 and 2026, by 967 ScholarsColab mentors (data updated 15 October 2026). The file you will load is anonymised: no paper titles, journal names or researcher identities.

Step 1: Load the data

In [1]:
import numpy as np
import pandas as pd
from datetime import date

# The ScholarsColab Research Methods Index (papers level): anonymised, updated daily
DATA = "https://scholarscolab.in/wp-json/scholarscolab/v1/research-methods-data?level=papers"
papers = pd.read_csv(DATA, comment="#")   # lines starting with # hold the citation

def p_text(p):
    return "< 0.001" if p < 0.001 else f"= {p:.3f}"

print(papers.shape)
papers.head()
Out [1]:
(41792, 15)
yearcitationsvenue_typevenue_grouptitle_wordstitle_colonreviewsystematicmachine_learningdeep_learningiotblockchaincovidsustainabilitydiscipline
02023.031journalOther journals12000000000Medicine & health
12024.024journalOther journals13000110000Medicine & health
22024.012journalOther journals16000000000Medicine & health
32024.09journalOther journals20010000000Medicine & health
42024.09journalGeneric "International Journal of"9000000000Medicine & health

Step 2: Describe the two groups

In [2]:
THIS_YEAR = date.today().year
window = papers[papers["year"].between(THIS_YEAR - 16, THIS_YEAR - 4)]
journal = window.loc[window["venue_type"] == "journal", "citations"]
conference = window.loc[window["venue_type"] == "conference", "citations"]
print(f"Journal papers:    n = {len(journal):,}, mean = {journal.mean():.1f}, median = {journal.median():.0f}")
print(f"Conference papers: n = {len(conference):,}, mean = {conference.mean():.1f}, median = {conference.median():.0f}")
Out [2]:
Journal papers:    n = 16,491, mean = 42.7, median = 8
Conference papers: n = 4,101, mean = 12.8, median = 5
Journal papers have a higher mean and a higher median than conference papers. In both groups the mean sits well above the median: both are heavily skewed.

Step 3: Look at the two groups with a box plot

In [3]:
import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(9, 3))
ax.boxplot([np.log10(journal + 1), np.log10(conference + 1)], vert=False, showfliers=False)
ax.set_yticks([1, 2], ["Journal", "Conference"])
ticks = [0, 10, 100, 1000]
ax.set_xticks(np.log10(np.array(ticks) + 1), ticks)
ax.set_xlabel("Citations (log scale)")
ax.invert_yaxis()
plt.show()
Out [3]:
Box plot of citations by venue group0101001000Citations (log scale)JournalConference
Each box covers the middle half of the papers in a group; the thick line is the median. Boxes further to the right mean more citations.

Step 4: Run the t-test and the Mann-Whitney U test

In [4]:
from scipy import stats
t_raw, p_raw = stats.ttest_ind(journal, conference, equal_var=False)
t_log, p_log = stats.ttest_ind(np.log(journal + 1), np.log(conference + 1), equal_var=False)
u, p_u = stats.mannwhitneyu(journal, conference, alternative="two-sided")
print(f"Welch t-test, raw citations: t = {t_raw:.2f}, p {p_text(p_raw)}")
print(f"Welch t-test, log citations: t = {t_log:.2f}, p {p_text(p_log)}")
print(f"Mann-Whitney U test:         U = {u:,.0f}, p {p_text(p_u)}")
Out [4]:
Welch t-test, raw citations: t = 10.71, p < 0.001
Welch t-test, log citations: t = 20.80, p < 0.001
Mann-Whitney U test:         U = 39,750,998, p < 0.001
How to read it: 3 of the 3 tests find a significant difference. Compare the t-values: 10.71 on raw citations against 20.80 on log citations. The raw t-test is weaker, because a few extremely cited papers inflate the variation. On skewed data like this, the Mann-Whitney U test (or a t-test on log-transformed data) is the trustworthy choice; with smaller samples, the raw t-test could easily give the wrong answer.
Run this analysis yourself. Download the notebook and open it in Jupyter or Google Colab (File → Upload notebook). It loads the same anonymised ScholarsColab data, so your results will match this page.Download notebook (.ipynb) →Data files: papers.csv · mentors.csv

How to choose between them

  1. Plot each group with a histogram or box plot.
  2. If both groups look roughly symmetric without extreme values, use the t-test (Welch’s version if spreads differ).
  3. If the data are skewed, have outliers or are ordinal (such as Likert ratings), use the Mann-Whitney U test.
  4. If the same people are measured twice, use the paired t-test or the Wilcoxon signed-rank test instead.
  5. Report the matching summary: means and standard deviations for the t-test; medians and interquartile ranges for Mann-Whitney.

Common mistakes reviewers point out

  • Using a t-test on skewed data without checking or transforming it.
  • Reporting means for a Mann-Whitney test instead of medians.
  • Using an independent test for paired data.
  • Reporting only the p-value without an effect size.

Frequently asked questions

When should I use a Mann-Whitney U test instead of a t-test?

Use the Mann-Whitney U test when data are skewed, contain outliers, are ordinal, or when samples are small and you cannot assume normality.

Does the Mann-Whitney U test compare medians?

Not exactly. It tests whether values in one group tend to be higher than in the other. When the two groups have similar shapes, this is close to comparing medians.

What is Welch’s t-test?

Welch’s t-test is a version of the t-test that does not assume the two groups have equal variances. It is often recommended as the default t-test.

Is the Mann-Whitney U test less powerful than the t-test?

Slightly, when data are truly normal. With skewed data or outliers, it is often more powerful and more reliable.

What is the non-parametric equivalent of a paired t-test?

The Wilcoxon signed-rank test.

Use this data in your own work

Use this data in your own research. Download the aggregated figures behind these charts (counts, medians and shares by type of paper, period, method and discipline) as a CSV file. Please cite it as:ScholarsColab (2026). ScholarsColab Research Methods Index [Data set]. Vidhilekha Soft Solutions Pvt Ltd. Retrieved 15 October 2026, from https://scholarscolab.in/research-methodology/. Licensed under CC BY 4.0.Download the data (CSV) →

Need help applying this to your research?

Discuss your research problem with a PhD mentor, hand the whole project to a research project manager, or post your project and let matched experts apply.

Hire a Research Project Manager Hire Experts Browse Mentor Profiles

Mentors whose published work uses this approach:

More research methodology guides

About the data on this page. The ScholarsColab Research Methods Index is built from the Google Scholar publication lists of PhD researchers who mentor on ScholarsColab, and is updated daily as mentors and publications are added. Methods, designs and paper types are identified from titles, so papers whose titles do not mention them are not counted in those groups. The mentors are not a random sample of all researchers (engineering and computer science are strongly represented), so the figures describe this group of researchers and may not apply to research as a whole.