Skip to content
Advertisements

As Predicted

January 25, 2019

Did you really think that?

Aspredicted.org (click here), created in 2015 by the University of Pennsylvania, Wharton School’s “Credibility Lab” (click here), provides a simple way for researchers to make post hoc analyses clear by registering experimental details prospectively, and creating a short, time-stamped, pre-registration document to share and read.  By publicly registered their study designs, clinical triallists have done something similar for years. Aspredicted.org makes it easy for others to do the same.

The reason is that a registered experiment, conducted and analysed as planned, is far more credible than one which cannot prove that it didn’t stop when the result was positive or select the most favourable outcomes to report.

A suggestion

Experts and opinion leaders could raise their credibility in a similar way; by registering their opinions, and indicating in advance whether they would alter in response to new data. There would be advantages both for the expert, for the general public and for researchers.

Imagine, for example that an expert judged that an experiment had a design flaw. If the expert revealed their objection only after the results appeared, perhaps in a direction not favoured by the expert, readers might suspect the expert was finding fault because he did not like the result. By committing to believe the results of a well-designed experiment the expert is indicating that his mind is open to new evidence. His reputation would rise.

The general public would have an objective way to judge who to believe; those who were prepared to commit (or not) to new evidence in advance.

Researchers would also gain, if the practice became popular, because expert criticism  of experimental design would sometimes allow them to alter the experiment before it was too late.

I’ve made a start.  Here are my first five predictions.

PRISM (click here), CSTICH (click here), EPPIC (click here), PHOENIX (click here), PITCHES (click here)

I will explain more about each one over the coming days.

Jim Thornton

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: