Scraping the SCOPE barrel
Predicting pre-eclampsia
Between 2004 and 2008, the Screening for Pregnancy Endpoints (SCOPE) Consortium collected blood samples from 3529 nulliparous pregnant women at 14-16, and at 19-21 weeks gestation. The idea was to see if they could predict the development of various nasty pregnancy outcomes. In September’s BJOG the consortium report on three biomarkers – PlGF, sFLT-1 and endoglin (click here or here angioPET_BJOG_2013), as predictors of pre-eclampsia. According to the paper only PlGF has any predictive power, and that was modest.
SCOPE was a lovely study. It involved obstetricians from New Zealand, Australia, UK and Ireland, and unlike many previous observational studies in the field the design was registered here before any analyses began. In theory this should have prevented the researchers picking and choosing among biomarkers, or subtly altering the definition of pre-eclampsia to make their favoured biomarker look good.
Unfortunately – I grumbled about it at the time – no biomarkers were prespecified at study registration, so readers have to take on trust that all those tested were reported. The SCOPE website (click here) reveals other reports of lipids, clusterin, vitronectin and high-molecular-weight kininogen, as well as various “proteomic” and “metabolomic” analyses in subsets of the data, so there may have been a bit of cherry picking.
More worryingly, according to the study registration, the consortium planned to test prediction of pre-eclampsia occurring “at any stage during pregnancy after recruitment until delivery or in the first 2 weeks after delivery”. They also pre-specified a secondary analysis for predicting “early onset pre-eclampsia” defined as “pre-eclampsia resulting in delivery at <34 weeks”. However, neither of these outcomes are reported in the paper. Instead they report “pre-term pre-eclampsia” defined as “pre-eclampsia before 37 weeks”. Search as I might I cannot find this endpoint in the registry.
Choosing a new endpoint would be fine if the primary analyses had been reported elsewhere. Two years ago a clinical prediction paper in the BMJ (click here) used the pre-specified definition of pre-eclampsia, but reported no biomarker results. Nor is there any mention of an earlier biomarker report in the present paper, and the SCOPE website doesn’t reveal one either. This looks like data dredging.
Alere, the manufacturer of the PlGF test, “funded the retrieval and shipping of specimens and measured the angiogenic biomarkers”. Did they influence the choice of endpoint?
Jim Thornton
Ripe-tomato.org 
This was data from a pilot case-cohort study, not the full study. Regarding the full study, a panel reviewed 220 markers from Alere’s antibody repository (developed by phage display). Forty-seven assays were considered plausible targets based on a possible biological role in the disease. All assays were measured by Alere in all 15-week samples, blinded to clinical data/endpoint. Analysis of data was completed by SCOPE investigators. All data was reported. To be published without exception of any data.
Comment by Alere.
Thank you Paul, You’ve answered one of my questions – namely the markers reported were selected from a panel of 47 assayed. Can you say how the three were chosen? At random, because they were already believed to be the most promising or – perish the thought – because they turned out best?
And what about the other issue. Choosing to predict PE <37 weeks, instead of total PE or PE <34 weeks as planned. I presume you are saying that was a SCOPE investigators decision. I still wonder why they did it.
The three (pilot) and the 47 (full study; including the 3 angiogenic) markers were measured in two phases. No cherry picking from data. Most Investigators agree that pre-eclampsia is not a single disorder (mechanism of disease) and you’ll appreciate that predictive performance deteriorates from <34 weeks, to <37 weeks, to all weeks. Remember, this (pilot) is not the full study analysis, in which all endpoints (<34 weeks, all pre-eclampsia), including the same <37 week endpoint, are to be reported. To the Investigators and to Alere, given what we now know about the disease, a 37-week endpoint (potentially still driven by poor placentation) is logical.
It’s reassuring all 47 markers, and prediction of the other two endpoints are forthcoming. Pity the paper said nothing about being a pilot.
What will editors say about separate papers for different definitions of PE based only on GA difference? Will they smell salami? But that’s another issue I guess.
We read the above blog and subsequent comments with interest and feel compelled to clarify a few points.
1. The BJOG study of angiogenic factors to predict preterm preeclampsia was a case-cohort study design (1:4) performed in the first 3529 recruits into SCOPE. This was a pilot collaboration study with Alere. The study was prospectively designed by the SCOPE Principle Investigators, independent of Alere, and approved by the SCOPE scientific steering committee. The end point of preterm preeclampsia was utilised, instead of preeclampsia <34 weeks, due to the number of cases available in the first 3500 women. Once designed and approved, the samples from specific patient IDs, selected according to the study design, were then sent to Alere for measurement of the angiogenic biomarkers. The study was not a retrospective dredge of data, as Professor Thornton incorrectly alleges. At the time of the study, there was only biomarker data on women selected according to the pilot study design, not in all women recruited into SCOPE.
2. Subsequent to the pilot and after completion of patient recruitment, the full set of biomarkers was measured in all women. To date we have not published on prediction of preeclampsia <34 weeks, but a manuscript describing biomarker prediction of preeclampsia <34 weeks is in preparation and will be submitted in the next few weeks. SCOPE is an example of an ethical collaboration between industry and six universities, involving retrieval and shipping of tens of thousands of frozen specimens, construction of a customised quality-assured multiplex immunoassays to measure almost 50 candidate biomarkers and time to perform robust analyses. Alere has not influenced data analysis, nor planned publications. This could have been explained to Jim, if he'd enquired.
Allegations of salami slicing are unfounded. The three angiogenic biomarkers in the pilot study were well established as potential biomarkers in the literature and reasonable candidates to investigate as predictors of preterm preeclampsia. The larger study, where approximately 50 biomarkers (including the 3 angiogenic biomarkers) have been measured in the whole cohort, has been performed as a completely separate study. Results of the biomarker study in the total cohort will be published in detail, ensuring the wealth of information is in the public domain for clinicians and researchers working in the area.
Robyn North, Louise Kenny and Lucilla Poston on behalf of the SCOPE Consortium
Thank you all. I’m reassured that Alere did not influence the analysis or choice of endpoints and that they performed the assays without knowledge of the clinical data. I also take the point that other data in the public domain supported the choice of PlGF, sFLT-1 and endoglin. But, given that there are 50 other biomarkers still unreported, it’s still a pity that these three weren’t registered prospectively.
Way back in 2007 I wrote to Robyn suggesting registering the analyses prospectively and got sent the trial registration link. Since then I’ve grumbled more than once about the lack of a detailed data independent analysis plan to Lucilla and Louise, and always been reassured that “the SOP covers it”.
With regard to the choice of a third unregistered endpoint to report, I’ve said my bit. Others can judge.
But I guess I can ask now. Which will be the primary endpoint for the 50 biomarker study?
I’m sorry if writing this blog seems aggressive but, in my humble outsider’s opinion, it is failure to rigorously avoid data-driven analyses that has held back pre-eclampsia screening, as compared with say, Downs screening, where the endpoint is unambiguous.
We can’t turn the clock back, but Louise tells me SCOPE2 is ongoing… .
This SCOPE paper will be discussed in the BJOG twitter journal club on Monday 14 October at 1830 BST. It will be hosted by @maireadblack. #bluejc to join in. The main journal club will last one hour but for those who miss it, BJOG will capture all tweets for a further week.
Lively discussion but no contributions from SCOPE researchers or Alere at #bluejc last night, and little support for this paper. But maybe I’m biased.