Archives for the 'Uncategorized' Category

Is ScienceExpress a bad idea?

The kerfufle about NASA’s arsenic DNA discovery a couple of weeks ago reminded me of the kerfufle in the summer about the centenarian GWAS study.  In both instances, the first thing I noticed was that I couldn’t download the papers.  I wouldn’t even classify myself as an open-access zealot, but when you’re at a big research university and you’re used to clicking on things and having them show up, that’s annoying as hell.  (Science eventually  relented and made the NASA paper available to all.  The longevity paper is still not available without a personal subscription, although the expression of concern is.)

I have no idea, but I wonder… do papers published via ScienceExpress have a higher rate of later significant revision or retraction than those published through the standard process?

It seems that the only real purpose of ScienceExpress is to try to force people to buy personal subscriptions to get early access to these “hot” papers.  The price we all pay is that they’re rushed to press, apparently without proper review, and then almost nobody can read them after they get breathlessly writen up in the newspapers.   It doesn’t serve anyone’s interests – not those of readers nor authors, and certainly isn’t good for Science.

12/16/2010 | Uncategorized | No Comments

VIII. clinical trial transparency references

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

[1]      “Clinical Research and the HIPAA Privacy Rule” National Institute of Health, US Department of Health and Human Services.

[2]      “Concept Paper: Quality in FDA-Regulated Clinical Research”

[3]      “Reporting on the findings of Clinical Trials: A Discussion Paper“ Buletin of the World Health Organization Vol 86 No 6 (June 2008)

[4]      Zarins, Deborah.  “Update on Clinical Trials Registration and Results Reporting Requirements; and Methodologic Issues and Next Steps in the Implementation of the Results Database”  Seminar.  February 24, 2009, Center for Health Policy, Stanford University.

[5]      Lenzer, J.  “Drug Secrets: What the FDA isn’t telling”

[6]      Office for Human Research Protections.  United States Department of Health and Human Services []

[7]      Public Meeting on U.S. Public Law 110-85.  National Institutes of Health.  Bethesda, MD.  April 20th 2009. []

[8]      McGoey L, Jackson E.  “Seroxat and the suppression of clinical trial data: regulatory failure and the uses of legal ambiguity”  J. Med Ethics 2009 35:109-112

[9]      Young D.  “New Clinical Trial Mandate Poses Concerns for Drug Firms”  March 24, 2008.  Bioworld Today.

[10]    PRS and U.S. Public Law 110-8


[12]    PHrmA.  “Principles on Conduct of Clinical Trials and Communication of Clinical Trial Results” April 2009

[13]    Bains, William.  “Fraud and Scandal in Biotech”  Nature Biotechnology 24, 745 – 747

[14]    Lurie P. and Pitts P.  “A Dialogue on Whether Disclosure Will Quiet The Industry’s Critics?” Oral Presentation.

[15]    Guo S-W, Hummelshoj L, Olive D, Bulun S, D’Hooghe TM, Evers JLH.  “A call for more transparency of registered clinical trials on endometriosis”  Human Reproduction 10.1093/humrep/dep045

[16]    Korn D., Ehringhaus S.  “Principles for Strengthening the Integrity of Clinical Research” PLoS Clinical Trials (2006)

[17]    H.R. 5605 “Physician Payments Sunshine Act of 2008.” [never became law]

[18]    H.R. 3580 / US Public Law110-85 121 STAT. 823 Sept 27 2007 “FDA Amendments Act of 2007″

[19]    McDonald D and Zisson S.  “Biopharmaceutical companies tackle clinical trial transparency”  in Clinical Trial Registries: A Practical Guide for Sponsors and Researchers of Medicinal Products.  Thomson Inc.  Boston, MA. (2006)  pp

[20]    Foote, MaryAnn. “Clinical Trial Registries and Publication of Results: A Primer” in Clinical Trial Registries: A Practical Guide for Sponsors and Researchers of Medicinal Products.  Thomson Inc.  Boston, MA. (2006)

[21]    Dwan, K et. al.  “Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias”  PLoS ONE. August 2008 Vol 3 No 8. e3081.

[22]    “GAP Client Exposes Flawed Procedure in Procter & Gamble Drug Study” Press Release.  Feb 22 2006.  Government Accountability Project, Washington DC.

[23]    Fleishacker, Sheila and Cohen, Mark.  “The ABCs of Drug Safety: Accountability, Balance, and Citizen Empowerment.  A Roadmap For Conducting Credible Trials and Protecting Drug Trial Participants”  Government Accountability Project, Washington, DC.  April 2009.

[24]    Washburn, Jennifer.  “Rent-a-Researcher: Did a British university sell out to Procter & Gamble?” Slate. 22 Dec 2005.

[25]    “Sponsorship, Authorship, and Accountability”  Annals of Internal Medicine.  18 Sept 2001. 135(6) 483-484.

[26]    Turner EH et al.  “Selective publication of antidepressant trials and its influence on apparent efficacy” New England Journal of Medicine.  2008 Jan 17;358(3):252-60

[27]    “Spitzer sues GlaxoSmithKline over Paxil”  Associated Press, June 2 2004.

[28]    “State of New York v. GlaxoSmithKline, No. 04401707″  New York State Supreme Court


[30]    Cahoy, Daniel. “Medical Product Information Incentives and the Transparency Paradox”  Indiana Law Journal Vol. 82:623-671.

[31]    Toigo, T.  “Food and Drug Modernization Act (FDAMA)  Section 113: Status Report on Implementation”  Biolaw and Business VOL 7, NUMBER 2.  2004.  US Food and Drug Administration.  Washington, DC.

[32]    Harris, Gairdner.  “Report Backs Up Warnings About Drug Avandia”  New York Times, July 27 2007.

[33]    Berensen, Alex.  “Data About Zetia Risks Was Not Fully Revealed” New York Times, Dec 21 2007.

[34]    Hughes, Sue. “Concerns raised on delay of ezetimibe data”  The Heart – HeartWire.  Nov 22 2007

[35]    McMaster W. et al.  “The Conduct of Clinical Research Under the HIPAA Privacy Rule”  Journal of Bone & Joint Surgery.  2006;88:2765-2770

[36]    Feuerstein A.  “FDA Lunches Inquiry Into Telik Trial”, June 13 2007.

[37]    FDAMA Section 113:  Status Report on Implementation.  US Food and Drug Administration.  August  2005

[38]    Turner, E, Moaleji N, Arnold B.  “Closing a Loophole in the FDA Amendments Act”  Science. Vol 322 (42-45)

[39]    “TGN1412” Wikipedia.  Accessed 26 April 2009.

05/18/2010 | Uncategorized | No Comments

VII. Conclusions: Towards Greater Clinical Trial Transparency

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

Transparency in clinical trial results is not a panacea.  New laws and policy measures to improve transparency must carefully balance the interests of the various stakeholders, including the industrial trial sponsors, investors in those companies, patients and trial subjects, doctors, and governmental agencies including the FDA and the NIH.  However, the numerous safety and reporting problems that have occurred suggest that our current bias should be towards greater transparency, rather than greater protections for industry.  Given the complexity and numerous constituencies involved in performing clinical trials, it unlikely that there is a single law or policy measure that could guarantee publication of trial data.  In fact, a strict law could potentially backfire and result either in a chilling of research altogether,  the movement of clinical research to different locales, or design of trials specifically to avoid reporting requirements.

Instead, a broad range of measures should be applied, both by gradual extensions of FDAAA requirements and pressure from the front lines of clinical trials, with the goal of gradually introducing a culture of transparency and availability of primary results.

Phase I Trials also be added to the reporting requirements of FDAAA, rather than being optional.  The preponderance of evidence from pharmaceutical company behaviour over the last decade seems to be that they cannot be relied upon to naturally do the right thing.  Given that Phase I trials are intended to evaluate safety, they are highly relevant to protecting patients from risks, and there is potentially much to be learned even from a drug whose development is terminated due to safety issues.  In the current environment, data from such a situation might still never become available, and future trials with similar interventions could be performed and put subjects at unnecessary risks.   In addition to the importance of publishing outcomes of even Phase I trials, protocols should also be made available as early as possible, so that experts outside of the FDA can review them.   For example, the tragedy involving the phase I trial of TGN1412 could have been prevented if the Phase I protocol had been subjected to more outside review, which could have predicted the possible risks and proposed a safer protocol in which subjects were not all dosed simultaneously.  The risk to the company of having competitors know about the existence and timing of the study would have been minimal, and avoiding this mishap would have been much better for the company overall, which failed along with the TGN1412.

FDA funding should be increased, so that the agency can operate with greater flexibility and independence.  At present, a significant portion of the FDA budget comes from user fees that are collected from companies submitting trials for evaluation.  Although there is no evidence that these fees directly influence the outcomes of evaluations, it is likely that the FDA would be able to operate with greater independence and confidence if its funding was more secure and if more inspections could be done.

Finally, the grassroots measures described in the previous section could be implemented on a limited scale.  Institutions, such as a university or hospital should set standards for clinical investigator agreements that require all data (including Phase I trials) to be made publicly available in at the conclusion of a trial.  Imposing such a standard would cost very little, but could be implemented quickly and will not require a protracted legislative process.  Individual physicians could also refuse to participate in trials that did not make data available.  For example, in the Actonel case described in the first section of this paper, the initial controversy arose because the clinical investigator had agreed contractually to cede publication rights to the company and to keep data confidential.  Stronger guidelines at the outset of a trial would prevent obfuscation and withholding of data.   Lastly, patients should also consider refusing to sign away access to their own clinical data.   Given the difficulties in finding and enrolling patients in clinical trials, patients are in a strong position to negotiate on some of the terms of their participation.  In conjunction with the full implementation of FDAAA and the possible extensions discussed, grassroots efforts to insist on greater clinical trial transparency could help ensure faster detection of safety risks, faster disclosure of drug development realities for investors, and better data with which doctors and the medical community can deliver evidence-based care.

05/18/2010 | Uncategorized | No Comments

VI. Bottom-Up Clinical Trial Transparency

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

Legislation, including the FDAAA of 2007, does appear to be leading to greater transparency in clinical research.  Legislation is relatively slow to modify, however, and its passage may be affected by the lobbying from the large and influential pharmaceutical industry.  A potential alternative approach to encouraging transparency may come from a recognition that the performance of clinical trials depend fundamentally on the voluntary participation of patients as research subjects, physicians who recruit patients and execute the trials, and the institutions that enable and provide permission, through Institutional Review Boards, for trials to occur.   The individual doctors who ultimately carry out clinical trials could insist on greater data transparency before agreeing to clinical research contracts.  In fact, if only a few individual doctors chose to refuse to conduct clinical trials without guarantees of transparency and submission of results to public database, they might exert considerable public relations pressure on the industrial sponsors, without necessarily causing disruption to the entire clinical research enterprise, as might occur if a large institution immediately adopted a similar position.  However, institutions do have a responsibility to help coordinate and legitimize these grassroots efforts to increase transparency.  Even a small institution could exert a significant influence by developing policies requiring that clinical trial investigator agreements.  Most institutions already provide checklists for investigators that must be signed off before and investigator joins a clinical trial; these checklists could be extended to make investigators more aware of trial data submission policies.

Patients could also exert a much greater influence on the clinical trials to which they subject themselves.  Informed consent should include information not only about the purpose and risks of a trial, but also information about outcomes.   HIPAA protects the information about patients that is stored electronically, and requires patient consent to use or transmit.  Under HIPAA, patients have access upon request to all of their own medical records that are stored electronically.  These health records include the results of clinical trials.  By default under HIPAA, patients will also have access to their own clinical trial data, unless a patient explicitly agrees to give up access while the trial is underway.  It is reasonable that patients should give up access for the duration of a trial; after the trial is complete, however, all medical information should be accessible.  Individual rights to information thus provide a route to clinical trial transparency that does not depend on the cooperation of trial sponsors, or even of clinicians.  Patients simply need to be better informed of their rights and refuse to contractually give up access to personal data when granting  informed consent.   If patients could access records after a trial had concluded, they could choose to make their own data available.  Since the information is accessible electronically, these records could be potentially be re-aggregated by third parties for analysis without further participation or cooperation of the trial sponsors.

05/18/2010 | Uncategorized | No Comments

V. Legislative Strategies to Encourage Clinical Trial Transparency

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

Is current legislation sufficient to ensure the clinical trials of high integrity?  I will review the US legislation that governs clinical trial reporting, starting with HOPE act in the 1980s and leading to the the FDA Amendments Act 2007 (FDAAA).

The HOPE Act of 1988 was the first major law intended to increase the transparency of clinical trials.  Section 2317 of the HOPE Act directed the Department of Health and Human Services to “collect and disseminate information about HIV research, treatment, and prevention” [31]  The HOPE Act led to the establishment of the first public, computerized databases for clinical trials, ACTIS, which recorded clinical trials for HIV/AIDS, and was intended to help both patients become aware of clinical trials of new HIV drugs and help physicians and researchers to more easily recruit subjects.

A decade passed before the next major legislative initiative to improve clinical trial transparency.  The Food and Drug Administration Modernization Act of 1997, known as FDAMA, contained important new directives to expand clinical trial registration.   Section 113 of FDAMA established and required registration of all clinical studies for “serious or life-threatening conditions,” which meant that not all clinical trials were necessarily submitted.  FDAMA did not insist on the publication of trial results — FDAMA allowed for results to be posted, but these results were required to have already been published in a peer reviewed journal.  The data that FDAMA required to be submitted to includes: “a description of the purpose of each experimental drug; patient eligibility criteria; a description of the location of clinical trial sites; and a point of contact for patients wanting to enroll in the clinical trial.”   Clinical was originally intended primarily as a clearinghouse to allow patients better access to clinical trials, rather than explicitly aiming to encourage dissemination of trial results.

FDAMA provided a valuable first step, but by 2005 it was apparent that not all parties were in full compliance with FDAMA.   An FDA investigation compared the trials submitted to to the primary registrations of trial protocols at CDER.  Only 35% (239/688) of the protocols that should have been submitted to were actually in the database.   Companies are also able to submit voluntary listings for trials that are not strictly required by FDAMA.  Only 35 voluntary listings had been contributed to the database, representing only a miniscule fraction of all Phase I, II, and III trials that had been performed between 1998 and 2005.  NIH, which also performs many clinical trials, had a much higher rate of registration with (90%) vs. industry (30%).  This indicates that industrial sponsors of trials are still somewhat reluctant to make even basic information about trials easily accessible.  It may be necessary to exact stricter penalties for non-compliance with the FDAMA requirements, or to encourage participation positively by offering benefits such as accelerated review to companies that file these mandatory submissions properly.

The most recent legislation affecting reporting of clinical trials is the FDA Amendment Act of 2007, or FDAAA.  Section VIII of FDAAA mandates registration of all trials other than phase I trials for drugs, biologics, and devices, going far beyond the “drugs for serious or life-threatening conditions” that were the main target of FDAMA.  The requirements of FDAAA are narrower than recommended by policy documents from the ICMJE and the World Health Organization [3],

In addition to requiring clinical trials to be registered, FDAAA requires that clinical trial registrations on be linked to any results already available on the FDA website, to make it easier for researchers to access results that are already available.  Going beyond mere clinical trial registration, FDAAA required that the “Basic Results Database” be set up by the National Institutes of Health by September 2008.  This database includes information about a) participant demographics and baseline characteristics;  b) primary and secondary outcomes and statistical analyses; c) disclosure of agreements between sponsors and nonemployees that restrict researchers from publishing results (although such agreements are not prohibited).  FDAA also sets time limits to ensure that trial data is reported promptly; data must be made available in the Basic Results Database within 12 months of trial completion or 30 days of FDA approval.  Data on adverse events must be collected and reported within 2 years.  Importantly, FDAAA specified a three year process to evaluate its performance and assess the need for additional regulations to further increase transparency.   The FDAAA doesn’t attempt to solve every possible issue – rules governing informed consent, research ethics and conduct are not covered, as they are assumed to be regulated by existing laws such as CFR 45 Part 46 and the Belmont Report.

FDAAA only works prospectively, for trials started after the bill became law.  The vast majority of drug trial data remains locked inside the FDA and pharmaceutical companies.  This represents a rich dataset that could be useful for evaluating the safety of current drugs (almost all of which were approved based on studies done before FDAAA) and predicating the safety and efficacy of new ones.  Although it may not be practical or cost-effective to mandate the wholesale publishing of all of this data, it would be beneficial to have some provision (other than the slow and relatively ineffective Freedom of Information Act requests) for researchers to access this important data [38].

The FDAAA also does not require the submission of Phase I data.  This is a major limitation, because Phase I studies constitute the largest group of investigations and explicitly assess safety, which is of primary concern to the FDA.   The results of Phase I studies are also closely watched by investors, and the failure to publicly disclose trial protocols and results can allow companies to sweep failures under the rug while trumpeting successes, and could give investors a misleading impression of a company’s success rate and prospects.

FDAAA does not cover observational studies, such as post-approval safety monitoring.  As more such studies are performed and more frequently used to guide clinical decisions, it will be important that the results of observational studies also be recorded in an accessible database like

Another area that the FDAAA does not address is provision of greater visibility into regulatory decision-making at the FDA.  Under FDAAA, the responsibility is on the trial sponsor to submit data to  Many of the problems with drug safety in recent years, however, have involved data submitted to the FDA.

One of the open questions surrounding open results databases is the extent of narrative descriptions of results that should be allowed.  There is considerable danger that a trial submitter would attempt to describe their results in as favorable a light as possible if given the opportunity to explicate their results in written narrative.  Such writing would likely be the first thing to be read by a user of the database, and would have undue influence because of the difficulty of understanding and interpreting raw data.  On the other hand, if narrative summaries are not allowed, the database becomes harder to use and understand.   One possible solution to this dilemma is to permit multiple narrative descriptions of results, layered on top of the core tabular database.  If the trial sponsor wishes to provide a narrative description, they may – however such a description could be commented on or challenged by other (validated and qualified) users of the database.  If  others had alternative interpretations of a dataset, they would be able to post and comment on those interpretations as well.  Encouraging both availability of the data and dialogue should help to mitigate the risk that narrative summaries could be used to mislead users.

Drug companies have responded to demands for greater transparency in public meetings with FDA and NIH [7], and have suggested that no additional data submission requirements should be imposed until trials have been conducted to prove value of result disclosure.  Of course, until results are disclosed systematically, it is virtually impossible to conduct such a trial.  The other major concern raised is that result submission to public databases might constitute prior publication, and prevent results from being submitted to academic journals.  However, journals have already agreed that database submissions to do not constitute publications, and will not affect publication decisions.  The industry has also created its own database,, although it is far from clear whether these efforts are sincere, or an attempt to create an illusion of voluntary transparency to avoid additional regulation.

05/18/2010 | Uncategorized | No Comments

IV. Benefits & Costs of Transparency

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

As Part II showed, there are frequent deficiencies in the publication of clinical trial results.  Increasing the transparency of trial data brings benefits and risks that must be balanced.  Potential costs of enforcing transparency include the disclosure of valuable, proprietary information that belongs to study sponsors, the possibility that disclosure rules could have a chilling effect on all clinical research by encouraging torts, the risk that trial sponsors would change study locations to evade regulation, and the practical problem that more transparent clinical trial data could be misinterpreted by other researchers or by the public, leading to unjustified fears or confusion.

The first concern about transparency is the need for drug developers to maintain the confidentiality of their products and plans.  Today, this is the major justification for the FDA refusing to release clinical trial data in response to FOIA requests.  However, since the information is not released, it is difficult to assess whether it legitimately has proprietary business value, and the proprietary information label might well be applied liberally to stonewall those seeking information.  We can consider three ways in which trial data might be proprietary, but will find that these arguments are unlikely to be justified.  First, a trial protocol could contain proprietary data about the drug or device under investigation, such as the chemical structure of a new chemical entity, or a diagram of a novel implantable device.  Such information is clearly of proprietary value.   However, by the time that a product is actually in clinical trials, the patents to protect these novel drugs or devices would long since have been applied for.  Only in rare cases would a product contain trade secrets for which the manufacturer was not seeking patent protection.   Since a patent application would have already been submitted, releasing information about new products in documentation about clinical trials would not constitute disclosure or jeopardize issuance of the patent.  It is possible, however, that a provisional patent had been applied for but had not yet made public as a full application.  Given the relatively long time that it takes to plan and execute a clinical trial, this seems unlikely, as a provisional patent would become public within 18 months.  However, if the situation did arise, it would be reasonable to permit non-disclosure of data until either the provisional application became a full application and was published.

One of the most important concerns about research transparency is the risk of creating a disincentive to do any research at all, because having accessible information could create tort liabilities.  A company would prefer not to know about a safety risk at all, than to conduct a study and expose itself to being sued because of what is learned.  [30]  This “transparency paradox” must be addressed if measures to increase transparency are to be effective.   In [30], Cahoy proposes two approaches to neutralize the transparency paradox.  The first approach is to tightly connect information production to limitations on tort liability.  In return for generating and publicizing information about a trial, a company would obtain immunity from tort liability based on the release of that information.  Companies would still be able to keep information secret if they so chose, but their interests would be better aligned with that of the public, because not keeping some damning data secret would have the advantage of reducing their liability, in addition to benefiting the public interest.  One possible way to do this would be, as Cahoy suggests, to create a statutory exception in federal evidentiary law to exclude the results of a controlled clinical trial from cases filed after the disclosure that claim a design defect, failure-to-warn, or negligence.

Another potential risk of enforcing clinical trial transparency is that sponsors might move trials to different locations in which they were not subjected to such rules.  Such a move could again paradoxically reduce transparency, and would also raise the risk that trials might not be adequately supervised in other respects.  For example, the Institutional Review Boards in a developing country might not be as stringent, or research subjects might not be as well informed of trial risks.  Although there are international guidelines governing both trial conduct and publication of results, these guidelines, which written by organizations such as the WHO and UN committees, cannot be easily enforced.   The risk of trials actually moving due to regulation is likely overstated, however.  In order to be considered for US marketing approval by the FDA, trials must be performed to the same standards even if conducted overseas.  Moreover, the US remains one of the largest medical markets, and offers access to more potential research subjects and doctors than most other regions.  Since subject recruitment is often the major bottleneck in conducting clinical trials, it is unlikely that a trial sponsor would give up access to the US patient population, even if new regulations encouraging transparency were to make operating in the US more difficult.

05/18/2010 | Uncategorized | No Comments

III. Clinical Trial Spin and Suppression

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

Why worry about transparency?  To justify new measures to enforce the reporting of clinical trials, it must first be shown that a problem exists with the status quo of results reporting.   We will consider both the global, statistical evidence that trial results are not adequately reported in the literature, and will then review a few specific instances of information suppression and biopharmaceutical malfeasance that might have been prevented by stronger incentives and rules to report trial results.

Several analyses have been done to assess the possibility of reporting and publication bias in clinical trials.  Dwan et al. [21] recently reported a comprehensive statistical evaluation of bias in controlled clinical trials.  Bias may take several forms.  Studies with positive results are much more likely to be published.   Dwan reports that even when negative results are published, they are susceptible to “time lag” or “pipeline” bias, which means that studies with negative results take longer to be published than positive studies.  The tendency to publish negative results may not even be malicious; negative results are generally seen as less interesting to the scientific community, so researchers may be less enthused about working to publish negative results, or feel that the negative results are not adequately interesting to justify publishing.  It is also very likely that trials with negative results are more difficult to get accepted for publication in top-tier peer-reviewed journals, so even if a clinical researcher makes their best effort to publish a negative result, doing so may be difficult or impossible.  In addition to bias in reporting the outcomes of trials as originally specified, another form of publication bias noted by Dwan is “within-study selective reporting bias,” which occurs when a subset of the original variables are chosen for reporting in a publication, such as reporting of different patient subgroups or time-points.

Dwan et al. performed a meta-analysis in which they reviewed the results of 16 different smaller studies that each evaluated a cohort of clinical trials to assess publication bias.  Eleven of these studies looked for publication bias, while five examined outcome reporting bias, by comparing final publication to the pre-specified trial protocols.  Approximately half of the trials evaluated were found to have at least one major outcome that was modified or omitted, suggesting that outcome reporting bias is prevalent and should be of concern.   Looking at publishing bias, across the sixteen cohorts, trials with positive results were consistently more likely to be published than those with negative results, and the total frequency of publication was less than 50%.  From the limited data available, it also appeared that this bias was due more to negative trials not ever being submitted for publication, rather than being more likely to be rejected after submission to a journal.

Turner et al. assessed evidence for selective publication of antidepressant trials [26].  Turner accessed trial results for drug approval studies directly from the FDA, and compared these results to outcomes reported (or not reported) by publication.   The fact that Turner was able to access FDA-reported results is itself unusual; it is difficult for researchers to even study whether or not publication bias exists when the existence of and data from unpublished studies is not available. Turner found that 31% of FDA-registered studies were not published at all.  37 studies with FDA-positive results were published; only a single study seen as positive was not published.  Most studies with negative or questionable results were either not published (22) or published in a way that conveyed a positive outcome (11 studies).  When primary outcome failed to have a positive significant result, a secondary result was often reported as the headline and the major failure was subordinated or omitted.  So according to the literature, it appeared as if 94% of the trials were positive, whereas FDA data analysis showed that only 51% were positive.

This statistical evidence shows that there are systematic problems in the reporting and publication of clinical trial results.  Several specific cases discussed below illustrate the potential harm that may arise when clinical trials are not transparent and their results not subject to adequate scrutiny.  Medicine generally attempts to be evidence-based, in which clinical methods are developed based on the empirical results reported in the literature; if the data in the published literature is incomplete or biased, evidence-based medicine cannot function effectively.  I will describe five cases that have occurred over the last decade which illustrate the risks and consequences of opaqueness in clinical trials.

Actonel, manufactured by Procter and Gamble, is a drug intended to help strengthen the bones of post-menopausal women, and competes with a similar drug called Fosamax made by Merck & Co.  In 2002, P&G contracted with a British physician, Dr. Aubrey Bluhmson, to conduct and analyze a study comparing the effectiveness of Actonel with Fosamax.  After the study concluded, P&G refused to grant Bluhmson access to the randomization codes required to analyze the double-blinded data, and provided Bluhmson with only a limited subset of the results that the researcher claimed biased the results to favor Actonel. [22]  P&G then published ghost-written publications about the Actonel studies under Bluhmson’s name, despite his disagreement with the study conclusions and lack of access to the dataset.   Even though there may be legitimate disagreements in the interpretation of such trials, it is impossible for third parties to assess the controversy in the absence of full, primary data set.  If the investigator’s allegations are true, patients taking Actonel may be at unnecessary risk and missing out on the benefits of a better drug.  This case illustrates the risks of clinical trial investigator agreements in which researchers agree to give up rights to access data, and to control authorship of publications, and reveals the difficulties that may derive from the conflicting interests of trial sponsors, individual investigators, their institutions, and the journals that publish results from clinical trials.  It also shows that even when the summarized results of a trial are published, it is difficult to evaluate the veracity of the conclusions without full access to the primary research data.  If the primary data had been deposited in a publicly accessible database, disagreements about the interpretation of results could be assessed by independent third-parties.

Glaxo Smith Kline has been involved in major controversies over two of its drugs.  Paxil (Paroxetine; Seroxat) is an anti-depressant medication in the family of selective-serotonin reuptake inhibitors.  Paxil was approved in 1992 for treatment of depression in adults, and grew to become a blockbuster that generated more than $1B in sales for GSK by 1997, and the set of indications  for which Paxil was marketed included panic disorder, social anxiety disorder, and premenstrual dysphoric disorder.  By 2006, evidence had come to light of significant problems with Paxil, including ineffectiveness and a significant increased risk of suicide when administered to children.

The New York State Attorney General sued GSK in June 2004, alleging that GSK withheld information from results of 5 internal studies showing that Paxil was ineffective and dangerous in children, despite having this information as early as 1999.  At the same time, Paxil was heavily promoted in for use in children.  GSK also found itself the target of a number of class-action suits related to Paxil safety issues.  The suit brought to light internal GSK memos that showed the company deliberately attempted to “manage the dissemination of data in order to minimize any potential negative commercial impact.” [27]

The suit against GSK was settled in August 2004 before going to trial [29].  Glaxo agreed to pay $14M and to modify their reporting behaviour.  GSK also agreed to create and update a public, internet-accessible database containing summaries of all sponsored clinical trials completed after Dec 2000.  Finally, GSK also consented to permit monitoring by the NY Office of the Attorney General, and to submit full clinical trial reports and raw data for inspection up request, subject to a reasonable confidentiality agreement.  The format and contents of the clinical trial summaries was specified in the agreement, and was based upon guidelines from recommendations of the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use, a joint effort between the USA FDA and European regulatory authorities.  This agreement provides a good framework for what should be industry-wide standards in reporting clinical trial data: a public, standardized database with full auditability.

Avandia (rosiglitazone) is another drug manufactured by GSK, used in the treatment of type II diabetes.  In 2007, Avandia was found to lead to a significant increase in risk of myocardial infarction, and some in the medical community claimed [32] that the FDA moved too slowly to warn doctors and patients about the risks from Avandia, despite having data about the risks as early as 2002.  Once again, while the scientific details about the true risks are debatable, it is impossible for physicians to objectively assess the data when it is not available.  In the case of Avandia, it is likely that the evasive behaviour of GSK actually did more damage than the results themselves; rather than engaging in a scientific debate about the risks and benefits of Avandia, GSK appeared to have something to hide, leading many doctors to switch patients to different drugs, and Avandia sales have been declining since 2006.

Zetia is a cholesterol-reducing drug marketed by Merck and Schering-Plough.   In 2007, the manufacturers came under criticism for failing to disclose evidence that Zetia posed serious risks of liver damage.  Although some of the data from the studies showing these risks had been reported to the FDA, neither the FDA nor the companies took any action.  Under pressure from the medical community, some of the data were disclosed via press releases. Concerns were also raised that registration of the trials were delayed, and that the primary outcome measures of the trial were modified after the trial was underway.   In this case, the data showing risks did not come from a clinical-outcome trial, showing that to protect patients, it is not sufficient to limit regulation to trials with a clinical outcome, as the FDAAA law currently does.

One of the most notorious instances in which patients were put at risk by a drug was Merck’s Vioxx.  A series of large clinical trials that showed a significant cardiovascular risk for Vioxx were performed, but the results of the trials were only made available to the FDA and not published for years after they had been completed.  Therefore independent researchers were not able to assess cardiovascular risk, and doctors continued to prescribe Vioxx widely.  The Vioxx case demonstrates the puzzling fact that the threat of tort liability is not a sufficient disincentive to pharmaceutical companies to prioritize patient safety and to publicize risks as soon as possible.  In the Vioxx case, the NEJM accused Merck of withholding certain data from publication that would have demonstrated a significant cardiovascular risk, while publishing the data that supported Vioxx efficacy.   If trial data was automatically deposited in public databases, it would relieve companies of these kinds of accusations.

Issues of drug safety risks and clinical trial non-disclosure are not limited to large pharmaceutical companies.  For small companies, whose businesses often depend on the development of only one or two products, the pressures to avoid exposing potential risks and failure may be even than greater than with larger companies that are much more diversified.   One case in which a smaller biopharmaceutical firm faced difficult disclosure issues was Telik, which was developing a cancer treatment called Telcyta.  After announcing at a conference that three of its trials had failed to show positive results and had shown patient deaths, Telik came under criticism from the FDA and investors for failing to disclose the findings earlier [36].

Finally, TGN1412 illustrates how poor communication of a trial protocol, even for a Phase I study, can place subjects at risk of dire harm.  TGN1412 was an immunomodulatory drug being developed by TeGenero Immuno Therapeutics for treatment of chronic lymphocytic leukemia and rheumatoid arthritis.   In phase I trials, conducted by a US contract research organization, PAREXEL at a site in England, healthy human volunteers were given a low dose, thought to be much lower than the dose found to be safe in animals.  In fact, the low dose caused a disastrous response that made all six volunteers gravely ill and nearly killed them.  After the incident, observers argued that the drug should not have been tested in multiple subjects simultaneously, and that the responses might have been predictable.  Had TeGenero made their study protocols available for outside review before the trial, these views might have been considered.

The cases described have illustrated the range of problems that are possible when clinical trial data is not accessible.   Patients can be placed at risk, because the economic incentives of companies are not aligned with maximizing patient safety.  Clinical trial results may be completely suppressed by sponsors, or they may be reported to the FDA but never published.  These examples show that solely relying on the FDA to scrutinize plans and results is insufficient.  The failures of companies and the FDA to promptly alert consumers of potential risks are not necessarily due to malfeasance; rather, the larger problem is simply that the FDA lacks adequate resources to adequately monitor and respond to all of the trial data to which it has access.  Even if there is no deliberate obfuscation or suppression of results, the limited resources at both FDA and at companies make it likely that data will not be sufficiently scrutinized and safety risks might not be promptly discovered.

05/18/2010 | Uncategorized | No Comments

II. Defining Clinical Trial Transparency

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

Defining Clinical Trial Transparency

There are varying degrees of transparency of clinical trials and the associated data.  We can distinguish three major types of trial transparency.  At the most basic level is trial registration.  Registration of clinical trials ensures that regulators and the public are aware of the existence of a trial.  Trial registration is a necessary for any of the other forms of transparency, but only in the last decade have there been serious efforts to encourage trial registration and open public access to registries.  Trial registries may contain only the most basic information describing the trial, the condition and intervention under investigation, and the investigators’ and trial sponsors’ identities.  A trial registry can also contain information about the trial protocol which should include a description of the trials’ intent, the mechanisms by which patients are selected, the pre-specified outcomes under investigation, and methods of analysis to be employed.  By requiring protocol registration before a trial starts, researchers can be prevented from changing the questions that they ask based on the data.

After a trial has been performed, the second level of transparency is reporting of summarized results.  An important consideration in the reporting of results is whether they are made available as data-only, as data with narrative summaries.   Although results databases help to make information more easily accessible, a risk of allowing narrative summaries is that they might make it possible for a submitter to introduce a biased interpretation of results without being subject to peer review.  Therefore, there have been strong arguments from the publishing community to restrict results databases to rigidly specified forms and tabular data, offered without interpretation, which is left “to the reader”.  The problem of how to balance the need for accessibility and comprehensible result summaries with the risk of allowing data to be subjected to spin and misinterpretation is an important open question that will likely require novel technical and policy solutions to fully solve.

Finally, the highest degree of clinical trial transparency is to make all primary data available for examination (with appropriate measures in place to ensure confidentiality for research subjects).  The advantage of making primary data available is that an outside scientist can assess whether analysis has been done correctly and potentially identify whether there are discrepancies or problems with the way that data was collected.  Full access to raw data poses many potential problems as well; external scientists (or worse, lay people) might misinterpret the data.  Exposing primary research data could also threaten patient privacy, unless any identifying information was carefully scrubbed.  Depending on the type of data recorded, it might even be difficult or impossible to fully anonymize the clinical data — genetic information such as SNP measurements (single-nucleotide polymorphisms) might be sufficient to uniquely identify a patient, even if biographical details are not available.

A related issue that I’m not going to address in this series is the question of disclosure of financial relationships between trial sponsors and the physicians who conduct studies [14].   Under the current system, the vast majority of clinical research is sponsored by companies that hope to have their investigational products eventually approved.  These companies compensate physicians directly for the costs of conducting the trials, and may also pay these clinical researchers for consulting or other services.  Not all research sponsored by companies is bad; indeed, there is evidence suggesting that industrially sponsored studies are better run and producer higher-quality data than studies funded solely from public sources, possibly because the company studies have better funding and larger incentives to be performed well in order to garner regulatory approvals.  However, there have been concerns raised that payments to doctors could be directly or indirectly affecting their scientific independence and clinical judgment.  In response to these concerns, the “Physician Payments Sunshine Act of 2008” [17] was a bill proposed by Rep. P. DeFazio in the 110th Congress.  The Sunshine Act would have amended the Social Security Act to “provide for transparency in the relationship between physicians and manufacturers of drugs, devices, or medical supplies for which payment is made under Medicare, Medicaid, or SCHIP, and for other purposes.”  The Sunshine Act would have required sponsors to make quarterly reports to the Department of Health and Human Services of all payments to physicians or professional medical organizations. The Sunshine act never became law, but illustrates how one element of the healthcare system, in this case the US Government as the funder of Medicaid and Medicare, can exert its fiscal power to encourage transparency.

05/15/2010 | Uncategorized | No Comments

‘internet addiction disorder’ and respecting attention

There’s a timely piece in the Times about taking a ‘secular Sabbath’ away from electronic distractions. Scoble and Roger Ehrenberg also recently lamented ‘attention thieves.’

I’m guilty too. There are precious few times in which I put away the blackberry: yoga class (@Karma) and dancing. And bed, I suppose. (usually).

Taking a break is obviously a good idea, but doesn’t address the core problem: what we really need are information tools that adequately respect and value our attention, and give us more control over our attention. I think there are three characteristics that many of these old and new applications need to adopt in order to let us focus better without cutting off cold-turkey: context, feedback, prioritization.


Traditionally, we had different physical places, times, and things for different tasks. To work, you went to the office. To read, you went to the library. If you were reading for work, you read the business section. If you were slacking off, you’d read the entertainment section.

We’re now free of such physical limitations, which is wonderful, but the result is that all of the streams of our activities flow to us simultaneously, and are presented to us merged.

We need to be able to set our context, and then have our software be humane and intelligent enough to respect it. Our IM status, for example, has busy / do-not-disturb settings. Such settings ought to be able to be applied across the whole gamut of information inputs, not just IM.

Feedback and Monitoring

Our information tools ought to help us monitor where our attention is going. Tools for analyzing clickstream, such as the early work from AttentionTrust and perhaps might help us see where attention is going. RescueTime is another neat application that lets us see where time/attention is going (or being wasted). And news.ycombinator has a noprocrastination setting that cuts one off after checking too many times. Making the behavioural changes to focus attention and lock out the attention theives is made much easier if software can provide the right sorts of feedback and incentives.


Software needs to do a better job at figuring out what messages are important and justify disruptions, and which can wait. The blackberry does a nice job of letting one configure different behaviours for different types of messages, such as ‘Level 1′ alerts that match a list of senders. But these tools are crude, and make use of little of the data which they might.

Humans are fallible, and media is often designed and evolve to steal attention, because it is valuable. Our software needs to be designed to recognize our finite cognitive limitations, avoid abusing our attention, and help us to stay disciplined.

Now back to real work!


03/03/2008 | attention, psychology, software, Uncategorized | No Comments

notes from the HBS Healthcare Conference

As all hell breaks loose in the markets, it’s probably healthy to step away from twitter for just a couple of minutes to take a longer-term perspective. I had the pleasure of attending the HBS Healthcare Club conference on Saturday, and thought I’d collect and post my notes on some of the big picture issues in healthcare and biotech that were discussed. I tended to stay at the product and technology focussed sessions, rather than those focussed on services and reimbursement, because, well, I’m a science guy at heart.

Sidney TaurelOutgoing Lilly CEO Sidney Taurel gave the keynote just after lunch, and he laid out the risks and changes facing the pharmaceutical industry (besides patent expirations!) and how Lilly is adapting to meet them.

  • Global aging trend / ‘inversion of the age pyramid’
  • Emergence of “Health Technology Assessment” agencies
    • limiting access to new technologies
  • Patent issues in developing countries
  • Legislative risks: government interference with Medicaid in the US
    • US is the last bastion of free markets
  • Changing perceptions of risk vs. benefit
  • Perceptions of drug costs
    • ~ 10% of healthcare spend, but consumers are much more exposed to drug costs through copays than to other costs in the system

Changes that are going to help address these challenges:

  • Individual patient outcomes
    • Measuring outcomes instead of outputs; and focussing on individuals
    • pharmaceuticals will become partly an information product
    • Better (adaptive) clinical trials; Regulators are slow to adapt.
  • Openness of information
  • R&D moving towards a virtual organization
    • FIBNET rather than FIBCO
    • Outsourcing examples at Lilly:
      • med chem moving to China and India (ChemExplorer in Shanghai doing 20% of global chemistry)
      • Innocentive
      • Chorus
  • Big opportunities to use IT to track outcomes after drugs come on the market
  • Open source models for biomarkers
  • Systems Biology (As a computational biologist myself, I’m somehow skeptical that this is going to come through for the industry in time to save them from patent expirations :-)
  • Sales: fewer people; more competent people
  • Merging of pharma, biotech (Biologicals are already a huge part of Lilly)

Given what Mr. Taurel had said about the importance increasing openness and better post-approval outcome monitoring, I was very tempted to ask something about how the industry was increasing transparency and whether Lilly did everything it could in making data available about drugs like Zyprexa, but unfortunately we ran out of time for questions.

Trends in Medical Devices

The most interesting thing to me was the near consensus across the panel about the inevitable convergence of drugs, devices, and data.

spineDr. Stephen Oesterle (one of the most provocative speakers, and certainly the funniest) suggested a few specific companies to watch. In drug delivery, Tempo Pharmaceuticals is developing nanoparticle delivery systems for (small molecule?) drugs that allow for control over release rate and timing of drug combinations. (Tempo just did a Series B for $8.1M from Polaris, Bessemer, and Lux Capital.) In biologics, he suggested having a look at Alnylam, which has most of the IP on RNAi therapeutics locked up. The importance of RNAi is motivated by a simplifed model of disease: all disease is caused by too much or too little protein. For those rare diseases caused by too little protein, we can give it back, via protein therapeutics, cell therapy, or gene therapy. Too much protein however, and we need RNAi, antisense, or antibodies. Biologics are going require clever and new delivery systems, and delivery technologies for biologics are still a big long-term opportunity. Finally, Oesterle suggested having keeping an eye on the spine. Back pain is huge, and current therapies (spinal fusion) are lacking, yet the spine is relatively simple and very accessible surgically, and therefore low hanging fruit for new devices and approaches.

Georg Nebgen said trends to watch included: disposable devices; battery powered device; and greater automation, diagnostics moving to point-of-care and away from central labs, and that acquisitions are moving later.

Another interesting trend proposed is the vanity of an aging population – a market that is happily self-paid by patients.

Communication between devices is going to be big: we’re going to see networks of monitors, and start to close the loop between diagnostics, treatment and discovery.  Better monitoring is going to let us give drugs episodically, excactly when they are needed and effective, instead of all the time.  Implanted sensors are going to start to happen. All these sensors and diagnostics are going to generate data that will require security, storage, and analysis of the resulting signals. Big opportunities; totally unclear who is going to do it yet. (Doctors? Google/Microsoft? Payors? NewCos?)

On the more mundane side of things, we’re going to continue to see big investments in informatics for clinical trials. Still a huge source of pain, with way too much paper.

The closing keynote was by Dr. Robert Langer, who somehow I haven’t heard speak before. Langer offered up a nice checklist of scientific characteristics of successful biotech startups, and then backed them up with examples of successful companies he had started, with the caveat that business issues — like the right team — generally matter more.

  1. Platform technology – applicable to multiple products
  2. Ideally a product company
  3. Seminal paper (Science / Nature)
  4. Seminal and blocking patents
  5. In vivo proof of principle

Overall, the conference was fantastic – congrats to the organizers for putting together a great panel of speakers and running everything so smoothly.

Tags: , , , ,

01/22/2008 | Uncategorized | No Comments