Resveratrol update

Maybe it’s just the bayesian in me, but where there’s smoke, there’s probably fire.

http://www.xconomy.com/boston/2010/12/01/glaxo-ends-resveratrol-drug-study/

12/16/2010 | drugdiscovery | No Comments

Is ScienceExpress a bad idea?

The kerfufle about NASA’s arsenic DNA discovery a couple of weeks ago reminded me of the kerfufle in the summer about the centenarian GWAS study.  In both instances, the first thing I noticed was that I couldn’t download the papers.  I wouldn’t even classify myself as an open-access zealot, but when you’re at a big research university and you’re used to clicking on things and having them show up, that’s annoying as hell.  (Science eventually  relented and made the NASA paper available to all.  The longevity paper is still not available without a personal subscription, although the expression of concern is.)

I have no idea, but I wonder… do papers published via ScienceExpress have a higher rate of later significant revision or retraction than those published through the standard process?

It seems that the only real purpose of ScienceExpress is to try to force people to buy personal subscriptions to get early access to these “hot” papers.  The price we all pay is that they’re rushed to press, apparently without proper review, and then almost nobody can read them after they get breathlessly writen up in the newspapers.   It doesn’t serve anyone’s interests – not those of readers nor authors, and certainly isn’t good for Science.

12/16/2010 | Uncategorized | No Comments

VIII. clinical trial transparency references

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

[1]      “Clinical Research and the HIPAA Privacy Rule” National Institute of Health, US Department of Health and Human Services.  http://privacyruleandresearch.nih.gov/clin_research.asp

[2]      “Concept Paper: Quality in FDA-Regulated Clinical Research” http://www.fda.gov/oc/initiatives/criticalpath/clinicalresearch.html

[3]      “Reporting on the findings of Clinical Trials: A Discussion Paper“ Buletin of the World Health Organization Vol 86 No 6 (June 2008) http://www.who.int/bulletin/volumes/86/6/08-053769/en/index.html

[4]      Zarins, Deborah.  “Update on Clinical Trials Registration and Results Reporting Requirements; and Methodologic Issues and Next Steps in the Implementation of the Results Database”  Seminar.  February 24, 2009, Center for Health Policy, Stanford University.

[5]      Lenzer, J.  “Drug Secrets: What the FDA isn’t telling” Slate.com http://www.slate.com/id/2126918/

[6]      Office for Human Research Protections.  United States Department of Health and Human Services [http://www.hhs.gov/ohrp/]

[7]      Public Meeting on U.S. Public Law 110-85.  National Institutes of Health.  Bethesda, MD.  April 20th 2009. [http://prsinfo.clinicaltrials.gov/public-meeting-april09.html]

[8]      McGoey L, Jackson E.  “Seroxat and the suppression of clinical trial data: regulatory failure and the uses of legal ambiguity”  J. Med Ethics 2009 35:109-112

[9]      Young D.  “New Clinical Trial Mandate Poses Concerns for Drug Firms”  March 24, 2008.  Bioworld Today.

http://www.bioworld.com/servlet/com.accumedia.web.Dispatcher?next=bioWorldHeadlines_article&forceid=47152

[10]    PRS and U.S. Public Law 110-8 ClinicalTrials.gov http://prsinfo.clinicaltrials.gov/fdaaa.html

[11]      http://scientific-misconduct.blogspot.com/search/label/Procter%20and%20Gamble

[12]    PHrmA.  “Principles on Conduct of Clinical Trials and Communication of Clinical Trial Results” April 2009

[13]    Bains, William.  “Fraud and Scandal in Biotech”  Nature Biotechnology 24, 745 – 747

[14]    Lurie P. and Pitts P.  “A Dialogue on Whether Disclosure Will Quiet The Industry’s Critics?” Oral Presentation. http://www.icvclients.com/ehcca/disclosure2009/1_500/

[15]    Guo S-W, Hummelshoj L, Olive D, Bulun S, D’Hooghe TM, Evers JLH.  “A call for more transparency of registered clinical trials on endometriosis”  Human Reproduction 10.1093/humrep/dep045

[16]    Korn D., Ehringhaus S.  “Principles for Strengthening the Integrity of Clinical Research” PLoS Clinical Trials (2006)

[17]    H.R. 5605 “Physician Payments Sunshine Act of 2008.” [never became law]  http://www.govtrack.us/congress/bill.xpd?bill=h110-5605

[18]    H.R. 3580 / US Public Law110-85 121 STAT. 823 Sept 27 2007 “FDA Amendments Act of 2007″ http://frwebgate.access.gpo.gov/cgibin/getdoc.cgi?dbname=110_cong_public_laws&docid=f:publ085.110.pdf

[19]    McDonald D and Zisson S.  “Biopharmaceutical companies tackle clinical trial transparency”  in Clinical Trial Registries: A Practical Guide for Sponsors and Researchers of Medicinal Products.  Thomson Inc.  Boston, MA. (2006)  pp

[20]    Foote, MaryAnn. “Clinical Trial Registries and Publication of Results: A Primer” in Clinical Trial Registries: A Practical Guide for Sponsors and Researchers of Medicinal Products.  Thomson Inc.  Boston, MA. (2006)

[21]    Dwan, K et. al.  “Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias”  PLoS ONE. August 2008 Vol 3 No 8. e3081.

[22]    “GAP Client Exposes Flawed Procedure in Procter & Gamble Drug Study” Press Release.  Feb 22 2006.  Government Accountability Project, Washington DC. http://www.whistleblower.org/content/press_detail.cfm?press_id=371

[23]    Fleishacker, Sheila and Cohen, Mark.  “The ABCs of Drug Safety: Accountability, Balance, and Citizen Empowerment.  A Roadmap For Conducting Credible Trials and Protecting Drug Trial Participants”  Government Accountability Project, Washington, DC.  April 2009.

[24]    Washburn, Jennifer.  “Rent-a-Researcher: Did a British university sell out to Procter & Gamble?” Slate. 22 Dec 2005.  http://www.slate.com/id/2133061/

[25]    “Sponsorship, Authorship, and Accountability”  Annals of Internal Medicine.  18 Sept 2001. 135(6) 483-484.  http://www.icmje.org/spon.pdf

[26]    Turner EH et al.  “Selective publication of antidepressant trials and its influence on apparent efficacy” New England Journal of Medicine.  2008 Jan 17;358(3):252-60

[27]    “Spitzer sues GlaxoSmithKline over Paxil”  Associated Press, June 2 2004. http://www.msnbc.msn.com/id/5120989/

[28]    “State of New York v. GlaxoSmithKline, No. 04401707″  New York State Supreme Court

[29]    “ASSURANCE OF DISCONTINUANCE PURSUANT TO EXECUTIVE LAW SECTION 63. SUBDIVISION 15″ http://www.oag.state.ny.us/media_center/2004/aug/aug26a_04_attach2.pdf

[30]    Cahoy, Daniel. “Medical Product Information Incentives and the Transparency Paradox”  Indiana Law Journal Vol. 82:623-671.

[31]    Toigo, T.  “Food and Drug Modernization Act (FDAMA)  Section 113: Status Report on Implementation”  Biolaw and Business VOL 7, NUMBER 2.  2004.  US Food and Drug Administration.  Washington, DC.

[32]    Harris, Gairdner.  “Report Backs Up Warnings About Drug Avandia”  New York Times, July 27 2007.

[33]    Berensen, Alex.  “Data About Zetia Risks Was Not Fully Revealed” New York Times, Dec 21 2007.

[34]    Hughes, Sue. “Concerns raised on delay of ezetimibe data”  The Heart – HeartWire.  Nov 22 2007 http://www.theheart.org/article/829507.do

[35]    McMaster W. et al.  “The Conduct of Clinical Research Under the HIPAA Privacy Rule”  Journal of Bone & Joint Surgery.  2006;88:2765-2770

[36]    Feuerstein A.  “FDA Lunches Inquiry Into Telik Trial”  TheStreet.com, June 13 2007.   http://www.thestreet.com/p/newsanalysis/biotech/10362270.html

[37]    FDAMA Section 113:  Status Report on Implementation.  US Food and Drug Administration.  August  2005  http://www.fda.gov/oashi/clinicaltrials/section113/113report/default.htm

[38]    Turner, E, Moaleji N, Arnold B.  “Closing a Loophole in the FDA Amendments Act”  Science. Vol 322 (42-45)

[39]    “TGN1412” Wikipedia.  Accessed 26 April 2009.   http://en.wikipedia.org/wiki/TGN1412

05/18/2010 | Uncategorized | No Comments

VII. Conclusions: Towards Greater Clinical Trial Transparency

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

Transparency in clinical trial results is not a panacea.  New laws and policy measures to improve transparency must carefully balance the interests of the various stakeholders, including the industrial trial sponsors, investors in those companies, patients and trial subjects, doctors, and governmental agencies including the FDA and the NIH.  However, the numerous safety and reporting problems that have occurred suggest that our current bias should be towards greater transparency, rather than greater protections for industry.  Given the complexity and numerous constituencies involved in performing clinical trials, it unlikely that there is a single law or policy measure that could guarantee publication of trial data.  In fact, a strict law could potentially backfire and result either in a chilling of research altogether,  the movement of clinical research to different locales, or design of trials specifically to avoid reporting requirements.

Instead, a broad range of measures should be applied, both by gradual extensions of FDAAA requirements and pressure from the front lines of clinical trials, with the goal of gradually introducing a culture of transparency and availability of primary results.

Phase I Trials also be added to the reporting requirements of FDAAA, rather than being optional.  The preponderance of evidence from pharmaceutical company behaviour over the last decade seems to be that they cannot be relied upon to naturally do the right thing.  Given that Phase I trials are intended to evaluate safety, they are highly relevant to protecting patients from risks, and there is potentially much to be learned even from a drug whose development is terminated due to safety issues.  In the current environment, data from such a situation might still never become available, and future trials with similar interventions could be performed and put subjects at unnecessary risks.   In addition to the importance of publishing outcomes of even Phase I trials, protocols should also be made available as early as possible, so that experts outside of the FDA can review them.   For example, the tragedy involving the phase I trial of TGN1412 could have been prevented if the Phase I protocol had been subjected to more outside review, which could have predicted the possible risks and proposed a safer protocol in which subjects were not all dosed simultaneously.  The risk to the company of having competitors know about the existence and timing of the study would have been minimal, and avoiding this mishap would have been much better for the company overall, which failed along with the TGN1412.

FDA funding should be increased, so that the agency can operate with greater flexibility and independence.  At present, a significant portion of the FDA budget comes from user fees that are collected from companies submitting trials for evaluation.  Although there is no evidence that these fees directly influence the outcomes of evaluations, it is likely that the FDA would be able to operate with greater independence and confidence if its funding was more secure and if more inspections could be done.

Finally, the grassroots measures described in the previous section could be implemented on a limited scale.  Institutions, such as a university or hospital should set standards for clinical investigator agreements that require all data (including Phase I trials) to be made publicly available in ClinicalTrials.gov at the conclusion of a trial.  Imposing such a standard would cost very little, but could be implemented quickly and will not require a protracted legislative process.  Individual physicians could also refuse to participate in trials that did not make data available.  For example, in the Actonel case described in the first section of this paper, the initial controversy arose because the clinical investigator had agreed contractually to cede publication rights to the company and to keep data confidential.  Stronger guidelines at the outset of a trial would prevent obfuscation and withholding of data.   Lastly, patients should also consider refusing to sign away access to their own clinical data.   Given the difficulties in finding and enrolling patients in clinical trials, patients are in a strong position to negotiate on some of the terms of their participation.  In conjunction with the full implementation of FDAAA and the possible extensions discussed, grassroots efforts to insist on greater clinical trial transparency could help ensure faster detection of safety risks, faster disclosure of drug development realities for investors, and better data with which doctors and the medical community can deliver evidence-based care.

05/18/2010 | Uncategorized | No Comments

VI. Bottom-Up Clinical Trial Transparency

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

Legislation, including the FDAAA of 2007, does appear to be leading to greater transparency in clinical research.  Legislation is relatively slow to modify, however, and its passage may be affected by the lobbying from the large and influential pharmaceutical industry.  A potential alternative approach to encouraging transparency may come from a recognition that the performance of clinical trials depend fundamentally on the voluntary participation of patients as research subjects, physicians who recruit patients and execute the trials, and the institutions that enable and provide permission, through Institutional Review Boards, for trials to occur.   The individual doctors who ultimately carry out clinical trials could insist on greater data transparency before agreeing to clinical research contracts.  In fact, if only a few individual doctors chose to refuse to conduct clinical trials without guarantees of transparency and submission of results to public database, they might exert considerable public relations pressure on the industrial sponsors, without necessarily causing disruption to the entire clinical research enterprise, as might occur if a large institution immediately adopted a similar position.  However, institutions do have a responsibility to help coordinate and legitimize these grassroots efforts to increase transparency.  Even a small institution could exert a significant influence by developing policies requiring that clinical trial investigator agreements.  Most institutions already provide checklists for investigators that must be signed off before and investigator joins a clinical trial; these checklists could be extended to make investigators more aware of trial data submission policies.

Patients could also exert a much greater influence on the clinical trials to which they subject themselves.  Informed consent should include information not only about the purpose and risks of a trial, but also information about outcomes.   HIPAA protects the information about patients that is stored electronically, and requires patient consent to use or transmit.  Under HIPAA, patients have access upon request to all of their own medical records that are stored electronically.  These health records include the results of clinical trials.  By default under HIPAA, patients will also have access to their own clinical trial data, unless a patient explicitly agrees to give up access while the trial is underway.  It is reasonable that patients should give up access for the duration of a trial; after the trial is complete, however, all medical information should be accessible.  Individual rights to information thus provide a route to clinical trial transparency that does not depend on the cooperation of trial sponsors, or even of clinicians.  Patients simply need to be better informed of their rights and refuse to contractually give up access to personal data when granting  informed consent.   If patients could access records after a trial had concluded, they could choose to make their own data available.  Since the information is accessible electronically, these records could be potentially be re-aggregated by third parties for analysis without further participation or cooperation of the trial sponsors.

05/18/2010 | Uncategorized | No Comments

V. Legislative Strategies to Encourage Clinical Trial Transparency

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

Is current legislation sufficient to ensure the clinical trials of high integrity?  I will review the US legislation that governs clinical trial reporting, starting with HOPE act in the 1980s and leading to the the FDA Amendments Act 2007 (FDAAA).

The HOPE Act of 1988 was the first major law intended to increase the transparency of clinical trials.  Section 2317 of the HOPE Act directed the Department of Health and Human Services to “collect and disseminate information about HIV research, treatment, and prevention” [31]  The HOPE Act led to the establishment of the first public, computerized databases for clinical trials, ACTIS, which recorded clinical trials for HIV/AIDS, and was intended to help both patients become aware of clinical trials of new HIV drugs and help physicians and researchers to more easily recruit subjects.

A decade passed before the next major legislative initiative to improve clinical trial transparency.  The Food and Drug Administration Modernization Act of 1997, known as FDAMA, contained important new directives to expand clinical trial registration.   Section 113 of FDAMA established clinicaltrials.gov and required registration of all clinical studies for “serious or life-threatening conditions,” which meant that not all clinical trials were necessarily submitted.  FDAMA did not insist on the publication of trial results — FDAMA allowed for results to be posted, but these results were required to have already been published in a peer reviewed journal.  The data that FDAMA required to be submitted to ClinicalTrials.gov includes: “a description of the purpose of each experimental drug; patient eligibility criteria; a description of the location of clinical trial sites; and a point of contact for patients wanting to enroll in the clinical trial.”   Clinical Trials.gov was originally intended primarily as a clearinghouse to allow patients better access to clinical trials, rather than explicitly aiming to encourage dissemination of trial results.

FDAMA provided a valuable first step, but by 2005 it was apparent that not all parties were in full compliance with FDAMA.   An FDA investigation compared the trials submitted to ClinicalTrials.gov to the primary registrations of trial protocols at CDER.  Only 35% (239/688) of the protocols that should have been submitted to ClinicalTrials.gov were actually in the database.   Companies are also able to submit voluntary listings for trials that are not strictly required by FDAMA.  Only 35 voluntary listings had been contributed to the database, representing only a miniscule fraction of all Phase I, II, and III trials that had been performed between 1998 and 2005.  NIH, which also performs many clinical trials, had a much higher rate of registration with ClinicalTrials.gov (90%) vs. industry (30%).  This indicates that industrial sponsors of trials are still somewhat reluctant to make even basic information about trials easily accessible.  It may be necessary to exact stricter penalties for non-compliance with the FDAMA requirements, or to encourage participation positively by offering benefits such as accelerated review to companies that file these mandatory submissions properly.

The most recent legislation affecting reporting of clinical trials is the FDA Amendment Act of 2007, or FDAAA.  Section VIII of FDAAA mandates registration of all trials other than phase I trials for drugs, biologics, and devices, going far beyond the “drugs for serious or life-threatening conditions” that were the main target of FDAMA.  The requirements of FDAAA are narrower than recommended by policy documents from the ICMJE and the World Health Organization [3],

In addition to requiring clinical trials to be registered, FDAAA requires that clinical trial registrations on clinicaltrials.gov be linked to any results already available on the FDA website, to make it easier for researchers to access results that are already available.  Going beyond mere clinical trial registration, FDAAA required that the “Basic Results Database” be set up by the National Institutes of Health by September 2008.  This database includes information about a) participant demographics and baseline characteristics;  b) primary and secondary outcomes and statistical analyses; c) disclosure of agreements between sponsors and nonemployees that restrict researchers from publishing results (although such agreements are not prohibited).  FDAA also sets time limits to ensure that trial data is reported promptly; data must be made available in the Basic Results Database within 12 months of trial completion or 30 days of FDA approval.  Data on adverse events must be collected and reported within 2 years.  Importantly, FDAAA specified a three year process to evaluate its performance and assess the need for additional regulations to further increase transparency.   The FDAAA doesn’t attempt to solve every possible issue – rules governing informed consent, research ethics and conduct are not covered, as they are assumed to be regulated by existing laws such as CFR 45 Part 46 and the Belmont Report.

FDAAA only works prospectively, for trials started after the bill became law.  The vast majority of drug trial data remains locked inside the FDA and pharmaceutical companies.  This represents a rich dataset that could be useful for evaluating the safety of current drugs (almost all of which were approved based on studies done before FDAAA) and predicating the safety and efficacy of new ones.  Although it may not be practical or cost-effective to mandate the wholesale publishing of all of this data, it would be beneficial to have some provision (other than the slow and relatively ineffective Freedom of Information Act requests) for researchers to access this important data [38].

The FDAAA also does not require the submission of Phase I data.  This is a major limitation, because Phase I studies constitute the largest group of investigations and explicitly assess safety, which is of primary concern to the FDA.   The results of Phase I studies are also closely watched by investors, and the failure to publicly disclose trial protocols and results can allow companies to sweep failures under the rug while trumpeting successes, and could give investors a misleading impression of a company’s success rate and prospects.

FDAAA does not cover observational studies, such as post-approval safety monitoring.  As more such studies are performed and more frequently used to guide clinical decisions, it will be important that the results of observational studies also be recorded in an accessible database like ClinicalTrials.gov.

Another area that the FDAAA does not address is provision of greater visibility into regulatory decision-making at the FDA.  Under FDAAA, the responsibility is on the trial sponsor to submit data to ClinicalTrials.gov.  Many of the problems with drug safety in recent years, however, have involved data submitted to the FDA.

One of the open questions surrounding open results databases is the extent of narrative descriptions of results that should be allowed.  There is considerable danger that a trial submitter would attempt to describe their results in as favorable a light as possible if given the opportunity to explicate their results in written narrative.  Such writing would likely be the first thing to be read by a user of the ClinicalTrials.gov database, and would have undue influence because of the difficulty of understanding and interpreting raw data.  On the other hand, if narrative summaries are not allowed, the database becomes harder to use and understand.   One possible solution to this dilemma is to permit multiple narrative descriptions of results, layered on top of the core tabular database.  If the trial sponsor wishes to provide a narrative description, they may – however such a description could be commented on or challenged by other (validated and qualified) users of the database.  If  others had alternative interpretations of a dataset, they would be able to post and comment on those interpretations as well.  Encouraging both availability of the data and dialogue should help to mitigate the risk that narrative summaries could be used to mislead users.

Drug companies have responded to demands for greater transparency in public meetings with FDA and NIH [7], and have suggested that no additional data submission requirements should be imposed until trials have been conducted to prove value of result disclosure.  Of course, until results are disclosed systematically, it is virtually impossible to conduct such a trial.  The other major concern raised is that result submission to public databases might constitute prior publication, and prevent results from being submitted to academic journals.  However, journals have already agreed that database submissions to ClinicalTrials.gov do not constitute publications, and will not affect publication decisions.  The industry has also created its own database, ClinicalStudyResults.org, although it is far from clear whether these efforts are sincere, or an attempt to create an illusion of voluntary transparency to avoid additional regulation.

05/18/2010 | Uncategorized | No Comments

IV. Benefits & Costs of Transparency

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

As Part II showed, there are frequent deficiencies in the publication of clinical trial results.  Increasing the transparency of trial data brings benefits and risks that must be balanced.  Potential costs of enforcing transparency include the disclosure of valuable, proprietary information that belongs to study sponsors, the possibility that disclosure rules could have a chilling effect on all clinical research by encouraging torts, the risk that trial sponsors would change study locations to evade regulation, and the practical problem that more transparent clinical trial data could be misinterpreted by other researchers or by the public, leading to unjustified fears or confusion.

The first concern about transparency is the need for drug developers to maintain the confidentiality of their products and plans.  Today, this is the major justification for the FDA refusing to release clinical trial data in response to FOIA requests.  However, since the information is not released, it is difficult to assess whether it legitimately has proprietary business value, and the proprietary information label might well be applied liberally to stonewall those seeking information.  We can consider three ways in which trial data might be proprietary, but will find that these arguments are unlikely to be justified.  First, a trial protocol could contain proprietary data about the drug or device under investigation, such as the chemical structure of a new chemical entity, or a diagram of a novel implantable device.  Such information is clearly of proprietary value.   However, by the time that a product is actually in clinical trials, the patents to protect these novel drugs or devices would long since have been applied for.  Only in rare cases would a product contain trade secrets for which the manufacturer was not seeking patent protection.   Since a patent application would have already been submitted, releasing information about new products in documentation about clinical trials would not constitute disclosure or jeopardize issuance of the patent.  It is possible, however, that a provisional patent had been applied for but had not yet made public as a full application.  Given the relatively long time that it takes to plan and execute a clinical trial, this seems unlikely, as a provisional patent would become public within 18 months.  However, if the situation did arise, it would be reasonable to permit non-disclosure of data until either the provisional application became a full application and was published.

One of the most important concerns about research transparency is the risk of creating a disincentive to do any research at all, because having accessible information could create tort liabilities.  A company would prefer not to know about a safety risk at all, than to conduct a study and expose itself to being sued because of what is learned.  [30]  This “transparency paradox” must be addressed if measures to increase transparency are to be effective.   In [30], Cahoy proposes two approaches to neutralize the transparency paradox.  The first approach is to tightly connect information production to limitations on tort liability.  In return for generating and publicizing information about a trial, a company would obtain immunity from tort liability based on the release of that information.  Companies would still be able to keep information secret if they so chose, but their interests would be better aligned with that of the public, because not keeping some damning data secret would have the advantage of reducing their liability, in addition to benefiting the public interest.  One possible way to do this would be, as Cahoy suggests, to create a statutory exception in federal evidentiary law to exclude the results of a controlled clinical trial from cases filed after the disclosure that claim a design defect, failure-to-warn, or negligence.

Another potential risk of enforcing clinical trial transparency is that sponsors might move trials to different locations in which they were not subjected to such rules.  Such a move could again paradoxically reduce transparency, and would also raise the risk that trials might not be adequately supervised in other respects.  For example, the Institutional Review Boards in a developing country might not be as stringent, or research subjects might not be as well informed of trial risks.  Although there are international guidelines governing both trial conduct and publication of results, these guidelines, which written by organizations such as the WHO and UN committees, cannot be easily enforced.   The risk of trials actually moving due to regulation is likely overstated, however.  In order to be considered for US marketing approval by the FDA, trials must be performed to the same standards even if conducted overseas.  Moreover, the US remains one of the largest medical markets, and offers access to more potential research subjects and doctors than most other regions.  Since subject recruitment is often the major bottleneck in conducting clinical trials, it is unlikely that a trial sponsor would give up access to the US patient population, even if new regulations encouraging transparency were to make operating in the US more difficult.

05/18/2010 | Uncategorized | No Comments

III. Clinical Trial Spin and Suppression

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

Why worry about transparency?  To justify new measures to enforce the reporting of clinical trials, it must first be shown that a problem exists with the status quo of results reporting.   We will consider both the global, statistical evidence that trial results are not adequately reported in the literature, and will then review a few specific instances of information suppression and biopharmaceutical malfeasance that might have been prevented by stronger incentives and rules to report trial results.

Several analyses have been done to assess the possibility of reporting and publication bias in clinical trials.  Dwan et al. [21] recently reported a comprehensive statistical evaluation of bias in controlled clinical trials.  Bias may take several forms.  Studies with positive results are much more likely to be published.   Dwan reports that even when negative results are published, they are susceptible to “time lag” or “pipeline” bias, which means that studies with negative results take longer to be published than positive studies.  The tendency to publish negative results may not even be malicious; negative results are generally seen as less interesting to the scientific community, so researchers may be less enthused about working to publish negative results, or feel that the negative results are not adequately interesting to justify publishing.  It is also very likely that trials with negative results are more difficult to get accepted for publication in top-tier peer-reviewed journals, so even if a clinical researcher makes their best effort to publish a negative result, doing so may be difficult or impossible.  In addition to bias in reporting the outcomes of trials as originally specified, another form of publication bias noted by Dwan is “within-study selective reporting bias,” which occurs when a subset of the original variables are chosen for reporting in a publication, such as reporting of different patient subgroups or time-points.

Dwan et al. performed a meta-analysis in which they reviewed the results of 16 different smaller studies that each evaluated a cohort of clinical trials to assess publication bias.  Eleven of these studies looked for publication bias, while five examined outcome reporting bias, by comparing final publication to the pre-specified trial protocols.  Approximately half of the trials evaluated were found to have at least one major outcome that was modified or omitted, suggesting that outcome reporting bias is prevalent and should be of concern.   Looking at publishing bias, across the sixteen cohorts, trials with positive results were consistently more likely to be published than those with negative results, and the total frequency of publication was less than 50%.  From the limited data available, it also appeared that this bias was due more to negative trials not ever being submitted for publication, rather than being more likely to be rejected after submission to a journal.

Turner et al. assessed evidence for selective publication of antidepressant trials [26].  Turner accessed trial results for drug approval studies directly from the FDA, and compared these results to outcomes reported (or not reported) by publication.   The fact that Turner was able to access FDA-reported results is itself unusual; it is difficult for researchers to even study whether or not publication bias exists when the existence of and data from unpublished studies is not available. Turner found that 31% of FDA-registered studies were not published at all.  37 studies with FDA-positive results were published; only a single study seen as positive was not published.  Most studies with negative or questionable results were either not published (22) or published in a way that conveyed a positive outcome (11 studies).  When primary outcome failed to have a positive significant result, a secondary result was often reported as the headline and the major failure was subordinated or omitted.  So according to the literature, it appeared as if 94% of the trials were positive, whereas FDA data analysis showed that only 51% were positive.

This statistical evidence shows that there are systematic problems in the reporting and publication of clinical trial results.  Several specific cases discussed below illustrate the potential harm that may arise when clinical trials are not transparent and their results not subject to adequate scrutiny.  Medicine generally attempts to be evidence-based, in which clinical methods are developed based on the empirical results reported in the literature; if the data in the published literature is incomplete or biased, evidence-based medicine cannot function effectively.  I will describe five cases that have occurred over the last decade which illustrate the risks and consequences of opaqueness in clinical trials.

Actonel, manufactured by Procter and Gamble, is a drug intended to help strengthen the bones of post-menopausal women, and competes with a similar drug called Fosamax made by Merck & Co.  In 2002, P&G contracted with a British physician, Dr. Aubrey Bluhmson, to conduct and analyze a study comparing the effectiveness of Actonel with Fosamax.  After the study concluded, P&G refused to grant Bluhmson access to the randomization codes required to analyze the double-blinded data, and provided Bluhmson with only a limited subset of the results that the researcher claimed biased the results to favor Actonel. [22]  P&G then published ghost-written publications about the Actonel studies under Bluhmson’s name, despite his disagreement with the study conclusions and lack of access to the dataset.   Even though there may be legitimate disagreements in the interpretation of such trials, it is impossible for third parties to assess the controversy in the absence of full, primary data set.  If the investigator’s allegations are true, patients taking Actonel may be at unnecessary risk and missing out on the benefits of a better drug.  This case illustrates the risks of clinical trial investigator agreements in which researchers agree to give up rights to access data, and to control authorship of publications, and reveals the difficulties that may derive from the conflicting interests of trial sponsors, individual investigators, their institutions, and the journals that publish results from clinical trials.  It also shows that even when the summarized results of a trial are published, it is difficult to evaluate the veracity of the conclusions without full access to the primary research data.  If the primary data had been deposited in a publicly accessible database, disagreements about the interpretation of results could be assessed by independent third-parties.

Glaxo Smith Kline has been involved in major controversies over two of its drugs.  Paxil (Paroxetine; Seroxat) is an anti-depressant medication in the family of selective-serotonin reuptake inhibitors.  Paxil was approved in 1992 for treatment of depression in adults, and grew to become a blockbuster that generated more than $1B in sales for GSK by 1997, and the set of indications  for which Paxil was marketed included panic disorder, social anxiety disorder, and premenstrual dysphoric disorder.  By 2006, evidence had come to light of significant problems with Paxil, including ineffectiveness and a significant increased risk of suicide when administered to children.

The New York State Attorney General sued GSK in June 2004, alleging that GSK withheld information from results of 5 internal studies showing that Paxil was ineffective and dangerous in children, despite having this information as early as 1999.  At the same time, Paxil was heavily promoted in for use in children.  GSK also found itself the target of a number of class-action suits related to Paxil safety issues.  The suit brought to light internal GSK memos that showed the company deliberately attempted to “manage the dissemination of data in order to minimize any potential negative commercial impact.” [27]

The suit against GSK was settled in August 2004 before going to trial [29].  Glaxo agreed to pay $14M and to modify their reporting behaviour.  GSK also agreed to create and update a public, internet-accessible database containing summaries of all sponsored clinical trials completed after Dec 2000.  Finally, GSK also consented to permit monitoring by the NY Office of the Attorney General, and to submit full clinical trial reports and raw data for inspection up request, subject to a reasonable confidentiality agreement.  The format and contents of the clinical trial summaries was specified in the agreement, and was based upon guidelines from recommendations of the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use, a joint effort between the USA FDA and European regulatory authorities.  This agreement provides a good framework for what should be industry-wide standards in reporting clinical trial data: a public, standardized database with full auditability.

Avandia (rosiglitazone) is another drug manufactured by GSK, used in the treatment of type II diabetes.  In 2007, Avandia was found to lead to a significant increase in risk of myocardial infarction, and some in the medical community claimed [32] that the FDA moved too slowly to warn doctors and patients about the risks from Avandia, despite having data about the risks as early as 2002.  Once again, while the scientific details about the true risks are debatable, it is impossible for physicians to objectively assess the data when it is not available.  In the case of Avandia, it is likely that the evasive behaviour of GSK actually did more damage than the results themselves; rather than engaging in a scientific debate about the risks and benefits of Avandia, GSK appeared to have something to hide, leading many doctors to switch patients to different drugs, and Avandia sales have been declining since 2006.

Zetia is a cholesterol-reducing drug marketed by Merck and Schering-Plough.   In 2007, the manufacturers came under criticism for failing to disclose evidence that Zetia posed serious risks of liver damage.  Although some of the data from the studies showing these risks had been reported to the FDA, neither the FDA nor the companies took any action.  Under pressure from the medical community, some of the data were disclosed via press releases. Concerns were also raised that registration of the trials were delayed, and that the primary outcome measures of the trial were modified after the trial was underway.   In this case, the data showing risks did not come from a clinical-outcome trial, showing that to protect patients, it is not sufficient to limit regulation to trials with a clinical outcome, as the FDAAA law currently does.

One of the most notorious instances in which patients were put at risk by a drug was Merck’s Vioxx.  A series of large clinical trials that showed a significant cardiovascular risk for Vioxx were performed, but the results of the trials were only made available to the FDA and not published for years after they had been completed.  Therefore independent researchers were not able to assess cardiovascular risk, and doctors continued to prescribe Vioxx widely.  The Vioxx case demonstrates the puzzling fact that the threat of tort liability is not a sufficient disincentive to pharmaceutical companies to prioritize patient safety and to publicize risks as soon as possible.  In the Vioxx case, the NEJM accused Merck of withholding certain data from publication that would have demonstrated a significant cardiovascular risk, while publishing the data that supported Vioxx efficacy.   If trial data was automatically deposited in public databases, it would relieve companies of these kinds of accusations.

Issues of drug safety risks and clinical trial non-disclosure are not limited to large pharmaceutical companies.  For small companies, whose businesses often depend on the development of only one or two products, the pressures to avoid exposing potential risks and failure may be even than greater than with larger companies that are much more diversified.   One case in which a smaller biopharmaceutical firm faced difficult disclosure issues was Telik, which was developing a cancer treatment called Telcyta.  After announcing at a conference that three of its trials had failed to show positive results and had shown patient deaths, Telik came under criticism from the FDA and investors for failing to disclose the findings earlier [36].

Finally, TGN1412 illustrates how poor communication of a trial protocol, even for a Phase I study, can place subjects at risk of dire harm.  TGN1412 was an immunomodulatory drug being developed by TeGenero Immuno Therapeutics for treatment of chronic lymphocytic leukemia and rheumatoid arthritis.   In phase I trials, conducted by a US contract research organization, PAREXEL at a site in England, healthy human volunteers were given a low dose, thought to be much lower than the dose found to be safe in animals.  In fact, the low dose caused a disastrous response that made all six volunteers gravely ill and nearly killed them.  After the incident, observers argued that the drug should not have been tested in multiple subjects simultaneously, and that the responses might have been predictable.  Had TeGenero made their study protocols available for outside review before the trial, these views might have been considered.

The cases described have illustrated the range of problems that are possible when clinical trial data is not accessible.   Patients can be placed at risk, because the economic incentives of companies are not aligned with maximizing patient safety.  Clinical trial results may be completely suppressed by sponsors, or they may be reported to the FDA but never published.  These examples show that solely relying on the FDA to scrutinize plans and results is insufficient.  The failures of companies and the FDA to promptly alert consumers of potential risks are not necessarily due to malfeasance; rather, the larger problem is simply that the FDA lacks adequate resources to adequately monitor and respond to all of the trial data to which it has access.  Even if there is no deliberate obfuscation or suppression of results, the limited resources at both FDA and at companies make it likely that data will not be sufficiently scrutinized and safety risks might not be promptly discovered.

05/18/2010 | Uncategorized | No Comments

II. Defining Clinical Trial Transparency

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

Defining Clinical Trial Transparency

There are varying degrees of transparency of clinical trials and the associated data.  We can distinguish three major types of trial transparency.  At the most basic level is trial registration.  Registration of clinical trials ensures that regulators and the public are aware of the existence of a trial.  Trial registration is a necessary for any of the other forms of transparency, but only in the last decade have there been serious efforts to encourage trial registration and open public access to registries.  Trial registries may contain only the most basic information describing the trial, the condition and intervention under investigation, and the investigators’ and trial sponsors’ identities.  A trial registry can also contain information about the trial protocol which should include a description of the trials’ intent, the mechanisms by which patients are selected, the pre-specified outcomes under investigation, and methods of analysis to be employed.  By requiring protocol registration before a trial starts, researchers can be prevented from changing the questions that they ask based on the data.

After a trial has been performed, the second level of transparency is reporting of summarized results.  An important consideration in the reporting of results is whether they are made available as data-only, as data with narrative summaries.   Although results databases help to make information more easily accessible, a risk of allowing narrative summaries is that they might make it possible for a submitter to introduce a biased interpretation of results without being subject to peer review.  Therefore, there have been strong arguments from the publishing community to restrict results databases to rigidly specified forms and tabular data, offered without interpretation, which is left “to the reader”.  The problem of how to balance the need for accessibility and comprehensible result summaries with the risk of allowing data to be subjected to spin and misinterpretation is an important open question that will likely require novel technical and policy solutions to fully solve.

Finally, the highest degree of clinical trial transparency is to make all primary data available for examination (with appropriate measures in place to ensure confidentiality for research subjects).  The advantage of making primary data available is that an outside scientist can assess whether analysis has been done correctly and potentially identify whether there are discrepancies or problems with the way that data was collected.  Full access to raw data poses many potential problems as well; external scientists (or worse, lay people) might misinterpret the data.  Exposing primary research data could also threaten patient privacy, unless any identifying information was carefully scrubbed.  Depending on the type of data recorded, it might even be difficult or impossible to fully anonymize the clinical data — genetic information such as SNP measurements (single-nucleotide polymorphisms) might be sufficient to uniquely identify a patient, even if biographical details are not available.

A related issue that I’m not going to address in this series is the question of disclosure of financial relationships between trial sponsors and the physicians who conduct studies [14].   Under the current system, the vast majority of clinical research is sponsored by companies that hope to have their investigational products eventually approved.  These companies compensate physicians directly for the costs of conducting the trials, and may also pay these clinical researchers for consulting or other services.  Not all research sponsored by companies is bad; indeed, there is evidence suggesting that industrially sponsored studies are better run and producer higher-quality data than studies funded solely from public sources, possibly because the company studies have better funding and larger incentives to be performed well in order to garner regulatory approvals.  However, there have been concerns raised that payments to doctors could be directly or indirectly affecting their scientific independence and clinical judgment.  In response to these concerns, the “Physician Payments Sunshine Act of 2008” [17] was a bill proposed by Rep. P. DeFazio in the 110th Congress.  The Sunshine Act would have amended the Social Security Act to “provide for transparency in the relationship between physicians and manufacturers of drugs, devices, or medical supplies for which payment is made under Medicare, Medicaid, or SCHIP, and for other purposes.”  The Sunshine Act would have required sponsors to make quarterly reports to the Department of Health and Human Services of all payments to physicians or professional medical organizations. The Sunshine act never became law, but illustrates how one element of the healthcare system, in this case the US Government as the funder of Medicaid and Medicare, can exert its fiscal power to encourage transparency.

05/15/2010 | Uncategorized | No Comments

Openness and Transparency in Clinical Trials

  1. Overview
  2. Defining Clinical Trial Transparency
  3. Clinical Trial Spin and Suppression
  4. Benefits and Costs of Transparency
  5. Legislative Strategies to Encourage Transparency
  6. Bottom-Up Strategies for Clinical Trial Transparency
  7. Conclusions: Towards Greater Transparency
  8. Clinical Trial Transparency References

Clinical trials are necessary to establish the safety and efficacy of new drugs, biotherapeutics, and medical devices.  Trials are typically sponsored by the companies that are developing new products, in order to support a  New Drug Appplication (NDA) to the Food and Drug Administration to obtain approval to sell and market a drug or device in the US.  The majority of the drug approval process operates behind closed doors.  While companies interact closely with the FDA through meetings and correspondence, neither the data that supports an NDA nor the full analysis and reasoning that supports an FDA regulatory decision is made public.  In some cases, a subset of information about a drug may be obtained from the FDA by filing a request under the Freedom of Information Act (FOIA).  However, submissions to the FDA are typically protected from public disclosure, even under FOIA, because FOIA allows the US government to withold publication of proprietary information such as trade secrets.  Therefore, the vast majority of information about clinical trials — until quite recently, even the existence of a clinical trial — has remained a secret shared between the sponsoring company and the FDA.  As a result, clinical trials and data are not subject to outside scrutiny until the trial sponsor elects to publish information about the trial in a scientific journal.

The need for increased scrutiny of clinical trials follows from an examination of the conduct of trial sponsors over the last decade.  The pharma, biotech, and device companies that sponsor trials have significant economic interests in their success, and negative or ambiguous results from a trial can lead to failure to gain approval, loss of market share, loss of credibility and difficulty raising additional capital.  Companies therefore have strong incentives to avoid publication of unsuccessful trials.   As a result of the progress in biotechnology over last two decades, many drugs are now developed, at least initially, by small biotechnology companies.  Biotechnology companies are particularly susceptible to the tension between managing expectations and publishing clinical trials. Small biotech companies are often under financial pressure and frequently need to raise capital and forge partnerships with larger companies. Small companies also have risk concentrated on a small number of products, and may lack sufficient human and financial resources to full analyze and interpret data, especially after disappointing trial results have been observed.

In this series of posts, I will attempt do the following:

  • Review current US law and administrative procedures that govern clinical trials and data publication
  • Highlight recent cases in which trial sponsors have attempted to suppress and spin clinical trial results, and which illustrate classes of problems with current trial reporting.
  • Look at how increased transparency of clinical trial result data could lead to better trials, more efficient investment decisions, better scientific research, more cost-effective healthcare, and reduced risk to trial participants and patients.
  • Evaluate the arguments against clinical trial transparency.  Over the last decade, a consensus has emerged both within the US regulatory community and among international health organizations that clinical trial transparency is important.  The major arguments against making clinical trial results available involve the risks that:
    a) data may be misinterpreted by the medical community or the public
    b) proprietary business information could  be made public, hurting the business prospects of a drug or company
    c) the costs of disseminating trial information would be burdensome
    d) companies generating and publishing additional information could expose themselves to significant tort liabilities, and transparency could therefore have a paradoxical chilling effect on clinical research.
  • I’ll argue that that the claims that publication results would damage companies by revealing trade secrets and proprietary information are not valid, and that the risks of misinterpretation of results are outweighed by the benefits of increased scrutiny, especially when the cost and difficulty of publishing data is decreasing due to the adoption of better information technology.

Having established that increased clinical trial transparency is both desirable and economically feasible, I will evaluate potential legislative and policy approaches that could encourage greater transparency.   Clinical trials are complex activities that depend on the cooperation of numerous stakeholders: sponsoring companies, regulatory agencies (FDA and NIH), clinical physicians, healthcare institutions, patients, and publishers.  In addition to the disclosure rules dictated by legislation and FDA regulations, the physicians who execute clinical trials and the patients who consent to trials are in a position to do much more to encouge the dissemination of trial results.

Trial sponsors ultimately depend on the cooperation of doctors and patients for trial recuritment.  Both physicians and patients hold considerable power in the clinical trial process, because clinical trials require explicit, contractual informed consent and because patient information rights are guaranteed by laws such as HIPAA.  I will propose that, in addition to the continued implementation and extension of the FDA Amendment Act of 2007 (FDAAA), institutions such as universities and hospitals should adopt policies to require enhanced clinical trial disclosure, and that such institutions are in actually in a strong position to exert pressure to encourage transparency much more quickly than can be accomplished by additional legislation or actions by regulatory agencies.

Tags: , ,

05/12/2010 | drugdiscovery, openness, transparency | No Comments