Drugs can fail in CTs but, by not showing proper diligence in the design, monitoring, analysis, and interpretation of a CT, investigators can fail drugs.

In recent reports, investigators identify how systematic and random measurement errors can potentially undermine the demonstration of efficacy in Alzheimer's disease (AD) and depression clinical trials (CT) [1-4].

The dependence on rating scales as outcome measures seems the problematic feature common to clinical research in these fields.

With the assistance of Elemer Piros (Senior Biotechnology Analyst, Rodman and Renshaw) and Neil Buckholz (Chief, Dementias of Aging Branch, National Institute on Aging, NIH) we identified nearly 100 compounds with over 40 different mechanisms of action that had been considered as potential treatments for AD.

We found 20 NIH Public Access


compounds with preliminary evidence for benefits in AD preclinical studies and in Phase II but that failed to show consistent successes in Phase III CTs.

We could not obtain exact information about many compounds due to commercial sponsors refusing to publish or release to reviewers information about negative outcomes from their drug development.

Our initial estimate is that this affects some 40% to 50% of the compounds and is a problem that, likewise, is encountered by others [5].

Our tabulated results of this review of AD compounds may thus never be entirely complete, and will form the basis of a separate article.


Using available reports about error effects in CTs, failed drug developments in AD and depression, and our own experiences of developing drugs for AD, we identified six issues that affect the success, costs, and subsequent clinical applications of CTs [6-11].

In this article, we discuss possible steps to overcome these impediments to successful AD drug development.

Specifically, we address how to avoid methodologically flawed trials and consequent failures in development of potentially viable AD drug candidates.

We consider the preclinical grounding needed to assure effective dosing of AD drugs; overcoming problems with the reliability of rated evaluations of CT research subjects; the need for close monitoring to assure quality of data obtained at research sites; how CTs can provide resources for effective clinical applications of results by practitioners; the importance of effective disease modeling in the CT; and the implications for effective research design for AD CTs.


Many different cholinesterase inhibitors (ChEIs) have shown both preclinical properties that predict efficacy in AD CTs and then in clinical trials (CTs) demonstrated efficacy with statistically significant improvements of drug treated patients compared to placebo treated patients [12,13].

Hence, the inhibition of acetylcholinesterase can now be considered a valid drug target consequent to the approval of four agents of this class by the U.S. Food and Drug Administration (US FDA) (tacrine (Cognex) 1993, donepezil (Aricept) 1997, rivastigmine (Exelon) 2000) and galantamine (Reminyl) 2001).

Recently, we were given the opportunity to review two ChEIs with preclinical profiles consistent with these approved drugs; yet, these drugs failed in large CTs and were abandoned or seem on the verge of being abandoned by developers.


In addition to reviews under confidentiality agreements of these two compounds, we also consider our direct preclinical and clinical research experience with heptylphysostigmine, phenserine, and metrifonate [14-19].

Heptylphysostigmine failed in CTs due to toxicity.

Do the failures of phenserine and metrifonate reveal flaws in the abilities of pre-clinical studies to characterize cholinesterase inhibitors as clinical candidates or did drug development planning and the environment for conducting AD CTs fail to provide an unbiased test of these AD drugs?

If current AD CT conditions can contribute to failed drug development, then new classes of drugs for AD may also be at risk [20].


The problem faced by developers of AD drugs is that the already failed drugs may, indeed, be drug failures; however, conditions of development may also have caused these failures and could haunt new drug developments in AD.

As we have already published and from publications addressing failures in drugs for depression, we find adequate evidence to suggest that current CT practices may cause or significantly contribute to commercial failures of AD drugs.

We review this evidence to remove methodological impediments that may deny AD drugs fair and unbiased CT tests of efficacy.


Becker et al. [ 21-23] and Perkins et al. [ 24] found a mean of multiple ratings necessary to overcome unreliability that interferes with the power in CTs.

Lachin [25] calls for all studies to report the reliability of measurements as a step towards improved CT designs.


Becker [1] presents evidence for how unexpectedly high levels of variance in data sets can explain the failures of CTs to support efficacy, and reviews work by statisticians and investigators of depression CTs who report how conditions of drug testing in CTs can account for failures to Becker and Greig Page 2 Curr Alzheimer Res.

Author manuscript; available in PMC 2008 October 20.

NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript demonstrate statistically supported efficacy [26-29].

Thase [30] calls for methods to improve the ability to discriminate between active antidepressants and placebo as needed to avoid dependence on meta-analyses for trials using outcome raters.


Muller and Szegedi [31] emphasize that lack of power due to unreliable assessments risks false-negative reports and ethical consequences.

CTs dependent on rated outcomes are widely suspect as not providing fair efficacy trials of drugs.


Demitrack et al. [ 32] found that, for some raters, unreliability of ratings in a depression trial could not be improved with extended training and investigators remain undecided about how problems of rater unreliability in CTs can be overcome [26,33,34].

It is evident from these sources that no simple formula assures the required careful preparations, design, training, implementation, analyses, and interpretations required for success in a CT.

Becker's [1] review reveals the vulnerabilities of CTs to inaccuracy and imprecision in data, and both justifies and describes steps to be taken in order not to repeat these sources for failure in future CTs.

In the CTs of ChEIs used as examples in Becker [1], limitations inherent in the drugs appear not to account for the failures of CTs to confirm efficacy.

Each of the ChEIs considered in Becker [1] were shown in preclinical research to be available in sufficient concentrations in brain and came from a class already shown to produce the molecular-kinetic changes in acetylcholine metabolism needed to improve cognitive performance and behavior in AD [9,22,23,35,36].


The failed CTs did not report higher rates of interfering adverse events than occurred in the successful CTs.

Each of the ChEIs showed in preclinical studies the required pharma-cokinetic and pharmacodynamic properties.

For these drugs the environment of CTs seemed to strongly affect outcomes.

An initial large commercial CT of phenserine failed due to excessive variance in observations [37] while a smaller academic and a more closely monitored, commercially sponsored CT yielded less variance and statistically significant efficacy (36, 38).

From experiences in other fields and developing evidence in AD drug testing, one is drawn to the conclusion that investigators, by choices in how they would develop their drugs, can fail their drugs by not providing fair tests for efficacy and adequate safeguards against compromises to safety.


This view, that in drug development and in CTs investigators can fail to provide conditions that fairly test their drugs, encourages reconsideration of preclinical preparations and aims of initial clinical phases of drug development.

For example, recent CTs of immunization against amyloid were interrupted by adverse events that could be interpreted as cautions against investigators proceeding too quickly to large scale human studies [39].

In vitro and animal models for the mechanism of action of the drug need to anticipate how investigators can avoid possible adverse events that in later stages of clinical development can interfere with efficacy or threaten humans sufficiently to require termination of the human research or withdrawal of a New Drug Application (NDA) approval by the FDA.

Drug development can usefully be thought of as a process of learning how to use a drug therapeutically as well as evidencing efficacy and safety in use.

Drugs may be failing in both CTs and after NDA approvals from inadequately disciplined development.

To overcome these lapses investigators, during preclinical in vitro and animal modeling of the drug's activities, can attempt to identify and avoid threats to success in CTs.

We suggest this view to replace more passive or less intense inquiries during CT preparations.

An aggressive yet cautious and conservative drug development seeks 'no surprises' in human phases; yet, achieves this goal while at the same time facilitating rapid and effective preparations for and then completion of clinical Phases I, II, and III CTs and uneventful Phase IV and prescription uses.


Often these aims of anticipating difficulties in CTs are not within our grasp as AD investigators; however, the costs from dosing failures, adverse events, and other factors interfering during CTs argue for efforts to exclude the unexpected prior to entering CTs.

As we argue below, if investigators do not more actively seek solutions to limitations in drug development planning and CTs we will not find solutions to the current limitations that negatively impact the effectiveness and costs of AD CTs [1].

Without a 'no surprises' quality control over prospective CT conditions, history teaches us to anticipate often unpleasant and costly surprises.


Two examples illustrate how insufficient controls over dosing of a drug can undermine CT success.

In a recent, 48-month, multi-center, double-blind, placebo-controlled CT studying 1018 persons with Mild Cognitive Impairment (MCI) at risk for AD [40], in spite of expert inputs into the design and execution of the research, investigators allowed sub-therapeutic doses of active drug to be tested against placebo [41].

At the other end of the spectrum of dosing, a commercially sponsored series of CTs testing the cholinesterase inhibitor, metrifonate, resulted in sufficiently serious adverse events to cause the drug development to be abandoned after completion of Phase III with demonstration of efficacy in two multi-center CTs, and submission of the NDA [16].

In this latter situation, there seems reason to associate the appearance of adverse events with the commercial sponsor's choice to dose this 'irreversible' enzyme inhibitor on a daily basis and to do so without independently established restraints against dosage increases [42-44] Even though the cholinesterase inhibition associated with the drug and its active metabolite, 2,2-dimethyl dichlorovinyl phosphate, is stable and may persist for in excess of a month [45], the ChEI used in these studies elicits only benign short-term adverse events that decrease if one allows time for the patient to accommodate to each dosing increase [8,46].

This accommodation by patients to adverse drug effects allowed commercial sponsors of development to use increased doses of the drug in their later studies [42].

In our research, this drug toxicity was shown to be associated with the highest levels of dosing reported in these studies [47].

Unfortunately, because of the proprietary ownership of human research results allowed under business law, full public access to these data and an unknown number of other negative CT results are not available for review [48].


In contrast to the commercially sponsored CTs, in an academic setting, in earlier development of metrifonate for use in AD carried out prior to its licensing for commercial development, investigators completed two Phase III clinical trials using weekly dosing designed, following in vitro studies, animal modeling and a Phase I-II human study, to avoid effects from excessive accumulated enzyme inhibition and to comply with an optimal dose-response relationship identified in the Phase I-II trial [21,22,49].

In drug exposure to metrifonate of approximately 150 patients for up to 5 years, these academic investigators encountered no serious adverse events even though one patient, unsupervised by her caretaker for three days, ingested a three months supply of her drug.


These dosing anecdotes provide figurative bookends for the concept of a range of acceptable and optimal dosing for a drug.


We interpret the above and similar problems with containing dosing within a safe and effective range as illustrating the importance for investigators that they obtain independent evidence to establish and defend the boundaries for optimal dosing of their investigational drug.


Under conditions of Phases III and IV, inadequate or toxic levels of dosing can occur because investigators unwittingly compromise existing dosing guidelines to achieve the aims of their Phase III or IV study.


In the MCI example it appears that an emphasis on obtaining data from an adequate patient number (N) within the time allowed to complete the study may have taken priority over and compromised dosage guidelines.


In the metrifonate example it appears, from discussions in symposia, that cautions about the commercial developer's already aggressive dosing were overlooked as hopes prevailed in development that higher doses would improve the magnitude or duration of drug effects and provide an advantage over competing ChEIs.


Important considerations thus arise for commercial developers: does positioning a new drug in relation to those already available for prescription undermine the efficacy and safety available from the new compound?


One does not develop a drug into an Becker and Greig Curr Alzheimer Res.

Author manuscript; available in PMC 2008 October 20.

agent with a level of efficacy that is better tolerated than competitor drugs or an agent with superior efficacy that is similarly tolerated; a drug reveals itself in the developmental studies and desirable but not inherent features are sometimes elicited by carefully defining conditions of its use.


The regulatory required studies are not hurdles to be surmounted but one context within which investigators explore drug properties.


Sound Phase II studies focused on finalizing dosing parameters are often put aside today to design Phase II-III combined studies.

Phase II studies need designs that focus on confirming dosing parameters without competition from other aims.

The properties of the drug in patients, not desires of or development related pressures on sponsors and investigators, need to control dosing.

In both Phases I and II, efficacy data can be usefully part of the studies which focus on the interrelationships of dosing, efficacy, and safety but, unlike Phase III, not a primary aim in these Phases.

Phase III confirms the efficacy of the optimal dosing range established in Phases I and II.

As the two examples offered illustrate, priorities to achieve Phase III or IV CT aims can easily undermine the safety or efficacy of the drug if optimal dosing requirements are not empirically established, set as study conditions prior to the start of Phase III, and observed in all CTs.


Other factors shown to influence success and failure in CTs, as demonstrated in the MCI study, need also to be as fully as possible understood for their interference prior to initiating a CT [41].

One among many examples of confounding problems noted as interfering in the MCI trial [41] is the high rates of other diseases in the prospective population, the consequent use by the study sample of prescribed medications with associated adverse events, and the resulting potential for high rates of adverse events, over 90% in the MCI study in both treatment and placebo arms.

The statistical assumptions behind CT designs open the investigational drug to association with high rates of adverse events and to dosage reductions by investigators blind to whether the adverse events show patterns characteristic of an association with the research drug or with the population being studied.

An increased integration of clinical phases, as we recommend for consideration below, provides sponsors opportunities to evaluate during earlier phases any potentially interfering effects from available study populations and to design Phase III with accommodations and protections against subject characteristics and conditions at research sites interfering with fair CT testing of the investigational drug.

In practical terms, smaller Ns seem needed in AD CTs to allow investigators selecting subjects to exclude those using other medications with adverse effects that could interfere with the study meeting its required dosing conditions.


For many investigational drugs being tested for efficacy in AD, there are currently no immediate or short-term indicators or markers that investigators can consult to estimate the long-term efficacy of the drug given at the specific dosing being studied.

We find having available one or more well-characterized animal models of dose to targeted molecular response relationships to be essential to our understanding the dose-response implications of Phase I and II subject responses [6,10].


For example, in preparation for administration of metrifonate to human subjects in a Phase I-II study with aims to establish an optimal dose, to characterize the dose-response curve, and to demonstrate the safety from toxicity at any dosing level or patient response disposition that could be encountered clinically from accidental overdosing, from deviant pharmacokinetics, from low body weight, and so forth, we studied indicators of pharmacokinetics and pharmacodynamics in over 5 species and using over 5 different routes of administration [6].

In addition we studied the in vitro kinetics of enzyme inhibition using animal and human cholinesterases [8,18].

These experiences established patterns of relationships among cholinesterase inhibition levels in red blood cells and plasma, in cerebrospinal fluid, physiological markers, adverse events, brain cholinesterase inhibition, and so forth.

We could then use changes in parameters that we could study in humans to estimate levels of brain cholinesterase inhibition in our human subjects.

Informed by these estimates Becker and Greig Curr Alzheimer Res.

Author manuscript; available in PMC 2008 October 20.


Prior to entry into the human species, we view as essential that investigators gain knowledge of animal models adequate to estimate targeted responses in humans with adequate reliability and certainty.

This use of modeling supports investigators to develop independent criteria for human dosing prior to Phase III and to explore for possible toxicity at dosing levels that might clinically occur above the optimal dose.

One is most fortunate if sufficiently direct and quantitative indicators are available for the molecular target, such that the target itself can be the dependent variable rather than clinical behaviors.

Given the molecular mediation of drug effects, the molecular specificity of drugs currently under development in AD, the limitations on the reliability of clinical ratings as we discuss below, and the rapidly evolving development of new distinctive and reliable biomarkers for AD pathologies, we conclude that drug development evaluated using drug activities against molecular disease mechanisms may offer important advantages over current dependence on rated clinical responses of research subjects.


Becker [1] compared four clinical trials and found that variance could account for the successful demonstrations of efficacy in two trials and the failure of two other trials.

Each of the failed trials used larger Ns than did the two successful CTs.

Larger Ns increase the statistical power of the trial to detect group differences.

Unfortunately, in spite of their using larger Ns, higher levels of variance appear to account for the failure of two trials to reach statistical significance.


Becker [1] focused mainly on one source for error variance, excessive random measurement error, and on methods to achieve sufficient control over this source of variance, such that statistical significance could be shown for treatments with relatively small numbers of patient subjects.

The two Becker et al. [ 21,22] CTs used to demonstrate the practicality of small Ns in CTs [1], compared to other published AD CTs, attained relatively tight standard deviations for patient ratings which standard deviations were further reduced by averaging three observations to obtain each data point entered into the data analysis [23].


As reported, but not discussed in detail in Becker [1], recently a series of articles in multi-center studies of depression, documented the presence of highly consequential inaccuracies in evaluations of research subjects completed by over 50% of trained research raters [2-4,50-52].

As a result of the prevalent inaccuracies and imprecision in ratings in which human judgments generate the data obtained from patient observations, it is our view that sponsors of CTs that use rating scales for AD, depression, neurological conditions, and so forth, must assure that systematic errors, bias, carelessness during periods of observation, inattention to the study protocol and imprecision in ratings do not interfere with or undermine findings of statistical significant differences for the investigational treatment.


Unfortunately, the required reliabilities may not be achievable using either trained research raters or practitioners working with individual patients.


In Becker [1] the partitioning of sources of variance and estimations of interventions aimed at controlling random measurement errors do not take into account problems of rater inaccuracy or the advantages gained with evaluations of molecular target changes (e.g. biomarkers) as the dependent variable.

A design with molecular target changes as the dependent variable should have inherently greater power given that the variance associated with the molecular target response can have tighter error distributions given laboratory and not human measurements, and will be independent of the variance added if clinical responses are rated or estimated by practitioners.

Given the problems with errors in rating scales, validated laboratory-based outcome measures should contribute to reductions in sizes of CTs and the successful completion of CTs and management of drugs by practitioners providing patient care [29].

Thus as the partitioning of variance model [1] suggests, measurements of a molecular target as the dependent variable should increase power over clinical measure outcomes both by the absence of the latter clinical measures' variance components and by the probably smaller standard deviation to change score proportions for measurements of a physical target as the dependent variable.


The clinical importance of molecular changes clearly remains a requirement for an NDA.

Engelhardt et al. [ 2] describe practical foundational environmental conditions needed for raters to operate within if investigators are to prevent inaccuracies by raters from adversely interfering with CT analyses.

Unfortunately, biases, inaccuracies, and other systematic errors by raters differ from random measurement errors such that accuracy requires external controls- external 'Gold Standard' assessments-to detect and correct rater inaccuracies [1,9].

Excessively random measurements can be detected by analyses internal to the CT data set, or by using data obtained independent of the CT [23].

Systematic errors, for example the inability of up to 50% of raters in two multi-center depression CTs to detect drug effects in patients, can in some cases be suspected from patterns in data but are established as present only by comparison of suspected raters' ratings with ratings by expert (Gold Standard) raters [2,34, 53].


In the model design for a drug development presented below, we recommend practical and ongoing training, evaluation, testing, and selection of raters during Phases I and II.

These revised procedures for preparing for CTs can use indicators of random measurement errors and systematic errors and comparisons with Gold Standard Raters, to assess and insure the quality of data in Phases I and II.

By screening out incompetent raters, sponsors of studies can create conditions favorable for the success of Phase III studies.

These steps, while unusual, seem both economical and scientifically indicated for consideration to avoid the unnecessary failures of Phase III CT studies and the abandonment of otherwise effective drugs, the ambiguity this introduces into our understanding of the role of molecular mechanisms in AD, the economic losses incurred by Phase III failures, loss of time under patent protection due to recruiting unnecessarily large Ns to counter uncontrolled variance, the business consequences of failures and fears by potential investors of possibilities of repeated failures, and so forth.


Of course, no training or supervision of raters can overcome inherent limitations in measurement.

For example, there is considerable reason to believe that, for conditions such as Parkinson's [54] and Alzheimer's diseases [55], relevant neurological circuitry that will be lost may be lost before definitive clinical signs and symptoms of disease appear.

This could mean that ceiling effects of AD cognitive and behavioral tests may reflect only the accelerating onset of clinical illness and the later floor effects reflect the absence of adequate numbers of still functional neurons to support cognition and behaviors.

In effect, some considerable percentage of function of relevant neurons may be lost while tests can not distinguish cognitive and behavioral differences between pre-AD and normals, consequent to the remarkable redundancy of the brain.

In this scenario, most clinically detectable abnormalities occur during the final stages of neuronal loss.

Preclinical functional and anatomical neuronal losses may explain problems using clinical rating scales as outcome measures for studies of MCI or other early AD related states.

Plagued already by problems of reliability, rated changes in clinical status during pre-Alzheimer's MCI may occur only after proportionally greater losses of neurons compared to post-AD diagnosis.

As an example, Clark et al. [ 56] found ceiling effects interfered with assessing the clinical status of mild AD patients and Feldman et al. [ 57] found floor effects with scales not especially designed for severe AD patients.


We therefore recommend that sponsors consider institutionalizing in their drug development processes first, relationships between basic scientists who will develop the animal models for drug dosing in human species and those who will use these models and, second, a continuity of sites and personnel who will carry out Phases I, II, and III.

A model proposal for this continuity with sites is presented in Figs. ( 1 and 2) and is discussed below.


Typically, in a commercially sponsored drug development, preclinical studies are carried out in commercial laboratories isolated from communications with clinical investigators, Phase I testing is carried out at one set of clinical research sites, Phase II at a second set of sites, and Phase III at a third set of sites.

The sponsor does not develop an ongoing relationship to the research sites on which the sponsor depends for crucial data and, because the conceptual focus in each step is on obtaining the data required for an investigation new drug (IND) application and then NDA, the sponsor typically receives a 'bottom line' report containing regulatory required data with little information about how this drug affects the diversity of patients or subjects exposed, except for case reports for serious adverse events.


We ask "Do current practices for carrying out preclinical, Phases I, II, and III studies make the most effective use of resources potentially available to support the demonstration of efficacy for a new drug?" Our answer is an emphatic "NO!" More constructive uses can be made of basic scientists to support clinical drug development and of Phases I and II to meet both current regulatory and pharmacological needs and to prepare for success in Phase III.

One important step is to facilitate interactions between laboratory and clinic to address issues surfaced by studies in the human species.

A second priority is to use preclinical and early clinical phases to insure the reliability of clinical data.


If the implications of inevitable problems with reliability of clinical data raised by Becker [1], Engelhardt et al. [ 2], Kobek [3], and Cogger [4] and others are correct, then it would be in a sponsor's interests to use all available opportunities to assure the quality of data obtained by their clinical raters.

Phases I and II provide just that opportunity to a sponsor at very little additional cost.

Phases I and II can be used to train and screen investigators who will later become involved in Phase III (Figs. 1 and 2).

If the problems of establishing optimal dose and the advantages of doing so are dependent on the availability of bench and animal models of drug kinetics and dynamics, as we have proposed above, then it is also in the sponsor's interests to have basic scientists, experienced with the drug from pre-clinical studies, available to interact with, consult with, and even carry out further in vitro or animal assessments for clinical investigators to address issues of drug effects in the human species that might unfold during clinical development.

Each of these two ongoing supportive environments-increased preclinical-clinical phase integration and site continuity during Phases I, II and III to improve rater performance in CTs- can be available if sponsors plan for their drug's development by not using a linear, stepwise completed plan of current development.

More useful is an overall conception of drug study and development, as carried out in various species with the aim of completing required regulatory steps but also, as developing investigators progressively more experienced with the drug and more skilled in the data acquisition that will be crucial to the success of Phase III.


The most efficacious or well marketed drug will not serve its commercial sponsor or the public well if the drug is not used at effective doses or if patients do not continue to use the drug because of interfering adverse events or unrelated experiences perceived as interfering adverse events (see for example Feldman et al. [ 40] and Petersen [41]).

Becker [1,9,58-60], in a series of articles, outlined methods to assess the effectiveness of specific doses of drugs, effectiveness in individuals, and helpful to assessing whether adverse events in an individual are drug related.

These interventions are not currently practical using rating scales in clinical practice given the evidenced inadequacy of clinical assessments in most patient care dependent on the clinician's evaluation of the patient's reports [2].

If well-trained, paid raters frequently perform inadequately, or do not have the tools to control random measurement error interference, then it is reasonable to expect even greater interfering errors with 'eyeball' clinicians' judgments not grounded with rating scales and protocols for their administration.

However, this does not mean that clinical decision rules based on laboratory clinical assessments could not contribute both to establishing a unique position in the market for a new drug, to countering hasty decisions by providers or patients to abandon treatment because of not obvious efficacy or apparent adverse events, and, most importantly, to the quality of care provided by medications [1,9].

These latter seem especially important in many chronic diseases where drugs provide no immediately obvious benefits for patients, but only abstract promises of long-term benefits and often uncomfortable adverse effects.


We therefore suggest, especially for sponsors of drugs with molecular targets, that drugs be developed to provide target-controlled clinical decision rules governing use [9].

This reassures the provider and consumer with empirical evidence of efficacy and lessens abuses from dosing that may lead to adverse events or unwarranted changes of drug.

Adverse events may cause the patient to refuse medication, the physician to change the prescription or, if prevalent and serious, the FDA to review the NDA.

Unfortunately a number of drugs recently lost FDA approval because of adverse events during prescription use.

Clinical pharmacology better serves drug development if it causes investigators to recognize how drugs are inherently potentially dangerous and that mastery of the conditions of use, not inherent properties of the drug, assure safety and efficacy.

We mistakenly see therapy as an inherent property of a drug when it is only a potentiality realized by carefully controlled and monitored use of the drug.

The wider population exposed following a drug's approval brings on a further set of potential problems [61].


Sponsors can use post-double blind and other open drug treatment data to develop evidence for long-term effectiveness.

With 'placebo periods' to demonstrate immediate drug efficacy, test data of effectiveness against molecular targets, and so forth, 'controlled' assessments of long-term outcomes can become available from open phase data and placebo arms can be avoided in some CTs [1,9].

A CT tested practice model for evaluating drug effectiveness in an individual using a period or periods of placebo administration or drug withdrawal can arm the practitioner with evidence of effectiveness needed to adjust dosing and to support the patient's continued use of the treatment.

In this regard, a design for placebo period evaluations during open treatment in research or clinical practice can be found in Becker [1].

Due to regulatory priorities, CTs are regarded as required tests of efficacy and sponsors neglect the equally important issues of demonstrating effectiveness in individual patients and providing models for rational clinical management of treatment in patients [1].


In two recent AD double-blind clinical trials placebo groups showed persistent secular trends of improvement during the double-blind phase [37; unpublished data].

In each of these trials the group improvements had a less than 5% chance occurrence based on the standard deviation for placebo group responses in CTs carried out in the 1980s and 90s and historical studies of AD.


Any trial that does not replicate the established clinical disease lacks credibility as a test for treatments aimed at the disease.

Based on the known pathology of AD, the specificity of diagnostic criteria, the past clinical courses of untreated AD, of placebo-treated AD, and of long-term drug treated AD patients, only random variation and bias can account for placebo group improvements.


Given the evidence of inaccuracy and bias surfaced by Woods et al. [ 27], Engelhardt et al. [ 2], Kobek et al. [ 3], and others in depression studies, placebo group improvements can be taken as evidence of error interference in the data obtained in a study until proven otherwise.

This evidence of bias and error undermines the entire data set.

Consequently, a sponsor will want to avoid the presumed bias that provides placebo group improvements in AD CTs.


The aims of the considerations raised already-the relationship of CT success to dosing, measurement precision and accuracy, flexible uses of professional resources over the whole course of drug development, standardization and quality control at research sites, and consideration of prospective drug effectiveness for individuals aided by clinical decision rules -are to support sponsors of drug development to avoid problems that may account for drugs failing in development or being underused or withdrawn from prescription use because of inadequate guidance and controls over use by practitioners.

As discussed, for drugs evaluated principally using the clinical judgments of practitioners and researchers, rather than with objectively quantifiable outcomes, much higher risks of error intrusions dispose development to failure.

The next and following sections present model proposals to overcome these risks of failure.

We present these models not as recommended for direct applications but as demonstrations that alternatives to current drug development practices are, indeed, available.

In these models, drug development designs are not merely responsive to minimum regulatory requirements and, from fragmentation of development into steps, piecemeal in how they meet those requirements.

These models and the discussions above aim to provide an overall concept of drug development as an optimal use of resources anticipating, and taking preventive action to avoid, difficulties in AD drug development.

Experience seems to prove that risks to efficacy and safety demonstrations are almost inevitable unless sponsors and investigators remain vigilant to the possibility that, in a complex drug development in any way permissive of wrongs, if something can go wrong, something will go wrong.


The following Model Drug Development Concept attends to four specialized areas to avoid failures that are present under current drug development practices.

Implicit in these steps is the recognition that regulatory criteria for an IND and NDA and discussions of drug development and of CTs in the literature leave the impression that a sponsor plans for and overcomes a series of hurdles to qualify for an IND and NDA.

In reality, success requires much reverse reasoning.

For example, the efficacy and safety required in Phases III and IV require dosing controls that avoid sub-therapeutic or toxic doses in individuals.

This requirement sets the Phase II goals of determining a safe and effective dosing range.

Phase II goals set Phase I priorities to achieve a safe dosing range.


Phase I and II require in vitro studies and preclinical animal models since drug concentrations and activities at targets in humans can not be determined but must be assumed from indirect indicators.

Problems with rater performance require preparations for Phase III to guard against failures and so forth.

The four foci chosen to implement this 'reverse planning for no surprises' are:


To execute in vitro laboratory and animal model supports for dose finding activities in human Phases I and II.


2. To enable interactions of basic and clinical researchers to support human phases of drug development.


To design and execute clinical studies to reach maximum power without excessive Ns.


To use Phases I and II to test outcome measure administrations for reliability, to recruit sites, and to train, evaluate, and screen site evaluators for participation in Phase III.

This Model Drug Development Concept, while it specifically addresses the factors limiting the effectiveness of current CTs to meet research and regulatory aims, does appear consistent with and prepared to provide data able to meet the requirements to demonstrate disease modifying effects from a treatment and able to justify the place of disease mechanism markers as surrogates for clinical efficacy [62].

These latter properties of this methodology are of importance since, by highlighting unique mechanistic advantages for a drug under development, these methods potentially offer routes to FDA accelerated approval of an investigational drug based on novel therapeutic potentials.


Prior to initiating clinical studies and in parallel to the toxicity and other studies required for human studies two areas of laboratory investigation lay the groundwork for Phases I and II clinical studies:


a. In vitro kinetic studies of drug-receptor or drug molecular target interactions.

These studies estimate important parameters such as required drug concentration at the target; duration of drug activity at the molecular target which will affect initial estimates of dosing intervals to maintain efficacy, to avoid accumulation and adverse effects; and so forth.

If anatomical material is available, comparisons should be undertaken between molecular target samples obtained from animals used as models for estimating human dosing and from humans.


b. Animal models, for example rats, mice, rabbits, dogs, monkeys, described by the kinetics of drug absorption by routes available for human administration, other routes that may explain characteristics of observed bioavailability and pharmacokinetics.

In these models pharmacokinetic and pharmacodynamic estimates establish dose-response relationships and estimate the variability associated with biological differences among and within species.

Since drug concentrations and dynamic effects at their target will not in all probability be available in human studies, a particular aim of the animal models is to provide patterns of relationships among indicators that can be measured in humans to allow argument by analogy from human observations to probable target concentrations and effects.

Adequate pharmacokinetic and dynamic animal models also indicate the range of bioavailability, rates of elimination, metabolism, and half-lives related to accumulation.

Microdosing with labeled compounds can confirm models or, 2.

To facilitate reaching solutions to problems that inevitably arise as drug development proceeds, sponsors may find useful expanding working relations among the pharmacologists, clinical pharmacologists and clinical researchers involved in pre- clinical and clinical phases of the drug development.

For example, predictions from animal models and human data from Phase I may not correspond and simple bench or animal investigations may clarify the issues providing a sounder base for design of Phase II.


For the smaller company with fewer resources to commit to a drug development, the more prevalent practices of separately contracting out to different vendors each of the activities required as steps in development may not be as advantageous as the more integrated and interactive professional relationships just described.

Continuity between preclinical and clinical adds resources for problem solving.

A functional interaction of Phases I, II, and III, rather than the customary steps from one vendor to the next for each phase, offers the opportunity for the sponsor to overcome, with preparations during Phases I and II, the error variance problems common in Phase III.


3. While most persons in drug development would agree with the concept of designing and executing clinical studies to reach maximum power without excessive Ns, the statistical authority, Joseph Cohen [65], describes the reality of current practices: "We have also lost sight of the fact that the error variance in our observations should challenge us to efforts to reduce it and not simply to thoughtlessly tuck it into the denominator of an F or t test." Cohen advocates against practices that have allowed an N = 1000 to 1500 to become acceptable in AD CTs.

Even putting aside the unnecessary tolerance of error that is not addressed directly and corrected, but rather compensated for and buried by a large N, the large N also increases expenses greatly.

For example, for a study of N = 1500 costs could increase by a factor of 3 or 4 over costs for an N = 300 to 400.

Large Ns increase the burdens of finding and monitoring additional sites.

With increased N the intercommunication and close monitoring for quality becomes difficult, if not impossible.

Excess numbers of sites, because close monitoring is more difficult, increase the risks of deviations from protocols and reemergence of the problems of error thought to be solved by N.


Important also, but not commonly discussed, are the degrees of selectivity available choosing subjects comparing large and small N studies.

Smaller studies can more easily find needed numbers of subjects without coexisting diseases and use of prescribed medications that may produce adverse events able to be confused as due to study drug, and thereby lead to compromises to adequate study drug dosing.

Problems recruiting large Ns lead to compromises that are rationalized as 'real world testing conditions' when, in fact, the CT in Phase III aims to confirm for the drug the conditions of use to achieve efficacy and, hopefully, effectiveness for individuals - not to document the shortcomings of the 'real world.' The proposals in this article aim to develop and support with CTs scientific conclusions that describe performance criteria for the 'real world' rather than have drug development suffer from questionable levels of bias and lacking skills in the 'real world.' Once the drug's potential benefits are established then the problems of achieving those benefits in the real world can be addressed without passing off the consequences of unrelated problems in the study population, such as adverse events unrelated to the study drug leading to inadequate dosing and so forth, onto the drug.

Science developed in CTs, not customs in patient care, must set the conditions for drug use if medicine is to become scientific.


"Three factors affect the power of a CT to detect a statistically significant difference between actively treated and placebo treated patients or episodes of patient care: 1.) mean differences among treatment arms, 2.) size of the variance and 3.) number of subjects" [1]. It could be claimed that mean differences among treatment arms can not be increased for a given drug since this property is drug limited.

However, Engelhardt and associates' recent reports document that poorly performing raters reduce the mean differences among treatment arms and, consequently, the power of a statistical test to detect a treatment effect that is actually present but inaccurately reported.

Becker's studies of increased error variance demonstrate how inherent limitations of measurement in methods of observation and rater's lack of skills, attention, inadequate inquiry of the patient, and so forth contribute to increases in variance, the denominator of statistical tests, reducing power and leading to the 'tucking' of the problems into increases in the number of subjects required to detect statistically significant treatment differences that are actually present in research subjects [1,9].


We recommend increased standardizations of assessment methods, monitoring the quality of how raters perform outcome evaluations, and reorientation of the sponsor's attitudes towards the phases of drug development to capitalize on opportunities that can be created to carry out the increased standardizations and monitoring for quality of outcome evaluations.

These operational changes are discussed, below, under 4.


The opportunity exists for sponsors to use Phases I and II to test outcome measure administrations for reliability, to recruit sites, and to train, evaluate, and screen site evaluators for participation in Phase III.

To achieve error control in clinical trials, during Phases I, II, and III we recommend the following changes in drug development clinical phase management practices:


i. Evaluators administer all outcome measures using the same administration protocol for the outcome scale (see for example Williams [66]).


ii. Selected evaluation sessions conducted by each evaluator are taped and the tapes reviewed for quality by a professional expert in the use of the outcome measure.


iii. This expert's rating of the evaluator's performance uses a standard evaluation form that addresses rater adherence to the protocol, use of follow-up questions, adequacy of the rater's clarification of ambiguous patient responses, the neutrality of the rater's attitude, rapport established with the subject, adequacy of information developed in relation to the requirements of the item being evaluated, and the duration of the interview (see for example, Lipsitz et al. [ 67] who provide a scale for use with the Hamilton depression scale and the use of this scale in Engelhardt [2]).


iv.

Rating of some subjects by expert raters to detect in the site evaluator's ratings


Current drug development practices tolerate fragmentation of investigations that overlook how work with the human species is, in the perspective of a well investigated compound, an extension of work in vitro and in other than human species.

In our view, the major problems in AD drug development are expressed by investigator's insufficient attention to dosing parameters and, for compounds whose clinical effects can be measured only with rating scales, lack of awareness of how imprecision and inaccuracy expand variance in outcome measurements.

Drug development is pulled in many different directions as investigators struggle for funding, to find and maintain subjects in CTs, attempt to accelerate into clinical phases with the cost that human responses can not be evaluated in the light of well characterized in vitro and animal models of drug actions, and so forth.


Our view is that greater recognition must also be given to how drugs act at molecular targets and, in turn, to characterizing how modifications in pathology at molecular targets can be measured for use as outcome variables and their relations to clinical outcomes in patients.

Certainly, molecular measures may not yet be acceptable indicators for outcomes of either drug efficacy or disease progression by the US FDA, but they may nevertheless prove of value to target therapeutic windows, optimize interpretations of clinical outcome measures and to avoid adverse actions.

AD is currently at best characterized as a multi-dysfunctional molecular condition with possibly interrelated and possibly independent pathologies in amyloid formation, tau abnormalities, cerebro-vascular amyloid accumulations, and loss of Nucleus Basalis acetylcholine modulation of cortical neuro-transmission.

Currently available drugs and Curr Alzheimer Res.

Author manuscript; available in PMC 2008 October 20.

drugs under development for AD each target one of these mechanisms, suggesting the need for a combination of symptomatic or disease modifying treatments as a next step.

CTs that evaluate treatment effects against one or more of these molecular targets potentially offer more rational clinical grounding for combination therapies in AD and experimentally evidence the contributions that specific mechanisms make to AD behavioral pathology.

With the advent of new resources to evaluate AD patients-structural MRI, molecular neuroimaging with PET, cerebrospinal fluid analyses, and so forth-molecular mechanisms of AD become more accessible as targets for treatment interventions [36,68].


The difficulties we identify as interfering with the successful conduct of drug development and CTs also raise two compromises to medical ethics that have practical negative consequences for drug development.

First, the Nuremberg Code [69] in its second requirement states "The experiment should be such as to yield fruitful results for the good of society…" Under current practices investigators and commercial sponsors can keep secret results from human research even though this practice has been acknowledged as "scientific misconduct" [49].

Whether the sciences of dose finding could benefit from the commercial studies of compounds not reported in the literature can not be evaluated because business interests to hold human research results as secrets prevail in law over the Nuremberg Code [69].

The full Nuremberg Code does not have status as international, national or regional law in any jurisdictions.

The World Medical Association notes that negative studies 'should' be published rather than requiring public reporting and access as a matter of medical and scientific ethics.

The scientific misconduct associated with not providing public access to results from human investigations and experimentation is not open to legal challenge and thus can be expected to continue except when investigators or sponsors elect to do otherwise.


The Nuremberg Code [69], incorporated into the Declaration of Helsinki and adopted by the World Medical Association, states "The experiment should be conducted only by scientifically qualified persons." The data from Engelhardt and colleagues, Becker, Targum and others suggest that many professionals recruited as evaluators in CTs are not skilled sufficiently to qualify for participation in scientific clinical research.

In view of the increasing competition for clinical sites for CTs and for patients as subjects, we can not expect commercial sponsors to risk alienating potential research sites by becoming more selective; yet, this appears to be exactly what drug development requires to overcome the problems of bias, inaccuracy and imprecision that have led to failed CTs and CTs in which placebo group behaviors do not model the established concept of AD as a deteriorating neurological condition.

AD CTs, like many other neurological CTs, may involve sites, such as eastern Europe and elsewhere, where AD demographics and the levels of health care may potentially be less well characterized, investigators less skilled and experienced in diagnosing, assessing, and managing AD, and clinical outcome measures are more susceptible to cultural and language differences.

As Cohen [65] notes, investigators can bury variance in the denominator of statistical tests by adding Ns to CTs but only with negative consequences.

Recently, Cogger [4] demonstrated the impact on effect sizes from burying variance due to inaccuracy, failures of raters to follow rating protocols, inattention, and bias on the part of clinical raters.

Following Kobak et al. [ 34] and Demitrack [32], drug development sponsors must be prepared to not use investigators who can not demonstrate the required levels of skills.


Large Ns increase the exposure of patients to placebos and investigational compounds, delay evaluations of new compounds in AD because of the increased time to completion of CTs, and drug development time and costs greatly expand with the need to recruit and monitor sites.

These pressures lead sponsors to enlistment of sites with personnel not scientifically skilled to provide the required precision and accuracy in observations, require planners of CTs to loosen criteria in order to complete recruitment, and lead to long recruitment times compared to more tightly designed and controlled studies not dependent on numbers recruited to develop the Curr Alzheimer Res.

Author manuscript; available in PMC 2008 October 20. required power to test the hypothesis of the study.

Insufficient in vitro and animal modeling contribute to investigators inadequately characterizing dosing parameters prior to Phase III CTs.

The ability of investigators to not publish and keep from public scrutiny negative results or even positive results considered 'business secrets' causes mistakes in drug development to be repeated and patients to be exposed to treatment conditions that do not provide maximum available benefits.

Current problems with human errors in clinical trials make it difficult to differentiate drugs that fail to evidence efficacy from apparent failures due to Type II errors.

AD drug development faces challenges that must be addressed if drug development is to become more effective, more economical, streamlined to provide more rapid feedback to medicinal chemists, pharmacologists and numerous other bench scientists responsible for new compounds, more fully grounded in science and investigators are to avoid scientific misconduct.


The present commentary is based on the thoughtful commentary by Becker and Greig (2008) in the August issue of Current Alzheimer Research, who describe a broken Alzheimer's disease drug development system.

Their theme is that it is not the drugs that fail clinical trials but potentially sponsors and investigators who fail the drugs.

They argue that “many current problems encountered in Alzheimer's disease drug development can be avoided by changing practices associated with drug development.

” According to the authors, we fail to give candidate drugs a fair shake by not finding the right dose, not evaluating patients properly, not using quality controls over our data, not effectively modeling the disease, and not using effective research designs.

They say sponsors and investigators fail drugs every which way.


It is hard to argue that mistakes are uncommon in AD drug development and easy for us to cherry pick and choose favorite miscues.

Becker and Greig with the help of a colleague at the investment firm Rodman and Renshaw identified 100 failed AD drug programs within which according to their analysis 30 compounds possibly showing initial efficacy failed in phase 3.

They imply that some of these compounds could possibly have succeeded in phase 3 and been marketed.

Given the various errors in development they express concerns that we can not know with adequate certainty whether drugs are failing or investigators are failing drugs.


This broad indictment is hard to counter because there are examples for the truth of every point Becker and Greig raise.

One or another drug development program has fallen short on one or more of the authors’ critical points [1].

In particular, they example the development programs for two drugs - metrifonate and phenserine – that they knew well as researchers, consultants or original inventors.

Metrifonate was developed by the large pharmaceutical company Bayer, and phenserine by a defunct boutique biotech, Axonyx.


There are other views however


Critics can look at the same drug development programs and see things differently. Whereas Becker and Greig see the failure of the cholinesterase inhibitor metrifonate as one of pharmacokinetics and inappropriate dosing others might see it as due to toxicity and commercial non-viability [2].

Did the world need another cholinesterase inhibitor, especially one that potentially caused respiratory depression?


Phenserine, on the other hand, is an easy target as perhaps an example of an unsuccessful drug development program [3].

It is a cholinesterase inhibitor that although demonstrated pre- clinically to have neuroprotective properties was developed in paradigms for symptomatic treatments.

The company's development program could serve as the poster child for how not to develop an AD drug.

Their errors are well documented in their financial filings (e.g. Axonyx Form 8-K, SEC file 0−25571, March 11 2005, Form 10-Q, June 30, 2006).


They spent much time in phase 1 with healthy subjects and not the Alzheimer's patients who would potentially benefit. They did not explore doses and find a proper dosage range.

They completed only one phase 2 trial, imperfectly modeled on the 12-week phase 2 trials of tacrine and donepezil.

They used only 72 patients in this trial, employed an allocation ratio that led to twice as many patients receiving the fixed phenserine dose than receiving placebo.

Despite a 25% dropout rate, imbalances in their randomized groups, and the failure to show statistical significance on their main outcomes they proceeded directly to a phase 3 trial.

This decision might have been informed by statistical trends on secondary neuropsychological batteries that were mainly due to the very few placebo patients unexpectedly worsening over the brief 12-week period (while the ADAS-cog did not).


Their 6-month long, phase 3 trial, also modeled on cholinesterase inhibitor pivotal trials, used the same 10 mg daily dose as in the phase 2 trial and only a slightly higher 15 mg daily dose for a dose-ranging comparison.

Rather stunningly, they used an allocation ratio of 2:2:1 that ensured that over 80% of the patients in the study received active drug and less than 76 patients received placebo.

The known high variability of the ADAS-cog and small placebo sample size greatly minimized the chances for the placebo group to deteriorate and consequently the chances for a significant drug-placebo difference to be observed [3].

The allocation ratio certainly contributed to a measurement bias, such that the majority of the patients improved over 6 months.

Moreover, it becomes a spin of the roulette wheel to show efficacy when there are so few placebo patients in a placebo-controlled trial.


Making one of the authors’ points, the trial was undertaken in several countries and several languages, and with inexperienced sites.

It is doubtful however that excellent rater training and patient cooperation could have compensated for a trial so flawed by design.

The signals seen on some CANTAB subscales in the phenserine phase 2 trial (http://findarticles.com/p/articles/mi_m0EIN/is_2001_Oct_30/ai_79540858) were not tested for in the phase 3 trial.

The more recently reported cognitive improvements in a small Karolinska Institute study by the Nordberg group [4] continue to support some level of efficacy consistent with the cholinesterase inhibitors.

Again, the early studies suggest a potential that was not optimized nor maximized in the phenserine clinical development program.


The phenserine experience also illustrates another barrier to drug development alluded to but not explicitly mentioned by Becker and Greig, and that is the lack of scale of small biotechs [5]. Having a drug for a new target is just the beginning but the main barriers are the costs of clinical development and the lack of available expertise and skill.

Excessive reliance on outside consultants and academics – each with their own limited experience, knowledge, points of view, and interests – might doom any drug.


When drugs for Alzheimer's disease are developed under such circumstances, without appreciation for sample selection, sample size, outcomes, study conduct, and competing interests, even a truly efficacious drug would be unlikely to show efficacy.

What phenserine needed was a better development program and intellectual commitment.

To this extent, the authors make their case and indeed, the phenserine experience might be more typical of the work of smaller, boutique biotech companies – essentially one-trick ponies – trying to gain the financial interest of a deep-pocketed partner than of drug development in general.


The Alzheimer's disease drug development business is tough and conflicted and there are no easy solutions.

It seems, however, as if both large and small companies keep repeating each others’ mistakes, deriving their development program from an earlier, often unproven one, and this fact may speak more to Becker's and Greig's points.

There has been a general lack of learning in the field.

One reason may be because the prior experiences of companies are kept confidential, and, clearly, companies should share their experiences.


In addition to the issues discussed above, genetics could influence outcomes of clinical trails, at least with respect to APOE genotype.

For example treatment outcome of cholinomimetic drugs was claimed to be dependent on APOE genotype [6] although a separate analysis on the data set showed a different result [7].

Elan and Wyeth claimed in a scientific presentation that the cognitive outcome of their phase 2 trials of bapineuzumab, a humanized monoclonal antibody, which did not reach statistical significance on the protocol-specified analyses (http://www.wyeth.com/news), nevertheless, showed that patients not carrying the high-risk E4 allele had a statistically significant benefit in the trial's outcome measures, whereas E4 carriers as a group had a trend that was not statistically significant.

At the safety front, too, APOE4 non-carriers fared better, the company states, although the data presented was not convincing.

While adverse events were very common in both placebo and treatment groups overall, non-E4 carriers in the treatment and placebo groups tended to have similar numbers of them.

The claim of a differential response to an experimental treatment based on APOE genotype is similar to that obtained in a in a phase 2 rosiglitazone trial [8].

Neurobiologically, the finding raises the debate whether the brains of APOE4 carriers may be more vulnerable, and by extension more difficult to treat, than those of non-carriers.

Yet in both studies there was observed no actual treatment interaction based of APOE carrier status and the apparent differences really rested with the apparently differing rates of placebo change, and not drug effect, and both trials, again, employed very small placebo groups that were further subdivided for analyses.


Notwithstanding all this, a formidable barrier to drug development is that we know very little about Alzheimer's disease, its pathogenesis, diagnosis, genetics and struggle with its clinical heterogeneity; we do not have full pre-clinical models on which to develop advanced therapeutic targets.

Under these circumstances, demonstrating the efficacy for even moderately efficacious drugs is challenging.

More rational development programs will help, but will not be sufficient.

Yet, to Becker's and Greig's points, none of this is reason not to exert best efforts in AD drug development.

No one argues that it is not important to develop a better clinical trials infrastructure and to train raters, but there will still remain substantial variability due to the instruments, patients, diagnoses, culture, other circumstances, and error.

Indeed, trials that show small variances in their outcomes might be suspect because of the implausibility of such precision with Alzheimer's disease clinical trials [3].


The fixes that Becker and Greig offer are helpful and necessary.

Correcting errors in drug development, trial designs, and execution will matter greatly when effective drugs come along.


Commercial priorities have been identified as negative factors in drug development.

We trace the problem to inattention to sound clinical pharmacology practices.

When properly applied, clinical pharmacology and associated drug development sciences can, hand in hand, facilitate success in commercial drug development.

Schneider and Lahiri, in 'The Perils of Alzheimer’s Drug Development' [1], expand on concerns [2] with how sponsors and investigators potentially fail drugs by using flawed methods.

In recent articles [2; 3], we described an attitude towards drug development that gives priority to the unfolding clinical pharmacology of the drug.

We argued for a clinical pharmacological science able to guide investigators from step to step in development.

In contrast, Schneider and Lahiri document how sponsors and investigators can respond to commercial priorities without adequate consideration of sound clinical pharmacological practices.

Perhaps this occurs in part because clinical pharmacology in recent decades developed into a commercially over-influenced technology that does not take advantage of advances in other medical sciences.

To document consequences from sponsors’ misplaced priorities Schneider and Lahiri discuss Axonyx’s commercial development of phenserine, a cholinesterase inhibitor with pre-clinical evidence for potential neuroprotection consequent to its ability to lower brain amyloid-β peptide production [4, 5].

They point out that the symptomatic development of phenserine neglects the very much sought after possible disease modifying effects suggested in preclinical research with phenserine.

This latter seems the scientifically and medically most consequential direction for development.

With Schneider and Lahiri, we can only ascribe the decision to undertake symptomatic development to commercial priorities that displaced assumptions essential to scientifically sound and clinically more consequential development.

We propose that it is important to build on Schneider’s and Lahiri’s observations, for example, that drugs like phenserine were developed 'trying to gain the financial interest of a deep-pocketed partner' [1] rather than with priority given to sound methodologies and to obtaining both clinically and scientifically consequential results.

Certainly the 'general lack of learning in the field' noted by Schneider and Lahiri derives in part from the majority of AD drug studies remaining unpublished because their negative outcomes do not benefit the commercial aims of sponsors [3].

We suspect as even more fundamental sources for lack of learning by sponsors and investigators underlying dispositions that allow commercial priorities to dismiss, without measured consideration, issues of clinical pharmacology likely to arise during a drug’s development.


The planning and design of the commercial metrifonate clinical trials (CT) illustrate this neglect of a drug’s clinical pharmacology.

Prior to university licensing of metrifonate for commercial development, preclinical and early clinical use provided strong evidence for toxicity.

We were aware that toxicity could potentially be mediated through excessive acetylcholinesterase (AChE) inhibition [6, 7], inhibition of other enzymes affected by the organophosphate class of drugs [8], by tolerance of low doses allowing too rapid dose escalation and irreversible enzyme inhibition producing cumulative drug effects [9, 10], other factors not known to us, and cautions we found in the literature.

Two pools of AChE were known in relation to the architectural structures in brain.

One pool circulates freely within the cytoplasm of neurons and in the cerebrospinal-interstitial fluid of the central nervous system.

A second pool consists of AChE inserted in or attached to the cell walls and concentrated on membranes within synaptic areas.

AChE is synthesized in neurons and then activated as cytoplasmic AChE, fixed to the exterior cell wall and secreted into the extracellular space spontaneously and during neurotransmitter release [14].

There was evidence for different rates of turnover for the unbound soluble AChE and the membrane bound AChE. Karprzak and Salpeter [15] found long stability of about 20 days for mouse muscle synaptic AChE although Fernandez and Stiles [16] reported total recovery of synaptic activity within about 3 days after organophosphate inhibition.

Rotundo and Fambrough [17] found a half-life of membrane-bound AChE in muscle cells of 50 hours and of intracellular AChE in chick muscle cells of 2–3 hours.

In addition to the two active pools there was an apparently more extensive pool of newly synthesized but inactive intracellular AChE, the majority of which is rapidly degraded before it becomes active [18].

AChE was known to exist in monomeric, dimeric, tetrameric, and asymmetric forms with the more complex forms assembled from the G1 monomer.

Forms have different distributions in the body [14].

Goosens et al. [ 19] reported no differences between the recovery rates of free G1 and membrane bound G4 forms in rat brain.

Moss et al. [ 20] found a recovery half-time after phenylmethylsulfonyl fluoride of 11 days in whole rat brain compared to 1, 3, and 6 days for ileum, heart, and pectoralis muscle, respectively.

Unni et al. [ 21] estimated a 2.21+/−1.22 day recovery half-time of CSF AChE in AD patients treated chronically with metrifonate.

We interpreted this data to imply that membrane bound synaptic AChE recovery and possibly recovery of other enzymes important for organophosphate toxicity may have a 14 day or longer half-time recovery in spite of more rapid recovery of free CSF AChE.


To insure safety while dosing metrifonate we assumed a functional brain membrane bound G4 pool of AChE with recovery half-time of 14 days.

Using the data published by Unni et al. [ 21] and others we calculated, for weekly and daily metrifonate administrations, the expected dose-AChE inhibition responses for RBCs, CSF, and brain. ( Figure 1).

We noted the risk for progression to high levels of AChE inhibition associated with daily dosing (Figure 1).

We regarded these risks as important because they were made possible by properties of metrifonate: cumulative irreversible enzyme inhibition; lack of adverse effects with stepwise dosing increases; and so forth.

Assuming a t1/2 recovery of 14 days, predicted brain AChE inhibitions and possible equivalent levels of inhibition of other important enzymes, such as the carboxyesterases, neurotoxin esterase, and so forth, provided strong reasons to develop a weekly dosing regimen if possible.

Later we considered these factors as possible explanations for the severe adverse events experienced with the daily dosing used in Bayer’s Phase III CTs.

In response to this understanding we developed a weekly dosing schedule to provide for regeneration of enzymes irreversibly inhibited with each dosing of metrifonate.

We showed for weekly dosing a U-shaped dose-response curve with maximum cognitive benefits at about 45% red blood cell (RBC) AChE inhibition [10, 11].

The decreased drug effectiveness at higher levels of inhibition fit our assumptions that receptor depolarization block might occur at high acetylcholine concentrations.

After initial clinical successes using weekly dosing in two small clinical trials [12, 13], we did not pursue higher dosing or replace weekly with daily dosing.

The licensee, Bayer, showed no evidence of increased efficacy with the daily dosing schedules they chose for commercial development.

We regard Bayer’s decision to use daily dosing, explained at the time of licensing as required to make development profitable, as an excellent additional example of how commercial orientations facilitate but also, if unsupported by sound clinical pharmacological facts and practices, potentially undermine AD drug developments.

Such decisions clearly are not made lightly by companies, whether multinational, like Bayer, or virtual, like Axonyx.


In the absence of large scale grant funding for new drug developments and academic centers committed to clinical pharmacology research, commercial realities will continue to drive almost all drug development programs.

We do not argue against commercial motivations for decisions, only against drug development decisions being reached without providing or taking adequate account of sound clinical pharmacological practices relevant to the decisions.

In sum, what do we recommend to render current AD drug development practices more effective? In our papers we have encouraged attitudes, assumptions, and practices that regard drug developments as informed by clinical pharmacological progressions through various animal species.

The human is the most problematic species for the clinical pharmacologist to study.

Consequently we argue for development in two or more non-human species of experimental conditions, assay methods, confirmations of dependent variable changes, and, as much as possible, other methods needed for a CT.

This work provides the model then applied in the human species [2].

This model assumes that many critical variables will not be measurable in humans—drug concentrations at brain targets, molecular changes induced by drug in brain, and so forth.

We see investigators using non-human species to establish robust patterns of change that can be applied to interpret human studies.

For hypothesis testing the investigator uses changes in variables accessible in human studies to argue for parallel changes in inaccessible variables.

These patterns ideally address variables able to confirm the concept of a drug’s therapeutic mechanism.

We now recognize how errors and biases can too easily lead to Type II errors in AD CTs [1–3].

Demonstrated patterns of mechanism changes induced by drug in multiple species make CTs tests of potential therapeutic mechanisms in AD and more than tests of one compound’s efficacy.

Standardizations of assays and investigative methods in non-human species remove effects on CT data due to changes in methods as one goes from bench and non-humans to human trials.

With stability of methods and conditions across experiments investigators anticipate consistency in humans with patterns documented in non-human animals and investigate inconsistencies before dismissing the research hypotheses.


This attitude—regarding humans as one of various animal species studied to understand the properties of drugs—and its recommended practices do not replace commercial considerations.

Properly applied, clinical pharmacology and associated drug development sciences such as medical statistics facilitate sponsors realizing commercial aims for drugs they bring to development.

The current failure to respect the teachings of these sciences concerns us.

Expanding lists of risks to drug development witness to the need for 1) more attention to what constitutes adequately sound clinical pharmacological AD drug development practices and 2) better understanding of how clinical pharmacology integrates with commercial realities of drug development.


Most organizational change efforts are overwhelming because of the many ways an organization can change.

To deal with organizational complexity, we assemble groups, spend countless hours in meetings, and too often make decisions that have no effect in the long run.

Here is an example: In an effort to reduce lines in the checkouts, a retailer attempted to implement the "next-in-line rule" in stores that hold up to 2000 customers at a given time and have about 40 cash registers.

The rule was that no line should have more than one customer waiting while a customer was being served at checkout.

Associates at the stores attempted to reduce or eliminate long checkout lines but compliance failed because many variables affecting checkout efficiency were not considered-traffic volumes at specific times of day, the effect of weather conditions on shopping patterns, number of staff required to manage variable volume of customers, service demands in other parts of the store, and so forth.


Many dynamic variables affect organizations and our work in them.

We suggest that understanding the nature of organizational complexity is essential, but not sufficient, if we hope to target meaningful change and avoid being overwhelmed by details.

In the next sections, we examine the nature of organizational systems, define the boundaries of organizations, and distinguish various types of complexity.


Behavior analysts working in the field of organizational behavior management must expand on the traditional activities of behavior analysts because their object of study is organizational behavior.

As it turns out, "organizational behavior" means both the behavior of individuals in organizations and the behavior of organizations as functioning entities in their own right. What must be managed is the relation between the behavior of the individuals in an organization and the behavior of the organization as a whole.


Organizations consist of the dynamic interaction of human behavior and its products that affect the behavior and products of other humans.

Behavior of all employees, like behavior in the experimental laboratory, is the result of behavioral selection contingencies, or behavioral contingencies for short.

These units of analysis are relations between antecedents, behavior and consequences.

Some behavioral contingencies make it more likely that behavior of the same kind will occur again.


For example, a production schedule (antecedent) cues the worker in a plastic manufacturing plant to set the mold (behavior) for plastic parts.

A mold setup is the product of this worker's behavior.

The worker locates the mold, loads it on a forklift, carries it to the press and places it in the press.

If the mold is fastened flush between the press doors, the task is completed (consequence).

If the mold isn't flush, the worker manipulates it until it is seated properly.

The relation between his actions and the properly loaded press (contingency) affects how mold setup will be performed on the next occasion.

Variants of the behavior that achieve properly set molds become increasingly frequent.


The repetitions of mold setting behavior of this worker constitute a behavioral lineage.

Mold setup is undergoing selection by reinforcement-a process by which a relation between behavior and its consequences increases the future likelihood of that behavior.

Mold setup is operant behavior because it operates on its environment.

An operant lineage consists of a sequence of operant instances that change over time as a result of behavioral selection contingencies.


Sometimes the behavior of Person A, or the product of that behavior, is the occasion for Person B to do something.

B's behavior, or its product, then may set the occasion for Person C to do something.

The behavioral contingencies of A, B, and C are intertwined.

The same event or object (e.g., A's product) is a consequence of A's behavior and sets the occasion for B's behavior.


For example, in manufacturing plastic parts, Worker A gets the mold from a tooling rack and sets it in the press.

Worker B sets the dials on the press according to engineering specifications.

Worker C molds parts.


The behavior of each person becomes part of the environment entering into the behavioral contingencies for others.

We call these kinds of relations among the behaviors of two or more people interlocking behavioral contingencies.

They are the building blocks of cultural complexity.


The behavior of A, B, and C may be part of a more inclusive set of interlocking behavioral contingencies that, together, result in an aggregate product: molded plastic parts.

These interlocking contingencies are repeated with each molding request; and the repetitions constitute a lineage of interlocking contingencies.

Variations in the elements of the interlocking contingencies may result in variations in the quantity or quality of plastic parts.

The interlocking contingencies determine the characteristics of the products; and the characteristics of the products determine the customers' acceptance of the product.

Customer acceptance is the external environment contingent on the product of the interlocking behavioral contingencies.


In organizations, we are interested in the products of the interlocked behavior of multiple individuals; so the behavior of individuals remains the fundamental component of organizations.

The evolution of an organization as a whole depends not only on its individuals' behaviors, but also on how those behaviors combine and form units of selection that evolve in their own right.

Given the inherent complexity of organizations, the behavior of any individual can rarely be isolated and managed without consideration of its interactions with the behavior of others.


We might start our organizational analysis by establishing the boundaries of an entity we want to study. What constitutes an organization? In the broadest sense, an organization consists of a group of people who perform tasks that achieve a particular product.

An organization is defined by its products.

XYZ manufacturing company consists of all the employees whose integrated activities result in XYZ's manufactured products.

If an internal department provided travel services for XYZ employees, its existence would necessarily depend on XYZ's manufactured product.

In contrast, a travel agency contracted by XYZ to provide travel services to XYZ's employees is a different organization than XYZ because the agency's existence does not necessarily depend on XYZ's manufactured product.


Organizations often comprise several systems that contribute to achieving their goals.

The term system is used for a variety of relationships between many kinds of separate elements arranged as a whole to achieve an outcome.

XYZ needs various systems to manufacture products, such as purchasing, sales, production and shipping.

Each system generates a product that relates to the operations of one or more other systems and thereby contributes to XYZ's aggregate product.

For example, products of XYZ's systems include purchase orders, purchased items, finished goods, and goods delivered.


Each system is composed of subsystems.

The production system could include molding, trimming, and packaging subsystems, each one producing a critical component-molded, trimmed and packed parts.

A subsystem may have its own subsystems.

For instance, molding includes plastic preparation, press set up and plastic injection, and these systems produce adequate plastic, proper set up and injected molds.


The least complex cultural system in an organization is one formed by an interlocking behavioral contingency in which two individuals each perform at least one recurring behavior.


Organizations are not static entities.

An organization as a whole and its systems are dynamic, always undergoing change.

Alterations in internal systems result in changes in the organization as a whole.

For instance, the interlocking behavior of a production team could be affected not only by the engineering team directly involved in production, but also by the purchasing, shipping, and other processes in the organization.

In addition to the internal dynamics of any process, that process is also affected by alterations in the environment external to the organization, such as changes in the customer and provider organizations.

The intrusions of parts of one system into the operations of other systems speak to the permeability of system boundaries.

The dynamic interactions between systems elements and the permeability of their boundaries make complexity hard to analyze.

Due to the permeability, boundaries of any system are somewhat arbitrary, but delineating boundaries helps to simplify overwhelming complexity.


Identifying boundaries does not mean that we can ignore the multitude of ongoing interactions between internal and external entities.

It only means that we set aside the more remote influences and focus on the most direct dynamics.

Systems analysis, no matter the size of the system, would require minimally the study of the dynamic interactions between its internal components, its relationships to critical systems in the organization, to the performance of the organization as a whole, and to customer demands.

( For an account of systems analysis in organizational change, see Gilbert, 1996; Malott, 2001-b; & Rummler and Brache, 1995.)


For example, the behavior of the sales force of a pharmaceutical company is integrally related to other organizational systems.

It would be short sighted to study only the behavior of the sales people and attempt to design new contingencies of reinforcement to increase sales.

We could not know if our changes would have a desired effect on other processes and thus on the organization as a whole.

So in addition to analyzing the behavior of the sales force, we must consider such things as the interrelations among sales forces in territories, districts and regions; the influence of marketing, product development, production processes; trends in customer purchases; and the impact of drug regulations on sales performance.


Organizations are like ecosystems, formed by a multitude of interdependencies.

Ecology offers an orderly view of nature that simplifies the study of the relationships between organisms and their physical environments, including other organisms.

Like ecologists, behavior analysts working in organizations need a way to order the complex interdependencies among organizational systems and their interlocking contingencies.


We have found it useful to consider three types of organizational complexity: environmental, component, and hierarchical complexities.

In this section we review these three types of complexity and their implications for organizational effectiveness.


In order to target any area for change, no matter how small, we should understand the organization as a whole.

The number of variables external to the organization that affect organizational performance determines environmental complexity.

The environment outside of the organization is constantly changing in ways that affect internal organization.

Some of the ways that the external environment can change are product and service development within an industry, government regulations, mergers, consolidations, bankruptcies and warfare.

Other external variables, such as changes in competition, providers and weather patterns may also affect organizations.

Figure 1 illustrates environmental complexity.


Millions of new products and services become available in the market every year, requiring changes in many internal systems of organizations such as production, quality control, training, and information technology.

Failure to upgrade products and meet or surpass competition standards eventually reduces sales and market share, threatening the long-term stability of the organization.


Regulations also impose tremendous changes in internal processes.

For instance, consider the impact of the 1994 food-labeling regulations designed to help consumers make healthful choices.

The United States Food and Drug Administration and the Department of Agriculture required nearly all packaged food to carry a standardized food label indicating the amounts of calories, vitamins, protein, fat, and fiber per serving.

Such regulations imposed changes in food manufacturing processes quality control, packaging, and others.


Mergers and consolidations often revive failing businesses, reduce competition, or diversify product lines, thus altering the landscape of the environment and creating new demand.

A merger is achieved when a company purchases the property of other firms, thus absorbing them into one corporate structure that retains its original identity.

A consolidation is achieved when two or more companies dissolve in order to form a completely new company.

Bankruptcies can result in the disappearance of competitors, vendors or clients.


Fluctuations in the economy have a significant impact on the environment of an organization.

In prosperity, production, employment, wages, and profits increase; more investments expand production.

As the upswing continues, however, production costs increase; shortages of raw materials may hamper production; interest rates rise; prices rise; and consumers react to increased prices by buying less.

As consumption starts to lag behind production, inventories accumulate, causing a decline in prices.

Manufacturers make cuts, investment decreases, production decreases, and unemployment increases.

Cycles of prosperity and depression affect most organizations.


Warfare also has a significant impact on organizational environments.

War affects economies, infrastructures of countries, development of military and intelligence systems, and so forth.

Environmental changes do not necessarily imply that organizations must become more complex.

Sometimes changes in the external environment force organizations to simplify their systems.

For instance, demand for faster delivery of orders may require an organization to streamline its processes so larger volumes are delivered in less time.


Changes in the external environment drive alterations inside the organization, but changes within organizations also affect the environment.

For instance, consider the impact on the economy of a single large merger, a company's hazardous waste, or a terrorist act.

Environmental complexity cannot be ignored.

Organizations that do not adjust to changes in their external environments are unlikely to survive.


The number of elements that constitute an organization determines component complexity.

The elements might be related to each other as equals or they might be at different levels in a hierarchy.

The smallest organizational units of interest are interlocking behavioral contingencies that generate critical products.


Organizations tend to be more complex when larger numbers of people participate in their processes.

Small businesses of a few employees are generally less complex than large businesses with thousands of employees.

Component complexity also depends on the number of processes each system subsumes.

For example, in a manufacturing company, the production process might be more complex than the advertising process.

This might be because production has more subsystems and/or larger numbers of interlocking behavioral contingencies.

The manufacturing process might contain all the interlocking contingencies involved in receiving of raw material, preparation of equipment, scheduling, production, and inventory management.


A simple way to relate to component complexity is by looking at process maps.

Think of a process map as a graphic depiction, where each box in the map represents interlocked behavior that generates a product.

A process map might include the behavior of thousands of people (Malott, 2001-b).

Processes with more boxes in the organizational chart are more complex than those with fewer boxes.

Figure 2 shows the partial process of selecting retail items to be advertised.


Studying component complexity is critically important.

Without systematic effort, organizations tend to increase in component complexity and become redundant and inefficient.

Unnecessary component complexity could be counterproductive to the overall goals of any organization.


Organizations are made up of systems that contain subsystems, which themselves might have subsystems, and so on.

Hierarchical complexity is determined by the number of system levels in the organization, or the number of part-whole relations that constitute an organization.

The levels of management in an organization are usually a good index of hierarchical complexity.


An organization increases in hierarchical complexity as it creates more layers of components.

Therefore, component complexity generally affects hierarchical complexity.

For instance, an organization that needs to increase production might acquire more plants; overseeing several plants requires another level of management.

When the number of plants increases considerably, organizations need regional directors to manage specific geographical areas.

As the company expands, the regional directors need more help to manage their territories, so they might create a new layer of management-district directors.

Figure 3 illustrates hierarchical complexity in a pharmaceutical company.


Organization structures often reflect the way they assemble their components in hierarchies.

The wholes might be defined by function (marketing-sales), geography (regions-districts), content (psychology, history, physics), form (cars and trucks), life expectancy (perishables-dry grocery) or seasonality (seasonal, non-seasonal).


Organizations often include multiple hierarchical layers.

For instance, pharmaceutical companies sometimes arrange their sales forces by territories, districts and regions.

They may also have layers of management grouped by product type.

So they have product specialists for specific types of disorders, for instance, central nervous system diseases.


An important implication of hierarchical complexity is that as the levels of management grow, the behavior of those in higher levels becomes increasingly unrelated to critical components of the interlocking contingencies of the lower levels.

Unfortunately, as levels of management grow in organizations, these disconnects between levels threaten the organizational success.

The performance of the lowest level ultimately determines the success, failure, and survival of the organization. But what happens at the Decisions at higher levels are often made without realizing the implications for the lower levels of the organization.

Making decisions takes less time than implementing them. And making poorly thought-out decisions takes much less time than implementing them - even implementing them poorly.

Managers usually ask for more change initiatives than can be realistically implemented.

Table 1 shows the three types of complexity discussed.


In this section we attempted to introduce organizations as ecosystems and to describe interdependencies in organizations by pinpointing three distinctive types of complexity: environmental, component, and hierarchical.

The next section addresses how organizations evolve through cultural and behavioral selection.


The biological characteristics of organisms, the learned behavior of individual organisms and the interlocking behavioral contingencies in organizations are very differ- ent kinds of things, but they all change over time as a result of selection.

Natural selection accounts for features of the organic world; behavioral selection for features of individual behavior; and cultural selection for features of organizations.

Although natural selection does not appear to play a current role in organizational change, the process of natural selection is well understood.

Therefore, we introduce the concept of selection by describing its role in biological evolution and its parallel to behavioral evolution.


Charles Darwin (1959) first suggested natural selection as a process that causes differential preservation of inherited (genetic) characteristics in a lineage of reproducing organisms.

Organism/gene characteristics that better fit the environment recur more frequently than others in their lineage.

For example, organisms of a particular species that live in a very cold environment may vary in their inherited propensities to grow thick fur.

Those more likely to grow thick fur are, on average, more likely to survive and reproduce than others with inherited propensity for less furriness.

So the average furriness of the organisms of this species increases in successive generations or reproductive cycles.

The average furriness of Generation #5,000 may differ considerably from the average furriness of Generation #20,000, but the change between any two successive generations would be negligible.


In this example, the relation between organisms' furriness and ambient temperatures affects the likelihood of furriness in future generations.

Similarly, the relation between a rat's lever presses and food deliveries affects the likelihood of future lever presses.

In summary, relations between characteristics of organisms or behavior and their environments determine future frequencies of those characteristics.

These relations have been called "contingencies of selection" (Skinner, 1981).

Contingencies of selection may also involve relations between organizations and their environments.

In the sections below, we attempt to provide a selectionist account of the evolution of organizations, where cultural selection and behavioral selection are both directly relevant.


Organizations are cultural entities that change over extended time while retaining their identity as "the same" organization.

In this sense, an organization is like a biological lineage.

It is composed of repeating generations of events having features that change over time as a result of the way in which variants in the current generation are "received" by their environment.