---

Abstract

Background: Both behavioral and psychological symptoms of dementia and the use of atypical antipsychotics among nursing home residents with dementia are epidemic. However, the evidence from systematic reviews and randomized controlled trials for the benefits of using atypical antipsychotics in this population is conflicting and inconsistent. This study examines the evidence from randomized controlled trials for the use of atypical antipsychotics to treat nursing home residents with behavioral and psychological symptoms of dementia.

Methods: PubMed, Cochrane review, the National Clinical Guideline Clearinghouse, previously published systematic reviews, and a search of references were used to find eligible randomized controlled trials on use of atypical antipsychotics to treat NH residents with behavioral and psychological symptoms of dementia. Inclusion and exclusion criteria were pre-defined. The papers were independently reviewed by two investigators. Risk of bias, applicability, and heterogeneity were assessed based on previously published methods.

Results: Among 1469 citations, the 12 trials met the inclusion criteria. The variability of the diagnostic criteria and outcome measurements, as well as that of these verity of dementia and behavioral and psychological symptoms of dementia, prohibited meta-analysis of the selected 12 original studies. Among the total 4352 enrolled subjects across the 12 trials, age ranged from 77 to 84 years old. Seventy two percent of participants were female, and 64% of participants completed the trials. Mean duration of the 12 trials varied from six to 12 weeks. Four out of the 12 trials (33%) showed positive results. The BPSD reduction varied from 7% to 72%. Risk of bias included low concealment (58%) and high attrition rate (20-42%). The trict exclusion criteria and low recruitment fractions (69%) among the 12 trials reduced the applicability of the trials. Additionally, there was significant clinical heterogeneity and methodological diversity across the 12 trials.

Conclusions: There is limited and inconsistent evidence to demonstrate the efficacy of atypical antipsychotics in the treatment of behavioral and psychological symptoms of dementia in nursing home residents. In addition, there are concerns about risk of bias, applicability, clinical heterogeneity, and methodological diversity among the 12 trials. We are less confident in the intervention effects of atypical antipsychotics on BPSD, and therefore do not recommend routinely prescribing atypical antipsychotics.

---

Keywords

Atypical antipsychotics; Behavioral and psychological symptoms of dementia; Dementia; Nursing home

---

Abbreviations

BEHAVE-AD: Behavioral Pathology in Alzheimer’s disease; BPRS: Brief Psychiatric Rating Scale; BPSD: Dementia and Behavioral and Psychological Symptoms of Dementia; CMAI: Cohen-Mansfield Agitation Inventory; CMS: Center for Medicare and Medicaid; CONSORT: Consolidated Standards of Reporting Trials; DSM: Diagnostic and Statistical Manual of Mental Disorders; FAST: Functional Assessment Staging Test; FDA: Food and Drug Administration; NH: Nursing Home; IRB: Institutional Review Board; MMSE: Mini-Mental State Examination; NINCDS-ADRDA: National Institute of Neurological and Communicative Diseases and Stroke-Alzheimer’s Disease and Related Disorders Association; NPS: Neuropsychiatric Symptoms; NPI: Neuropsychiatric Inventory; NPI-NH: Neuropsychiatric Inventory-Nursing Home; OIG: Office of the Inspector General; PANSS-EC: Positive and Negative Syndrome Scale-Excitement Component; RCTs: Randomized Controlled Trials

---

Introduction

Behavioral and psychological symptoms of dementia (BPSD) are defined as “symptoms of disturbed perception, thought content, mood or behavior that frequently occur in patients with dementia” according to the International Psychogeriatric Association [1]. The International Psychogeriatric Association also states that “behavioral symptoms are usually identified on the basis of observation of the patient, including physical aggression, screaming, restlessness, agitation, wandering, culturally inappropriate behaviors, sexual disinhibition, hoarding, cursing and shadowing. Psychological symptoms are usually and mainly assessed on the basis of interviews with patients and relatives. These symptoms include anxiety, depressive mood, hallucinations and delusions and psychosis” [1]. Because 23 BPSD scales and checklists have been published in the literature [1], we use BPSD as a term to cover behavioral and psychological symptoms or NPS in this review. BPSD among nursing home (NH) residents are common and eventually occur to every NH resident [1-4] and potentially cause significant distress and burden to both the individuals with dementia and their caregivers [5-10]. Consequently, antipsychotics, mainly atypical agents, have been widely used in the NH setting [11-17], resulting in concerns for adverse effects and regulations from several government agencies including the Office of the Inspector General (OIG), the Food and Drug Administration (FDA), and the Center for Medicare and Medicaid (CMS) [18-20]. Atypical antipsychotic use is not only associated with high cost, but also with multiple adverse effects including increased mortality, falls, strokes, and myocardial infarction [1,18-30]. Additionally, the evidence from systematic reviews and randomized controlled trials (RCTs) for the benefit of atypical antipsychotics use in this vulnerable population is inconsistent and conflicting [31-43]. Ten systematic reviews examining BPSD suggest that atypical antipsychotics are effective [31-40]. Incontrast, three systematic reviews on BPSD suggest that atypical antipsychotics are ineffective [41-43].

Most importantly, all of these systematic reviews [31-40,42,43] except one [41] do not focus on NH residents with BPSD. One systematic review without a meta-analysis included15 studies examining atypical and typical antipsychotics to treat BPSD in NH residents and found atypical antipsychotics to be ineffective for treating BPSD in NH residents [41-43]. In the absence of consistent good evidence, many general review articles have recommended the use of atypical antipsychotics for BPSD in older patients [44-45], though the guidelines provide inconsistent and conflict recommendations [56-60]. Importantly, a systematic review focusing on treating BPSD in NH residents with atypical antipsychotics has not been previously reported. Therefore, we decided to perform this systematic review. The PICOS (patient, intervention, comparison, outcome, study design) question [61,62] of this systematic review is used to examine whether the use of atypical antipsychotics reduces BPSD in NH residents compared to placebo or usual care based on the evidence from RCTs. Specifically, the objectives of this systematic review are to address the following four questions: 1) Can atypical antipsychotics reduce BPSD among NH residents? 2) Are the RCT sexamining atypical antipsychotics to treat BPSD in NH residents valid? i.e., is the internal validity good? 3) What is the role of the placebo effect of atypical antipsychotics in the treatment of BPSD for NH residents? 4) Are the results of RCTs applicable to the population in the real world? i.e., is the external validity good? We also provide a list of BPSD measurements and scales, which were not reported in all previous systematic reviews [31-43].

In performing this systematic review, we followed the standards of the Cochrane Handbook for Systematic Reviews of Intervention [61] and Preferred Reporting Items for Systematic Reviews and Meta-analyses [62] with the following modifications. We did not use scale scores such as Jahad score to assess the quality of RCTs in our previous study [63]. Instead, we adopted a content evaluation method to assess the internal validity and risk of bias of the trials [61-64]. Assessment of the internal and external validity of the RCTs followed the Consolidated Standards of Reporting Trials (CONSORT) [64] and methods from our previous study [63] and others [65].

The first two authors had several meetings to discuss pre-defined research questions, eligible criteria, search strategy, data collection, assessment of internal validity and external validity, analytic approach, and summary of results during the planning phase. However, we did not write a full protocol and complete the online registration.

We defined the eligibility criteria of this systematic review as following based on our PICOS research question: 1) Participants: NH residents with BPSD. The rationale is that BPSD in NH patients is epidemic and a significant problem [2-4] and atypical antipsychotics are widely used in this vulnerable population [11-17]. Additionally, the evidence from previous systematic reviews based on RCTs is inconsistent and conflicting [41-43]; 2) Intervention: atypical antipsychotics as the intervention; 3) Comparison: placebo or usual care; 4) Outcome measures: BPSD used as the primary outcome explicitly or implicitly; 5) Study design: parallel RCTs that lasted at least six weeks. RCTs are well-known to be the gold standard for testing the efficacy of a given intervention. Full papers (not the abstract) were published in English. Non-randomized trials, trials without a control, crossover or head-to-head designs, open-label trials, duplicate reporting of the same trial and reporting in the format of an abstract were excluded.

The Medline database (PubMed) was used. This search was conducted from the inception to October 19, 2013. Additional research included Cochrane review, the National Guideline Clearinghouse database, previously published systematic reviews [31-43], and a hand search for references. Unpublished studies were not examined.

Starting with PubMed, we searched for relevant articles using MeSH terms, key words, and text words. The search was conducted in the following six steps: Step1 for the RCTs Domain: using the following term “randomized controlled trial or controlled clinical trial or placebo or randomly or trial or group or drug therapy or randomized” to retrieve all RCT-relevant citations; Step 2 for Atypical Antipsychotics Domain: using the following term “ziprasidone or quetiapine or olanzapine or aripiprazole or risperidone or clozapine or amisulpride or sertindole or zotepine or atypical antipsychotics” to retrieve all citations relevant to atypical antipsychotics; Step 3 for the BPSD Domain: using the following term “neuropsychiatric symptoms or psychiatric disorders or behavior symptoms or behavioral symptoms dementia or psychological symptoms or behavioral and psychological symptoms of dementia or behavioral disturbances or agitation or aggression or delusion or hallucination or disinhibition or wandering or irritability or delirium or psychosis or euphonia or anxiety or apathy or aberrant motor behavior or night time behavior disturbances or appetite and eating abnormalities” to retrieve all BPSD-related citations; Step 4 for the Dementia Domain: using the following term “dementia or delirium oramnestic dementia orcognitive disorders orvascular dementia or AIDS dementia complex or multi-infarct dementia or front temporal dementia or Alzheimer vaccine or Alzheimer’s disease or cognition disorders or cognitive impairment or mild cognitive impairment” to retrieve all dementia-related citations. Step 5 for NH domain: nursing homes, assisted living facilities, homes for aged, assisted living, residential facilities, housing for the elderly, skilled nursing facilities, long term care, skilled nursing, long term care facilities; Step 6 for combining all citations from Step 1 to 5: Using “AND” to combine RCTs Domain, Atypical Antipsychotics Domain, BPSD Domain, Dementia Domain, and NH domain together and get all related citations for screening eligible papers. The search strategy was guided by an experienced medical librarian from our institution. The search terms and search domains for this systematic review were saved in PubMed (available as requested).

To avoid any missing studies, we also searched Cochrane review, the National Guideline Clearinghouse database, and previously published systematic reviews [31-43], and conducted a hand search for references as well.

All possible citations were retrieved via PubMed. One investigator (HYC) screened and identified all citations for potentially eligible studies. Study selection included the following two steps: 1) Examine the title and abstract for possible inclusion for this study; 2) Review the original article for definite inclusion into this study (Figure 1).

We modified the data extraction sheet based on our previous study [63]. Studies that met the inclusion criteria were reviewed independently. The data sheet was completed independently by two investigators (HYC and TNH). The same two investigators met and compared the completed data sheets, and any discrepancies were resolved by face to face discussion. One author (TNH) entered the data into the Excel data set, which was further reviewed by another author (HYC) for accuracy. The data in Excel was imported to SPSS by one author (HYC) who also performed data cleaning, coding and analysis. The content in all tables was independently reviewed by all authors.

The data sheet included the first author name of the publication, year of the publication, the journal name, methods and scales of dementia and depression diagnostic criteria, demographics (age, gender, and race), measurements and scales of primary outcome measures, secondary outcomes, intervention and comparison, participant recruitment processes including the number of eligible and enrolled participants, drug interventions, trial duration, power calculations, randomization status, blinding status, intention-totreat, outcome measures and statistical significance, sponsorship, Institutional Review Board (IRB) approval, and informed consent. Placebo effects from the intervention and comparison groups from the 12 trials were later added to the sheet and Table 5 by one investigator (HYC). Summary of exclusion criteria and adverse effects were added to Tables 4 and 5.

We assessed risk of bias by assessing the internal validity which is defined as whether the study results in true findings and minimizes systematic error. The internal validity of the trials was determined based on the quality assessment of content from the previously published methods as well as from a previous study of one of the authors (HYC), and included randomization, double blinding, concealment, attrition, intention-to-treat analysis, and a power calculation [61-64]. Assessment of these items covered selection bias (concealment), performance bias (blinding), attrition bias (attrition), and detection bias (blinding) [61].

We assessed the applicability of individual studies by examining external validity i.e., whether the results of the studies in the research setting can be applied to the population in the real world. External validity was assessed by examining the recruitment process, i.e., the percentage of patients in the daily practice that was enrolled through the recruitment process and the exclusion of research subjects [61-65]. The eligible index was defined as the percentage of patients who were screened and met the inclusion criteria. The enrollment index was defined as the percentage of eligible patients who were enrolled in the trial. The recruitment index was defined as the percentage of patients who were screened and enrolled in the trial [63-65]. The exclusion criteria reported among the trials included certain medications, medical diseases, psychiatric and mental illness, and other reasons which are summarized in Table 4.

Heterogeneity is defined as any kind of variability among studies. It incudes 1) clinical diversity (also called clinical heterogeneity), i.e., variability in the research participants, intervention, outcomes; 2) methodological diversity, i.e. variability in study design and risk of bias; and 3) statistical heterogeneity, i.e., variability in the intervention effects [61]. Our inclusion criteria cover clinical heterogeneity e.g., intervention, outcome measurements, and external validity and methodological diversity e.g., risk of bias, internal validity. We did not perform a meta-analysis. Therefore, we did not include an assessment of statistical heterogeneity.

We used percentage of BPSD reduction as the primary outcome measure for efficacy of atypical antipsychotics or placebo across the 12 trials. We reported qualitative descriptions and estimates of the intervention effect among the 12 RCTs and followed one previous systematic review [41] to examine the consistency of the intervention effects by point estimates across the 12 trials. Because we did not perform meta-analysis, we did not have I2 and CIs.

We decided not to perform a meta-analysis because the diagnostic criteria of dementia and BPSD, BPSD outcome measurements, and rating scales varied across the 12 trials. Instead, we provided qualitative descriptions and estimates following one previous systematic review without a meta-analysis [41], which is the only systematic review focusing on NH patients with BPSD. Descriptive analyses were performed in SPSS (Version 18). Data was analyzed by one author (HYC) and discussed with another author (TNH).

Because we decided not to perform a meta-analysis, risk of bias across the 12 RCTs was not assessed. Missing studies, missing outcomes, and detection of missing information were not examined. We used descriptive approach to describe the differences of risk of bias across the 12 trials which are summarized in Table 2 .

The placebo effect is a common phenomenon in pharmacological trials of treating depression and other conditions [66-72] and was assessed across the 12 RCTs in this study.

Diagnostic criteria, outcome measurements, and scales of dementia and BPSD were obtained via searching the original studies and a standard textbook [61] and are listed in the footnote of Tables 1 and 5. Secondary outcomes were also collected and are summarized in Table 5 , which is not the focus of our systematic review.

From PubMed we found 1469 relevant citations using the predefined search criteria. Of those, 1449 were excluded because they did not meet the inclusion criteria. Twenty original papers were fully reviewed. Twelve papers [73-84] met the inclusion criteria (Figure 1). Searches through Cochrane review, the National Guideline Clearinghouse database, previously published systematic reviews [31-43], and a hand search for references did not find additional papers that met the inclusion criteria. We identified more RCTs on our ICOS question than did previous systematic reviews [31-43]. The excluded RCTs and the reasons for exclusion from our systematic review are not reported here, but are available as a supplemental fileonline.

The 12 selected RCTs are summarized in Table 1. All 12 RTCs [73-84] were published between 1999 and 2008. Nine of the 12 RCTs were conducted only in the NH setting [73-81]. Three of the 12 RCTs were conducted either in NH and in clinics [82], or in NH and in hospitals [83,84]. Most trials were sponsored by industry and conducted in the United States. All trials except one [76] used either the Diagnostic and Statistical Manual of Mental Disorders (DSM) [73-75,77,79-84] or the National Institute of Neurological and Communicative Diseases and Stroke-Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA) criteria [78,80], or both [75], to diagnose dementia. The Mini-Mental State Examination (MMSE) was used in 11 trials [73-77,79-84] to assess the severity of dementia and the Functional Assessment Staging Test (FAST) was used in one trial [78]. One trial used both MMSE and FAST [84]. Severity of dementia ranged from mild to severe across the 12 trials. The instruments used to measure BPSD as the primary outcome varied across the 12 trials (Table 1) and included the Neuropsychiatric Inventory- Nursing Home (NPI-NH) [73,74,80,83], the Behavioral Pathology in Alzheimer’s Disease (BEHAVE-AD) [76,81,84], the Cohen- Mansfield Agitation Inventory (CMAI) [78,79], and the Positive and Negative Syndrome Scale-Excitement Component (PANSS-EC) [75]. The Neuropsychiatric Inventory (NPI) and the Brief Psychiatric Rating Scale (BPRS) were used together with other instruments [77,81]. BPSD severity was not defined due to lacking well-accepted scale criteria. However, the scores of BPSD severity are generally at low levels of the BPSD scales across the 12 trials. Among the total 4352 enrolled subjects across the 12 trials, the age of the enrolled participants ranged from 77 to 84 years old. Fifty eight percent to 72% of the participants were female and 80% to 100% of participants were white. The sample size of the enrolled participants ranged from 62 to 652. Among the enrolled participants, 2924 received an atypical antipsychotic agent as the intervention and 1425 received placebo (three research subjects dropped out before the interventions from one trial) [83]. Sixty seven percent and 70% completed the trial in the intervention and placebo groups, respectively. Five of the 12 trials [74,79,81-83] did not report the number of individuals screened. Diagnostic instruments of dementia and BPSD measurements and scales are listed in the footnote of Tables 1 and 5. Finally, IRB approval and informed consent were obtained among all 12 trials.

The internal validity of the 12 selected RCTs is summarized in Table 2. All trials used randomization and double blinding (performance bias and detection bias). Seventy five percent of the trials included power calculations. Fifty eight percent of trials reported concealment (selection bias). The attrition rate ranged from 20% to 42% across the 12 trials (attrition bias). Power calculation was performed in 75% of the 12 trials. Intention-to-treat analysis was used in 83% of the 12 trials.

The recruitment process indicators are summarized in Table 3. Seventy nine percent of the trials reported eligible fractions, which ranged from 57% to 86%. Eighty seven percent of trials reported enrollment fractions, which ranged from 41% to 100%. Sixty nine percent of trials reported recruitment fractions. Among reported trials, recruitment fractions ranged from 33% to 86%. Exclusion criteria included certain medications, co-existing medical and other psychological diagnoses, and other reasons and are summarized in Table 4.

Heterogeneity includes three type of diversity [61]. 1) Clinical diversity (clinical heterogeneity): Table 1 shows all research participants had dementia and BPSD. However, different atypical antipsychotics were used. Severity of dementia and BPSD and BPSD outcome measurements varied across the 12 trials. 2) Methodological diversity: All trials used randomized controlled trial design. However, there were significant variations of risk of bias including concealment and attrition rate across the 12 trials (Table 2). 3) Statistical heterogeneity: The intervention effects from atypical antipsychotics varied across the 12 trials (Table 5). We did not perform a metaanalysis, and so summary of statistical heterogeneity are not available.

We did not perform a meta-analysis for the intervention effects. Therefore, we reported qualitative descriptions and estimates of intervention effect among the 12 RCTs following one previous systematic review [41] to examine the consistency of the intervention effects among the 12 trials.

The atypical antipsychotics tested among the 12 RCTs included risperidone (N=4) [76,79,81,84], quetiapine (N=3) [75,77,78], olanzapine (N=3) [80,82,83], and aripiprazole (N=2) [73,74]. The four RCTs that demonstrated a statistically significant BPSD reduction utilized aripirazole (N=1) [74], risperidone (N=2) [79,84], and olanzapine (N=1) [80]. The three trials with quetiapine [75] and five other trials [73,76,81-83] did not demonstrate statistically significant BPSD reduction. There was inconsistent BPSD reduction among trials testing risperidone, quetiapine, olanzapine, and aripiprazole. The intervention effects on BPSD reduction varied from 7% to 72% and were inconsistent among the 12 trials. There were no statistically significant differences in the number of enrolled participants or in attrition rates between the eight negative and four positive trials (unadjusted).

All trials reported adverse effects, which were similar between the intervention and placebo groups. A brief list of adverse profiles is presented in Table 5, which is not the focus of this systematic review. Most trials tended to report multiple secondary outcomes in Table 5, which is not the focus of this systematic review.

Because we decided not to perform a meta-analysis, we did not analyze the risk of bias across the 12 trials. Instead, we reported the risk of bias for individual studies above (Table 2). Missing studies, missing outcomes, and detection of missing information were not examined in this systematic review.

The effects of placebo on BPSD reduction were reported in the 11 trials except one [74] for which we were unable to calculate a placebo effect. In this trial, the author used a figure for the primary outcome [74]. The placebo effect varied from 14% to 34% among the four RCTs that showed statistically significant BPSD reduction [74,79,80,84,79,80,84] and from 10% to 61% among the seven RCTs that showed no statistically significant BPSD reduction [73,75,77,78,81-83,75,77,78,81-83] (Table 5).

---

Discussion

Though both the FDA and CMS have released warnings and recommendations regarding cautious use of atypical antipsychotics due to the risk of adverse effects [19,20], these agents are still widely used in NH residents [11-17]. Based on the limited evidence of efficacy of atypical antipsychotics, this systematic review further supports these recommendations. In this review, we addressed the following four questions.

First, can atypical antipsychotics reduce BPSD among NH residents? Our major findings indicate that atypical antipsychotics did not reduce BPSD among NH residents in eight of the 12 RCTs. This is consistent with a previous systematic review that focused on NH residents [41]. However, it is inconsistent with previous systematic reviews that were not specific to NH settings [31-40]. This suggests that the results of systematic reviews from the non-NH setting might not be relevant to the NH population. One of reasons for the negative results in these eight RCTs may be the low level of BPSD scales in these participants, despite the severity of BPSD not being defined. We have provided the BPSD measurements and rating scales on the footnote of Tables 1 and 5 which may help the readers to assess the BPSD severity. For example, the mean score of BPSD in the intervention arm in one trial was 40 on a scale of 0-126 [77], while in another study, the mean score of the CMAI in the intervention arm was 59 on a scale of 29-203 [78]. These trials may have shown greater efficacy if investigators had recruited participants with higher levels of BPSD. However, four positive trials showed a statistically significant BPSD reduction. These inconsistent findings reduce our confidence in the intervention of atypical antipsychotics for BPSD reduction. Whether some patients might benefit from the use of atypical antipsychotic agents needs to be confirmed in NH residents with BPSD. In this systematic review, there was no difference in adverse effects between the intervention and the placebo groups, which is most likely due to the small sample size. NH residents are often frail and have multiple co-existing conditions [85] and tend to take multiple medications [86-88], while older adults in general are sensitive to antipsychotic agents [89]. Due to the limited evidence for the use of atypical antipsychotics, the potential for adverse effects [1,18-30], and risks of polypharmacy [86-88], we fully understand the concerns, warnings, and regulations from the FDA and CMS [18-20]. Atypical antipsychotics should be used cautiously for NH residents with BPSD.

Second, are the RCTs examining atypical antipsychotics to treat NH residents with BPSD valid, i.e. is the internal validity good? The internal validity is defined as whether the study results in true findings and minimizes systematic error [61-64]. The internal validity is critically important in RCTs [61-64]. Randomization and blinding among the 12 RCTs were well reported. However, we are very concerned about the high attrition rates among the 12 trials, which may results in attrition bias [61]. High attrition rates in RCTs among the elderly population are expected because older participants might die or drop out due to pre-existing conditions or side effects, but researchers have proposed multiple ways to retain elderly participants [90]. We are also concerned with the low allocation concealment among the 12 trials, which can be an indication of selection bias [61]. A good trial of antipsychotics to treat NH residents with BPSD with a high retention of participants and high allocation concealment should be conducted to reduce the bias and to improve the internal validity.

Third, what is the role of the placebo effect of atypical antipsychotics in the treatment of BPSD in NH residents? Placebo effects in the treatment of depression and other conditions are well documented in RCTs [61,66-72], but the mechanism of the placebo effect is complex and unknown [68-70]. To our knowledge, we may be the first to examine the placebo effect in both the negative and positive trials examining treatment of BPSD in NH residents. The positive trials had a larger placebo effect than did the negative trials, which indicates that the intervention effect in the positive trials was not less likely due to a small placebo effect. While previous studies that showed significant placebo effects often used patients’ self-reported subjective symptoms and research subjects’ responses to interviews, such as with pain or depression [61,66-71,72], BPSD across the 12 trials was observed and measured by researchers. Therefore, placebo effects in these trials cannot arise from the NH residents themselves, but more likely arise from the researchers. Another potential mechanism of the placebo effect could be due to the therapeutic effect of the interaction between the researchers and participants, which needs to be tested. Finally, these trials lasted from six to 12 weeks. BPSD natural history is toward a reduction over time, which may also explain the observed placebo effect.

Fourth, are the results of the RCTs applicable to the population in the real world, i.e. is the external validity good [91-93]? External validity has been an issue in conducting RCTs. For example, research subjects from RCTs in treating depression may have only represented the minority of the patients in the real world [94-98]. Our results showed that the research subjects across the 12 RCTs in treating BPSD in NH patients with dementia were not representative of the realworld patients because many participants were excluded from the trials, including those on certain medications, or having co-existing medical diseases and other psychological conditions. The highly selected populations in these RCTs are likely significantly different from real NH patients with dementia who often have multiple coexisting conditions [99-101]. In addition, the participant recruitment process across the 12 RCTs was under-reported, and therefore it is difficult for the readers to know from where the enrolled research subjects came. Taken together, all these factors could significantly reduce the external validity of these 12 RCTs. Better reporting methods and better recruitment processes are needed in RCTs examining treatment of NH residents with BPSD. Recruiting typical NH patients via a pragmatic RCT design should be used to improve the applicability and guide daily practice [102-105].

Heterogeneity is a crucial part of the assessment of RCTs and other types of studies [61]. Heterogeneity includes three types of diversity [61]. 1) Clinical diversity (clinical heterogeneity) is present among the 12 trials because four different atypical antipsychotics (risperidone, quetiapine, olanzapine, and aripiprazole) with different doses were used, and because the severity of dementia and BPSD and BPSD outcome measurements varied across the 12 trials. 2) Methodological diversity is present among the 12 trials because of significant variations of the risk of bias including concealment and attrition rates. 3) Statistical heterogeneity is present among the 12 trials because the intervention effects from four atypical antipsychotics varied. We did not perform a meta-analysis, so a summary of statistical heterogeneity are not available. We are concerned with the heterogeneity of the 12 studies analyzed, as it reduces the internal and external validity of the findings.

The discussion of other findings is worthwhile. Based on our own experience, primary care providers are often uncomfortable using research instruments such as BPSD outcome measurements as they are less commonly used in their daily primary practice. This could limit their understanding and consequent application of the results from these RCTs. Additionally, the details of BPSD outcome measurements and the rating scales were not fully reported in previous systematic reviews. This systematic review provides the various BPSD outcome measurements and scales in order to help primary care providers understand these measurements and resultsof the 12 RCTs.

Since the Cochrane Handbook for Systematic Reviews of Intervention was published in 2008 [61], The Grades of Recommendations, Assessment, Development, and Evaluation (GRADE) has been rapidly evolved to help systematic reviewers, clinical practice guideline developers, and health technology assessments [106]. The first part of GRADE (evidence profiles) is well developed to rate the quality of evidence-based risk of bias, publication bias, imprecision (random error), inconsistency, and indirectness and to classify the confidence of effect estimates into four categories from high, moderate, low to very low quality of evidence [106-115]. The Evidence-based Practice Center (EPC) program funded by the U.S. Agency for Health Research and Quality (AHRQ) has updated their grading system accordingly based on GRADE [116,117]. Because we were not familiar with the evolution of GRADE since 2010, we did not integrate GRADE into our systematic review. However, we wouldlike to briefly summarize the quality of evidence for the 12 selected RCTs in our systematic review based on the latest GRADE framework as following. 1) Risk of bias. It is good to have blindness and report primary and secondary outcomes among the 12 trials. However, the low concealment rate (58%) and high drop-out (20-42%) among the 12 trials indicates significant risk of bias. 2) Publication bias. The small number of selected RCTs, the small sample size of some RCTs, and possible missed studies including unknown unpublished studies increases the risk of publication biasin our systematic review. 3) Imprecision. CIs and optimal information size (OIS) were not obtained, which indicates the imprecision might be present among the 12 trials. 4) Inconsistency. The point estimates of intervention effects on BPSD reduction varied widely from 7% to 72% among the 12 trials, which is considered an inconsistency. We did not perform a meta-analysis and unable to provide information on statistical heterogeneity and I2. The inconsistency in our systematic review is uncertain. 5) Indirectness. Highly selected research participants (poor applicability), different atypical antipsychotics with different doses and trial durations, and participants from academic settings among the 12 trials indicate indirectness. Taken together, we rate the quality of evidence from the 12 trials as low.

We admit our systematic review has several limitations. GRADE framework was found to be reproducible [118] and recently used to assess the quality of evidence via the format of evidence profile and summary of findings [118,119]. We did not integrate this format with our systematic review in the beginning. In our opinion, this format will be fully accepted by the Cochrane Handbook for systematic review of interventions in the future. We did not write a full search protocol, however we did have a search plan. A true pre-specification of methods was not done because one of the authors wrote an invited chapter on BPSD and knew few RCTS on BPSD. This review had a small number of RCTs. A meta-analysis was not performed due to the different diagnostic criteria and severity scales of dementia and BPSD, as well as different BPSD outcome measurements and rating scales among the 12 trials. Non-English papers were excluded, and unpublished data were not collected. PubMed was used to search and identify relevant papers, but other datasets were not used. Despite our extensive search terms and search of Cochrane reviews, the National Guideline Clearinghouse database, previously published systematic reviews, and hand search, we may have missed some relevant RCTs which results in publication bias. Missing studies, missing outcomes, and detection of missing information were not fully examined.

---

Conclusions

There is limited and inconsistent evidence to demonstrate theefficacy of atypical antipsychotics in the treatment of behavioral and psychological symptoms of dementia in nursing home residents. In addition, there are concerns about the risk of bias, applicability, clinical heterogeneity, and methodological diversity among the 12 trials. We are less confident in the intervention effects of atypical antipsychotics on BPSD in nursing home patients and therefore do not recommend routinely prescribing atypical antipsychotics for this indication.