你的位置:论文发表 >> 论文下载 >> 医药论文 >> 医学论文 >> 详细内容 在线投稿

Spanish Version of Test MIS (*) with Delayed Memory Recall Normative Values and

浏览200次 时间:2017年4月27日 09:14
Spanish Version of Test MIS (*) with Delayed Memory Recall Normative Values and Results in a Population with Mild Cognitive Impairment Labos Edith1, Trojanowski Sofía1, Seinhart Daniel2, Schapira Marcelo2, Renato Alejandro1 1 Laboratory of Cognitive Functions Research-Faculty of Medicine, University of Buenos Aires, Buenos Aires, Argentina 2 Italian Hospital of Buenos Aires, Buenos Aires, Argentina Email: elabos@arnet.com.ar Received 24 April 2015; accepted 26 June 2015; published 30 June 2015 Copyright © 2015 by authors and Scientific Research Publishing Inc. This work is licensed under the Creative Commons Attribution International License (CC BY). http://creativecommons.org/licenses/by/4.0/ Abstract Background: Cognitive impairment becomes more common with ageing and may benefit from intervention. In a Spanish speaking population, detection of cognitive impairment by a general practitioner in Primary Care can be a problem, as many of the standard tests target English speaking populations. The Memory Impairment Screen (MIS-A) is a validated test using English words to detect Alzheimer’s Disease (AD) and other dementias. We have modified this test to suit a Spanish speaking population and added a new component, delayed recall. We have called our new test the Memory Impairment Screen with Delayed Recall (MIS-D). Objectives: 1) To test a Spanish version of MIS-A and MIS-D. 2) To assess the discriminative validity of MIS-D as a screening tool for the amnestic variant of Mild Cognitive Impairment (aMCI) in a group of Spanish speaking people aged 65 years old and over. Methods: A case-control study of a cohort of 739 native Spanish speaking residents of Buenos Aires aged 65 years old and over, of whom 436 were healthy controls and 303 had a diagnosis of aMCI. Measurements: Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NVP) were estimated for MIS-D and MIS-A. Results: Normative values for MIS-A and MIS-D were obtained from the control population. Both age and education significantly affected these values (p < 0.0001). Control participants showed significant differences for both modalities, MIS-A and MIS-D. The cut-off for MIS-A should be 7.5 and for MIS-D, 5.5. Comparison between control population and aMCI population using ROC curve gave a result of 5.5 in MIS-D, with 97% specificity and 76% sensitivity. Conclusion: MIS-D was positively predictive of aMCI, with 97% specificity and 76% sensitivity in a sample of Spanish speaking patients aged 65 years old and over in Buenos Aires. L. Edith et al. 46 Keywords Amnestic Mild Cognitive Impairment, Memory Test, Delayed Recall 1. Introduction Memory complaints make up an important presenting clinical entity in neuropsychology, especially in patients in the very early stages of Alzheimer’s Disease. In 2.3% to 6.6% of older people with subjective memory complaints there will be progression to MCI or dementia each year [1]. Cognitive impairment and dementia are increasing globally. The probability of developing Alzheimer’s Disease (AD) rises exponentially with age, doubling every 5 years after the age of 65. Projections indicate that by 2050 the number of individuals older than 60 years old will be approximately 2 billion and will account for 22% of the world’s population [2]. According to the World Health Organization [3], 7.7 million new cases of dementia are registered every year, with an important contribution from Asia (46%). Even more striking, the World Alzheimer Report 2014-ADI [4] estimated that 44 million people worldwide were living with dementia. What is more, the report of First WHO ministerial Conference on Global Action against Dementia [5] states that dementia is one of the major causes of disability and dependency among older people worldwide. Recent research [6] raises concern when it comes to identifying early markers of AD disease, as early diagnosis could potentially improve response to treatment. While a late diagnosis of AD is a problem for primary health care practitioners [7] [8] as most patients make frequent consultations regarding their cognitive complaints and do not receive an accurate diagnosis, an early diagnosis would be of great help in medical decision making concerning the patient’s overall health; in fact, it may avoid the risk of exposure to unnecessary pharmacological treatments, exposure to anaesthesia and others [9] [10]. According to “Memory Matters”, a report released by the Alzheimer’s Foundation of America (AFA-2008), there has been a growing interest in screenings for memory problems because most persons with dementia remain undiagnosed by their primary care providers, and families often fail to recognize the significance of early cognitive symptoms. Alarmingly, only one in four patients with AD is diagnosed at an early stage [11]. In highincome countries, only 20% - 50% of dementia cases are recognized and documented in primary care [12]. This finding shows the need to identify population at an early stage of the disease. The International Working Group proposed three steps in the development of AD: 1) the asymptomatic, atrisk, stage of AD (AD pathology evidenced by biomarkers and no symptoms); 2) prodromal AD deficit of episodic memory, the ability to learn and retain new information, together with impaired cued recall that can be isolated or in association with other cognitive changes and biomarker evidence for AD; and, 3) AD dementia (dementia and biomarker evidence for AD) [13] [14]. It was Bruno Dubois [15] who introduced the concept of Prodromal AD, characterized by the presence of mesiotemporal (amnesia linked to the hippocampus and surrounding parahippocampal and entorrhinal areas) plus one of the biomarkers. The mesiotemporal amnestic deficit is characterized by a failure of verbal episodic memory (VEM), as patients find it hard to recover verbal information after a delay, which may signify abnormal semantic representations. In contrast, the main clinical manifestations of the Mild Cognitive Impairment (MCI) that may envolve into AD include gradually progressive episodic memory impairment and objective evidence of a retrieval deficit that will not improve with verbal cueing. This deficit may appear alone or in association with other cognitive changes [16]. Between 3% and 19% of adults over 65 years old suffer from MCI, and more than a half of them progress to dementia within 5 years [17]. This is why aMCI represents a high risk factor for developing AD, and may in fact represent a new indicator for Prodromal AD. According to Dubois [18], the amnestic syndrome of the hippocampal type defined by very poor free recall and a decreased total recall due to an insufficient effect of cueing can identify prodromal AD in patients with MCI with a sensitivity of 79.7% and a specificity of 89.9%. The neuropsychological evaluation is crucial for establishing the nature of memory impairment at the prodromal stage. Specific memory tests are useful for distinguishing an amnestic syndrome from the type of memory deficits that result from attentional and executive dysfunction. Many reports point out that memory tests that control for encoding deficits and provide semantic cueing to facilitate retrieval can improve the accuracy of AD diagnosis [19] [20]. Wagner et al. [21] confirmed the predictive value of cognitive scales, including cued recall of verbal events, in a longitudinal study involving monitoring of MCI patients. The National Institute on Aging and the Alzheimer’s Association have developed criteria for diagnosing the symptomatic pre-dementia phase of L. Edith et al. 47 AD [10], which they call “Mild Cognitive Impairment due to AD”. They reported that impairment of episodic memory is most commonly seen in MCI patients who subsequently progress to a diagnosis of AD. When it comes to cognitive assessment, the authors suggest using episodic memory tests that assess both immediate and delayed recall. Impairment after a delay in MCI patients increases the likelihood of progressing to AD dementia within a few years. Developing a neuropsychological test that can make a useful prediction as to whether cognitive impairment will evolve into Alzheimer’s disease would be especially useful in clinical settings where access to expensive imaging of amyloid in the brain and spinal fluid analysis for abeta42 and tau is limited. A second advantage would be to use these tests in research settings, including clinical trials. Neuropsychological tests like Buschke’s Free and Cued Selective Reminding Test (FTCSR) [22] showed high specificity for identifying Prodromal Alzheimer’s two years prior to the diagnosis of dementia by measuring Free Recall performance and total recall of verbal stimuli. Dubois [23] considers the FCSRT to be the best predictor of AD, which provides specificities of 90%, and was more sensitive to incidence and prevalence of dementia than others measures. In sum, MCI represents a useful clinical entity as once for of it, aMCI, may augur AD. An early diagnosis of aMCI could encourage patients to seek interventions that could delay or prevent progression to AD. The frequent limitations to implementing biomarker testing for AD, because of the high cost of the testing and often unavailable technology, also have to be considered when it comes to dealing with diagnostic criteria. A brief test should fulfill the following needs: 1) to be fast and easy to administer; 2) to be accepted by professionals and patients; 3) to be easy to score; and, 4) to be independent from the test language, culture and educational level. Furthermore, it should fulfill a series of methodological requirements such as: a) having good internal consistency; b) having a high inter and intra evaluator reliability; c) having concurrent validity; d) having a good predictive validity and criteria; and, e) having comparative norms [24]. Ideally, this brief test would be a screening neuropsychological test that allows for identification of the individualat risk for aMCI in a primary health care setting. Memory evaluation is usually included as a sub-item in traditional screening tests like the Mini Mental Scale [25]. Nevertheless, tests limited to three-word items in free recall, which are commonly used in clinical practice, are of low specificity for early diagnosis [26] [27]. In 1999 Buschke [28] created the Memory Impairment Screening Test (MIS), which includes: a “four-item delayed free and cued-recall memory impairment assessment that uses controlled learning and cued recall to optimize encoding specificity”. The test was designed to obtain greater sensitivity and to be more specific when it came to detecting episodic memory disorders. According to the cognitive model, Buschke makes reference to the benefits of using the same cues at acquisition and retrieval, in agreement with Tulving, who is quoted to say that “specific encoding operations performed on what is perceived determine what is stored and what is stored determines what retrieval cues are effective in providing access to what is stored” [29]. The quality of retrieval relies on a good semantic codification of stimuli. In this way, potential attentional issues that could act as confounders in an immediate word-list retrieval test can be clearly identified. The Memory Impairment Screening Test has a great potential for clinical utility considering that in 75% of cases, Alzheimer’s Disease is not identified in primary care centers [30]. Buschke’s original report claims a sensitivity of the test of 0.87%, with a specificity of 0.96 for a cut-point of 4, demonstrating its efficacy to detect AD in early stages. MIS has been validated in Spain in three separate centers specializing in cognitive disorders [31] [32], and similar results were obtained in relation to the original version, with equal or superior validity to MMSE. Bhom et al. [33] reported a sensitivity of 0.74 with a specificity of 0.96 for detecting AD, and a cut-off score less than or equal to 4 points. Galeno Rojas et al. [34] introduced a Spanish version of the MIS Test, with normative data applied to a sample of 183 participants (88 patients with dementia, 54 with AD and 41 controls). They observed that in dementia as a whole, the most effective cut off point was <6 points, sensitivity of 82% and specificity of 90%, whereas in AD a cut-off of <5 gave a sensitivity of 85% and specificity of 95%. We have used psycholinguistic criteria for selecting the stimuli in present version of MIS. We took into account the freqcuency of use of Spanish words in our population, relying on previous research [35] carried out by our team, where specificity values for 22 semantic categories are represented in the sample. The stimuli employed in the test correspond to examples of medium freqcuency of words use in argentinian population. We considered the length and syllabic complexity of the stimuli to optimize the phonologic and syllabic variables of words. Then, keeping in mind the same criteria, L. Edith et al. 48 we designed a second list of stimuli to be used as a retest. We suggest that the present version is optimized for South-American Spanish. We have maintained in our design the same semantic and linguistic stimulus categories as those in the original version. In previous work [36] 180 healthy participants over the age of 50 were studied, according to the pre-established inclusion criteria: 186 patients with cognitive impairment (of which 77 with MCI, 76 with AD and 33 with major depression) were evaluated neurologically and neuropsychologically. The MIS test was administrated as a single-blind test to a population of patients with MCI or AD. Keeping in mind the prevalence of AD, a control cohort of participants aged 65 years and over was selected, matched with the patients for age and level of education. The results obtained were corroborated with other studies with a cut-off ≤4 that evinced a high sensitivity (92.3%) and a specificity greater than 70% for detecting dementia. A positive correlation with ADAS Cognitive Memory Scale was obtained. The final results in the control population were 7.49 ± 0.84, with no significant differences between the two versions. Our findings placed the higher sensitivity in a cut-off of 6, in contrast to other studies that located it at a lower cut-off. The clinical implications of these results are that participants that obtain a lower score can be considered to be at risk for developing dementia of the Alzheimer type. The present version is currently being validated in other Latin American countries in order to see whether our present results can be generalized. To optimize the sensitivity and specificity of the test for the purpose of detecting patients with MCI, we added a delayed recall register (DR) of verbal stimuli. Patients are asked to remember, after a 30-minute delay, the four-item free and cued list. We call this version the MIS-D, while reserving the name MIS-A for the original version without the delayed recall component. The DR addresses a phase of memory processing, it is included on a standard neuropsychological evaluation, and serves as a predictor of AD in patients with MCI [37]-[39]. Our modification optimizes the evaluation of verbal episodic memory, which allows for the early identification of persons with aMCI and who is also at risk for AD. In fact, the screening of at-risk populations for aMCI should be considered an important contribution for the early treatment and prevention of cognitive decline. The objectives of this work were 1) to introduce a version of MIS-A and MIS-D in the Spanish language and in native Spanish speaking residents of Buenos Aires, and 2) to determinate sensitivity and specificity values for MIS-D in patients with aMCI (ROC curve). We are aware that screening does not provide a sure diagnosis, but it may guide there ferral of patients for further evaluation aiming at confirming or discarding a diagnosis of aMCI. 2. Methods We studied a population of 739, of which 436 were healthy controls (Group A) and 303 had aMCI (Group B). All participants gave informed consent to participate. For the Group A the following criteria of inclusion and exclusion were established. The participants were at least 60 years old and were able to read and write, and they should also demonstrate cognitive integrity. They were also required to score = 24 on the Mini Mental State Exam (MMSE: Folstein et al, 1975), with no significant impairment on activities of daily living (ADL) or on instrumental activities of daily living (IADL). All potential participants having neurological, psychiatric, psychological and/or sensory disorders (without compensation) that could alter their cognitive performance were excluded. The control participants who were recruited came from surgical and medical units of the Italian Hospital of Buenos Aires, from administrative offices of the school of Medicine (University of Buenos Aires), from referrals by general practitioners, and also patients who had consulted a physician for memory complaints but whose cognitive evaluation did not show significant deficits. The MCI group was patients from the Geriatric Unit for the Elderly at the Italian Hospital of Buenos Aires staffed by geriatricians. All patients were tested with the standard Cognitive Evaluation (see below). Procedures were approved by the local institutional review board. All the participants were recruited between January 2008 and March 2011. 2.1. Assessments The cognitive evaluation at the Italian Hospital’s Functional Study Unit for the Elderly include: the Anamnesis, Clinical Dementia Rating, the Blessed Dementia Scale, the Functional assessment stages for AD , the ADL, the IADL, Hamilton Depression Rating Scale, the Rosen modified Hachinsky Score, the Trail Making Test and the L. Edith et al. 49 ADAS Cog. with Delayed Recall [40]. Versions of the MIS Test (A and D) are usually given in our Cognitive Evaluation. The MIS test (A and D), together with a neuropsychological screening battery including the MMSE, the ADAS Cog., and the Verbal Fluency test, were administrated to both samples. In order to diagnose MCI, the Petersen criteria [41] were used: 1) memory complaints, preferably corroborated by an informant other than the patient; 2) impaired memory function for age and educational level on formal testing; 3) preserved general cognitive functions; 4) intact activities of daily living; and 5) no evidence of dementia. MIS-A and MIS-D Tests (See Appendix 1): Procedures The participant must read aloud four items from different categories (game, crockery, message and job). Then, he has to identify the categorical cue of the stimuli introduced (these are learnt when shown on a card at the moment of introduction of its categorical cue). Memory is evaluated in free recall after an interval of 2 or 3 minutes when a distractive non-semantic task is applied, for example, counting from 1 to 20 and repeating the sequence as time goes by. If one of the items is not assimilated as free recall, the categorical cue correspondent to that item is introduced again so as to stimulate his memory through cued recall. 30 minutes after Free recall takes place, delayed recall of items is performed (free and cued). The words and cues used in MIS-A and MIS-D are the same ones. Punctuation: In the first place, words must be added to each column: cued and free recall. Then, the number of words listed in the Free recall column must be multiplied by 2 and finally, this result must be added to the number of words listed in the Cued recall column (2 × Free recall + Cued recall) in order to obtain the MIS score. We can estimate any operation using the same scoring for MIS delayed recall (Appendix 1). 2.2. Statistical Analysis An ANOVA was carried out together with the Student’s t-test for the means for age groups, education and gender both for control group and for MCI group. Then, the values of sensitivity and specificity were verified through ROC curve, as well as positive predictive values (PPV) and negative predictive values (NPV) for the original MIS (MIS-A) and MIS-D. Tukey’s Honestly Significant Difference (HSD) Analysis was designed in order to identify significant differences between groups for age and education items. Statistic software R. was used for ANOVA analysis and the Student’s t-Test for ROC curve analysis. The specific package “pROC” [42] was used for ROC curve, which was developed in the Swiss Institute of Bioinformatics, and the package “Psych” [43] for other analysis. 3. Results The sample was divided into three age bands: (60 - 69, 70 - 79, and 80 - 90 years old). The educational level was divided into three categories: Elementary (up to 7 years education), Medium (from 8 to 12 years) and High (>13). Group A (Table 1 and Table 3) consisted of healthy Spanish speaking residents of Buenos Aires older than 60 who fulfilled the pre-established inclusion criteria. The value of MMSE for the healthy participants was 28.61 (SD ±2.43). Group B (Table 2 and Table 3) consisted of participants diagnosed with aMCI. We confirm a non- significant difference in means related to age and education. Study of “covariate balance” was carried so as to know if both population were properly balance. We used “Matching” package [44] observed in the “R” distribution. The results showed a non-significant difference for age (Estimate = −0.32412, p-value = 0.311) educational level (Estimate = −015413, p-value = 0.81929) and gender (Estimate = −0.019829, p-value = 0.08594). 3.1. Analysis of Control Group Results from Analysis of Variance (ANOVA) for MIS-A and MIS-D were analyzed within the control group (A) according to variables such as age, education (Table 4) and gender. Age and education proved to be significant variables both for MIS-A and MIS-D. The gender variable showed no relevance within the groups. The results obtained from Tukey’s Honestly Significant Difference (HSD) analysis for both varieties, MIS-A L. Edith et al. 50 Table 1. Descriptive data of control group (A). N Mean SD Median Trimmed Mad Min Max Range Skew Kurtosis AGE 436 72.99 6.72 73 73.03 8.90 61 91 30 −0.05 −0.85 EDUC. 436 13.23 7.20 12.5 13.43 9.64 1 24 23 0.07 −1.44 MMSE 436 28.61 2.43 29 28.78 1.48 1 60 59 1.37 99.39 MIS_A 436 7.72 0.52 8 7.82 0.00 6 8 2 −1.71 2.02 MIS_D 436 7.16 0.82 7 7.23 1.48 5 8 3 −0.63 −0.38 Table 2. Descriptive data of MCI group (B). N Mean SD Median Trimmed Mad Min Max Range Skew Kurtosis AGE 303 79.43 5.69 80.00 79.69 5.19 63.00 96.00 33.00 −0.38 0.11 EDUC. 303 15.58 7.06 21.00 16.00 1.48 3.00 24.00 21.00 −0.31 −1.74 MMSE 303 25.45 3.71 26.00 25.98 2.97 2.00 30.00 28.00 −2.30 9.35 MIS_A 303 5.35 2.16 6.00 5.60 1.48 0.00 8.00 8.00 −0.85 −0.04 MIS_D 303 3.84 2.06 4.00 3.88 1.48 0.00 8.00 8.00 −0.17 −0.71 Table 3. Distribution of participants according to age and education. Elementary Medium High Elementary Medim High G.A G.B G.A G.B G.A G.B 60 - 70 57 11 42 6 60 9 70 - 80 78 74 51 46 95 18 80 - 90 15 79 18 32 20 27 Table 4. ANOVA: MIS-A and MIS-D for control group. MIS-A MIS-D AGE Anova DF Sum Sq Mean SQ F Value Pr (>F) Means 60 - 70 (N = 159) 70-80 (N = 224) 80 - 90 (N = 53) Grand Mean (N = 436) 2 5.81 2.91 11.03 <0.0001 7.86 7.67 7.53 7.72 2 24.76 12.3 20.21 <0.0001 7.45 7.04 6.77 7.15 EDUCATION Anova DF Sum Sq Mean SQ F Value Pr (>F) Means Elementary (N = 161) Medium (N = 142) High (N = 133) 2 5.31 2.65 10.04 <0.0001 7.6 7.71 7.81 2 6.17 3.08 4.7 <0.0001 7.02 7.16 7.32 and MIS-D, show that significant differences occur among groups of age 70 - 80 and 80 - 90 in relation to the 60 - 70 group, but not between the former ones (Table 5). This evinces an abrupt fall as years go by, being the high- L. Edith et al. 51 est fall close to the seventies. Figure 1 shows different values of means of MIS-A and MIS-D for age variable. Tukey’s analysis (HSD) of MIS-A and MIS-D variables shows a significant difference between Elementary and High education levels for both MIS-A and MIS-D (Table 6). As regards MIS-A there is a difference between Elementary and High levels and between Medium and High ones, while for MIS-D the significant difference occurs between High and Elementary levels. Finally, an analysis of ANOVA with Type III error was performed in order to observe the interaction between variables Age and Education in relation to Target categories in MIS-A and MIS-D (Table 7). For MIS-A and MIS-D we verify that variables Age and Education are significant, but not the interaction between them (Figure 2). The analysis of normative data (Table 8) shows that the means decline according to age and education for both MIS-A and MIS-D, being more abrupt for MIS-D. This is a reasonable finding because episodic verbal memory declines with age and its delayed phase demands a significant effort to the system in elder Participants. Table 5. Tukey’s Honestly Significant Difference (HSD) analysis for variables MIS-A and MIS-D according to age. AGE MIS-A MIS-D 70 - 80 60 - 70 80 - 90 60 - 70 80 - 90 70 - 80 60 - 70 70 - 80 80 - 90 DIFF −0.196 −0.333 −0.137 Means 7.86 7.67 7.53 Lower −0.322 −0.525 −0.321 Sd 0.40 0.57 0.58 Upper −0.071 −0.142 0.047 Min 6 6 6 P Value 0.00073** 0.00014** 0.1893901 Max 8 8 8 Letter A B B DIFF −0.413 −0.679 −0.266 Means 7.45 7.04 6.77 Lower −0.603 −0.971 −0.548 Sd 0.71 0.85 0.49 Upper −0.222 −0.387 0.015 Min 5 5 5 P Value <0.0001** <0.0001** 0.0673965 Max 8 8 8 Letter A B B Table 6. Tukey’s Honestly Significant Difference (HSD) Analysis for variables MIS-A and MIS-D according to Education. (a) EDUCATION MIS-A DIFF Lower Upper P Value Medium-Elementary 0.11 −0.03 0.25 0.16 High-Elementary 0.27 0.13 0.41 0.01** High-Medium 0.16 0.01 0.31 0.03 Means Sd Min Max Letter Elementary (N = 161) 7.60 0.62 6 8 B Medium (N = 142) 7.71 0.50 6 8 B High (N = 133) 7.87 0.38 6 8 A (b) EDUCATION MIS-D DIFF Lower Upper P Value Medium-Elementary 0.14 −0.08 0.36 0.31 High-Elementary 0.29 0.07 0.51 0.01** High-Medium 0.15 −0.08 0.38 0.26 Means Sd Min Max Letter Medium (N = 161) 7.02 0.87 5 8 B Medium (N = 142) 7.16 0.81 5 8 AB High (N = 133) 7.31 0.73 5 8 A L. Edith et al. 52 Table 7. Anova type III error between age and education for MIS-A and MIS-D. Sum Sq Df F Pr (>F) MIS-A (Intercept) Age Education Age: Education 17423.4 4.8 3.7 1.3 1 2 2 4 68832.64 9.56 7.25 1.26 <0.0001 <0.0001 0.0007996 0.2847387 MIS-D (Intercept) Age Education Age: Education 23 5.2 4 1 2 2 4 24643.56 19.12 4.36 1.68 <0.0001 <0.0001 0.01336 0.1538 Table 8. Normative values for MIS-A and MIS-D compared by age and education. Age Education N Variable Mean SD Min Max Median Lower 95% CL for Mean Upper 95% CL for Mean t Value Pr > |t| 60 - 70 Elementary 48 MIS-A MIS-D 7.83 7.46 0.48 0.77 6 5 8 8 8 8 7.69 7.23 7.97 7.68 113.92 67.05 <0.0001 <0.0001 Medium 57 MIS-A MIS-D 7.82 7.47 0.43 0.66 6 6 8 8 8 8 7.71 7.30 7.94 7.65 138.11 85.82 <0.0001 <0.0001 High 54 MIS-A MIS-D 7.92 7.42 0.26 0.72 7 6 8 8 8 8 7.85 7.23 8 7.62 220.33 76.17 <0.0001 <0.0001 70 - 80 Elementary 92 MIS-A MIS-D 7.54 6.91 0.65 0.87 6 5 8 8 8 7 7.41 6.73 7.68 7.09 110.77 75.98 <0.0001 <0.0001 Medium 72 MIS-A MIS-D 7.64 7.00 0.54 0.88 6 5 8 8 8 7 7.51 6.79 7.76 7.21 120.30 66.88 <0.0001 <0.0001 High 60 MIS-A MIS-D 7.88 7.28 0.37 0.74 6 5 8 8 8 7 7.79 7.09 7.98 7.47 163.96 76.39 <0.0001 <0.0001 80 - 90 Elementary 21 MIS-A MIS-D 7.33 6.52 0.58 0.60 6 5 8 7 7 7 7.07 6.25 7.60 6.80 58.21 49.70 <0.0001 <0.0001 Medium 13 MIS-A MIS-D 7.61 6.69 0.50 0.48 7 6 8 7 8 7 7.31 6.40 7.92 6.98 54.22 50.23 <0.0001 <0.0001 High 19 MIS-A MIS-D 7.68 7.10 0.58 0.74 6 6 8 8 8 7 7.40 6.75 7.96 7.46 57.51 42.00 <0.0001 <0.0001 Normative values should rank between 6 and 8 for MIS-A and between 5 and 8 for MIS-D, depending on age and education. (For example, in ranks 70 - 80 years and 80 - 90 years with elementary education, values of 5 in MIS-D are acceptable.) 3.2. ROC Analysis of MIS-D for Population with MCI Sensitivity and Specificity parameters were determined for different cut points considering MCI population versus control population (Table 9 and Table 10). The comparison between control population and population with MCI using the ROC curve gave a result of 5.5 in MIS in delayed time, 97% of specificity and 76% of sensitivity. The software used to obtain the ROC curve uses decimal numbers for the cut offs. This means that a cut off of 5.5 suggests that whoever obtains 6 is over 5.5, and likewise, who ever obtains 5 is below 5.5. The choice of the cut-off point depends either on clinical objectives or on research ones. The choice of a cut-off point with high specificity is more suitable to be used in studies aiming to identifypotential cases of the disease in a given population. On the contrary, the choice low specificity cut-off point aims to include the largest quantity of the diseased individuals below the cut-off point. Since this study attempts, on the one hand, to account for the normative values in the population context and on the other, for its clinical application, the choice of a 5.5 value is the best cut-off point which maximises both specificity and sensibility. L. Edith et al. 53 Figure 1. Age groups for MIS-A (on the left) and Age Groups for MIS-D (on the right). Figure 2. Interaction between means of age and education for MIS-A and MIS-D. Table 9. MIS-D: values of sensitivity, specificity, PPV, and NPV for different cut-off points. Threshold 7.5 6.5 5.5 4.5 3.5 2.5 Sensitivity Specificity 1-ppv 1-npv 0.97 0.39 0.47 0.04 0.91 0.79 0.25 0.07 0.76 0.97 0.05 0.15 0.61 1.00 0.00 0.21 0.41 1.00 0.00 0.29 0.25 1.00 0.00 0.34 Table 10. MIS-D: Different cut-off points with confidence intervals and means. Sp. low Sp. median Sp. high Se. low Se. median Se. high 7.5 6.5 5.5 4.5 0.35 0.75 0.95 1.00 0.39 0.79 0.97 1.00 0.44 0.83 0.98 1.00 0.95 0.88 0.72 0.56 0.97 0.91 0.76 0.61 0.99 0.94 0.80 0.66 In order to inter pretate the results in clinical terms, three variables must be considered: age, education and the probability of occurrence of the disease within the population. In our case, the probability of occurrence reflects the features of the population who attend hospital outpatient consultations, and in this case it would not be expected to find too many differences between the sample hereby obtained and the probability of occurrence in the L. Edith et al. 54 clinical context. Therefore we observe that the 5.5 cut-off point for the MIS-D is the most suitable one in statistical terms and accounts for the validity of the proof. The 5.5 cut-off point has a specificity of 97%, which guarantees a minimal fraction of the population below this value not carrying the disease. In this case the 1-PPV value is 0.05, indicating only the 5% of the normal population being below the 5.5 of the cut-off value. The sensitivity value is 76% for the 5.5 cut-off point, with a 1-NPV of 0.15, which indicates that the 15% of the patients who carry the disease and who are above this value are not properly detected as aMCI. A higher cutoff point, such as 6.5, has a 1-NPV of 0.07, which indicates that the non-detected aMCI population is halved, that is, only a 7%. However, the amount of individuals without the pathology who are considered positive, increases by 25%. A higher cut-off would lose specificity and would improve in sensitivity, even though we consider it better to attract a major quantity of subjects with aMCI (Figure 3). The area under the ROC curve covered a 93.3%, with a confidence interval of 91.4%, 95.2%, as shown in Figure 4. The softened ROC curve was also estimated, and 94% was obtained under the area of the curve. Sensitivity values were of 83%, specificity of 92%, with a PPV of 89% and a negative predictive value of 90% (Figure 4). 3.3. Curve ROC Analysis of MIS-A for aMCI A ROC curve analysis was carried out for MIS-A test on the same population. Data reveals that the test does not detect MCI patients in a reliable way with MIS-A. The area under the curve is of 87.9%, which constitutes a method enabled to capture a proper percentage of population (Figure 5). Nevertheless, when it comes to analyzing the cut-off, we observed that the cut-off for MIS-A is 7.5, with a sensitivity of 87.08% and a specificity of 75.68%, a PPV of 0.71 and a NPV of 0.89 (Table 11). The software used to obtain the ROC curve uses decimal numbers for the cut offs. This means that a cut off of 7.5 suggests that whoever obtains 8 is over 7.5, and likewise, whoever obtains 7 is below 7.5. Values showed that the cut-off was too high, because a 7.5 value out of a maximum of 8 would mean that only subjects obtaining a value of 8 could have been considered normal. The truth is that a score of 7 was acceptable for that population, especially for older age groups. On the other hand, by selecting this score, many patients with MCI would not have been registered by the test. Figure 3. Cut-off point with selected specificity and sensitivity values. L. Edith et al. 55 Figure 4. ROC curve. Figure 5. Area under the ROC curve, cut off point and confidence intervals for MIS-A. L. Edith et al. 56 Table 11. MIS-A: Values of sensitivity, specificity, PPV, and NPV for different cut-off points. Cut-off Cut-off Cut-off Cut-off Threshold --- 7.5 6.5 5.5 Sensitivity 1.00000 0.8708609 0.6026490 0.4503311 Specificity 0.00000 0.7568807 0.9633028 1.000000 PPV 0.4092141 0.7127371 0.9191919 1.000000 NVP ---- 0.8943089 0.7777778 0.7242525 At this point we noticed a low specificity (75%) with which nearly a 72% of patients with aMCI was detected, while the remaining 28% wasn’t, according to a PPV value of 0.71 (Table 11). As for NPV, we noticed that almost 11% of normal patients could be diagnosed with aMCI. The choice of a cut-off in 6.5 would show a higher specificity (96%) but an acceptable sensitivity (60%). 4. Conclusions and Discussion The MIS-A proved to be able to detect population with AD, but did not achieve proper specificity to detect patients with mild impairment in episodic recall, as discussed in point 3 above. Verbal episodic impairment is a cognitive marker of great value in MCI diagnosis and is found specifically in the clinical sub-group of an MCI. As Herman Buschke showed, [28] the MIS uses category cues controlled learning and cued recall to optimize encoding specificity to ensure attention, induce semantic processing, and optimize encoding specificity. With the addition of delayed recall, the MIS-D becomes a useful screening tool for the detection of episodic memory impairment in non demented population. The MIS-D provides supporting evidence to identify aMCI subjects by testing memory recall in delayed time under controlled learning conditions. In our study, discriminative validity of the MIS-D for aMCI was assessed in a large sample of individuals with and without an MCI diagnostic. The specificity of MIS for detecting an MCI is 0.97 when sensitivity is set to 0.76 (see Table 9). These data suggest that the MIS-D is a valid clinical instrument for the detection of episodic memory impairments in participants who consult for mnesic failures in primary health care centers. In Herman Buschke’s original study [28], the demographic variables such as age, education, and gender as well as their interactions terms was significant at the 0.05 level. However, in our sample and in agreement with other studies [16], age and education were the most significant variables in the MIS-A and the MIS-D scores. According to Bhom et al., and due to the significant influence of age and education on the total MIS results in cohort participants, it became necessary to correct the MIS variable by adding 1 point per number of years to any 76 years old participant or older. In our study there was no need to adjust the scores to counter balance that effect because the study population introduced differences in means of less than 1 point. A major amount of years of education is related to a greater cognitive reserve, a new concept to have in mind when it comes to evaluate the impact of these variables in the scores of cognitive scales. Certainly, a major cognitive reserve will help the subject to handle better compensating strategies when confronted with a deficit, and that is why these populations are expected to achieve better scores in simple and brief tests like MIS, than in older participants with lower educacional level. This cognitive behavior could mask an incipient impairment of episodic recall, so that the administration of tools of higher complexity like the FCSRT [45] [46] is recommended. On the other hand, it is known that memory is a system that declines with age, and it is precisely the episodic memory sub-system the one that declines earlier. As well, it is also the last one to be attained throughout the ontogenetic process. That is to say, the impact of both education and age variables is expected when specific tests are administrated as our results prove. They verify significant differences for both modalities, MIS-A and MIS-D, among age groups of 70 - 80 and of 80 - 90 years old. The score interpretation of the MIS-D proof in the clinical context must take into account the normative re- L. Edith et al. 57 sults as shown in Table 8. It must be taken into account that the choice of a higher cut-off point raises the sensitivity level, that is to say, the chances of including the largest amount of individuals who carry the disease to the detriment of raising the number of those who are not carriers. A score of 5.5 would be acceptable for 80 - 90 years old. In this case, the means for age are 6.52 for elementary level of education, 6.69 for intermediate level and 7.10 for advanced level. Due to the fact that the elementary, intermediate and advanced educational levels have confidence limits of 6.25, 6.40 and 6.75 respectively, it is convenient to use a 6.5 cut-off point for the whole age range. The 6.5 cut-off point is acceptable for participants of an age range of 60 - 70 years old, in which the mean of the different levels of education is: 7.46 for elementary level, 7.47 for intermediate level and 7.42 for advanced level. Likewise, it must be considered that the confidence limits for the means are above 7.2 in all cases. It should as well be mentioned that the choice of this cut-off point raises the chances that these individuals (who represent the 25% of the total population) will not account for the pathology, as they show a 1-NPV of 0.25. As we can see, the cut-off for MIS-A is very high (7.5), that is, close to the normal population’s score. This is why the MIS-A is not intended to discriminate aMCI participants versus normal population. On the other hand, the 7.5 score is an average of the whole population. We need to consider the fact that while age progresses and education level decreases, the differences with the normal population become narrower (see Table 8). Instead, the MIS-D score (5.5), is more conclusive when it comes to make a difference between MIS-A population and population with no cognitive impairment (see Table 8). As it has already been mentioned, this current Spanish version has the advantage of having a careful selection of verbal stimuli that respect the categories of the original version and, at the same time, of word frequency for our population. The words are of mid range frequency, a fact that ensures that stimuli will not be too easy or too difficult at the codifying moment. The short time required to administrate this test and the results obtained render it a proper tool to be used in primary health care centers whose practitioners aim to detect risk participants who might have aMCI . The time for administration of the test ranges from 3 to 5 minutes in its initial stage, and from 3 to 5 minutes in order to evaluate the recovery of verbal stimuli in delayed time. The results obtained in the current study show that the Spanish Speakers version of MIS-D is a valid screening tool for identifying the population with episodic memory impairment, which is an early symptom of MCI due to AD. A low score may indicate a deficit in the long-term memory system (episodic sub-system), but clinical diagnosis will require further tests as well as neurological and neuropsychological extended studies. 5. Limitations and Future Perspectives Even if the sample is appropriate (303 registered cases with an MCI), the amount of participants classified according to age range and educational level is quite small so it is not always possible to have a real dimension of the influence that these variables exert on the scores and the cut-off points that provide the data obtained in this study. The significant percentage of the participants who constituted the sample’s control group derived from all those individuals who attended the hospital to consult for cognitive complaint and did not evince objective disorders. This feature partly reflects the conditions of the clinical context in which the screening test will be carried out in order to identify those patients with an MCI risk. In this sense, the percentage of patients with aMCI in the frame of primary health care attention is much higher than that of a random sample. However, an evolutionary follow-up of the progression of the disorder as well as the detection of the patients who develop AD would ameliorate the quality and the relevance of the data obtained from the present study. Acknowledgements This research was carried out with the support of the UBACYT, University of Buenos Aires, Argentina and the Italian Hospital of Buenos Aires, Argentina. References [1] Mitchell, A.J., Beaumont, H., Ferguson, D., Yadegarfar, M. and Stubbs, B. (2014) Risk of Dementia and Mild Cognitive Impairment in Older People with Subjective Memory Complaints: Meta-Analysis. Acta Psychiatrica Scandinavica, 130, 439-451. http://dx.doi.org/10.1111/acps.12336 L. Edith et al. 58 [2] Luengo-Fernandez R., Leal J. and Gray A. (2010) Dementia 2010: The Economic Burden of Dementia and Associated Research Funding in the United Kingdom. A Report Produced by the Health Economics Research Centre, University of Oxford for the Alzheimer’s Research. [3] Prince, M., Prina, M., Guerchet, M., et al. (2013) World Alzheimer Report. Dementia: A Public Health Priority. Alzheimer’s Disease International (ADI), London. [4] Prince, M., Prina, M., Guerchet, M., Albanese, E., et al. (2014) World Alzheimer Report: Dementia and Risk Reduction. Alzheimer’s Disease International (ADI), London. [5] Chan, M. (2015) Article Title. First WHO ministerial Conference on Global Action against Dementia. 16-17 March 2015, Geneva. [6] Prince, M., Bryce, R., Ferri, C. (2011) World Alzheimer Report 2011: The Benefits of Early Diagnosis and Intervention. Alzheimer’s Disease International (ADI), London. [7] Bond, J., Stave, C., Sganga, A., Vincenzino, O., O’connell, B. and Stanley, R.L. (2005) Inequalities in Dementia Care across Europe: Key Findings of the Facing Dementia Survey. International Journal of Clinical Practice, 59, 8-14. http://dx.doi.org/10.1111/j.1368-504X.2005.00480.x [8] Wilkinson, D., Stave, C., Keohane, D. and Vincenzino, O. (2004) The Role of General Practitioners in the Diagnosis and Treatment of Alzheimer’s Disease: A Multinational Survey. Journal of International Medical Research, 32, 149- 159. http://dx.doi.org/10.1177/147323000403200207 [9] Pravat, K., Ritchiec, K. and Fodaled V. (2014) Anesthetics and Its Impact on the Brain and Alzheimer’s Disease. Journal of Alzheimer’s Disease, 39, 223-225. [10] Liu, Y., Pan, N., Ma, Y., Zhang, S., Guo, W., Li, H., Zhou, J., Liu, G. and Gao, M, (2013)Inhaled sevoflurane may promote progressionof amnestic mild cognitive impairment: A prospective,randomized parallel-group study. Am J Med Sci, 345, 355-360. http://dx.doi.org/10.1097/MAJ.0b013e31825a674d [11] Alzheimer Europe. National Dementia Plans: Finland. http://www.alzheimer-europe.org/EN/Policyin-Practice2/National-Dementia-Plans/Finland [12] (2011) UK Elder Care. In the Know E-Newsletter. [13] Dubois, B., Feldman, H.H., Jacova, C., Dekosky, S.T., Barberger-Gateau, P., Cummings, J., et al. (2007) Research Criteria for the Diagnosis of Alzheimer’s Disease: Revising the NINCDS-ADRDA Criteria. The Lancet Neurology, 6, 734-746. http://dx.doi.org/10.1016/S1474-4422(07)70178-3 [14] Dubois, B., Feldman, H.H., Jacova, C., Cummings, J.L., Dekosky, S.T., Barberger-Gateau, P., et al. (2010) Revising the Definition of Alzheimer’s Disease: A New Lexicon. The Lancet Neurology, 9, 1118-1127. http://dx.doi.org/10.1016/S1474-4422(10)70223-4 [15] Dubois, B. (2004) Amnesic MCI or Prodromal Alzheimer’s Disease? The Lancet Neurology, 3, 246-248. http://dx.doi.org/10.1016/S1474-4422(04)00710-0 [16] Albert, M., et al. (2011) The Diagnosis of Mild Cognitive Impairment Due to Alzheimer’s Disease: Recommendations from the National Institute on Aging—Alzheimer’s Association Workgroups on Diagnostic Guidelines for Alzheimer’s Disease. Alzheimer’s & Dementia: The Journal of the Alzheimer’s Association, 7, 270-279. http://dx.doi.org/10.1016/j.jalz.2011.03.008 [17] Gautier, S., Reisberg, B., Zaudig, M., Petersen, R.C., Ritchie, K., Broich, K., et al. (2006) Mild Cognitive Impairment. The Lancet, 367, 1262-1270. http://dx.doi.org/10.1016/S0140-6736(06)68542-5 [18] Dubois, B., Picard, G. and Sarazin, M. (2009) Dialogues in Clinical Neuroscience. Journal of Clinical Neuroscience, 11, 135- 139. [19] Tounsi, H., Deweer, B., Ergis, A.M., et al. (1999) Sensitivity to Semantic Cuing: An Index of Episodic Memory Dysfunction in Early Alzheimer Disease. Alzheimer Disease and Associated Disorders, 13, 38-46. http://dx.doi.org/10.1097/00002093-199903000-00006 [20] Sarazin, M., Berr, C., De Rotrou, J. et al. (2007) Amnestic Syndrome of the Medial Temporal Type Identifies Prodromal AD: A Longitudinal Study. Neurology, 69, 1859-1867. http://dx.doi.org/10.1212/01.wnl.0000279336.36610.f7 [21] Wagner, M., Wolf, S., Reischies, F.M., Daerr, M., Wolfsgruber S., Jessen F., et al. (2012) Biomarker Validation of a Cued Recall Memory Deficit in Prodromal Alzheimer Disease. Neurology, 78, 379- 386. http://dx.doi.org/10.1212/WNL.0b013e318245f447 [22] Buschke, H. (1986) Cued Recall in Amnesia. Journal of Clinical and Experimental Neuropsycho- logy, 6, 433-440. http://dx.doi.org/10.1080/01688638408401233 [23] Dubois, B. (2012) The New Criteria for Alzheimer Disease. ADI, London. [24] Villarejo, G.A. and Puertas-Martín, V. (2011) Utilidad de los test breves en el cribado de demencia. Neurology, 26, L. Edith et al. 59 425-433. http://dx.doi.org/10.1016/j.nrl.2010.12.002 [25] Folstein, M.F., Folstein, S.E. and McHugh, P.R. (1975) Mini-Mental State: A Practical Method for Grading the Cognitive State of Patients for the Clinician. Journal of Psychiatric Research, 12, 189-198. http://dx.doi.org/10.1016/0022-3956(75)90026-6 [26] Cullum, C.M., Thompson, L.L. and Smernoff, E.N. (1993) Three-Word Recall as a Measure of Memory. Journal of Clinical and Experimental Neuropsychology, 15, 321-329. http://dx.doi.org/10.1080/01688639308402566 [27] Buschke, H., Sliwinski, M., Kuslansky, G. and Lipton, R.B. (1995) Aging, Encoding Specificity and Memory Changes in the Double Memory Test. Journal of the International Neuropsychological Society, 1, 483-493. http://dx.doi.org/10.1017/s1355617700000576 [28] Buschke, H., Kuslansky, G., Katz, M., Stewart, W.F., Sliwinski, M.J., Eckholdt, H.M. and Lipton, R.B. (1999) Screening for Dementia with the Memory Impairment Screen. Neurology, 52, 231-238. http://dx.doi.org/10.1212/WNL.52.2.231 [29] Tulving, E. and Thomson, D. (1973) Encoding Specificity and Retrieval Processes in Episodic Memory. Psychological Review, 80, 352-372. http://dx.doi.org/10.1037/h0020071 [30] Gifford, D.R. and Cummings, J.L. (1999) Evaluating Dementia Screening Tests: Methodological Standards to Rate Their Performance. Neurology, 52,224-227. http://dx.doi.org/10.1212/WNL.52.2.224 [31] Pérez-Martínez, D.A., Baztán, J.J., González-Becerra, M. and Socorro, A. (2005) Evaluación de la utilidad diagnóstica de una adaptación española del Buschke Memory Impairment Screen para detectar demencia y deterioro cognitivo. Revista de Neurología, 40, 644-648. [32] Barrero-Fernández, F.J, Vives-Montero, F. and Morales-Gordo, B. (2006) Evaluación de la versión española del MemoryImpairmentScreen. Revista de Neurología, 43, 15-19. [33] Böhm, P., Peña-Casanova, J., Gramunt , N., Manero, R.M., Terrón, C. and Quiñones-Ubeda, S. (2005) Versión española del MemoryImpairmentScreen (MIS): Datos normativos y validez discriminativa. Neurología, 20, 402-411. [34] Rojas, G., Serrano, C.M. and Allegri, R.F. (2008) Versión Argentina del Memory Impairment Screen (MIS) y Métodos de detección de demencias: Análisis comparativo de curvas ROC. Revista Neurológica Argentina, 33, 33-39. [35] Renato, A., Labos, E., Atlasovich, C. and Vanotti, S. (1996) Características de la producción de cate- gorías semánticas en español rioplatense. En: Aires, B., Ed., Actas de las Jornadas Universitarias El lenguaje y sus alteraciones, Facultad de Medicina, Buenos Aires, 155-160. [36] Labos, E., Mauriño, A., Cristalli, D., Arguello, G., Seinhart, D., Etchepareborda, J., Cruz, D., Martinez, L. and Trojanowski, S. (2010) Versión adaptada en lengua hispana del Test MIS para hablantes sudamericanos. Estudio multicéntrico en la República Argentina. Resultados Preliminares. Alzheimer’s Association-International Conference, Honolulu, 21 July 2010, 21. [37] Sano, M., Raman, R., Emond, J., Thomas, R.G., Petersen, R., Schneider, L.S. and Aisen, P.S. (2011) Adding Delayed Recall to the Alzheimer Disease Assessment Scale Is Useful in Studies of Mild Cognitive Impairment but Not Alzheimer Disease. Alzheimer Disease and Associated Disorders, 25, 122-127. http://dx.doi.org/10.1097/wad.0b013e3181f883b7 [38] Takayama, Y. (2010) A Delayed Recall Battery as a Sensitive Screening for Mild Cognitive Impairment: Follow-Up Study of Memory Clinic Patients after 10 years’. Journal of Medical and Dental Sciences, 57, 177-184 [39] Gauthier, S., Dubois, B., Feldman, H. and Scheltens P. (2008) Revised Research Diagnostic Criteria for Alzheimer’s Disease. The Lancet Neurology, 7, 668-670. http://dx.doi.org/10.1016/S1474-4422(08)70146-7 [40] Labos, E., Renato, A., Trojanowski, S., Seinhart, D., Camera, L. and Schapira, M. (2013) Com- paring the Memory Impairment Screen (MIS)* Test with the Memory Impairment Screen Deferred (MISD) for Detection of Amnesic Mild Cognitive Impairment (aMCI). Proceedings of the Alzheimer’s Association International Conference, Boston, 13-18 July 2013, 21. [41] Petersen, R.C., Doody, R., Kurz, A., Mohs, R.C., Morris, J., Rabins, V., Ritchie, K., Rossor, M., Thal, L. and Winblad, B. (2001) Current Concepts in Mild Cognitive Impairment. Archives of Neurology, 58, 1985-1992. http://dx.doi.org/10.1001/archneur.58.12.1985 [42] Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J. and Markus, M. (2011) pROC: An Open-Source Package for R and S+ to Analyze and Compare ROC Curves. BMC Bioinformatics, 12, 77. http://dx.doi.org/10.1186/1471-2105-12-77 [43] Revelle, W. (2009) An Introduction to Psychometric Theory with Applications in R. Springer. http://personality-project.org/r/book [44] Sekhon, J.S. (2011) Multivariate and Propensity Score Matching Software with Automated Balance Optimization. L. Edith et al. 60 Journal of Statistical Software, 42, 1-52. http://www.jstatsoft.org/v42/i07/ [45] Labos, E., Trojanowski, S. and Ruiz C. (2008) Prueba de recuerdo libre/facilitado con recuerdo inmediato. Versión Verbal de la FCSRT-IR. Adaptación y normas en lengua española. Revista Neurológica Argentina, 33, 50-66. [46] Van der Linden, M., Coyette, F., Pointrenaud, J., et al. (2004) L’Epreuve de Rappel Libre/ Rappel indicé á 16 Items’ (RL/RI-16). In: Van der Linden, M., Ed., L’Evaluation des troubles de la mémoire, Solal, Marseille.
上一篇 下一篇

论文发表与咨询

论文发表 写作指导 职称论文 毕业论文 客服联系方式:
投稿信箱:lunww@126.com
在线咨询客服QQ:站点合作85782530
在线咨询客服QQ:站点合作82534308
联系电话:18262951856
点击进入支付宝支付(支付宝认可网络诚信商家)
点击进入财付通支付(财付通认可网络诚信商家)
点击进入支付方式---->>>>

论文发表 诚信说明

论文发表 论文投稿 热点图片