SciCombinator

Discover the most talked about and latest scientific content & concepts.

Concept: Validity

177

A common approach for determining musical competence is to rely on information about individuals' extent of musical training, but relying on musicianship status fails to identify musically untrained individuals with musical skill, as well as those who, despite extensive musical training, may not be as skilled. To counteract this limitation, we developed a new test battery (Profile of Music Perception Skills; PROMS) that measures perceptual musical skills across multiple domains: tonal (melody, pitch), qualitative (timbre, tuning), temporal (rhythm, rhythm-to-melody, accent, tempo), and dynamic (loudness). The PROMS has satisfactory psychometric properties for the composite score (internal consistency and test-retest r>.85) and fair to good coefficients for the individual subtests (.56 to.85). Convergent validity was established with the relevant dimensions of Gordon’s Advanced Measures of Music Audiation and Musical Aptitude Profile (melody, rhythm, tempo), the Musical Ear Test (rhythm), and sample instrumental sounds (timbre). Criterion validity was evidenced by consistently sizeable and significant relationships between test performance and external musical proficiency indicators in all three studies (.38 to.62, p<.05 to p<.01). An absence of correlations between test scores and a nonmusical auditory discrimination task supports the battery's discriminant validity (-.05, ns). The interrelationships among the various subtests could be accounted for by two higher order factors, sequential and sensory music processing. A brief version of the full PROMS is introduced as a time-efficient approximation of the full version of the battery.

Concepts: Psychometrics, Skill, Validity, Reliability, Sound, Music, Test

168

Abstract Purpose: To develop a patient reported outcome measure of active and passive function in the hemiparetic upper limb. Methods: Potential items for inclusion were identified through (a) systematic review and analysis of existing measures and (b) analysis of the primary goals for treatment in a spasticity service. Item reduction was achieved through consultation with a small, purposively selected multi-disciplinary group of experienced rehabilitation professionals (n = 10) in a three-round Delphi process. This was followed by a confirmatory survey with a larger group of clinicians (n = 36) and patients and carers (n = 13 pairs). Results: From an initial shortlist of 75 items, 23 items were initially identified for inclusion in the arm activity measure (ArmA), and subsequently refined to a 20-item instrument comprising 7 passive and 13 active function. In common with the six measures identified in the systematic review, a five-point ordinal scaling structure was chosen, with ratings based on activity over the preceding 7 days. Conclusions: The ArmA is designed to measure passive and active function following focal interventions for the hemiparetic upper limb. Content and face validity have initially been addressed within the development process. The next phase of development has involved formal evaluation of psychometric properties. Implications for Rehabilitation In clinical practice or research, outcome measures in rehabilitation need to have face and content validity. Following stroke or brain injury, goals for rehabilitation of the hemiparetic upper limb may be: to restore active function, if there is return of motor control or to improve passive function making it easier to care for the limb (e.g. maintain hygiene) if no motor return is possible, measurement of both constructs should be considered. This study describes the systematic development of the ArmA, a measure of active and passive function in the hemiparetic upper limb.

Concepts: Traumatic brain injury, Psychometrics, Validity, Upper limb, Motor skill, Content validity, Face validity, Armas

54

Background The consensus definition of severe sepsis requires suspected or proven infection, organ failure, and signs that meet two or more criteria for the systemic inflammatory response syndrome (SIRS). We aimed to test the sensitivity, face validity, and construct validity of this approach. Methods We studied data from patients from 172 intensive care units in Australia and New Zealand from 2000 through 2013. We identified patients with infection and organ failure and categorized them according to whether they had signs meeting two or more SIRS criteria (SIRS-positive severe sepsis) or less than two SIRS criteria (SIRS-negative severe sepsis). We compared their characteristics and outcomes and assessed them for the presence of a step increase in the risk of death at a threshold of two SIRS criteria. Results Of 1,171,797 patients, a total of 109,663 had infection and organ failure. Among these, 96,385 patients (87.9%) had SIRS-positive severe sepsis and 13,278 (12.1%) had SIRS-negative severe sepsis. Over a period of 14 years, these groups had similar characteristics and changes in mortality (SIRS-positive group: from 36.1% [829 of 2296 patients] to 18.3% [2037 of 11,119], P<0.001; SIRS-negative group: from 27.7% [100 of 361] to 9.3% [122 of 1315], P<0.001). Moreover, this pattern remained similar after adjustment for baseline characteristics (odds ratio in the SIRS-positive group, 0.96; 95% confidence interval [CI], 0.96 to 0.97; odds ratio in the SIRS-negative group, 0.96; 95% CI, 0.94 to 0.98; P=0.12 for between-group difference). In the adjusted analysis, mortality increased linearly with each additional SIRS criterion (odds ratio for each additional criterion, 1.13; 95% CI, 1.11 to 1.15; P<0.001) without any transitional increase in risk at a threshold of two SIRS criteria. Conclusions The need for two or more SIRS criteria to define severe sepsis excluded one in eight otherwise similar patients with infection, organ failure, and substantial mortality and failed to define a transition point in the risk of death. (Funded by the Australian and New Zealand Intensive Care Research Centre.).

Concepts: Inflammation, Systemic inflammatory response syndrome, Intensive care medicine, Acute respiratory distress syndrome, Validity, Multiple organ dysfunction syndrome, Septic shock, Sepsis

43

We hypothesized that individuals may differ in the dispositional tendency to have positive vs. negative attitudes, a trait termed the dispositional attitude. Across 4 studies, we developed a 16-item Dispositional Attitude Measure (DAM) and investigated its internal consistency, test-retest reliability, factor structure, convergent validity, discriminant validity, and predictive validity. DAM scores were (a) positively correlated with positive affect traits, curiosity-related traits, and individual preexisting attitudes; (b) negatively correlated with negative affect traits; and © uncorrelated with theoretically unrelated traits. Dispositional attitudes also significantly predicted the valence of novel attitudes while controlling for theoretically relevant traits (such as the Big 5 and optimism). The dispositional attitude construct represents a new perspective in which attitudes are not simply a function of the properties of the stimuli under consideration, but are also a function of the properties of the evaluator. We discuss the intriguing implications of dispositional attitudes for many areas of research, including attitude formation, persuasion, and behavior prediction. (PsycINFO Database Record © 2013 APA, all rights reserved).

Concepts: Scientific method, Prediction, Futurology, Prophecy, Psychometrics, Validity, Reliability, All rights reserved

42

The list of motives by Kanin (1994) is the most cited list of motives to file a false allegation of rape. Kanin posited that complainants file a false allegation out of revenge, to produce an alibi or to get sympathy. A new list of motives is proposed in which gain is the predominant factor. In the proposed list, complainants file a false allegation out of material gain, emotional gain, or a disturbed mental state. The list can be subdivided into eight different categories: material gain, alibi, revenge, sympathy, attention, a disturbed mental state, relabeling, or regret. To test the validity of the list, a sample of 57 proven false allegations were studied at and provided by the National Unit of the Dutch National Police (NU). The complete files were studied to ensure correct classification by the NU and to identify the motives of the complainants. The results support the overall validity of the list. Complainants were primarily motivated by emotional gain. Most false allegations were used to cover up other behavior such as adultery or skipping school. Some complainants, however, reported more than one motive. A large proportion, 20% of complainants, said that they did not know why they filed a false allegation. The results confirm the complexity of motivations for filing false allegations and the difficulties associated with archival studies. In conclusion, the list of Kanin is, based on the current results, valid but insufficient to explain all the different motives of complainants to file a false allegation.

Concepts: Psychology, Validity, Criminal law, Motivation, Pleading, Legal terms, Computer file

31

Working dog organisations, such as Guide Dogs, need to regularly assess the behaviour of the dogs they train. In this study we developed a questionnaire-style behaviour assessment completed by training supervisors of juvenile guide dogs aged 5, 8 and 12 months old (n = 1,401), and evaluated aspects of its reliability and validity. Specifically, internal reliability, temporal consistency, construct validity, predictive criterion validity (comparing against later training outcome) and concurrent criterion validity (comparing against a standardised behaviour test) were evaluated. Thirty-nine questions were sourced either from previously published literature or created to meet requirements identified via Guide Dogs staff surveys and staff feedback. Internal reliability analyses revealed seven reliable and interpretable trait scales named according to the questions within them as: Adaptability; Body Sensitivity; Distractibility; Excitability; General Anxiety; Trainability and Stair Anxiety. Intra-individual temporal consistency of the scale scores between 5-8, 8-12 and 5-12 months was high. All scales excepting Body Sensitivity showed some degree of concurrent criterion validity. Predictive criterion validity was supported for all seven scales, since associations were found with training outcome, at at-least one age. Thresholds of z-scores on the scales were identified that were able to distinguish later training outcome by identifying 8.4% of all dogs withdrawn for behaviour and 8.5% of all qualified dogs, with 84% and 85% specificity. The questionnaire assessment was reliable and could detect traits that are consistent within individuals over time, despite juvenile dogs undergoing development during the study period. By applying thresholds to scores produced from the questionnaire this assessment could prove to be a highly valuable decision-making tool for Guide Dogs. This is the first questionnaire-style assessment of juvenile dogs that has shown value in predicting the training outcome of individual working dogs.

Concepts: Scientific method, Psychometrics, Validity, Test validity, Criterion validity, Construct validity, Dog, Test

28

OBJECTIVE: The purpose of this study was to develop a short and reliable measure of hypersexuality that could be used in everyday practice in patients with Parkinson’s disease (PD). DESIGN: The original questionnaire containing twenty-five-items, the Sexual Addiction Screening Test (SAST), was shortened and tested in a PD population. METHODS: Successive reductions were performed until a final set of items satisfied the model fit requirements. The testing phase consisted of administering the SAST questionnaire to 159 PD patients. It included i) acceptability, ii) dimensionality construct validity, and iii) a complete general correlation structure of data. Finally, criterion validity of the final version of the instrument was assessed. RESULTS: The initial questionnaire was reduced to five items (PD-SAST) with a cut-off score of 2. Psychometric analysis revealed three factors corresponding to “Preoccupation”, “Cannot stop” and “Relationship disturbance”. The discriminant validity of the PD-SAST was high (ROC area under the curve: 0.96). CONCLUSIONS: The PD-SAST performs well as a screening instrument. It has been found to be acceptable to patients and is ready for use. Moreover, it tests multidimensional aspects of hypersexuality.

Concepts: Assessment, Psychometrics, Addiction, Validity, Test validity, Criterion validity, Construct validity, Reliability

28

ABSTRACT Background: As ageing is a personal experience, an attitude to ageing questionnaire is essential for capturing the most realistic appraisal of this important stage of life. Our aim was to study the psychometric properties of the Attitudes to Ageing Questionnaire (AAQ) in a sample of Spanish older people. Methods: A total of 242 participants aged 60 years and older were recruited from community centers, primary care centers, and family associations for the mentally ill and dementia. In addition to the AAQ, participants provided information on demographics, self-perception of health, comorbidity, health status (SF-12), depressive symptoms (GDS-30), and quality of life (WHOQOL-BREF and WHOQOL-OLD). Analysis was performed using standard psychometric techniques with SPSS v15.0. Results: No floor and ceiling effects were found, and missing data were low. The internal consistency measured by Cronbach’s alpha for AAQ subscales were 0.59, 0.70, and 0.73. Exploratory Factor Analysis produced a three-factors solution accounting for 34% of the variance. A priori expected associations were found between some AAQ subscales with WHOQOL-BREF domains, with WHOQOL-OLD, SF-12, and the GDS-30 indicating good construct validity. In general, AAQ subscales differentiated between participants with lower and higher levels of education, and between a priori defined groups of older people (non-depressed vs. depressed; those with higher vs. lower physical comorbidities, and non-carers vs. carers). Conclusions: The Spanish version of the AAQ questionnaire showed acceptable psychometric properties in a convenience sample of Spanish older people. It is a useful measure of attitude for use with older people in social and clinical services.

Concepts: Educational psychology, Psychometrics, Factor analysis, Validity, Reliability, Cronbach's alpha, SPSS, Internal consistency

28

The Falls Efficacy Scale-International (FES-I) is a highly reliable questionnaire for assessing fear of falling in elderly individuals with increased fall risk and has low or no convergent validity with balance performance tests and health-related quality of life (HRQL) among elderly women with osteoporosis, which indicates that both measurements should be included as they are measuring different components. INTRODUCTION: Fear of falling is increased in elderly individuals with osteoporosis and FES-I is a widely used questionnaire to assess fear of falling. There is limited evidence of the reliability and convergent validity in elderly with increased risk of falling and osteoporosis. METHODS: Reliability and validity study of the FES-I. Community-dwelling elderly with increased fall risk, 59 subjects, were recruited to the reliability assessment, and 81 women with osteoporosis, in the validity assessment. For the reliability assessment, two postal surveys were used. For the validity assessment, we used baseline data from an on-going study in women with osteoporosis. The FES-I was correlated to a single-item question regarding fear of falling, self-reported history of falls, balance performance tests and health-related quality of life. RESULTS: The FES-I had very good relative reliability (intra-class correlation 0.88) and internal consistency reliability (Cronbach’s alpha 0.94). The value for absolute reliability was a standard error of measure 2.9 (10.6 %), smallest real difference 7.9 (29 %). There was “little if any” to “low” correlation between the FES-I and the single-item question regarding fear of falling and self-reported history of falls, HRQL and balance performance tests. CONCLUSION: The FES-I seems to be a highly reliable questionnaire for assessing fear of falling in elderly with increased fall risk but has low relation to/convergent validity with balance performance and HRQL among elderly women with osteoporosis.

Concepts: Measurement, Assessment, Psychometrics, Validity, Reliability, Cronbach's alpha, Internal consistency, Kuder-Richardson Formula 20

27

OBJECTIVE Recent advances in optics and miniaturization have enabled the development of a growing number of minimally invasive procedures, yet innovative training methods for the use of these techniques remain lacking. Conventional teaching models, including cadavers and physical trainers as well as virtual reality platforms, are often expensive and ineffective. Newly developed 3D printing technologies can recreate patient-specific anatomy, but the stiffness of the materials limits fidelity to real-life surgical situations. Hollywood special effects techniques can create ultrarealistic features, including lifelike tactile properties, to enhance accuracy and effectiveness of the surgical models. The authors created a highly realistic model of a pediatric patient with hydrocephalus via a unique combination of 3D printing and special effects techniques and validated the use of this model in training neurosurgery fellows and residents to perform endoscopic third ventriculostomy (ETV), an effective minimally invasive method increasingly used in treating hydrocephalus. METHODS A full-scale reproduction of the head of a 14-year-old adolescent patient with hydrocephalus, including external physical details and internal neuroanatomy, was developed via a unique collaboration of neurosurgeons, simulation engineers, and a group of special effects experts. The model contains “plug-and-play” replaceable components for repetitive practice. The appearance of the training model (face validity) and the reproducibility of the ETV training procedure (content validity) were assessed by neurosurgery fellows and residents of different experience levels based on a 14-item Likert-like questionnaire. The usefulness of the training model for evaluating the performance of the trainees at different levels of experience (construct validity) was measured by blinded observers using the Objective Structured Assessment of Technical Skills (OSATS) scale for the performance of ETV. RESULTS A combination of 3D printing technology and casting processes led to the creation of realistic surgical models that include high-fidelity reproductions of the anatomical features of hydrocephalus and allow for the performance of ETV for training purposes. The models reproduced the pulsations of the basilar artery, ventricles, and cerebrospinal fluid (CSF), thus simulating the experience of performing ETV on an actual patient. The results of the 14-item questionnaire showed limited variability among participants' scores, and the neurosurgery fellows and residents gave the models consistently high ratings for face and content validity. The mean score for the content validity questions (4.88) was higher than the mean score for face validity (4.69) (p = 0.03). On construct validity scores, the blinded observers rated performance of fellows significantly higher than that of residents, indicating that the model provided a means to distinguish between novice and expert surgical skills. CONCLUSIONS A plug-and-play lifelike ETV training model was developed through a combination of 3D printing and special effects techniques, providing both anatomical and haptic accuracy. Such simulators offer opportunities to accelerate the development of expertise with respect to new and novel procedures as well as iterate new surgical approaches and innovations, thus allowing novice neurosurgeons to gain valuable experience in surgical techniques without exposing patients to risk of harm.

Concepts: Surgery, Psychometrics, Validity, Cerebrospinal fluid, Neurosurgery, Hydrocephalus, Content validity, Special effects