In clinical psychological and neuropsychological settings such as are the concern of this volume, the most common cognitive tests are intelligence tests, other clinical neuropsychological measures, and performance validity measures. Many tests used by clinical neuropsychologists, psychiatrists, technicians, or others assess specific types of functioning, such as memory or problem solving. Performance validity measures are typically short assessments and are sometimes interspersed among components of other assessments that help the psychologist determine whether the examinee is exerting sufficient effort to perform well and responding to the best of his or her ability.
Most common non-cognitive measures in clinical psychology and neuropsychology settings are personality measures and symptom validity measures. Some personality tests, such as the Minnesota Multiphasic Personality Inventory MMPI , assess the degree to which someone expresses behaviors that are seen as atypical in relation to the norming sample. Symptom validity measures are scales, like performance validity measures, that may be interspersed throughout a longer assessment to examine whether a person is portraying him- or herself in an honest and truthful manner.
Somewhere between these two types of tests—cognitive and non-cognitive—are various measures of adaptive functioning that often include both cognitive and non-cognitive components. Psychometrics is the scientific study—including the development, interpretation, and evaluation—of psychological tests and measures used to assess variability in behavior and link such variability to psychological phenomena.
In evaluating the quality of psychological measures we are traditionally concerned primarily with test reliability i. This section provides a general overview of these concepts to help orient the reader for the ensuing discussions in Chapters 4 and 5. In addition, given the implications of applying psychological measures with subjects from diverse racial and ethnic backgrounds, issues of equivalence and fairness in psychological testing are also presented.
Reliability refers to the degree to which scores from a test are stable and results are consistent. When constructs are not reliably measured the obtained scores will not approximate a true value in relation to the psychological variable being measured. It is important to understand that observed or obtained test scores are considered to be composed of true and error elements.
A standard error of measurement is often presented to describe, within a level of confidence e. Consistency of test scores over time stability, temporal consistency ;.
Parallel or alternate forms: Consistency of scores across different forms of the test stability and equivalence ; and. Consistency of different items intended to measure the same thing within the test homogeneity. A special case of internal consistency reliability is split-half where scores on two halves of a single test are compared and this comparison may be converted into an index of reliability.
A number of factors can affect the reliability of a test's scores. These include time between two testing administrations that affect test-retest and alternate-forms reliability, and similarity of content and expectations of subjects regarding different elements of the test in alternate forms, split-half, and internal consistency approaches. In addition, changes in subjects over time and introduced by physical ailments, emotional problems, or the subject's environment, or test-based factors such as poor test instructions, subjective scoring, and guessing will also affect test reliability.
It is important to note that a test can generate reliable scores in one context and not in another, and that inferences that can be made from different estimates of reliability are not interchangeable Geisinger, While the scores resulting from a test may be deemed reliable, this finding does not necessarily mean that scores from the test have validity. In discussing validity, it is important to highlight that validity refers not to the measure itself i. To be considered valid, the interpretation of test scores must be grounded in psychological theory and empirical evidence that demonstrates a relationship between the test and what it purports to measure Furr and Bacharach, ; Sireci and Sukin, Historically, the fields of psychology and education have described three primary types of evidence related to validity Sattler, ; Sireci and Sukin, Construct evidence of validity: The degree to which an individual's test scores correlate with the theoretical concept the test is designed to measure i.
Content evidence of validity: The degree to which the test content represents the targeted subject matter and supports a test's use for its intended purposes; and. Criterion-related evidence of validity: The degree to which the test's score correlates with other measurable, reliable, and relevant variables i. Other kinds of validity with relevance to SSA have been advanced in the literature, but are not completely accepted in professional standards as types of validity per se.
The degree to which psychological tests are truly aiding in the formulation of an appropriate diagnosis. The degree to which test scores represent everyday levels of functioning e. The degree to which test content and procedures accurately reflect the sociocultural context of the subjects being tested. Each of these forms of validity poses complex questions regarding the use of particular psychological measures with the SSA population.
For example, ecological validity is especially critical in the use of psychological tests with SSA given that the focus of the assessment is on examining everyday levels of functioning. Measures like intelligence tests have been sometimes criticized for lacking ecological validity Groth-Marnat, ; Groth-Marnat and Teal, More recent discussions on validity have shifted toward an argument-based approach to validity, using a variety of evidence to build a case for validity of test score interpretation Furr and Bacharach, In this approach, construct validity is viewed as an overarching paradigm under which evidence is gathered from multiple sources to build a case for validity of test score interpretation.
Five key sources of validity evidence that affect the degree to which a test fulfills its purpose are generally considered AERA et al. Does the test content reflect the important facets of the construct being measured? Are the test items relevant and appropriate for measuring the construct and congruent with the purpose of testing? Relation to other variables: Is there a relationship between test scores and other criterion or constructs that are expected to be related?
Does the actual structure of the test match the theoretically based structure of the construct? Are respondents applying the theoretical constructs or processes the test is designed to measure? What are the intended and unintended consequences of testing? As part of the development of any psychometrically sound measure, explicit methods and procedures by which tasks should be administered are determined and clearly spelled out. This is what is commonly known as standardization.
Typical standardized administration procedures or expectations include 1 a quiet, relatively distraction-free environment, 2 precise reading of scripted instructions, and 3 provision of necessary tools or stimuli. All examiners use such methods and procedures during the process of collecting the normative data, and such procedures normally should be used in any other administration, which enables application of normative data to the individual being evaluated Lezak et al. Standardized tests provide a set of normative data i. Norms consist of transformed scores such as percentiles, cumulative percentiles, and standard scores e.
Without standardized administration, the individual's performance may not accurately reflect his or her ability. For example, an individual's abilities may be overestimated if the examiner provides additional information or guidance than what is outlined in the test administration manual. Conversely, a claimant's abilities may be underestimated if appropriate instructions, examples, or prompts are not presented. When nonstandardized administration techniques must be used, norms should be used with caution due to the systematic error that may be introduced into the testing process; this topic is discussed in detail later in the chapter.
It is important to clearly understand the population for which a particular test is intended. The standardization sample is another name for the norm group. Norms enable one to make meaningful interpretations of obtained test scores, such as making predictions based on evidence. Developing appropriate norms depends on size and representativeness of the sample.
In general, the more people in the norm group the closer the approximation to a population distribution so long as they represent the group who will be taking the test. Norms should be based upon representative samples of individuals from the intended test population, as each person should have an equal chance of being in the standardization sample. Stratified samples enable the test developer to identify particular demographic characteristics represented in the population and more closely approximate these features in proportion to the population.
For example, intelligence test scores are often established based upon census-based norming with proportional representation of demographic features including race and ethnic group membership, parental education, socioeconomic status, and geographic region of the country. When tests are applied to individuals for whom the test was not intended and, hence, were not included as part of the norm group, inaccurate scores and subsequent misinterpretations may result.
Tests administered to persons with disabilities often raise complex issues. Test users sometimes use psychological tests that were not developed or normed for individuals with disabilities. It is critical that tests used with such persons including SSA disability claimants include attention to representative norming samples; when such norming samples are not available, it is important for the assessor to note that the test or tests used are not based on representative norming samples and the potential implications for interpretation Turner et al.
Performance on psychological tests often has significant implications high stakes in our society. Tests are in part the gatekeepers for educational and occupational opportunities and play a role in SSA determinations. As such, results of psychological testing may have positive or negative consequences for an individual.
Often such consequences are intended; however, there is the possibility for unintended negative consequences. It is imperative that issues of test fairness be addressed so no individual or group is disadvantaged in the testing process based upon factors unrelated to the areas measured by the test.
Biases simply cannot be present in these kinds of professional determinations. Moreover, it is imperative that research demonstrates that measures can be fairly and equivalently used with members of the various subgroups in our population. It is important to note that there are people from many language and cultural groups for whom there are no available tests with norms that are appropriately representative for them.
As noted above, in such cases it is important for assessors to include a statement about this situation whenever it applies and potential implications on scores and resultant interpretation. While all tests reflect what is valued within a particular cultural context i. Bias leads to inaccurate test results given that scores reflect either overestimations or underestimations of what is being measured. When bias occurs based upon culturally related variables e.
Relevant considerations pertain to issues of equivalence in psychological testing as characterized by the following Suzuki et al. Whether the construct being measured occurs with equal frequency across groups;.
19 psychological tricks that will help you ace a job interview In their book " Crazy Good Interviewing," John B. Molidor, Ph.D., and Barbara Parus write that you Baby Boomer interviewers (between 50 and 70): Show that you work hard . the strongest elements of your case," Cialdini told Business Insider. 6 Psychological Tricks to Help You Nail Your Next Interview. Lindsay Kolowich. Written by Lindsay Kolowich In fact, research shows that interviewers reach final decisions about . What other interviewing tips do you have?.
Whether the item information is familiar across groups and means the same thing in various cultures;. Whether average score differences reflect the same degree, intensity, or magnitude for different cultural groups;. Whether the language used has similar meaning across groups; and.
Whether the scale measures the same behavioral qualities or characteristics and the measure has similar psychometric properties in different cultures. It must be established that the measure is operating appropriately in various cultural contexts. Test developers address issues of equivalence through procedures including.
Cultural equivalence is a higher order form of equivalence that is dependent on measures meeting specific criteria indicating that a measure may be appropriately used with other cultural groups beyond the one for which it was originally developed. Trimble notes that there may be upward of 50 or more types of equivalence that affect interpretive and procedural practices in order to establish cultural equivalence. For most of the 20th century, the dominant measurement model was called classical test theory. This model was based on the notion that all scores were composed of two components: The model further assumes that all error is random and that any correlation between error and some other variable, such as true scores, is effectively zero Geisinger, The approach leans heavily on reliability theory, which is largely derived from the premises mentioned above.
Since the s and largely since the s, a newer mathematically sophisticated model developed called item response theory IRT. The premise of these IRT models is most easily understood in the context of cognitive tests, where there is a correct answer to questions. The simplest IRT model is based on the notion that the answering of a question is generally based on only two factors: Computer-adaptive testing estimates scores of the test-taker after each response to a question and adjusts the administration of the next question accordingly.
For example, if a test-taker answers a question correctly, he or she is likely to receive a more difficult question next. It has been found that such computer-adaptive tests can be very efficient. IRT models have made the equating of test forms far easier. Equating tests permits one to use different forms of the same examination with different test items to yield fully comparable scores due to slightly different item difficulties across forms.
To convert the values of item difficulty to determine the test-taker's ability scores one needs to have some common items across various tests; these common items are known as anchor items. Using such items, one can essentially establish a fixed reference group and base judgments from other groups on these values. As noted above, there are a number of common IRT models. Among the most common are the one-, two-, and three-parameter models. The one-parameter model is the one already described; the only item parameter is item difficulty.
A two-parameter model adds a second parameter to the first, related to item discrimination. Item discrimination is the ability of the item to differentiate those lacking the ability in high degree from those holding it. Such two-parameter models are often used for tests like essay tests where one cannot achieve a high score by guessing or using other means to answer currently. The three-parameter IRT model contains a third parameter, that factor related to chance level correct scoring. This parameter is sometimes called the pseudo-guessing parameter, and this model is generally used for large-scale multiple-choice testing programs.
These models, because of their lessened reliance on the sampling of test-takers, are very useful in the equating of tests that is the setting of scores to be equivalent regardless of the form of the test one takes. The test user is generally considered the person responsible for appropriate use of psychological tests, including selection, administration, interpretation, and use of results AERA et al. Test user qualifications include attention to the purchase of psychological measures that specify levels of training, educational degree, areas of knowledge within domain of assessment e.
Test user qualifications require psychometric knowledge and skills as well as training regarding the responsible use of tests e. In addition, test user guidelines highlight the importance of understanding the impact of ethnic, racial, cultural, gender, age, educational, and linguistic characteristics in the selection and use of psychological tests Turner et al.
Test publishers provide detailed manuals regarding the operational definition of the construct being assessed, norming sample, reading level of test items, completion time, administration, and scoring and interpretation of test scores. Directions presented to the examinee are provided verbatim and sample responses are often provided to assist the examiner in determining a right or wrong response or in awarding numbers of points to a particular answer.
Ethical and legal knowledge regarding assessment competencies, confidentiality of test information, test security, and legal rights of test-takers are imperative. Resources like the Mental Measurements yearbook MMy provide descriptive information and evaluative reviews of commercially available tests to promote and encourage informed test selection Buros, To be included, tests must contain sufficient documentation regarding their psychometric quality e.
Many instruments, such as those discussed throughout this report, would be considered qualification level C assessment methods, generally requiring an advanced degree, specialized psychometric and measurement knowledge, and formal training in administration, scoring, and interpretation. However, some may have less stringent requirements, for example, a bachelor's or master's degree in a related field and specialized training in psychometric assessment often classified level B , or no special requirements often classified level A for purchase and use.
While such categories serve as a general guide for necessary qualifications, individual test manuals provide additional detail and specific qualifications necessary for administration, scoring, and interpretation of the test or measure. Given the need for the use of standardized procedures, any person administering cognitive or neuropsychological measures must be well trained in standardized administration protocols. He or she should possess the interpersonal skills necessary to build rapport with the individual being tested in order to foster cooperation and maximal effort during testing.
Additionally, individuals administering tests should understand important psychometric properties, including validity and reliability, as well as factors that could emerge during testing to place either at risk. Many doctoral-level psychologists are well trained in test administration; in general, psychologists from clinical, counseling, school, or educational graduate psychology programs receive training in psychological test administration. For cases in which cognitive deficits are being evaluated, a neuropsychologist may be needed to most accurately evaluate cognitive functioning see Chapter 5 for a more detailed discussion on administration and interpretation of cognitive tests.
The use of non-doctoral-level psychometrists or technicians in psychological and neuropsychological test administration and scoring is also a widely accepted standard of practice APA, ; Brandt and van Gorp, ; Pearson Education, Psychometrists are often bachelor's- or master's-level individuals who have received additional specialized training in standardized test administration and scoring. They do not practice independently or interpret test scores, but rather work under the close supervision and direction of doctoral-level clinical psychologists or neuropsychologists.
Interpretation of testing results requires a higher degree of clinical training than administration alone. Threats to the validity of any psychological measure of a self-report nature oblige the test interpreter to understand the test and principles of test construction. In fact, interpreting tests results without such knowledge would violate the ethics code established for the profession of psychology APA, Most doctoral-level clinical psychologists who have been trained in psychometric test administration are also trained in test interpretation.
Modification of procedures, including the use of interpreters and the administration of nonstandardized assessment procedures, may pose unique challenges to the psychologist by potentially introducing systematic error into the testing process. Such errors may be related to language, the use of translators, or examinee abilities e.
For example, if one uses a language interpreter, the potential for mistranslation may yield inaccurate scores. Use of translators is a nonpreferred option, and assessors need to be familiar with both the language and culture from which an individual comes to properly interpret test results, or even infer whether specific measures are appropriate. The adaptation of tests has become big business for testing companies, and many tests, most often measures developed in English for use in the United States, are being adapted for use in other countries.
Such measures require changes in language, but translators must also be knowledgeable about culture and the environment of the region from which a person comes ITC, For sensory, perceptual, or motor abilities, one may be altering the construct that the test is designed to measure. In both of these examples, one could be obtaining scores for which there is no referenced normative group to allow for accurate interpretation of results. While a thorough discussion of these concepts is beyond the scope of this report and is presented elsewhere, it may be stated that when a test is administered following a procedure that is outside of that which has been developed in the standardization process, conclusions drawn must recognize the potential for error in their creation.
As noted in Chapter 2 , SSA indicates that objective medical evidence may include the results of standardized psychological tests. Given the great variety of psychological tests, some are more objective than others. Whether a psychological test is appropriately considered objective has much to do with the process of scoring. For example, unstructured measures that call for open-ended responding rely on professional judgment and interpretation in scoring; thus, such measures are considered less than objective. In contrast, standardized psychological tests and measures, such as those discussed in the ensuing chapters, are structured and objectively scored.
In the case of non-cognitive self-report measures, the respondent generally answers questions regarding typical behavior by choosing from a set of predetermined answers. With cognitive tests, the respondent answers questions or solves problems, which usually have correct answers, as well as he or she possibly can. Such measures generally provide a set of normative data i. Therefore, standardized psychological tests and measures rely less on clinical judgment and are considered to be more objective than those that depend on subjective scoring.
Unlike measurements such as weight or blood pressure standardized psychological tests require the individual's cooperation with respect to self-report or performance on a task. The inclusion of validity testing, which will be discussed further in Chapters 4 and 5 , in the test or test battery allows for greater confidence in the test results. Standardized psychological tests that are appropriately administered and interpreted can be considered objective evidence. The use of psychological tests in disability determinations has critical implications for clients. As noted earlier, issues surrounding ecological validity i.
Two approaches have been identified in relation to the ecological validity of neuropsychological assessment. Establishing ecological validity is a complicated endeavor given the potential effect of non-cognitive factors e. Specific concerns regarding test performance include 1 the test environment is often not representative i. Activities of daily living ADLs and the client's likelihood of returning to work are important considerations in disability determinations. Occupational status, however, is complex and often multidetermined requiring that psychological test data be complemented with other sources of information in the evaluation process e.
Table highlights major mental disorders, relevant types of psychological measures, and domains of functioning.
Determination of disability is dependent on two key factors: As discussed in detail in Chapter 2 , applications for disability follow a five-step sequential disability determination process. At Step 3 in the process, the applicant's reported impairments are evaluated to determine whether they meet or equal the medical criteria codified in SSA's Listing of Impairments.
This includes specific symptoms, signs, and laboratory findings that substantiate the existence of an impairment i. If an applicant's impairments meet or equal the listing criteria, the claim is allowed. If not, residual functional capacity, including mental residual functional capacity, is assessed. This includes whether the applicant has the capacity for past work Step 4 or any work in the national economy Step 5. SSA uses a standard assessment that examines functioning in four domains: Psychological testing may play a key role in understanding a client's functioning in each of these areas.
Box describes ways in which these four areas of core mental residual functional capacity are assessed ecologically. Psychological assessments often address these areas in a more structured manner through interviews, standardized measures, checklists, observations, and other assessment procedures. Your expertise and knowledge are what distinguish you from other guests and the host of the program. The producer who calls you has limited knowledge about your field. It is therefore important that you take control by shaping the interview in a professional, nonaggressive manner.
Brevity and tightly constructed responses are even more important in electronic radio and television interviews than print interviews. In either forum, you have less than 30 seconds to make your point. Preparation is the key to success. If you are part of a discussion panel with other experts, each of you will be competing for airtime. Do not wait to speak until a question is directed to you. You were invited on the program because of your expertise, and you should participate and raise key points or clarify ones made by other panelists when appropriate. Be assertive; most interview segments last only a few minutes, so claim airtime before the interview is over.
Take advantage of any media training opportunities that are available to you. With each workshop or lecture, you will learn something new and helpful. These errors are rarely deliberate. Your concern should be that the meaning of what you said to the reporter was conveyed accurately, not so much whether the exact words were used. If the reporter completely missed the point, let the reporter know in as helpful a manner as possible.
In a case of serious misstatement of your data or your views, you may certainly request a correction phone or send a letter to the editor. A common error when being interviewed on television is to allow a reporter's false or inaccurate statements to stand uncorrected.
If a reporter creates a false premise to a question, correct that first and then reframe and answer the question. If a reporter cites information or statistics with which you are not familiar, do not assume they are being reported correctly. Simply state that you are unfamiliar with the information. After an interview, you may ask the journalist if you can contact him or her with more information you think of later.
How to Work With the Media. Where do you begin? Understanding the Media Always remember that the media are in the business of making money. Interviews to Turn Down Choose wisely in accepting interviews. You do not need to grant every request. Carefully consider whether to participate in an interview that would compromise you in any way is out of your range of expertise is in a panel format Beware: Know Yourself Grant a request for an interview only if you want to give the interview. Your answers to these questions will help you determine whether you should do the interview: What do you want to accomplish with this interview?
What do you want to say about this subject? What do you have to gain by doing it? Will there be more opportunities for you in the future? Know Your Message Before you start an interview, know exactly what you want to say.
Do Your Homework Watch the television programs and read the publications for which you would like to be interviewed. The Preinterview Because reporters assigned to psychology and mental health stories do not know the subject area very well, they often will conduct a preinterview. Take the preinterview as seriously as the interview. Never underestimate this step. The three types of media — print, radio, and television — handle the preinterview differently.
Print There is no automatic preinterview. Make this call a preinterview by asking the reporter about the area of interest and telling him or her you will have to call back. Reporters are always working against deadlines, so promise to get back quickly, within the hour or later that day, and make sure you do. Use the time between the first call and your return call to think about what you want to say about the subject area.
Radio A radio reporter will also telephone and most often want to do an interview right then and there. Television When you receive a call from a television producer, consider this preinterview an audition. Identifying Yourself Establish how you want to be identified. Remember also that preliminary research is often more newsworthy than "known science.
Many news directors will accept interviews conducted by telephone, both live and taped. Try to fly in the day before your interview. What Should I Wear? Speak in short, succinct sentences "sound bites". State your overall message and the supporting points. Youth violence is preventable because it is a learned behavior. It is important to recognize the early experiences that can lead to violence and confront them. For instance, exposure to violence in the media, effects of prejudice and hostility in the home or community, access to firearms.
Psychologists are helping to develop school programs that reduce aggression and prevent violence. Examples of successful programs that have been implemented in schools and communities across the country.
Let your passion for your topic show. Maintain good eye contact with the interviewer, and avoid looking into the camera or monitors. Controlling the Interview Keep in mind that your knowledge and expertise are what distinguishes you from other guests and the host of the program. Help shape the interview in a professional, assertive manner. You can steer the direction of the interview effectively by using transition phrases: The Value of Your Expertise You are the expert who has been called to give information for a program.
Know your key points, and say them throughout the course of the interview. Do not expect to be given equal time if you are in a debate situation. Do not expect the producer or the host to have the same amount of knowledge that you have about your field. You are the expert, and the audience wants to hear your information. Panel Interviews If you are part of a discussion panel with other experts, each of you will be competing for airtime. Workshops Take advantage of any media training opportunities that are available to you.
Print Most journalists strive for accuracy in their reporting, but you may be misquoted at some point. Television A common error when being interviewed on television is to allow a reporter's false or inaccurate statements to stand uncorrected. Always keep a record of interviewers. Good journalists are interested in all the facts. Do your homework keep up with your topic read the publicatio watch the program for which you will be interviewe Don't take cold calls get information and call back within the reporter's deadlin Whether to interview Don't do an interview outside of your range of knowledge Don't do an interview if you haven't rehearsed Preparing for the interview Create 3 to 5 talking points Anticipate the questions, and practice the answers Practice being brief, think "sound bites".
What will happen if I turn down an interview? It's okay — others will call Will I receive questions in advance? No What is meant by "off the record"? No such thing What should I say if I'm asked something about which I'm unfamiliar? Probably not, but you can offer to review What should I do if I am misquoted?
Ask for a correction if it's serious How much time will I have in my radio or television interview to get my point across? Figure on no more than 30 seconds What should I wear to a television interview? Simple, conservative clothing in dark or solid colors, simple makeup, no tinted glasses.