Practical guidance on the use of the MMPI instruments in remote psychological testing.

Abstract

The coronavirus-2019 pandemic was declared a national emergency in the United States on March 13, 2020. The disruptions resulting from subsequent unprecedented mitigation efforts have limited and, in some cases, eliminated the ability of psychologists to meet in person with their patients, clients, and examinees. This has led to a broad and sudden reliance on synchronous (live) videoconferencing (also commonly referred to as telehealth, telepsychological practice, and telepractice) to deliver clinical services. A vital component of psychological practice involves the administration of psychological testing, particularly in clinical settings and specialties that rely heavily on the use of assessment instruments. Remote administration of psychological testing presents challenges that cannot be ignored, even in a crisis, without risking the violation of ethical standards and without compromising the reliability and interpretability of test results and the security of test instruments. With these considerations in mind, we provide practical guidance for remote test administration, using the Minnesota Multiphasic Personality Inventory instruments.

The coronavirus-2019 (COVID-19) pandemic caused by the novel coronavirus (SARS-CoV-2) was declared a national emergency in the United States on March 13, 2020. In response to public health warnings about the high virulence and transmissibility of the SARS-CoV-2 virus and the potential for catastrophic loss of life, more than nine in 10 American residents are under government instructions to stay home as of this writing. The sweeping disruption in human travel, economic activity, and health care resulting from these mitigation efforts and from the surging infection rates that threaten to overwhelm the U.S. health care system has been staggering.

This disruption extends also to psychologists’ and other health service providers’ ability to meet in person with their patients, clients, and examinees and has resulted in a broad and sudden reliance on synchronous (live) videoconferencing (also commonly referred to as telehealth, telepsychological practice, and telepractice) to deliver clinical services. For practitioners who have little or no previous experience with delivering psychological services via telehealth, this could mean running afoul of federal and state laws for protecting the security of electronically transmitted personal health information.

The Security Rule provisions of the Health Insurance Portability and Accountability Act (HIPAA; 1996) and its various state progeny require covered entities to develop and follow policies and procedures to restrict access to, protect the integrity of, and guard against unauthorized access to electronically transmitted personal health information, including specifications for integrity control and encryption. The Office for Civil Rights (OCR; 2020) of the U.S. Department of Health and Human Services (HHS) is responsible for federal enforcement of violations of the HIPAA Security Rule technical safeguards provisions (2013). However, in recognition that health care professionals who would ordinarily be subject to the HIPAA Security Rule may, during this national emergency, provide telehealth services through remote communications technologies that “may not fully comply with the requirements of the HIPAA Rule,” the OCR issued a Notification of Enforcement Discretion for Telehealth Remote Communications During the COVID-19 Nationwide Public Health Emergency (Notification; OCR, 2020) that effectively waives penalties for noncompliance with the HIPAA Security Rule “in connection with the good-faith provision of telehealth using such nonpublic facing¹

audio or video communication products during the COVID-19 nationwide public health emergency.” The OCR noted that “[t]his exercise of discretion applies to telehealth provided for any reason, regardless of whether the telehealth service is related to the diagnosis and treatment of health conditions related to COVID-19.” Some state licensing boards also have suspended or relaxed regulations prohibiting interjurisdictional telehealth practice by psychologists not licensed or registered in the state in which the patient or examinee resides, although other boards have not (Association of State and Provincial Psychology Boards, 2020). Practitioners are urged to check local laws in relevant jurisdictions before engaging in interjurisdictional practice and to keep current with evolving pertinent federal and state regulations.

In the context of these and other national and state public health emergency waivers, it may be tempting to ignore or attempt to skirt psychological standards of care, but practitioners do so at their own, and perhaps also their clients’, peril. In contrast to aspirational standards of practice, a standard of care reflects the usual and customary professional practice in the community. In medicine, “crisis standards of care”-defined as “a substantial change in usual health care operations and the level of care it is possible to deliver, which is made necessary by a pervasive (e.g., pandemic influenza) or catastrophic (e.g., earthquake, hurricane) disaster” (Institute of Medicine, 2009, p. 3)-are invoked by a formal governmental declaration of such an emergency to facilitate “the necessary tasks of allocating and using scarce medical resources and implementing alternate care facility operations” (p. 3). But, as nominally implied by the concept of crisis standards of care, even in a crisis, there remain established standards for delivering care. “Ethical norms in medical care do not change during disasters-health care professionals are always obligated to provide the best care they reasonably can under given circumstances” (Institute of Medicine, 2009, p. 2).

A vital component of psychological practice involves the administration of psychological testing, particularly in clinical settings and specialties that rely heavily on the use of assessment instruments (e.g., forensic psychology, neuropsychology, police and public safety psychology). Remote administration of psychological testing presents challenges that cannot be ignored, even in a crisis, without risking the violation of ethical standards and without compromising the reliability and interpretability of test results. We illustrate this point and provide practical guidance for the remote administration of the Minnesota Multiphasic Personality Inventory (MMPI) instruments, both because they are the assessment instruments with which we have the greatest experience and because remote administration of the MMPI instruments was not available prior to the COVID-19 crisis. Although the principles underlying our advice are generalizable to other assessment instruments with remote administration capability, readers are advised to consult the test publisher for specific direction on how to achieve similar protections.²

Remote psychological testing involves Internet-delivered administration of a psychological test at a remote location rather than in person and administered by a practitioner or a trained assistant. The Joint Task Force for the Development of Telepsychology Guidelines for Psychologists (Joint Task Force) published general guidance on telepsychological practice, Guidelines for the Practice of Telepsychology (Joint Task Force, 2013). Although these professional practice guidelines provide little practical instruction on how to ensure adherence to applicable ethical standards when administering psychological tests remotely, they do highlight an issue that is fundamental to understanding the common challenge in remote psychological testing. Guideline 7 states,

When a psychological test or other assessment procedure is conducted via telepsychology, psychologists are encouraged to ensure that the integrity of the psychometric properties of the test or assessment procedure (e.g., reliability and validity) and the conditions of administration indicated in the test manual are preserved when adapted for use with such technologies. (p. 798)

There is ample evidence that the reliability and validity of MMPI test scores are preserved when the test is administered by computer or tablet (e.g., Finger & Ones, 1999; Forbey & Ben-Porath, 2007; Menton et al., 2019; Roper, Ben-Porath, & Butcher, 1995). The second recommendation, that the conditions of administration indicated in the test manual are preserved, is the key challenge associated with remote MMPI administration.

Administration guidelines articulated in the various MMPI instrument manuals identify the need for supervised test administration. For example, the MMPI-2 Restructured Form (MMPI-2-RF) administration guidelines indicate:

The Standards for Educational and Psychological Testing (Standard 5.6) require that test users make reasonable efforts to protect the integrity of test scores by eliminating opportunities for test takers to obtain scores fraudulently. Although the MMPI-2-RF is a self-administered test, completion of the inventory should be supervised by a qualified user or a technician working under the supervision of a qualified user. Adequate supervision ensures that the test taker completes the inventory on his or her own, that any unusual events that may occur during testing are recorded and can be considered in the interpretation of the test results, and that conditions conducive to obtaining optimally valid information are maintained. Supervision does not require that the individual administering the test be in the same room as the test taker throughout the session, although it is desirable that the test taker be within the supervisor’s line of sight (emphasis added). MMPI-2-RF materials should not be sent home with test takers, nor, in institutional settings, should test takers be allowed to complete the instrument in their rooms or anywhere else in which supervision is not possible. The standards (5.7) also state that test users are responsible for protecting the security of test materials at all times. (Ben-Porath & Tellegen, 2008/2011, p. 19)

These administration guidelines reflect three ethical standards (American Psychological Association [APA], 2017) pertaining to assessments: (a) Standard 9.02 (requiring psychologists to administer tests or instruments “in a manner” that is “appropriate in light of the research on or evidence of the usefulness and proper application of the techniques”); (b) Standard 9.06 (requiring that, when interpreting assessment results, “psychologists take into account . . . the various test factors . . . that might . . . reduce the accuracy of their interpretations”); and (c) Standard 9.11 (requiring psychologists to “make reasonable efforts to maintain the integrity and security of test materials”). When relying on remote test administration, these ethical standards can best be followed by reliance on an on-site proctor. The Joint Task Force (2013)Guidelines for the Practice of Telepsychology indicate that psychologists may consider the use of a trained assistant (e.g., proctor) to be on premise at the remote location in an effort to help verify the identity of the client/patient, provide needed on-site support to administer certain tests or subtests, and protect the security of the psychological testing and/or assessment process.

If feasible, the availability of an on-site proctor who can carry out the tasks just described would best meet the need for supervised MMPI administration. However, the use of an on-site proctor can be accomplished properly only in settings in which a noninterested party is available to perform the proctoring tasks (e.g., a trained proctor at a public safety agency for whom the psychologist is conducting a preemployment psychological evaluation). If such an arrangement is not feasible, remote supervision-using an appropriately trained and supervised proctor-can be accomplished via audio-visual monitoring, with the aid of a synchronous (live) teleconference application. The test taker should be seated such that they can be observed by the individual supervising remote administration of the test and audio (should be enabled and kept on so that the supervisor and test taker can communicate with one another and the supervisor is able to hear any communications or possible disruptions that occur during the testing session). Appropriate precautions should be taken to properly identify the test taker and their location (in the event of an emergency requiring notification of local authorities). Any unusual events that may occur during testing should be noted and considered in the interpretation of the test results (e.g., a brief loss of Internet connectivity that is quickly restored may not be expected to meaningfully affect the interpretation of test results, in contrast to an interruption that precludes completion of the remote assessment until a day or more later). With these principles in mind, we offer the following specific recommendations for remote administration of the MMPI instruments using Pearson’s (the test distributor) Internet-based platform, Q-global. Although, as we noted earlier, users of other assessment instruments are advised to consult the test publisher for recommendations on how to achieve similar protections, the following guidance applies generally to self-report instruments with remote administration capabilities.

Practical guidance for remote proctoring with Pearson’s Q-global Remote On-Screen Administration (ROSA) system when proctored via telehealth follows.

Select a Teleconference Platform That Conforms as Close as Possible to Health Insurance Portability and Accountability Act (HIPAA) Security Rule. (2013) Requirements

Although the HHS announced that the OCR will not take enforcement action against HIPAA violators, there is no reason or need to willfully choose a nonsecure telecommunications technology when a HIPAA-compliant option is readily available. Indeed, notwithstanding the HHS notification of the OCR’s discretionary waiver of enforcement action, it is important to note that the waiver applies only “in connection with the good faith provision of telehealth using such nonpublic facing audio or video communication products during the COVID-19 nationwide public health emergency.” This implies that practitioners are expected to put forth a good-faith effort to provide telehealth services that conform to HIPAA requirements while waiving penalties for the use of technology that may unknowingly or unwittingly fall short.

An APA practice advisory (American Psychological Association Services, Inc., 2020), FAQs Practicing Psychologists Have About COVID-19, includes a response to the question, “What is the effect of HHS waiving HIPAA fines?” The advisory points out that “the notice is vague about what other HIPAA provisions related to telehealth HHS will not enforce. Despite this notice, the APA recommends that practitioners use a telehealth platform vendor that will sign a [business associate agreement] and that claims to be HIPAA compliant.” The practice advice cites five reasons for this recommendation: (a) “The notice still recommends doing so to better protect patient privacy. It also lists such vendors”; (b) “using a HIPAA-compliant vendor now will save practitioners from having to switch vendors and learn a new system once the crisis is over” (indeed, it is likely that many practitioners will continue to integrate telehealth into their practices even after the national health emergency ends); (c) “[t]he HHS Office of (sic) Civil Rights does not have authority to waive any similar requirements in state law”; (d) “the notice doesn’t suspend enforcement regarding your overall practice and HIPAA compliance”; and (e) “psychologists should not count on nonenforcement of any HIPAA requirement not directly related to providing telehealth.”

Concerns about the security of some teleconferencing platforms have been well publicized and warrant careful consideration when selecting a vendor. In addition to strong encryption capability, other features that enhance security and reduce the likelihood of hackers or uninvited participants from gaining access to the telehealth session include (a) requiring participants to use a unique password to participate in the session; (b) the ability to lock the session once the examinee and the proctor have joined the session, thereby preventing others from entering; (c) the use of a waiting room in which participants initially enter after identifying themselves and allowing the proctor to deny access to uninvited persons; (d) the option to prohibit participants from renaming themselves; and (e) disabling the share screen feature.

We emphasize in this context that Pearson’s Q-global system does not include an integrated telehealth delivery platform. Remote administration of the MMPI instruments requires that the examiner combine use of Q-global to administer the test (as described next) with a HIPAA-compliant telehealth service such as those that were just described.

Ensure That the Proctor Is Able to Observe the Test Taker Before Providing Access to the Test

One of the security features of Pearson’s Q-global ROSA web-based application-instructions for which are available on its website³

-is that a test administrator first creates a record for an examinee and then assigns one or more specific assessments. The assessment can be given in one of three ways: (a) traditional paper-pencil format, with the scores entered by the test administrator into the Q-global system (manual entry option); (b) on a computer or tablet provided by the test administrator or proctor (on-screen administration option); or (c) on a remote computer or tablet away from the test administrator (ROSA option). It is this last option that is the focus of our guidance.

In the ROSA option, the test taker is administered the MMPI after clicking on a link, provided within an e-mail invitation that immediately connects to the test. A test administrator can e-mail the invitation either to the examinee or to someone other than the examinee. It is important to never send the examinee the invitation until the test administrator or proctor is present (physically or via teleconference) and able to confirm:

* the examinee’s identity,

* the examinee completes the inventory on his or her own,

* any unusual events that may occur during testing are noted so that they can be considered in the interpretation of the test results,

* the testing environment is (and remains) conducive to obtaining optimally valid information, and

* test security is maintained.

In practical terms, this means either sending the invitation to someone other than the examinee (i.e., the person who will be proctoring the remote testing session) or sending it to the test taker only after establishing the telehealth connection. Confirmation that the testing environment is private, conducive to testing, unaided by a third party, and secure may also be facilitated by an environment check at the start of the test session, in which the proctor asks the examinee to temporarily reposition the webcam to display the surrounding area.

Document the Use of Remote Test Administration and Telehealth Proctoring Both in Your Informed Consent Procedures and in Any Written Report, and Obtain All Necessary Consent for Remote Test Administration and Proctoring

In addition to the obligation all psychologists have for obtaining and documenting informed consent when providing professional services (APA, 2017; Standard 3.10), state law may also impose specific requirements when engaged in telehealth. For example, California Business and Professions Code § 2290.5(b) stipulates:

“Before the delivery of health care via telehealth, the health care provider initiating the use of telehealth shall inform the patient about the use of telehealth and obtain verbal or written consent from the patient for the use of telehealth as an acceptable mode of delivering health care services and public health. The consent shall be documented.”

Psychologists should always consider whether remote test administration is viable or advisable for a particular examinee or situation, irrespective of informed consent. Individuals with significant cognitive impairments, with severe mental illness, or who are at risk for suicide, may not be suitable for remote test administration. In addition, some forensic or other evaluations conducted in an adversarial environment may be ill suited for remote test administration.

Finally, evaluations that are governed by regulatory or institutional procedures may impose additional consent requirements. For example, when using telehealth to conduct any portion of a preemployment psychological evaluation of a California police officer candidate, the California Commission on Peace Officer Standards and Training (POST) advises psychologists to obtain both the advance approval of the hiring agency and the consent of the candidate (California Commission on Peace Officer Standards and Training, 2020). In addition, POST guidance includes the recommendation to “[s]tate in your written report when any portion of the psychological evaluation was conducted remotely” (California Commission on Peace Officer Standards and Training, 2020, p. 7).

Consider Using a Checklist to Ensure That Issues Unique to Remote Test Administration Are Addressed

In his 2009 bestseller, The Checklist Manifesto: How to Get Things Right, Atul Gawande (2009) made a compelling case for checklists to ensure that important actions are taken, and for pause points at strategic moments in a sequence of actions to confirm that the important actions have been taken. In Table 1,

we provide a checklist that may be useful in a variety of practice settings when administering the MMPI via remote on-screen test administration and when proctoring the testing session via teleconference.

Closing Comments

We conclude this brief guidance document with our response to concerns we have heard that remote proctoring as described in this document may not afford sufficient supervision of test administration, particularly with regard to maintaining test security, or that test users may opt not to adhere to practices such as those described here. We acknowledge that for some types of assessments (e.g., in the adversarial context of forensic assessment) remote test administration could create more problems than it solves. It is ultimately the responsibility of every practitioner to weigh the costs and benefits of this practice, which may vary from one case to another. In our opinion, the steps recommended here are sufficient to maintain the integrity of test results and the security of the MMPI test items. It is possible, of course, that unscrupulous users may fail to follow the guidance we have offered, but this concern is not unique to remote administration. The MMPI manuals indicate that in-person administration of the test should be supervised. This admonition also cannot prevent practitioners from sending booklets home with test takers or in the mail, for example. Introduction of new technology does present new options for mispractice. We do not believe that this risk should prevent the vast majority of ethically scrupulous providers from adapting their practice to challenging times and new technology. Nevertheless, remote MMPI administration does introduce some unique challenges, which this paper seeks to address.

Footnotes

¹ The use of public-facing communication products, such as Facebook Live, Twitch, and TikTok, that are designed for public viewing is not protected by the OCR discretionary enforcement waiver.

² A wide range of other instruments is available for remote administration using Pearson’s Q-global platform, as are instruments licensed to JRA, Inc. (e.g., the California Psychological Inventory and Personality Assessment Inventory) for use in evaluations of police and other public safety personnel, and tests published by PAR, Inc. and Multi-Health Systems, Inc., to name just two additional test publishers.

³ https://qglobal.pearsonclinical.com/qg/static/Platform/Customer/en/WebHelp/Assessments/Remote_On_Screen_Administration.htm.

⁴ Although Q-global assessments can be administered by use of a tablet, the camera may be disabled when the test taker alternates from the teleconference application to the Q-global browser. In this case, using a smartphone for conducting the teleconference proctoring, while the examinee completes the test on a tablet, may be an alternative.

Corresponding Author

David M. Corey received his PhD in clinical psychology from Fielding Graduate University. He is the founder of Corey & Stewart in Portland, Oregon, a psychology practice limited exclusively to assessment-related services in forensic psychology and police and public safety psychology. He is board certified in both specialties by the American Board of Professional Psychology. His research interests are focused primarily on the use of psychological testing to predict posthire behavior in police and other public safety personnel.

Yossef S. Ben-Porath received his PhD from the University of Minnesota. He has been involved extensively in MMPI research for the past 35 years. He is a codeveloper of the MMPI-3, MMPI-2-RF, and MMPI-A-RF and coauthor of test manuals, books, book chapters, and articles on the MMPI instruments. He is a board-certified clinical psychologist (American Board of Professional Psychology-Clinical) whose clinical practice involves supervision of assessments at Kent State’s Psychological Clinic, consultation to agencies that screen candidates for public safety positions and provision of consultation and expert witness services in forensic cases.

David M. Corey receives royalties on the sale of the MMPI-2-RF Police Candidate Interpretive Report. Yossef S. Ben-Porath receives royalties on MMPI-2-RF sales.

Correspondence concerning this article should be addressed to Yossef S. Ben-Porath, Department of Psychological Sciences, Kent State University, Kent, OH 44242

Email: [e-mail]

Publication History

Received April 6, 2020

Revision received April 15, 2020

Accepted April 15, 2020

References

American Psychological Association. (2017). Ethical principles of psychologists and code of conduct. Retrieved fromhttps://www.apa.org/ethics/code/ethics-code-2017.pdf

American Psychological Association Services, Inc. (2020). FAQs practicing psychologists have about COVID-19. Retrieved fromhttps://www.apaservices.org/practice/news/psychologists-questions-covid-19?_ga=2.92503197.1217006363.1585700505-1165606630.1583016895

Association of State and Provincial Psychology Boards. (2020). Temporary/telepsychological practice and COVID-19. Retrieved fromhttps://drive.google.com/file/d/1jPGbDi8bh5K945xTxAiQe6ZqquDSjwez/view

Ben-Porath, Y. S., & Tellegen, A. (2008/2011). MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2 Restructured Form) manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press.

California Business and Professions Code, Division 2, chap. 5, Article 12, § 2290.5.

California Commission on Peace Officer Standards and Training. (2020). Compliance with POST selection standards during coronavirus (COVID-19) emergency. Bulletin No. 2020–18. Sacramento, CA: Author.

Finger, M. S., & Ones, D. S. (1999). Psychometric equivalence of the computer and booklet forms of the MMPI: A meta-analysis. Psychological Assessment, 11, 58–66. https://doi.org/10.1037/1040-3590.11.1.58

Forbey, J. D., & Ben-Porath, Y. S. (2007). Computerized adaptive personality testing: A review and illustration with the MMPI-2 Computerized Adaptive Version. Psychological Assessment, 19, 14–24. https://doi.org/10.1037/1040-3590.19.1.14

Gawande, A. (2009). The checklist manifesto: How to get things right. New York, NY: Henry Holt & Company.

Health Insurance Portability and Accountability Act of 1996 (HIPAA), Public Law 104-191.

Health Insurance Portability and Accountability Act (HIPAA) Security Rule. (2013). Technical safeguards. 45 CFR § 164.312(e), as amended on January 25, 2013.

Institute of Medicine. (2009). Guidance for establishing crisis standards of care for use in disaster situations: A letter report. Washington, DC: The National Academies Press. https://doi.org/10.17226/12749

Joint Task Force for the Development of Telepsychology Guidelines for Psychologists. (2013). Guidelines for the practice of telepsychology. American Psychologist, 68, 791–800. https://doi.org/10.1037/a0035001

Menton, W. H., Crighton, A. H., Tarescavage, A. M., Marek, R. J., Hicks, A. D., & Ben-Porath, Y. S. (2019). Equivalence of laptop and tablet administrations of the Minnesota Multiphasic Personality Inventory-2 Restructured Form. Assessment, 26, 661–669. https://doi.org/10.1177/1073191117714558

Office for Civil Rights. (2020). Notification of enforcement discretion for telehealth remote communications during the COVID-19 nationwide public health emergency. Retrieved fromhttps://www.hhs.gov/hipaa/for-professionals/special-topics/emergency-preparedness/notification-enforcement-discretion-telehealth/index.html

Roper, B. L., Ben-Porath, Y. S., & Butcher, J. N. (1995). Comparability and validity of computerized adaptive testing with the MMPI-2. Journal of Personality Assessment, 65, 358–371. https://doi.org/10.1207/s15327752jpa6502_10