877.334.8512 |      

EHR and practice management for mental / behavioral health

Health Services Research, Part 5: Writing Survey Questions

PIMSY mental health practice management system shares part 5 of a series about health services research

by Susan Dess, 7.27.17

The final major step in writing excellent health services research surveys is writing data collection questions that truly grasp the complexity of the assessment domains indicated by the statement of research purpose. While some clinical outcomes are easy to measure objectively, the other general types of outcomes we have discussed (behavioral, cognitive, and affective) must often be measured in more creative ways. Several strategies for creating data collection questions that can measure these assessment domains in greater complexity are available to survey designers. This entry discusses the multi-item scale, in particular.

Multi-Item / Likert Scale

Most individuals who have taken a survey are familiar with a Likert scale. This scale gives the survey respondent the option to choose between a series of values that indicate how much they agree or disagree with a statement. For example, “Indicate how much you agree with the following statement on a scale of 1 to 4 (1 being agree and 4 being disagree).” This can be a very useful type of data collection question and creating multi-item scales allows the survey analyst to discern differences across respondents in much more detail. As opposed to asking one question, the survey designers can ask several, with the goal of aggregating the results to arrive a one global measure of the assessment domain that is the focus on of the research.

To measure the beliefs of a specific group of clients about fruits and vegetables (which can be considered a goal of a program designed to increase healthy dietary habits) a survey design team may use a multi-item scale like the following:

For all of the statements below indicate how much you agree (1) or disagree (4).

health services research part 5 writing survey questions

In this example, we can see how the assessment domain “beliefs about fruits and vegetables” is represented by 10 elements. Some elements are phrased positively (ex: “I like most vegetables”) while others are phrased negatively (ex: “I don’t like fruit). The items that are phrased negatively are “reverse-scored,” meaning that a score of 1 indicates a low level of the desired outcome, while a 4 represents a high level. Before processing the results all items that are reverse scored need to be adjusted so that all “Agree” ratings are scored as 4, “Slightly Agree” ratings as 3, “Slightly Disagree” ratings as 3, and “Disagree ratings” as 1.”

Once adjustment to reverse scored items is completed, the results can be aggregated. Aggregating the results, a survey respondent could score from 10 (Agree on all 10 items) to 40 (Disagree on all 10 items). A score of 5 would indicate very positive beliefs, while a score of 40 would be very negative beliefs about fruits and vegetables.

Alternatively, subsets of the questions in a multi-item scale can be aggregated to create different indicators of the belief being measured. In the example above, the 6 questions about vegetables could form one scale, while the 4 questions about fruits could make another.

By using a multi-item scale, the survey designer has measured this assessment domain in high detail which will strengthen the data team’s ability to make more complex and meaningful statistical analysis than would a single item scale, or another type of more simple measure. For example, if the same multi-item scale is administered to a group of people at the beginning and end of a program meant to impact them in a certain way, such as to increase their belief in the accessibility and desirability of eating fruits and vegetables, a simple statistical procedure (T-test for Paired Samples) can be conducted to determine if changes associated with participation in the program are statistically significant.

Resources Include 

https://www.cdc.gov/policy/hst/hra/frameworkforhra.pdf

https://www.ahrq.gov/cahps/surveys-guidance/index.html

http://www.hosonline.org/en/survey-instrument/

http://www.rand.org/health/surveys_tools.html

Aday, Lu Ann and Llewellyn J. Cornelius, 2006. “Designing and Conducting Health Surveys: A Comprehensive Guide,” Jossey-Bass: San Francisco, CA.

The Series

Focusing specifically on developing surveys for HSR studies designed to measure health outcomes, the entries that follow in this series detail more specifically how to create surveys that successfully measure health outcomes:
> Creating meaningful research questions
> Conceptualizing and operationalizing variables
> Developing sophisticated survey questions

Part 1: Introduction to Health Services Research
Part 2: Types of Surveys + Outcomes
Part 3: Research Questions
Part 4: Conceptual & Operational Definitions

For more information, check out: 
Crestline Advisors
Compliance Resource Center
Business Management Resource Center

 

sue-dessAs an experienced health care professional, Susan (Sue) Dess brings a wide range of experiences to Crestline. Her 15 year administrative and executive management background spans the operations of both managed care and provider organizations.

Additionally, Sue spent 25 years as an Emergency Room and Intensive Care Registered Nurse, further rounding out her ability to understand the "big picture." Sue is intimately involved with each Crestline project, collaborating closely with consultants and clients.

 

Resources - Business Management Resource Center

  • Default
  • Title
  • Date
  • Random
load more hold SHIFT key to load all load all

Kudos from Clients

  • Seth H.

    “PIMSY more than pays for itself by streamlining my office, improving efficiency and reducing billing times. I would recommend PIMSY to anyone looking for a good EMR company that will help you implement its program and help you with any questions you have along the way.”

    ~ Seth H., Business Owner

  • Karen B.

    “Love PIMSY! So much quicker to complete notes and easier for everyone working with clients to know current authorizations and track units.”

    ~ Karen B., Therapist

  • Dr. Carmen L.

    “I am extremely appreciative and am so glad I decided to go with PIMSY versus the other options I was considering. I was singing your praises to a colleague of mine today who is feeling overwhelmed with her paper process. I highly recommend all of you.”

    ~ Dr. Carmen L., Program Director

  • Kim T.

    “We are now functioning at a 50% faster recovery rate for money and a 50% lower denial rate. You should really give the PIMSY team time to demonstrate for you personally.”

    ~ Kim T., Business Director

Subscribe To Our Newsletter

captcha