Health Services Research, Part 5: Writing Survey Questions
Health Services Research Survey
The final step in writing an excellent health services research survey is writing data collection questions. These questions should grasp the complexity of the assessment domains indicated by the statement of research purpose. While clinicians can easily measure some clinical outcomes objectively, they often must measure other general outcomes. This includes behavioral, cognitive, and affective—creatively. Several strategies for creating data collection questions that can measure these assessment domains in greater complexity are available to survey designers. This entry discusses the multi-item scale, in particular.
Multi-Item / Likert Scale
Most individuals who have taken a survey are familiar with a Likert scale. This scale allows the survey respondent to choose between values that indicate how much they agree or disagree with a statement. For example, “Indicate how much you agree with the following statement on a scale of 1 to 4 (1 being agree and 4 being disagree).”
Creating multi-item scales allows the survey analyst to discern in-depth differences across respondents. The survey designers can ask several questions to aggregate the results. This is to arrive at a single global measure of the assessment domain that is the focus of the research.
Example
A survey design team may use a multi-item scale to measure the beliefs of a specific group of clients. The following is an example of fruits and vegetables.
All statements below indicate how much you agree (1) or disagree (4).
We can see that the assessment domain “beliefs about fruits and vegetables” includes 10 elements. Some elements use positive phrasing (e.g., “I like most vegetables”), while others use negative phrasing (e.g., “I don’t like fruit”). Scorers reverse-score the negatively phrased items. A score of 1 indicates a low level of the desired outcome. A 4 represents a high level.
Results
Before processing the results, all reverse-scored items must be adjusted to be scored.
The results can be aggregated once the adjustment to reverse-scored items is completed. Aggregating the results, a survey respondent could score from 10 (Agree on all items) to 40 (Disagree on all items). A score of 5 indicates very positive beliefs. A score of 40 indicates very negative beliefs about fruits and vegetables.
Question Subsets
Alternatively, subsets of the questions in a multi-item scale can be aggregated to create different indicators of the measured belief. In the example above, the 6 questions about vegetables could form one scale, while the 4 questions about fruits could form another.
Using a multi-item scale, the survey designer has measured this assessment domain in high detail, strengthening the data team’s ability to make more complex and meaningful statistical analyses than a single-item scale or another type of simpler measure. For example, if the same multi-item scale is administered to a group of people at the beginning and end of a program meant to impact them in a certain way, such as to increase their belief in the accessibility and desirability of eating fruits and vegetables, a simple statistical procedure (T-test for Paired Samples) can be conducted to determine if changes associated with participation in the program are statistically significant.
The Series
Focusing specifically on developing surveys for Health Services Research studies designed to measure health outcomes, this series of articles covers many topics.
- Various types of health-related surveys and outcomes that can be measured
- Creating meaningful research questions
- Conceptualizing and operationalizing variables
- Developing sophisticated survey questions
Part 1: Introduction to Health Services Research
Part 2: Types of Surveys & Outcomes
Part 3: Research Questions
Part 4: Conceptual & Operational Definitions
Part 5: Writing Survey Questions
For more information, check out:
Crestline Advisors
Compliance Resource Center
Resources Include
https://www.ahrq.gov/cahps/surveys-guidance/index.html
http://www.hosonline.org/en/survey-instrument/
http://www.rand.org/health/surveys_tools.html
Aday, Lu Ann, and Llewellyn J. Cornelius, 2006. “Designing and Conducting Health Surveys: A Comprehensive Guide,” Jossey-Bass: San Francisco, CA.
As an experienced health care professional, Susan (Sue) Dess brings many experiences to Crestline. Her 15-year administrative and executive management background spans the operations of both managed care and provider organizations.
Sue spent 25 years as an Emergency Room and Intensive Care Registered Nurse, further rounding out her ability to understand the “big picture.” Sue is intimately involved with each Crestline project, collaborating closely with consultants and clients.
