Page 314 - Contributed Paper Session (CPS) - Volume 2

CPS1873 Ferenc M. et al.
prior rehearsal of the interview protocol of a given questionnaire is important
even for experienced cognitive interviewers. The researchers can get feedback
on how to adjust the test protocol by summarising the experiences gained
during the initial interviews (Willis, 1999, Shafer & Lohse, n.d., Gray, 2015).
All three researchers conducting the CIs of the HBLS questionnaire were
experienced in qualitative interviewing, and all of them took part in the design
process of the interview protocol. In the initial phase of the project, the
researchers did a prior research on the methodology of cognitive
questionnaire testing, and they gathered knowledge on the HBLS survey.
Before proceeding to the cognitive interviewing phase, practical experience
was built up by discussing the practical issues in regular meetings and by
conducting a rehearsal interview with a volunteering colleague. After each CI,
the major experiences and issues were discussed that gave a continuous
feedback of practical information for the team as well as it facilitated the
consistency across the interviews conducted by the three researchers.
Furthermore, as interview experiences built up, the discussions enabled
harmonised, flexible modifications of the probes (see Földvári & Mújdricza,
2018).
Our initial goal was to test the entire questionnaire. However, it became
apparent during the rehearsal interview that administering the full interview
protocol would far exceed the maximum of 1.5 hours timeframe per interview
recommended by the literature (e.g. Willis, 1999). Hence, given the fact that
the CI is rather burdensome for the interviewees as well as for the interviewers,
the pool of the tested questions had to be reduced. To this end, based on
expert review of the complexity and difficulty of understanding of the
questions, each was sorted into one of the following categories:
1. easy to understand questions assumed not to cause any hardships;
2. questions expected to cause problems;
3. especially difficult to understand questions that were expected to
require more attention.
In the test project of the HBLS questionnaire, scripted probes were
composed to the questions sorted in the third category, that is, to the ones
that had been assumed to be especially difficult to understand. As described
above, the scripted probes were treated flexibly, though, and whenever it
seemed necessary, either different, additional spontaneous probes were
composed befitting to the actual interview situation or topic, or irrelevant
scripted probes were simply skipped. For the questions sorted in the second
category, no scripted probes were composed. Probing was only done if the
researcher sensed difficulties in the response process, and these probes were
composed as spontaneous probes adequate to the sensed difficulty.
The first category questions that were deemed unproblematic for the
respondents were not tested. This expert categorisation served design

303 | I S I W S C 2 0 1 9