Page 313 - Contributed Paper Session (CPS) - Volume 2

CPS1873 Ferenc M. et al.
to each quota cell was also created to ensure replacement in case of an
unforeseeable absence of any selected test subject (Mújdricza, 2018).
During the cognitive test semi-structured in-depth interviews were
conducted on the subjects’ mental processes and considerations as they form
answers to survey questions (understanding what the question is asking,
recalling relevant information, formulating an answer, mapping the answer
onto the provided response categories). Of the two main practices used in
cognitive interviewing (think aloud and verbal probing approach), the verbal
probing method was chosen for the testing of the HBLS questionnaire. In
verbal probing, the interviewer facilitates the revealing of thought processes
by composing situation-specific or post-situation questions as probes. There
are two different practical approaches within the verbal probing method:
concurrent or retrospective probing. They can also be used in combination,
enabling the separate testing of the questions as well as getting an overall
general picture on the questionnaire at the same time (Willis, 1999, 2015,
Collins, 2015). Such combined probing was done in the testing of the HBLS
questionnaire. On the one hand, probes on specific questions were
administered concurrently with the asked questions of the questionnaire, on
the other hand, retrospective probes were administered after finishing the
questionnaire about how it was to answer the questions in general. Some
probes (especially for questions difficult to understand) were designed pre-
interview (scripted probes), however, the each interviewer was authorised to
ask spontaneous probes in the entire course of the interview to further clarify
certain topics in case s/he deemed it necessary. In line with the most recent
methodological recommendations (Miller et al. 2014, Willis, 2015, Collins,
2015), the pre-composed probes, too, were flexibly modified simultaneously
with the progress of the interviewing phase when sufficient experience was
gathered on the usability of the pre-scripted probes, and also during the
interviews in order to make them appropriate to the actual interview situation.
During the (usually) semi-structured interviews, the success of acquiring
relevant information depends on the active, ‘on-the-spot’ analytic attention of
the researcher in case of the verbal probing method. So the researcher has to
provide adequate, facilitative frames and support for the interviewees so that
they can reveal how they interpret and reply a question in the light of their
experiences. Only the interviewees can give information on it, but it is of great
importance to ask their experiences rather than their opinions on the
questions. Asking the interviewees’ opinion on the questionnaire is
unreasonable and might well be misleading, for they usually lack the necessary
expertise to assess the quality of a questionnaire or its questions. Interviewees
should be treated as knowledgeable informants rather than evaluators (Miller
et al. 2014, Willis, 2015). If multiple researchers do the cognitive interviewing,
the consistency of the interviewing must be maintained. The opportunity to

302 | I S I W S C 2 0 1 9