Page 315 - Contributed Paper Session (CPS) - Volume 2

CPS1873 Ferenc M. et al.
purposes primarily, so it was treated flexibly in the light of the interview
experiences. However, our pre-categorisation of the questions proved to be
typically correct with a few exceptions in the course of the testing, that is, the
prior categorisation was quite efficient. The analysis of the interviews and,
based on the results of the analysis, elaboration of the revisions to develop
the tested questionnaire is of major importance in every questionnaire pre-
test using cognitive interviewing methods. Based on the advantages and
disadvantages, we applied the latest inductive interview analysis method,
namely, Theme Coding based on Grounded Theory (GT) elaborated by the
interpretive school of cognitive interviewing. The main steps of data reduction
and data analysis (they have two be carried out simultaneously) are the
following in this method (based on Miller et al. 2014):
1. Conducting the interviews, recording narratives of how the interviewees
answer the questions.
2. Making interview excerpts and then summaries based on the recordings:
how did the subjects answer the questions, were there any difficulties?
The starting point of the analytic phase of the HBLS questionnaire test
project was the audio recordings. Two steps lead from the recordings to the
summaries on the level of the individual questions. First, the seemingly
relevant parts of the recording regarding a question were transcribed
verbatim, and then the experiences on the question were summarised in three
analytic categories: (1) interpretations, (2) difficulties in the response process,
and (3) issues with the question. This two-step process was re-done with each
question. The first step (the transcription of the interview excerpts) was done
by the researcher who had conducted the given interview, thus s/he could also
record reflections on her/himself and on the interviewee regarding the specific
situation that were important supplementary information for the analysis. The
second step, the actual analysis was done by the researcher responsible for
the analysis of the given topic of the HBLS questionnaire.
3. Comparison of the summaries: developing a thematic schema, that is, can
common interpretational patterns be discovered concerning a given question?
The common themes (Willis, 2015 refer to them as ‘codes’) across the test
subjects’ personal narratives (represented by the interview summaries) were
identified by reviewing the specific section on a particular question in all
interview summaries searching for interpretive patterns. In case of a given
question, typically multiple themes emerge during the comparison process.
These themes represent the key elements of the question. In the HBLS
questionnaire testing – which was a special project due to the vast number of
tested questions, so the analysis, too, demanded unique solutions – the
majority of the themes were written down in text. The themes were only
visualised in tree diagrams – recommended by Miller et al. (2014) – in case the
complexity of the interpretive patterns necessitated it.

304 | I S I W S C 2 0 1 9