Page 316 - Contributed Paper Session (CPS) - Volume 2

CPS1873 Ferenc M. et al.
4. Comparison of specific respondent groups: do the experiences of
respondents with different economic/social/cultural background differ?
(Developing an advanced schema)
On the fourth level of analysis, systematic comparison of test subject
groups was carried out in order to discover whether any particular themes are
more apparent among specific groups. By performing cross-group
comparison of the themes, advanced schemas were developed (see Miller et
al. 2014) in text format or in complex cases, with tree diagram visualisation.
5. Summary of the overall functioning of each question among respondents
with different economic/social/cultural backgrounds
It is important to note that the first three analytic steps occur, to a certain
degree, simultaneously, because the identification of the themes (step 3) may
spontaneously begin during interviewing (step 1) or when the analyst is
summarising the interview excerpts (step 2). The ‘constant comparison’
principle of GT (Glaser & Strauss, 1967) requires the analyst to continuously
move back and forth between raw textual data, themes, and conceptual claims
(Miller et al. 2014, Mújdricza & Földvári, 2018).

3. Result
The cognitive questionnaire testing method presented above produced
plenty of valuable results for the HBLS questionnaire testing project as well as
for the general purpose of the pilot: the improvement of the HCSO’s
methodological protocols for questionnaire testing. Numerous directions for
improvement were outlined in both aspects, of such efficiency, quality, and
quantity that is unprecedented in previous similar test programmes of the
HCSO.
Almost half of the total HBLS questionnaire: 171 questions were tested in
the project, with pre-scripted probes for 31 questions. Spontaneous probes
were used by the rest of the tested questions (140 questions) in case of any
respondent uncertainty. Based on the experiences of the cognitive testing,
recommendations for modification, for altering the composition, or for
2
rewriting were made – staying within the frames of the EU-SILC Description –
in case of 99 questions total in order to enhance their interpretability and
answerability (i.e. ease of response). The results clearly demonstrate that if the
ultimate goal of a cognitive test is to formulate a question that serves the
purpose of the survey research as best as possible, the methodology for
exploring the respondents’ thought processes applied in the present study
outperforms alternative testing opportunities by far.

Methodological Guidelines and Description of EU-SILC Target Variables, hereinafter referred
2
to as: Description.
305 | I S I W S C 2 0 1 9