Conference report from ASC 2007 Southampton
Internet research is facing a credibility crisis that technology alone cannot solve – though skilful application of the emergent technologies and new methodological understanding presented at ASC 2007 may offer the best hope in curing its current malaise. This was a key message cutting across the forty-some papers presented at the Association for Survey Computing’s fifth international conference, held at the University of Southampton last month.
ASC’s twin focus on methodology and technology and inclusion of voices from government, social, academic and market research tends to lead to more candid revelations than you will hear at the big MR conferences. Reg Baker from Market Strategies in the US went beyond recent revelations about the extent of ‘professional respondents’ in panels and reports of online surveys giving contradictory results and presented his analysis of ‘panelist pathologies’. Hyperactives, inattentives and fraudulents all exhibit slightly different behaviours which, with some intervention, can be detected and weeded out provided you know what you are looking for.
The concept of respondents ‘satisficing’ rather than responding emerged in other papers too – a strategy of answering in a way that is sufficient to satisfy the survey instrument, but not to give results of any acceptable quality – such as ‘straightlining’ through grids. Several ingenious technical interventions were shown to anticipate and counteract this tendency or even design it out by using novel questioning methods. Michael Johnston from AT&T Labs presented how future interview might look that use ‘agents’ combining avatars and voice generation. Using a repertoire of prompts, and offering automated clarifications when hesitation or errors are detected can offer a new interviewing mode that plays to the strengths of both assisted and self-completion interviewing. While that may remain a distant promise, Nicola Stanley (Silver Dialogue) and Steve Jenkins (Snap Surveys) gave convincing evidence of the benefits of moving away from essentially text-based online surveys, which predominate, to questions based on graphical and interactive elements, in terms of improved respondent engagement.
Yet Mick Couper (University of Michigan) advised caution here in the closing keynote, as he deconstructed the various aspects of Web 2.0, searching for what may be of enduring benefit here to the survey researcher. His more limited tests using interactive components had showed no improvement in engagement, and at a cost of greater effort in development, and loss of some respondents whose browsers or bandwidth prevents them from participating. Identifying three strands to Web 2.0 – the social web, the interactive web and the mobile web, he counselled against the obsession with the social web and the ‘the big numbers’ involved in Facebook, Second Life and so on. In reality, these were and probably always would be elite groups of limited value to the genuine survey researcher. He was much more excited by the potential of the interactive web, based on new hybrid technologies such as data-driven animation and multi-source data mash-ups and the mobile web for building better surveys in the future.
ASC 2007 Conference Proceedings (400pp) £50 incl UK P&P available from ASC while stocks last. Tel 01453 511511 or email firstname.lastname@example.org.