Tim is a world-renowned specialist in the application of technology in the field of market and opinion research and is probably the most widely-published writer in the field. His roots are in data analysis, programming, training and technical writing. These days, as principal at meaning he works with researchers, users of research data and technology providers around the globe, as an independent advisor. He is quite passionate about improving the research process and empowering people through better use of technology.

In Brief

What it does

Handheld interviewing for Windows mobile devices, with capabilities for mystery shopping, public transport measurement and self-completion diary surveys

Supplier

Cluetec, Germany

Our ratings

Score 4 out of 5Ease of use

Score 3 out of 5Compatibility with other software

Score 5 out of 5Value for money

Cost

Pay-per-use model from €0.30 per interview, with support contracts from €100 (inc 1 hr phone support). Other pricing plans available for longer-term users.

Pros

  • QuestEditor authoring tool very simple and easy to use
  • Special ‘traffic’ version is ideal for transport and travel surveys
  • Robust and reliable in the field
  • Cluetec offers PDA rental and per-survey cost

Cons

  • Cannot pre-populate interviews with case data
  • No support for quotas
  • Lacks tools to manage the allocation of fieldwork to individuals and devices
  • Windows Mobile/Pocket PC devices only, not Palm

In Depth

“Great idea, but not really practical.” This is a common reaction from experienced fieldwork managers to handheld interviewing for long and complex surveys – and it is fair to say that it has also been the experience of some users when trying to do demanding surveys such as mystery shopping or travel audits. Long surveys with complex routing are often no problem, but surveys where the interviewer needs to follow a routing determined by what they are observing rather than a pre-determined flow of questions can be very tricky to present on a PDA.

Cluetec, a German software company, has developed a palmtop interviewing system which aims to provide support for these problem children of mobile interviewing, as well as the more standard fare of face-to-face interviews.

Surveys are created in a Windows-based tool called QuestEditor. It is fairly standard fare, with a tree-view on the left and tabbed forms where you define your question texts, answers, conditions and validation. It is virtually syntax-free, though some of the routing and validation logic can be a bit cryptic, and branches are achieved using a ‘goto’ type construct, which has the potential to become very confusing for complex routings. A block-structured approach would be safer all round.

Cluetec offers a special ‘Traffic’ version of mQuest which is aimed at public transport service measurement, though it could be equally useful in a variety of other mystery shopping situations. Traffic lets the auditor toggle between two surveys, which is ideal for on-board measurement on a train, tram or bus, as a full audit of the service and the vehicle can be carried out when the vehicle is moving. However, when the vehicle reaches a stop, the auditor can toggle to a second survey which lets them count those boarding and alighting, and in a number of categories such as by age, disability, with bikes, pieces of luggage and so on.  The boarding survey is already populated with information about the route, which is downloaded to all the devices. All the auditor needs to do is enter the route code and the survey will then anticipate each stop in sequence. There appears to be no practical limit on how many routes can be loaded on the palmtop device for any transportation region.

An auto-completion feature, available in mQuest, makes travel and mystery shopping very effective, and overcomes the constraint of typing letters with a stylus from a pop-up keyboard on screen. As each letter is entered, the list will narrow down to the most likely candidates that begin with or contain those letters. It will make light work of several thousand destination or transport interchange points.

The software is strong on validation – both rigorous error checking and lighter-touch plausibility checking, and contains several other features to make completion both fast and reliable in the field. Single click mode moves to the next question as soon as an answer is given – and this can be turned on and off as required. Global variables allow some questions, such as location, to be filled in automatically, until the interviewer changes it. Also ideal for mystery shopping is its ability to integrate any pictures or short movies taken using the PDA’s built-in camera with the interview, or audio, so that these images or recordings are passed back along with the rest of the data. There are special question types for each of these, making programming a breeze.

There are also some surprising lacks in the product. There is no support for quota control, which is perhaps forgivable if the emphasis is mystery shopping. Less easy to understand is the absence of the means to pre-populate surveys with case-specific data, such as data pertaining to the places being mystery shopped. Neither is there a decent solution for allocating work to fieldworkers or respondents yet. Improvements are promised here for a version due out in June. Though multiple languages are supported, the authoring tool only shows one at a time. As you enter the translation, the original disappears from view, which is an accident waiting to happen, in my view.

Underneath all of this, the software is developed in Java, and Cluetec have written their own emulator and delivery platform, which means the software has the potential to run on any platform that supports Java, not just Windows. Market demand, however, means that mQuest is currently only supported on Windows mobile devices and support for PalmOS has recently been withdrawn. But the emulator, which you can use to test surveys, works in any Java environment device. It worked faultlessly on my Mac, for example.

The server, which the handhelds connect to for up- and downloading, works under Linux or UNIX as well as Windows. But the QuestEditor authoring tool insists that you are a Windows user. Yet the experience of users seems to point to the product being rock solid for reliability when used in the field. Data transfer can be achieved wirelessly, using cellular telephony for distributed fieldwork, or Bluetooth for short distances such as at a conference centre, or simply by docking the devices’ flash memory cards.

Conveniently. Cluetec also maintains a large stock of loan devices, and allows renters to use its servers for data transfer and fieldwork management. With rental charged on a per-interview basis, it can offer both a low-cost and a low-risk was to dip a toe into the fast-moving waters of mobile interviewing.

Media company MindShare has made extensive use of mQuest for its MindSet media survey in Germany. Here, Christian Franzen, Director Advanced Techniques Group and Christian Maerten, Project Manager MindSet, speak of their experiences.

Customer Perspective: MindShare, Germany

Media company MindShare has made extensive use of mQuest for its MindSet media survey in Germany. Here, Christian Franzen, Director Advanced Techniques Group and Christian Maerten, Project Manager MindSet, speak of their experiences.

CF: “We belong to a media agency, so we do research on media usage and ad effectiveness. The aim of our MindSet study is to evaluate total media usage of people, because all we have are highly separated studies in this market. We do not have something that does not cover all media – not just TV, radio and newspapers but all the other media such as posters, beer mats, internet, email and so on.

CM: “Our aim was to do a study where people filled in a questionnaire every hour for three days. The idea was to have something that was something between a questionnaire and measurement – very near to measurement. We wanted to do this via PDA so [respondents] could fill in the questionnaire on their own. We wanted a solution that was simple for them to use. The software did not do all of this, but Cluetec were very open to change, and created a special version for us.

CF: “With our system we could have a very detailed look at what the person is watching or has seen, and also ask them about their attention and general feeling. This provides a very good combination and overcomes the problem of people either forgetting things or changing things. A lot of things get lost [with traditional methods]. We chose mQuest on the basis that it would be very good for our research.

CM: “The technology for respondents is very easy to use. We did a pilot and in that asked people if they would participate again and 70 per cent said ‘yes’. Only two per cent said they did not like it. The technique is amazingly stable and Cluetech have made it very robust. For example, the PDA crashes for some reason – and that usually does not happen – immediately the software starts again. This is very important if you are handing out a PDA to your respondent.

Working with this software is really easy. It’s no problem to create a questionnaire and it does not take much time to learn – you can learn what you need to make the questionnaire after one or two hours.

CF: “We have been amazed by the technology – when we started, our concerns were about whether people could work with it and if it would was stable enough to do a quick study. It was. We eventually had 200 PDAs out in the field.”

CM: “Our questionnaire is huge and contains a lot of complex filtering, but this means is not time-consuming for the respondent. It takes about a minute and a half to complete each time. But if you print it out, the questionnaire is 120 pages.”

CF: “We are definitely going to do more studies this way.

A version of this review first appeared in Research, the magazine of the Market Research Society, May 2007, Issue 492

Share This