The latest news from the meaning blog


Tim Macer talks about the future of Data Collection at ESOMAR 2015

There is a future for collecting data in market research, contrary to what some Big Data analysts may say, but that future will be very different from the data collection practiced today, said Tim Macer at the first session of the day at ESOMAR’s 2015 Congress in Dublin. Invited by research technology provider Qualtrics, Macer focused on the two disruptive influences he considers are going to reshape data collection in the future.

He recalled how, 15 or more years previously, the introduction of online research had acted as a major disrupter to established research (then predominantly CATI), although looking back, it is easy to see it now as a logical, incremental evolution. Yet now, the industry faced two significant disrupters, from mobile or smartphone participation in research – whether research companies actively allow it or not – and the extent to which Big Data and data analytics from existing data sources is likely to bypass the need to collect new data.  ‘Research cannot afford to ignore the tornado that these forces will unleash on data collection”, he said. The future, he continued, lay in providing “high frequency data” that aligns better with the rhythm of the marketplace and constant availability of Big Data.

Embracing mobile, going ‘high frequency’ and achieving better integration were the key recommendations from Macer’s presentation

“For too long, the survey has been treated as a silo, and that is no longer sustainable, and it does not need to be,” he said, urging research companies to focus on how they can integrate both their data and their information systems with those of their clients, as well as their network of suppliers and collaborators.

  • View Tim Macer’s 2015 ESOMAR presentation here.

Big data? Big problem

keyboard showing special custom buttons for social media activitiesThe Economist recently ran an article on “Big Data” in a special report on International Banking. Its assessment of banking elsewhere in the report is that the industry has been surprisingly resistant to embracing the Internet as an agent of change in banking practice. It reveals, counter-intuitively, that number of bank branches has actually risen by 10-20% in most developed economies during a period when most customers pass through their doors once a year rather than once a week.

The newspaper explains this paradox thus: banks with a denser branch network tend to do better, so adding more branches is rewarded by more business. But it’s business on the bank’s terms, not necessarily the customer’s. It does not increase efficiency – it increases cost. And, as The Economist points out, banks’ response in general to customers using mobile phones for banking has been lacklustre, even though customers love it and tend to use it to keep in daily contact with their accounts. It’s a level of engagement that most panel providers would envy.

All of which is to say that there are parallels here with our own industry. Here at Meaning, we have just released the findings of the latest annual MR Software Survey, sponsored by Confirmit. In a sneak peek, Confirmit blogger Ole Andresen focuses on an alarming finding about the lack of smartphone preparedness among most research companies.

But what interests me is the Big Data – both in The Economist’s report and our own. The former offered a fascinating glimpse into the way banks were using technology to read unstructured text and extract meaning, profiling some of the players involved and the relative strengths of different methods. This is technology which is improving rapidly and can already do a better job than humans.

In our annual survey this year, we have asked a series of questions on unstructured text. Research companies, in embracing social media, “socialising” their online panels and designing online surveys with more open, exploratory questions in them, are opening the floodgates to a deluge of words that need analysing: at least that was what we suspected.

Analysis methods cited by research companies for handling unstructured text, from the 2011 Confirmit MR Software Survey by meaning ltd

In our survey we asked a series of questions on unstructured text. Research companies – in embracing social media, “socialising” their online panels and designing online surveys with more open, exploratory question – are opening the floodgates to a deluge of words that need analysing: at least, that was what we suspected.

It turns out that half of the 230 companies surveyed see an increase in the amount of unstructured text they handle from online quant surveys, and slightly more (55%) from online qual and social media work. Yet the kinds of text analytic technologies that banks and other industry sectors now rely on are barely making an impact in MR.

Even a quick glance at the accompanying chart shows that most research companies are barely scratching the surface of this problem. It’s not the only area where market research looks as if technology has moved on, and opened a gap between what is possible and what is practised. There’s much more on this in our report, which will be publishing in full on the 30th May. Highlights will also be appearing in the June issue of Research magazine.

SurveySwipe reviewed

In Brief


Survey Analytics, USA
Date of review: February 2012

What it does

Survey platform for creating and administering mobile survey apps for participants or panel members to download to their mobile device, in order to participate in surveys.  Works across a range of mobile devices and integrates with panels.


Survey Analytics

Our ratings

Score 3.5 out of 5Ease of use
Score 4 out of 5Compatibility with other software
Score 4.5 out of 5Value for money


Standard ‘co-branded’ package for SurveySwipe mobile apps, panel, up to 30,000 members and one admin user: $8,000 annually plus $2,000 one-off set-up fee. Premium package, including custom apps, custom panel, ideation module, unlimited members and three admin users, from $23,000 annually, plus $2,000 one-off.


  • App-based surveys on Android, BlackBerry, iPhone, and Windows Phone
  • Location-triggered surveys are easy to do
  • Tightly integrated with easily-defined custom panels


  • Location-based surveys may drain participant’s phone batteries
  • Limited set of community engagement tools
  • Documentation and help files are inadequate

In Depth

Mobile research – or more specifically, self-completion surveys on participants’ smartphones – remains something of a conundrum for many professional market researchers. The opportunities it offers are tantalizing, with respondent-centric benefits such as convenience, immediacy, intimacy (as it is a more personal device) and even fun balanced by some great benefits for the researcher, including better engagement, quicker response, sharper and fuller insights, greater candor and less distortion from delayed recall – which have all been reported by practitioners. The problem is that these gains have to be paid for through a ruthless commitment to brevity. This is the brave new world of the five-to-10-question survey and it is one that calls for a fundamental rethink not just of survey design but of the technology required to support these surveys.

Seattle-based Survey Analytics is one technology provider that has embraced mobile research with gusto. Its mobile offer is styled as a solution for creating mobile communities – comprising four complementary modules for deployment to mobile devices, mobile panel and community, a mobile quali-quant ideation tool and, of course, survey management, design and analysis.

There is always the dilemma with mobile research as to whether the mobile survey should use the smartphone or tablet’s built-in browser or run as an app that the participant first needs to download. Survey Analytics lets you choose because its SurveySwipe will let you deploy your survey as an app that participants can download on to any of the four main smartphone platforms – Android, iPhone, BlackBerry or Windows Phone 7 or above, or to the device’s browser, or mix modes between handheld and desktop/laptop devices. If that is not enough, yet another program in the suite, SurveyPocket, is designed for iPads for offline data collection or where a network connection is intermittent.

Location-based services

A great strength of SurveySwipe is its use of location-based services, which means a survey can be triggered by the participant reaching a particular place – which could be a city, a retail outlet within the city or even a particular aisle within that outlet. All you need is the latitude and longitude of each trigger location and to then set the size of the active zone, which can be as little as a few tens of feet.

For location triggering to work, the participant needs to have the app on his or her phone and to have agreed to allow the app to use location services. Then, when the participant strays into the defined zone, SurveySwipe will ping an alert to the phone with a message to say there is a survey to take and cue the relevant survey within the app. It seems to be as seamless and foolproof as it can be, both for participant and for survey creator.

The drawback at present with all location-based services is they eat up the battery life of your participants’ devices when switched to the more accurate GPS mode. Cell-based location, which is kinder on the power consumption, can only help to pinpoint locations to within a mile or two.

Sophisticated tool

The survey editor is not specific to mobile surveys and can be used to design conventional online surveys too – it is a sophisticated tool with a wide range of question types and options. Routing logic, randomizations, dynamic answer list masking, text piping and most other advanced survey features seem to be well catered for. You can also start your survey off in Word and then import it. It allows you to designate various question options by putting keywords in braces within your text. This is rather fiddly in practice and the import seems to be most helpful when used simply to import very long questions or simple, unformatted text. However, this lack of focus on mobile at the editing stage means you need to plan your pocket-sized surveys very carefully and select judiciously from a range of options that do not all apply to mobile surveys. Neither does there appear to be quick, simple way to preview the survey as you are writing it to see how it is likely to appear on the target devices.

Though the editor does not make skip logic explicit, as it is largely hidden within the question where the branch occurs, there is an extremely useful diagram you can call up which reveals the logical structure of your questionnaire as a flowchart.

Overall, the survey editor is reasonably intuitive, though being Web-based, it can feel somewhat hesitant and lumpy to use, even on a fast connection. Context-sensitive help is available but it tends to be rather verbose on the obvious points and less forthcoming when more obscure information is sought. For more general questions, an FAQ approach has been taken. For such a vast and sprawling application, this is inadequate and does not do it justice. Professional users need their infrequently-asked questions answered too.

Work with a panel

The developers have rightly anticipated that most survey designers will be creating mobile surveys to work with a panel or research community. MicroPanel is provided as another integrated Web-browser-based module with the aim of making it easy to create your own custom panels or communities. It is certainly very easy to create either a one-off panel or, by adding facilities for user-contributed microblogs and polls, the panel can be run as a community. In the next release, Survey Analytics will be adding a “Badge Farm” that will allow community members to earn kudos and recognition from their contributions as well as, or as an alternative to, points.

Panels/communities can be co-branded with some limited artwork modifications to demonstrate your own identity within the standard pricing but you can pay extra for the creation of a custom fully-branded panel. The same approach applies to the survey app itself, which can either be a generic SurveySwipe app, within the standard price, which may mean sharing surveys with other researchers, or Survey Analytics will create and register a custom app with the different download sites (e.g., Apple’s App Store and the Android Market).

Researchers are increasingly finding that mobile, as a research channel, sits at the junction of quant and qual – especially with the ease by which participants can upload pictures or videos taken with their smartphone and then provide commentary or captions for these. This capability is well-supported in SurveySwipe and there is even some support for analyzing unstructured text within the Survey Analytics analysis module. But this can be taken further with another add-on module, IdeaScale, which is a co-creative idea generator that allows panelists to contribute ideas and vote on other’s ideas. The module will work both as an app on the same range of mobile devices or on a Web browser.

At this point, beyond IdeaScale, no other advanced engagement tools are offered on this platform, though more are surely bound to follow.

SurveySwipe in action

Dhaval Shah is project manager for business applications and a member of the innovation team at Ipsos Loyalty in Parsippany, N.J. He has recently guided the company through the process of implementing SurveySwipe and related technologies from Survey Analytics. “Our clients were coming out with their own mobile apps to reach their customers, especially for their retail brands,” he says. “We knew we wanted to do something similar with our own Ipsos Loyalty mobile app to help clients engage directly with customers. We also wanted to bring together the power of two-way communication and the potential of location services to create a mobile app with strong community capabilities.

“We looked to the market to identify existing technology solutions available for this kind of research. We found most companies were behind the times on location services and they did not share our vision for creating an app that would lend itself to building strong communities. But the team at Survey Analytics shared our enthusiasm for combining two-way communication and location services and they were excited about working with us to build a robust solution.”

To build this solution for Ipsos Loyalty, Survey Analytics integrated SurveySwipe with IdeaScale. It also customized back-end analysis tools to extract and process the results. “We are able to host IdeaScale within the community and trigger location-based surveys. For example, we can trigger surveys when panel members enter a particular store. Members can then take a short survey, take pictures and post them as part of their feedback. This information helps generate meaningful analytics and also helps us provide real-time feedback to the store manager.”

Asked about the reliability of the location-based triggers, Shah reports, “So far we are seeing good accuracy to within 50 to 300 feet. But we can also expand the zone to 500 feet based on the requirements of the study.”

The response from participants has also been encouraging. Panel members can choose how they want to participate – using mobile phones or via a PC. “So far, we are getting great responses on the Ipsos Loyalty mobile phone app – it’s a very useful mechanism for giving prompt feedback and people find it easy to respond immediately,” he says.

“Not only is the software performance up-to-standards, it is also easy to use while setting up surveys and administering them. Survey design and setup is largely done by the client service teams at Ipsos Loyalty, with occasional consultation from the firm’s technology group. Training new users to write surveys and administer them is an easy process and rarely takes more than a day. The solution has proved to be very productive for some of our research at Ipsos Loyalty.

“We can get a survey out to panel members within the hour and get insightful and timely results back to the client within 24-48 hours. It is a great example of what we mean by ‘point-in-time’ research,” Shah says.

Each survey engagement is carefully restricted to five minutes or less. “We can ask a couple of open-ended questions or maybe up to 10 closed questions in this time,” he says, admitting that this does restrict the kinds of research the channel is suitable for. “But with this you get immediate feedback on whatever is happening at that time. We think of it as a ‘flash mob’ survey. Combined with traditional research, this feedback can prove to be an invaluable tool to measure customer loyalty while creating a rich dialogue with customers.”

A version of this review was first published in Quirk’s Marketing Research Review in February 2012, p. 24. Copyright © 2012 meaning ltd. Reproduction prohibited. All rights reserved. 

SODA adds a dash of mobile fizz to Confirmit platform

SODA adds a dash of mobile fizz to Confirmit platform

The prize for Confirmit is a dedicated app for online/offline mobile interviews, to integrate with its other web-browser offerings

Not one but two technology acquisitions in the same week: Confirmit, providers of one of the most widely used web interviewing platforms for research agencies, announces it is buying mobile survey specialist Techneos, while Kantar Group acquires panel and technology provider GMI.

Robert Bain covered the GMI acquisition in a blog piece in Research Live. My focus here is on Confirmit and what it plans to do with SODA, Techneos’s flagship software product for both self-completion and interviewer-administered surveys on smartphones and handheld devices.

Confirmit’s chief technology officer Pat Molloy tells me: “The basic plan is the same as with Pulse Train… to bring the functions of Techneos SODA and all the best features into our Confirmit Horizons platform.” Techneos will be rebranded as Confirmit in the New Year, and Confirmit will be adding to Techneos’ development resources in Vancouver to start work on building an integrated platform. Existing SODA users will see development continue on the existing SODA platform in the short term, but eventually it will mean a switch to Confirmit for them.

“We hope we’ll do a good enough job that existing SODA users will want to move over. We are not going to mothball SODA for a considerable time,” says Molloy. “But eventually we will want to roll those customers over at no charge on to the new platform.”

Confirmit has recently added pretty advanced support for mobile research on smartphones into its flagship Horizon software. What it lacked was a dedicated app for mobile that could be run natively on the mobile device – surveys were still delivered via the device’s web browser and required a stable internet connection. Using an app offers many different benefits. I’m presenting a paper at the forthcoming ASC Conference in Bristol on the very subject (and will share the link to the paper after the event). There are pros and cons in each approach, but an app allows for much greater sophistication in survey design.

Techneos has been providing mobile apps before we even understood the term – starting with its interviewer-only Entryware product, originally designed for Palm Pilots. Techneos had appeared to lag behind on mobile self-completion until it brought out SODA at the end of 2008, which was a big leap forward.

Entryware, however, will be left to wither on the vine, on the premise that SODA provides most of its functions. This will be unwelcome news for any Entryware customers that have not yet switched to SODA, as they will be faced with two migrations.

Mobile research warrants more attention

Hands holding out a collection of mobile phones

June’s Quirks magazine is examining the state of mobile interviewing. “I don’t see a lot of researchers getting terribly excited about mobile research,” says Tim Macer, MD of meaning, in an interview in the magazine. “And I think they could perhaps be a bit more excited because there are some situations where it can really improve engagement among respondents.”

Tim also speculates on impact of Apple’s new iPad on research, the mobility of mobile research and some of the constraints that the mobile channel places on survey design.

  • Read an excerpt from the article which will be published in full in June.