The latest news from the meaning blog

 

Tim Macer talks about the future of Data Collection at ESOMAR 2015

There is a future for collecting data in market research, contrary to what some Big Data analysts may say, but that future will be very different from the data collection practiced today, said Tim Macer at the first session of the day at ESOMAR’s 2015 Congress in Dublin. Invited by research technology provider Qualtrics, Macer focused on the two disruptive influences he considers are going to reshape data collection in the future.

He recalled how, 15 or more years previously, the introduction of online research had acted as a major disrupter to established research (then predominantly CATI), although looking back, it is easy to see it now as a logical, incremental evolution. Yet now, the industry faced two significant disrupters, from mobile or smartphone participation in research – whether research companies actively allow it or not – and the extent to which Big Data and data analytics from existing data sources is likely to bypass the need to collect new data.  ‘Research cannot afford to ignore the tornado that these forces will unleash on data collection”, he said. The future, he continued, lay in providing “high frequency data” that aligns better with the rhythm of the marketplace and constant availability of Big Data.

Embracing mobile, going ‘high frequency’ and achieving better integration were the key recommendations from Macer’s presentation

“For too long, the survey has been treated as a silo, and that is no longer sustainable, and it does not need to be,” he said, urging research companies to focus on how they can integrate both their data and their information systems with those of their clients, as well as their network of suppliers and collaborators.

  • View Tim Macer’s 2015 ESOMAR presentation here.
SODA adds a dash of mobile fizz to Confirmit platform

SODA adds a dash of mobile fizz to Confirmit platform

The prize for Confirmit is a dedicated app for online/offline mobile interviews, to integrate with its other web-browser offerings

Not one but two technology acquisitions in the same week: Confirmit, providers of one of the most widely used web interviewing platforms for research agencies, announces it is buying mobile survey specialist Techneos, while Kantar Group acquires panel and technology provider GMI.

Robert Bain covered the GMI acquisition in a blog piece in Research Live. My focus here is on Confirmit and what it plans to do with SODA, Techneos’s flagship software product for both self-completion and interviewer-administered surveys on smartphones and handheld devices.

Confirmit’s chief technology officer Pat Molloy tells me: “The basic plan is the same as with Pulse Train… to bring the functions of Techneos SODA and all the best features into our Confirmit Horizons platform.” Techneos will be rebranded as Confirmit in the New Year, and Confirmit will be adding to Techneos’ development resources in Vancouver to start work on building an integrated platform. Existing SODA users will see development continue on the existing SODA platform in the short term, but eventually it will mean a switch to Confirmit for them.

“We hope we’ll do a good enough job that existing SODA users will want to move over. We are not going to mothball SODA for a considerable time,” says Molloy. “But eventually we will want to roll those customers over at no charge on to the new platform.”

Confirmit has recently added pretty advanced support for mobile research on smartphones into its flagship Horizon software. What it lacked was a dedicated app for mobile that could be run natively on the mobile device – surveys were still delivered via the device’s web browser and required a stable internet connection. Using an app offers many different benefits. I’m presenting a paper at the forthcoming ASC Conference in Bristol on the very subject (and will share the link to the paper after the event). There are pros and cons in each approach, but an app allows for much greater sophistication in survey design.

Techneos has been providing mobile apps before we even understood the term – starting with its interviewer-only Entryware product, originally designed for Palm Pilots. Techneos had appeared to lag behind on mobile self-completion until it brought out SODA at the end of 2008, which was a big leap forward.

Entryware, however, will be left to wither on the vine, on the premise that SODA provides most of its functions. This will be unwelcome news for any Entryware customers that have not yet switched to SODA, as they will be faced with two migrations.

Read the results: Globalpark Annual MR Software Survey 2009

We’re excited to be able to share with you the results of the Globalpark Annual Market Research Software Survey, which is published today. We have been conducting this international survey – which looks at software and technology used in the market research industry – every year since 2004, the results of which we make freely available to all.

Pie chart showing the perceived viability of moble self-completion research

In 2009 we introduced some new questions about mobile research and communities.  These reveal the MR industry is split over the merits of self-completion mobile-based research, with 45% seeing it as viable 48% only seeing it as close to becoming viable (and not really viable at present) with a further 7% never expecting it to be viable (chart to left). It is also very striking, that large companies have a far more optimistic opinion about the viability of mobile research.

Our questions on communities also proved to be interesting. For example, we learnt that only one in six research firms was operating a community in 2009, and those that do are operating very few communities.

Four- and five-year trends

Chart showing the growth of the mixed-mode integrated platform

Over the years, our tracker questions have revealed some interesting trends. For example, we ask respondents whether they use an integrated platform for their multimode research or whether they switch between modes (chart to right). We first asked this question in 2006 and since then we have detected a gradual but consistent trend to wards integrated platforms. In another question we found that nearly all respondents (84%) thought that multimode data collection was either essential, very important or moderately important when choosing a new data collection tool. Software developers, please take note!

Member Login

Confirmit Horizons Reviewed

In Brief

What it does

A complete mixed mode interviewing system with high-quality support for CATI, CAPI and web interviewing, integrated with panel and community management, and telephony integration for CATI, operating in a web-based or cloud computing environment.

Supplier

Confirmit

Our ratings

Score 5 out of 5Ease of use

Score 4.5 out of 5Compatibility with other software

Score 3.5 out of 5Value for money

Cost

Confirmit Horizons CATI hosted solution. Entry-level system from £8,000 per year. Web surveys and other pricing on application.

Pros

  • Captive application for web CATI interviewers and supervisors (rather than a web browser interface)
  • Switch easily from Web to CAPI or CATI and back again
  • Clean, modern and customisable look throughout
  • Open system capabilities through a range of API extensibility kits

Cons

  • Sample loading capabilities are rudimentary
  • No offline scripting capabilities

In Depth

There is a new professional telephone interviewing system on the block with a surprising name to those familiar with it – Confirmit. Confirmit Horizons, released earlier this year, extends this granddaddy of a Web survey platform into the phone room and also the street and shopping mall, with full online/offline CAPI too.

Presumptions that CATI would dwindle away as an interviewing channel, in the face of Internet research have so-far been wide of the mark, rather like those predictions about long-awaited paperless office. Industry stats show decline, but CATI is far from collapsing. For many years Confirmit (in its earlier FIRM days) appeared to procrastinate over whether it would or would not introduce CATI into its Web survey offering. Then two years ago, it bought up Pulse Train, which appeared to have two jewels in its crown – a superior reporting platform, Pulsar, and a real workhorse of a CATI system, Bellview. The combined firm set about merging together the two product lines – an initiative that is not for the feint-hearted.

In a surprisingly short period of time, a merged set of tools for data collection has emerged, and in doing so, the Confirmit product line has matured into a very comprehensive offering for the whole spectrum of quant research. The manufacturer has clearly learned from the mistakes of others. First, there is an upgrade route for Bellview customers to convert their legacy QSL scripts into Confirmit, and secondly, the revised platform is not a hotch-potch of legacy modules and new modules – the capabilities of the old Bellview system have almost all been reproduced within Horizons without compromising the Confirmit environment or way of working.

Bellview was a very flexible system for CATI, giving a lot of control to supervisors over the management of interviewers, sample and callbacks through its admin interface and offered limitless possibilities to the script writer. Scripts were written either using a proprietary script language called QSL or a graphical (GUI) authoring tool called Visual QSL — though in reality, most people wrote QSL syntax. Confirmit, on the other hand, has always been a GUI system, and the developer took the brave decision to go with GUI alone. There are those who will defend the syntax approach to the bitter end on the grounds that it is ‘more efficient’ and quicker to do certain tasks. But QSL was quirky and took time to learn – and it was always easier to make mistakes than it was to notice them and correct them.

The Confirmit authoring GUI is, in practice, highly efficient and the developer has worked on optimizing it. There is often a fear that a Web-based editing interface will be sluggish in operation. Not so with Horizons – moving from screen to screen is instant and effortless. There will be a fairly steep learning curve for anyone experienced in QSL that has not worked with Confirmit, as the design interface is completely different.

The legacy code bridge will not convert everything – it seems to get you around 80 to 90 per cent of the way to a working CATI script in Confirmit, and it deals with all of the more tedious aspects of converting texts, even in multiple languages, variables and logic, but you will probably have to patch up any complex execution logic or clever scripting by hand. Another bonus is that a lot of the concepts are very similar between the two systems, such as the block-structured skip logic, and those that were missing from Confirmit (such as QSL’s ‘pblocks’ for handling beginnings, endings and handover from one mode to another) all now appear in Confirmit as ‘call blocks’. QSL users will, however, miss some of the syntax tricks that were possible – the Confirmit interface is not as slick as QSL in writing logic and performing operations on variables.

Better-looking CATI

There is no separate CATI-specific module – all of the interviewing and supervision capabilities are also implemented now through the main Confirmit platform, which has been skilfully enhanced to provide a true CATI experience to users, within a Web-based environment. However, to avoid the problem of Interviewer and Supervisors having to work in a Web browser, both of these interfaces are provided as dedicated applications, or ‘consoles’. This is easily set up on Interviewer machines, or if interviewers and supervisors are working remotely, they can be emailed a link to download and install the package.

CATI interviewer screens now take on a very web-like appearance – you can control their exact look and feel through the same template gallery as a Web interview, and there are some templates optimised for CATI too. However, the console environment also allows interviewers to control all aspects of the interview from the keyboard, rather than the mouse, which is typical in a Web interview – though they can also use the mouse if they wish.

Clever reporting

The web environment certainly improves the look of the CATI screen – they can be branded, for instance, with the clients logo, or made distinctive for different projects. These are not merely cosmetic matters – theming the screens can help with project recognition, and better screen design can aid concentration and legibility. They can also contain many more answer options.

The supervisor console similarly packs a punch, with added style, for supervisors. Again, how it does things is a little different for those familiar with Bellview, but the concepts are similar, and a lot of effort has gone into making the features intuitive to use. A real innovation here is the alerting report – triggered by events such as an interviewer not entering any data, or an appointment falling overdue. This is potentially a big money-saver for CATI units, as it makes it feasible for a single supervisor to supervise 15 or more interviewers as effectively as 6-10 with a more conventional system. This could even free up a few monitoring stations for interviewer use instead.

The next day CATI centre

Telephony integration is also possible through Magnetic North, a telephony specialist, with interfaces for both conventional telephone lines or VOIP. It is a web-based dialler, which means it will support either conventional call centres or satellite offices and homeworkers. When an interviwer logs in, they are asked to enter the number of the phone on their desk, and a call is placed to that number. The line is then kept open, and the dialler then starts placing calls. The dialler will support every level of automation from click-to-dial through to full predictive dialling, with the supervisors able to monitor and control the nuisance call rate through the alerting reports.

The telephony support also allows for recording of chosen segments of the interview, such as an open-ended question, or the entire interview. The web telephony also means that you could set up a real or a virtual call centre using the cloud computing model pretty much overnight.

Where does it end?

The sheer depth of functionality becomes apparent when you take a look at the other interfaces offered, to allow you to integrate Confirmit with other software. The software is developed using Microsoft SQL Server and using the Microsoft .NET framework, which already makes integration relatively straightforward for other Microsoft users. Confirmit have also embraced Web Services with a passion – this allows applications to exchange data and drive one-another across the Internet, effectively going in through the back-door but in a highly secure and also efficient way. There are currently nine different APIs for such touch-points as reporting, data transfer, quotas and panel integration. There is also an ‘extensibility framework’ to make it easy to develop specialist applications within Confirmit. One has already been designed by a third party for fraud detection and another is being developed to support interviewing on iPhones.

It is going to be hard in future for the humble reviewer to find anything that this software cannot do, since there are likely to be many more of these third party plug-ins. Indeed, the few niggles I do have are relatively minor in scope – sample management is still a bit too Web interview-oriented, and you have to be a Windows user to exploit the software. Confirmit are cagey about the pricing too, but it appears to be towards the top end.

Confirmit have pulled off quite a feat with Horizons – it’s a serious product with a friendly face that makes for a hard act to follow.

A version of this review first appeared in Research, the magazine of the Market Research Society, September 2009, Issue 520.

Catglobe reviewed

In Brief

Catglobe version 5.5

Catinét a/s, Denmark Date of review: February 2009

What it does

Web-based software-as-a-service (SaaS) product for mixed mode data collection and analysis, including CATI, CAPI, CAWI and integrated panel management.

Our ratings

Score 3 out of 5

Ease of use Score 4.5 out of 5

Compatibility with other software Score 4 out of 5Value for money

Cost

Variable cost based upon usage. Start-up costs typically €3,500 for configuration and training then €0.015 per panel member and €0.03 per interview minute, with some additional charges applicable

Pros

  • Completely web browser-based – supports Internet Explorer or Firefox on PC or Mac
  • Simple GUI for most operations with a powerful scripting language in the background
  • Strong on panel management and sampling capabilities
  • Good range of imports and exports including Triple-S, SPSS and Excel

Cons

  • Data analysis is inflexible and limited in scope
  • GUI questionnaire editor is cumbersome to use
  • Some performance issues – complex sample queries can be slow to run

In Depth

It may seem as if we are spoilt for choice with data collection software packages, but if you are looking for a multimodal interviewing solution that is also web-based, the choice is relatively narrow – especially if web-based CATI is part of the mix. So it’s good to welcome a new entrant on the scene in the guise of Catglobe, a mixed mode interviewing system offered as a SaaS (software as a service) product by the software division of the Danish fieldwork company Catinét. The SaaS model makes it very easy to get started with little infrastructure in-house – all that is needed is a reasonable internet connection and a web browser. It is not fussy about which one: Firefox on Windows or Mac, or Internet Explorer on Windows works equally well. Catglobe is a surprisingly vast system, and the fact that it has been extensively road-tested by Catinét’s in-house fieldwork team is evident in the range of capabilities and options provided. There are different modules for sampling, questionnaire authoring, fieldwork management, reporting and some report automation. The interviewing module supports CATI, laptop CAPI and CAWI and even has a special Hall Test mode for a temporary local network of interviewing stations. All of these modules are accessed from a central home page through a pop-up menu similar to the Windows start button. Behind all of this is a single relational database which holds all of the assets or resources relating to your surveys – questionnaires, survey responses, respondents or panellists, interviewers, reports and so on. This is one of those advanced systems that moves you away from rigid boundaries of the survey to define how data are organised. The concept of the survey still exists – however more as a workflow concept. The system presents the surveys you have available to work on as a folder structure, which you can model as you wish. However, in the background, all the survey does is provide a convenient, organising view of the central data repository. Questions and response data from one survey are easily accessible from others, if you can make a connection through questions or respondents in common. This opens up endless possibilities for using your data more intelligently both in sampling and in analysis, and it makes the logistics of running one or more panels really simple. Panel management is an area Catglobe handles particularly well. At the sample selection stage, there is a wonderful tool for building ‘groups’ – which are effectively a database query. You use a group to pull a sample from the respondent database. However, this is a query tool that understands concepts such as key demographics, sample frames, frequency of previous response and interviewer resting rules. It then ties in seamlessly with the ‘communications’ module that serves invitations and reminders for web surveys. These work directly from a library of templates, so it is very quick to set up an invitation from an existing project and adapt it slightly for the survey. The system is fully multilingual, so invitations can be templated in several languages then dispatched in the appropriate one for each respondent. As it is also truly multimodal, samples can be drawn for CATI or CAWI in parallel. The workflow is well-designed, so it is not only quick to run through the process from end to end, but also flexible when changes are needed, or if the sample requires a boost part-way through the fieldwork. Panel recruitment works equally well, and there is considerable scope to automate this, including ongoing top-up recruitment. Recruitment can be by web or by telephone, and a phone recruit can be used to trigger an immediate web survey invitation for new panellists to complete their profile data. There is also an elaborate points allocation and redemption capability too, if you wish to incentivise your panel. Access to surveys and functionality is managed from the HR module, which allows you to define roles and allocate individuals to roles. Respondents and panellists are treated in the same way: everyone from the system administrator to the panel member is registered as a user and has usage rights associated with them. The majority of the system has a very cohesive appearance, which is simple to follow – it passes the test of deceiving you that you are using a desktop program, when in fact it is a browser-based web app. At the bottom of the screen are two buttons – one labelled Tools, which is the ‘start’ button that gets you to all the different modules; the other is the Folders button, which takes you to a tree view of folders containing surveys, questionnaires, templates – essentially all your data. The questionnaire editor has a somewhat different feel, and is not as well-crafted as the other modules. It does provide pretty much all of what you need, but it feels clumsy to use. You can view a list of the questions, but important details, such as the answers, can’t be seen without going into the question itself. There is no overview of the logic or routing, which makes like difficult for the scripter. There is a powerful scripting language available, and parts of the questionnaire created in the GUI can be exported out into this too, which makes writing it (and learning it) much easier. This is an excellent capability. Unfortunately, you are likely to need it more than you should if writing web or CATI interviews of even a medium level of complexity, such as creating a constrained sum set of questions. Really, more options should be built into the questionnaire editor. Also on the downside, actual users have reported sluggish performance with some of the database operations such as drawing sample or exporting results once the number of records is in the hundreds of thousands – though Catinét report that they have worked to improve this. The reporting capabilities are also unlikely to meet most users’ needs at present – there are some nice features there, but Catinét have ambitious development plans for the reporting side, so this is likely to be improved over the current year.

Customer viewpoint: Ólafur Thor Gylfason, Market and Media Research, Reykjavik

MMR moved to Catglobe a year ago, in order to move to a single interviewing platform for CATI, CAPI and web interviewing replacing a range of different packages in use to that point. As Ólafur explains: “The good thing is we have been able to use this platform for everything we do from CATI recruiting of panels to CATI phone interviews, CAPI and CAWI. “There is a powerful programming language within the software so when we do complicated surveys such as international surveys where you have to produce an exact data map afterwards, we can write the data handling programs in advance, so that when the survey is finished, we can export the data in the exact format the client requires straight away. With this programming language there is nothing you can’t do with the software, provided you have a little bit of programming experience. “Another positive thing about the software is that we use it for open-ended coding and this capability is very powerful – we can do this on the fly so that the turnaround time on projects can be reduced considerably. “We use it for CATI recruiting, and once the phone phase is completed, the automated CAWI questionnaire is sent out immediately, and everything is always interlinked, so it is very good for us. “With panel management, there are two key points. Firstly, because it is completely multimodal, and all recruitment is done by phone but recruits are immediately served a web survey to complete their profile. Everything happens at the same time. Secondly, their sampling is very easy to work with. It makes sure there is the right load across the sample and making sure that panellists get the right number of invitations, keeping track of invitations and reminders . Their ‘group builder’ is very powerful and very easy to use and the communicator, which is the email part of the system, links in with the group builder. “We have run into some problems, but the main thing for us has been that the support has been excellent – they are almost acting as a division within our company when it comes to support, so you forget about the bad things very quickly.” Published in Research, the magazine of the Market Research Society, February 2009, Issue 512