The latest news from the meaning blog

 

New book on Online Panels – with a chapter from meaning MD Tim Macer

Online Panel Research book coverHow good is the quality of the access panel you are using to feed participants into your online research? How would you begin to assess quality? How can you tell good practice from bad practice? How do you create and sustain a panel that will create robust and reliable samples for market research or social research?

These are the kinds of questions that a new book published by Wiley sets out to answer. The book comprises of 19 chapters which form an encyclopaedia of the issues relating to the use and also the operation of panels for online research. These chapters were curated by a team six editors: Mario CallegaroReg BakerJelke BethlehemAnja S. GöritzJon Krosnick, and Paul J. Lavrakas and contain the contributions of 50 authors around the world with a wide range of experience and expertise in the field of online research.

Each chapter is based on original research, and in the same spirit of transparency that the book espouses in the operation of online panels, all of the datasets are also made public for anyone to use in their own research.

Lead editor and contributor to the book, Mario Callegaro said: “The book is trying to answer many questions on the quality of such data. It is amazing that online panels have been used for the past 15 years and yet there is no textbook out there. This is the first book to focus on online quality, something that everyone is struggling with.”

Managing Director of meaning ltd, Tim Macer, was asked to provide the chapter on the technology of online panels. He carried out a survey of all of the technology providers offering software or technology for operating panels, and put together a model of good practice in the technology support for panels, based around ESOMAR’s 26 Questions To Help Research Buyers Of Online Samples.

Tim Macer commented: “As Mario says, little had been published on good practice in the use of online panels, but still less was known about the software people were using, and whether they were up to the task of supporting quality panels with the kind of tools, data recording and reports required. This chapter attempts to provide a framework for panel operators, panel users and software developers to use to promote best practice.”

The book may be purchased online and individual chapters may also be purchased for download directly from the publisher.

Marsc.net 1.1 reviewed

In Brief

What it does

Web-based panel and sample management tool, based on a subset of the most useful features in the desktop/server version of the MARSC sampling software.

Supplier

MARSC Ltd

Our ratings

Score 4.0 out of 5Ease of use

Score 4.5 out of 5Compatibility with other software

Score 4.5 out of 5Value for money

Cost

Prices start from around £175 per month for a small-scale operation, plus hosting fees if required; £250-£300 per month for a mid-scale operator with multiple panels. Price determined by volumes.

Pros

  • Allows precise targeting of samples with extremely accurate incidence calculations and estimates
  • No limit on size, demographics or history kept
  • Can reuse sample jobs to draw fresh samples, create ‘favourites’  and define default sample templates
  • Provides resources for creating multiple panel members’ sites

Cons

  • Does not support non-interlocked (margin defined) quotas yet
  • Some reports rather cryptic and confusing
  • Windows and IE only for admin interface
  • Panellist portal module needs programming skills to configure

In Depth

MARSC has always been the heavyweight among the sampling and panel platforms, but now a new software-as-a-service version of the tool has emerged with the aim of lowering the bar to entry. It makes it easier to get up and running and offers something of a break on the price too, which should appeal to the smaller-scale operator.  Just how useful this is will depend on how sophisticated your requirements are, but if you need something with all the bells and whistles, the desktop and server-based MARSC is still available and being developed.

These days, most users tend to look on MARSC as a panel management tool but wasn’t always that way: MARSC started out as a sophisticated sampling tool to allow corporate research clients to draw balanced, quota-controlled samples directly from their own CRM databases.  It was a program that was in the right place at the right time when researchers realised that the most efficient way to do research online was to have a panel of pre-screened, actively managed respondents over which there was a known pedigree, both in terms of demographics and past participation.

MARSC maintains its own database of contacts and therefore, compiling and revealing all of this history is second nature to it. It is not an interviewing system – to use MARSC you will also need survey data collection platform, though it is agnostic about which one, and supports SPSS Data Collection and Confirmit directly, and many others via Triple-S.

The new .net version rationalises the process of sample selection by setting out all the options across six tabbed screens. You start by stating who you do want – referring to any of the profile variables in your panel, overall targets and incidence estimates. In the filters tab, you set exclusions, which can also be on demographics or past participation, such as people who have already received a survey invitation to another study in the last two days, or who have been interviewed on a previous wave of the same study in the last year.

In the third tab, you choose the variables you want to pass across the interviewing system, and in the fourth one you define your quotas – the quota targets you are aiming for. While you may also need to reinforce these with quotas applied in the interviewing module, a great advantage of MARSC is that, over time, it can build up very detailed response history and it will use this for each sample segment you are selecting, to predict just how much sample you need to quota target without over-inviting respondents or wasting sample.

The fifth screen handles notifications – who the reports of the sample jobs get emailed to, and last one ‘properties’ (misnamed in my view) which are not job options but the metadata for the job – the type of project, its name, who is the client and the exec, and also where you specify the reward points for participation.

All of this set-up is saved as a job, which you can give a name to and save in a folder structure that is always visible on the left of the screen. Once a job is saved it can be queued for execution, when it will draw a sample and mark them in the database as having been selected for a survey. You can also do a trial run, when it will simply report on what it would draw – a useful prior step in ensuring you do have enough sample to run with.

Saved jobs can also be stored as “favourites”– a nice web-like touch. Indeed, generally, the program has ported well to the web environment. However, the report displays could be improved as they present a mass of data and tend to use rather cryptic two-letter codes as column headers taken straight from the desktop version, whereas so much more is possible using dynamic HTML on the web.

The respondent portal is a vital part of the panel management tool, and MARSC provides a versatile portal module and set of tools for configuring it. The module is common to desktop and .net versions. In this, participants can update their profile, review what surveys are available for them to take, review and redeem incentive points and so on. The tool is designed for those with developer skills, however – sadly, there is no simple a point-and-click interface for creating or customising panellist portals, which the SaaS user is likely to expect and other systems now provide. Those without an in-house web programmer are likely to need to buy some consulting services from MARSC to get set up.

MARSC say that it is their intention in time to move all of the desktop functionality over to  the .net interface – at the moment, it lacks a handful of the more advanced features, the most serious one being the ability to interlock quotas, where an iterative model is used as a direct counterpart to using rim weighting on tables. MARSC.net won’t appeal to everyone yet, but it does go a long way to democratising efficient panel management by making it available to smaller operators without the expense of dedicated servers and teams of specialist programmers.

Client perspective: Robert Favini, Research Results, Massachusetts, USA

Research Results, a research company in Massachusetts, uses desktop MARSC to host a number of custom panels for clients in a number of consumer sectors incuding the entertainment industry.  Robert Favini, VP Client Services discusses some of the changes that MARSC has enabled.

“Originally, we had had our own internal systems with screener surveys attached which we used in a rudimentary way to pull sample, but it was a bit of a patchwork of things and wasn’t very sophisticated.  About three years ago we started to look at what other people were doing, and we came across MARSC. Shortly before that, we had also had decided to use SPSS as our main survey authoring environment, and as the two tools fit together really seamlessly, that was a big draw for us. As a result, our level of sophistication in what we can offer to our customers for managed panels has jumped up a lot.

“Our clients need something very robust because often they are doing quite a lot of analysis on the data that we provide. With our home-grown tools, the problem we were having was compatibility. This has a nice agnostic format that talks to everything.

The full implementation took place over a two-month period, including converting all of the data, though setup and training required only a few days.

“It was relatively easy to get it up and running: we had a couple of training sessions and someone from MARSC came over to work with our in-house developers. They are in the industry so they are aware of what we are trying to do and use the same terminology as us.

“About the time MARSC came along, I think the industry started using sample in a different way. Gone were the days when people would happily take surveys – we were having to use sample much more carefully. What we were looking for with MARSC was something that would use sample wisely, and let us treat it as a precious resource.”

“What we found was that as we used it, we appreciated it more and more. We like its ability to gain intelligence within the panel. We find we can target sample really precisely, and the incidence calculations it provides – the ‘guesstimates’ of how much sample to broadcast – have been phenomenally accurate. The bottom line is we are not over-using sample: we are basically being very efficient, which is where we were looking to be.”

Robert welcomes MARSC’s strategy to migrate the product to the web, although the lack of interlocked samples along with some other advanced features make it unviable for his firm yet.  “We’d still be interested in using a web-based product because of the portability it brings. Sometimes we have staff scattered all over the place and at the moment we have to use VPN to give them remote access. It would be useful for client users, but we too would like to have that bit of greater flexibility to be able to work out of the office.“

A version of this review first appeared in Research, the magazine of the Market Research Society, March 2010, Issue 526.

Globalpark EFS 7 Panel and Communities Reviewed

In Brief

What it does

Fully hosted software-as-a-service online research suite that offers a high level of performance and flexibility, with tightly integrated panel management capabilities. The panel module now offers support for online research communities

Supplier

Globalpark

Our ratings

Score 3.5 out of 5Ease of use

Score 4 out of 5Compatibility with other software

Score 4.5 out of 5Value for money

Cost

Three components: Set-up and customisation fee for panel typically £10,000-£14,000; plus, annual company-wide licence fee for survey module: £2,700 and for panel on sliding scale, from £6,830 (10,000 members) up to £20,630 (half a million or above); plus, usage fee per complete interview on a sliding scale, e.g. 49p for 10,000-20,000 in a year; 12p for 2 million.

Pros

  • A captive application for CATI interviewers and supervisors rather than a web browser interface
  • Integrated question and media library for rapid survey development
  • Works with any modern browser or OS
  • Provides a full web content management system (CMS) for multiple panel/community sites
  • Panel can work standalone with other interviewing software, e.g. for other modes

Cons

  • Online and mobile interviewing are the only survey modes supported
  • Steep learning curve
    A lot of web technical knowledge needed to fully exploit panel customisation
  • Contains quant research elements but no obvious survey workflow for quant projects

In Depth

How a panel differs from a community has become a bit of a topic among the research profession of late: how to avoid influence, whether incentives should be paid or not, or even whether the two differ at all. It’s clear that there is diversity in understanding and practice, and in introducing community support to the Globalpark EFS interviewing suite (the EFS stands for enterprise feedback management) this research software provider leaves those decisions to the individual. You could use the software to run multiple communities, multiple panels or any combination of the two, with different websites for members to use for each, and behind the scenes you may choose to keep all your panel members in one database, and segregate them logically, or physically segregate them into separate databases.

Globalpark EFS splits the task into three essential components: panel (or panels), projects and websites (the panel members’ portal). Therefore, if you had a panel of customers, and wanted to create a community of premium customers, as an elite group drawn from the panel, you could create a special website for these customers. Surveys are deployed through the respondent-facing website, and can be deployed to more than one site. They can even be skinned differently, so the survey the premium customers get be the same survey as in the general panel, but could take on a different look, consistent with the premium site’s theme. It also makes this a very appropriate pick for research companies, alongside the corporate EFM customers that Globalpark target, since panels and surveys can easily be branded for different customers or contexts.

The real power of the system is in its ability to create multiple panel and community websites, and for these sites to contain dynamic content driven from a number of sources. It means that once the site has been configured, no further technical tweaking is required, provided you do not fundamentally change the scope of what you are doing. All the routine activities such as putting surveys live, inviting panellists to participate, collecting demographics and contact updates from members, reward redemption, and the more community-oriented capabilities such as adding content to news feeds, featuring snap polls and results of surveys are simply managed through a set of attractive and straightforward control panels.

The site builder is another matter though – this is something aimed squarely at the web technician, and even then it will tax even the specialist, as there is a lot to learn and a lot of layers to work through. What Globalpark give you is a fully functioning web content management system (or CMS) which conveniently happens to understand surveys and panels. It is HTML and PHP based, browser-independent and, following best web practice, rigorously separates presentation from function. In an attempt to make it a little less complex, rather than having to write any PHP code, most text content can be written in Smarty, a text markup system. This makes it easy to pull fields from the panel database for display, and put logic into the text too.

It’s a highly accomplished implementation of a CMS and you could certainly use this software to build big fast-moving content-rich sites in which the survey activity was only a small component. It is a clever stance to take, though the trade-off is that all this flexibility is the time and expertise required to create a new site. This will not let you pop-up a new community in a couple of hours. To be fair, the people at Globalpark recognise that only a minority of customers would be able to do the configuration from scratch and tend to quote for doing the initial configuration work with new customers.

Version 7 also introduces a number new Web 2.0-style ‘community’ building blocks. Forums allow you to create threaded discussions, with members contributing responses, or optionally, defining new topics too. Whiteboards let you create a simple single-topic forum. Blogs let you turn the commenting over to your participants, who can add their own content and upload images, documents and so on. You can also feature selected blogs on the home page. Chat lets you hold one-to-one or group discussions in real time, to a limited extent, though stops well short of a full online group.

You can restrict access to forums and all the community components, so you can work with an invited subset of members only. Whenever content upload is an option, you can restrict the files you permit, e.g. only to allow JPEG images or Word documents, and the size can be limited too. It’s all very sensible, but it does not really jive yet for the qualitative researcher wanting to pull panel members into open, semi-structured research. There is no built-in workflow in the way there is for a quant survey and your data is likely to end up scattered all over the place. This needs more thinking through, and no doubt later versions will improve the situation.

However, praise must go to Globalpark for providing these features and making the software entirely DIY, if you have the skills to do the CMS configuration work behind the scenes, because many other community tools do not give you this degree of control or flexibility. You could do a lot of novel and interesting community-based and collaborative research with what this offers.

Customer perspective

Sony Music in Germany started using Globalpark EFS a year ago for a range of research activities carried out in-house using their own panel. These include new product and concept testing, as well as song cover and artist image tests for upcoming artists or newcomers. Michael Pütz, Director CRM, Web Strategy and Research explains: “We also create target group profiles, including information about media usage, which is useful for developing marketing and media plans, later on, and we use it to gain additional overall consumer insights.

“It is sometimes said that the music industry is failing to meet consumers’ needs and adapting too slowly to new business models and technologies; our activities with our online panel at www.musikfreund.de (along with other initiatives) shows evidence to the contrary. For some years now, our consumers have become regular part of a&r [artist and repertoire] and marketing decisions and our reliable partners in developing new business models and proofs of concepts.”

The market research team was therefore seeking something that would let them create well-structured and well-designed surveys and offer integrated panel management capabilities too – and to expand some of these into communities – something else EFS offered.

Mr Pütz continues: “The possibilities with EFS are huge. We are constantly challenging EFP and the Globalpark team, and they nearly always come up with good ideas on how to transfer what we want to do into solutions.” He notes in particular the ways in which Globalpark allows users to save time and improve consistency through the use of both standardized ready-made types of questions and the ability to set up a media library to make it easy to insert audio and video clips, which are fundamental to the research he does.

“The basic functionalities of EFS are easy to learn and to teach, however, configuration and tool menus of EFS can be a little bit confusing to beginners – it is not self-explanatory, which is when the help of Globalpark support teams and experts is needed.”

A version of this review first appeared in Research, the magazine of the Market Research Society, October 2009, Issue 521

Catglobe reviewed

In Brief

Catglobe version 5.5

Catinét a/s, Denmark Date of review: February 2009

What it does

Web-based software-as-a-service (SaaS) product for mixed mode data collection and analysis, including CATI, CAPI, CAWI and integrated panel management.

Our ratings

Score 3 out of 5

Ease of use Score 4.5 out of 5

Compatibility with other software Score 4 out of 5Value for money

Cost

Variable cost based upon usage. Start-up costs typically €3,500 for configuration and training then €0.015 per panel member and €0.03 per interview minute, with some additional charges applicable

Pros

  • Completely web browser-based – supports Internet Explorer or Firefox on PC or Mac
  • Simple GUI for most operations with a powerful scripting language in the background
  • Strong on panel management and sampling capabilities
  • Good range of imports and exports including Triple-S, SPSS and Excel

Cons

  • Data analysis is inflexible and limited in scope
  • GUI questionnaire editor is cumbersome to use
  • Some performance issues – complex sample queries can be slow to run

In Depth

It may seem as if we are spoilt for choice with data collection software packages, but if you are looking for a multimodal interviewing solution that is also web-based, the choice is relatively narrow – especially if web-based CATI is part of the mix. So it’s good to welcome a new entrant on the scene in the guise of Catglobe, a mixed mode interviewing system offered as a SaaS (software as a service) product by the software division of the Danish fieldwork company Catinét. The SaaS model makes it very easy to get started with little infrastructure in-house – all that is needed is a reasonable internet connection and a web browser. It is not fussy about which one: Firefox on Windows or Mac, or Internet Explorer on Windows works equally well. Catglobe is a surprisingly vast system, and the fact that it has been extensively road-tested by Catinét’s in-house fieldwork team is evident in the range of capabilities and options provided. There are different modules for sampling, questionnaire authoring, fieldwork management, reporting and some report automation. The interviewing module supports CATI, laptop CAPI and CAWI and even has a special Hall Test mode for a temporary local network of interviewing stations. All of these modules are accessed from a central home page through a pop-up menu similar to the Windows start button. Behind all of this is a single relational database which holds all of the assets or resources relating to your surveys – questionnaires, survey responses, respondents or panellists, interviewers, reports and so on. This is one of those advanced systems that moves you away from rigid boundaries of the survey to define how data are organised. The concept of the survey still exists – however more as a workflow concept. The system presents the surveys you have available to work on as a folder structure, which you can model as you wish. However, in the background, all the survey does is provide a convenient, organising view of the central data repository. Questions and response data from one survey are easily accessible from others, if you can make a connection through questions or respondents in common. This opens up endless possibilities for using your data more intelligently both in sampling and in analysis, and it makes the logistics of running one or more panels really simple. Panel management is an area Catglobe handles particularly well. At the sample selection stage, there is a wonderful tool for building ‘groups’ – which are effectively a database query. You use a group to pull a sample from the respondent database. However, this is a query tool that understands concepts such as key demographics, sample frames, frequency of previous response and interviewer resting rules. It then ties in seamlessly with the ‘communications’ module that serves invitations and reminders for web surveys. These work directly from a library of templates, so it is very quick to set up an invitation from an existing project and adapt it slightly for the survey. The system is fully multilingual, so invitations can be templated in several languages then dispatched in the appropriate one for each respondent. As it is also truly multimodal, samples can be drawn for CATI or CAWI in parallel. The workflow is well-designed, so it is not only quick to run through the process from end to end, but also flexible when changes are needed, or if the sample requires a boost part-way through the fieldwork. Panel recruitment works equally well, and there is considerable scope to automate this, including ongoing top-up recruitment. Recruitment can be by web or by telephone, and a phone recruit can be used to trigger an immediate web survey invitation for new panellists to complete their profile data. There is also an elaborate points allocation and redemption capability too, if you wish to incentivise your panel. Access to surveys and functionality is managed from the HR module, which allows you to define roles and allocate individuals to roles. Respondents and panellists are treated in the same way: everyone from the system administrator to the panel member is registered as a user and has usage rights associated with them. The majority of the system has a very cohesive appearance, which is simple to follow – it passes the test of deceiving you that you are using a desktop program, when in fact it is a browser-based web app. At the bottom of the screen are two buttons – one labelled Tools, which is the ‘start’ button that gets you to all the different modules; the other is the Folders button, which takes you to a tree view of folders containing surveys, questionnaires, templates – essentially all your data. The questionnaire editor has a somewhat different feel, and is not as well-crafted as the other modules. It does provide pretty much all of what you need, but it feels clumsy to use. You can view a list of the questions, but important details, such as the answers, can’t be seen without going into the question itself. There is no overview of the logic or routing, which makes like difficult for the scripter. There is a powerful scripting language available, and parts of the questionnaire created in the GUI can be exported out into this too, which makes writing it (and learning it) much easier. This is an excellent capability. Unfortunately, you are likely to need it more than you should if writing web or CATI interviews of even a medium level of complexity, such as creating a constrained sum set of questions. Really, more options should be built into the questionnaire editor. Also on the downside, actual users have reported sluggish performance with some of the database operations such as drawing sample or exporting results once the number of records is in the hundreds of thousands – though Catinét report that they have worked to improve this. The reporting capabilities are also unlikely to meet most users’ needs at present – there are some nice features there, but Catinét have ambitious development plans for the reporting side, so this is likely to be improved over the current year.

Customer viewpoint: Ólafur Thor Gylfason, Market and Media Research, Reykjavik

MMR moved to Catglobe a year ago, in order to move to a single interviewing platform for CATI, CAPI and web interviewing replacing a range of different packages in use to that point. As Ólafur explains: “The good thing is we have been able to use this platform for everything we do from CATI recruiting of panels to CATI phone interviews, CAPI and CAWI. “There is a powerful programming language within the software so when we do complicated surveys such as international surveys where you have to produce an exact data map afterwards, we can write the data handling programs in advance, so that when the survey is finished, we can export the data in the exact format the client requires straight away. With this programming language there is nothing you can’t do with the software, provided you have a little bit of programming experience. “Another positive thing about the software is that we use it for open-ended coding and this capability is very powerful – we can do this on the fly so that the turnaround time on projects can be reduced considerably. “We use it for CATI recruiting, and once the phone phase is completed, the automated CAWI questionnaire is sent out immediately, and everything is always interlinked, so it is very good for us. “With panel management, there are two key points. Firstly, because it is completely multimodal, and all recruitment is done by phone but recruits are immediately served a web survey to complete their profile. Everything happens at the same time. Secondly, their sampling is very easy to work with. It makes sure there is the right load across the sample and making sure that panellists get the right number of invitations, keeping track of invitations and reminders . Their ‘group builder’ is very powerful and very easy to use and the communicator, which is the email part of the system, links in with the group builder. “We have run into some problems, but the main thing for us has been that the support has been excellent – they are almost acting as a division within our company when it comes to support, so you forget about the bad things very quickly.” Published in Research, the magazine of the Market Research Society, February 2009, Issue 512

Vovici 4.0 reviewed

In Brief

Vovici Community Builder and Feedback Intelligence, version 4.0

Vovici, United States
Date of review: November 2008

What it does

Web-based suite for building online research communities and custom panels, for both quantitative and qualitative research. Allows you to create fully branded respondent community portals easily, using an online point-and-click interface. Feedback Intelligence module offers sophisticated dashboard and drilldown reporting systems for individualised reporting to stakeholders across the enterprise from multiple data sources, and integrated with Business Objects

Our ratings

Score 4.5 out of 5Ease of use

Score 5 out of 5Compatibility with other software

Score 3.5 out of 5Value for money

Cost

Annual fees in US dollars: Community Builder module starts at $24,995 for up to three named users; Enterprise Feedback Management module starts at $24,995 plus $1,500 for each portal user.

Pros

  • Standardise and co-ordinate surveys, questions and measurements across all a company’s survey activities
  • Requires no web programming or HTML skills for the most part
  • Platform independent – Windows, Mac or Linux with any modern browser
  • Integrates with Oracle CRM and a range of industry standard CRM systems

Cons

  • No built-in incentive or reward management
  • Can only execute surveys created in the Vovici EFM survey module
  • Relatively expensive

In Depth

Today more and more companies are realising the benefit of building online panels of customers to involve in research. The idea is simple, but the reality can be complex and costly to deliver from a technical standpoint. Whether firms try to build them for themselves or park the problem with a research agency, it is an area crying out for an off-the-shelf solution like the new Vovici Community Builder, which was released last month and effectively relaunches the concept of the panel management tool for the demands of Web 2.0-style research.

The product is a completely web-based suite which sits astride a database of contacts or panellists, and allows you to interface directly with the Vovici EFM survey engine as well as other enterprise platforms or CRM systems – such Siebel or Hyperion – so that sample selections can refer to real behavioural data from recent transactions for that customer. Configuring the interfaces with other enterprise data sources is, understandably, beyond the lay user, but once these have set up a customer’s purchase history can be used just like any other piece of panel profile data, such as age or location, or be used to drive sample selections for just-in-time research.

The Portal Builder

At the heart of the software is the Portal Builder, in which you design the pages of the community site your target panel members will visit. It is effectively a content management system which allows you to lay out pages with placeholders for content that will be streamed in from other sources, and which you can arrange neatly in different columns and boxes in the way most websites are organised these days. So, in the centre you could choose to put a list of the surveys the respondent is invited to some introductory text above, headlines from the current community newsletter on the left, highlights of recent survey results on the right and so on. The portal has built-in support for just about all of the objects you are likely to need to add when building a research community site: a profile editor so panellists can view and update their personal data; current survey invitations; past surveys taken; containers for welcome messages, help and links to more information or contacts. The list reaches far into the Web 2.0 milieu: you can add forums for collaborative discussion, blogs for respondents to view and react to, data mash-ups.

There is also a wealth of collaborative tools, from a simple suggestion box to access-controlled forums that can be used for asynchronous focus groups, so that quantitative surveys can be backed up by some selective qual work or vice versa. The highly modular approach means that any tool can be access controlled, and only available to invited participants. And if you are concerned that this portal page is getting a bit busy, it is easy to spread it across a series of tabbed pages, which you can title and organise how you like. There is a large template library, and it is very simple to create an overall theme with your own imagery and branding.

You can also publish results through the portal and make these relevant to the respondent – you could present each member with a report showing their answers compared to the survey as a whole, for instance, show highlights and add commentaries. Vovici emphasise this as the means to build interest and engagement, and work on the assumption that the kind of interest a community member will derive from the experience as a whole will eliminate the need to offer financial inducements. As a consequence, there is no built-in incentive and reward management capability in the product – something that will not go down well with agencies wishing to build panels.

Though the Vovici name may be unfamiliar to many, what is now branded as Vovici EFM was originally developed as Perseus EFM. The main web survey capabilities and engine are an incremental development of the Perseus EFM software which Interface reviewed in Research July 2006 when it was already a mature and capable offering for online research. Vovici has recently established sales and support offices in London and Singapore alongside three existing locations in the United States.

The other major addition since Vovici took over is in reporting. There is now a dashboard reporting system largely in place, with some development ongoing. It follows a similar philosophy to the Community Builder by allowing you to arrange graphical and tabular reports across the screen in columns and rows – as designer, you choose what reports to show simply by pointing and clicking, selecting them from menus and so on. Again, the overall appearance is controlled by externally defined templates and stylesheets, so the entire reporting experience can be themed and branded to match a corporate intranet site.

Reporting in Business Objects

The reporting system is, in fact, built on Business Objects (using Crystal Reports), which is a widely used reporting tool in the mainstream corporate database and business intelligence sector. However, Business Objects is typically of limited use with survey data, because it does not understand common survey concepts such as multiple-response data, respondent bases that may differ from the number of responses to a given question, or one-off data formats for each short ad hoc survey. The breakthrough with Vovici is that the developers have created a data model and accompanying metadata to make research data comprehensible to Business Objects.

The beauty of this is that any reporting can be a composite of hard commercial data alongside softer attitudinal and intentional survey data. Questions too can be analysed across different surveys. By smashing through the old silo approach, Vocivi is also working towards delivering true benchmark capabilities. The idea is that any question can be reused across any survey, and once the same question has been reused, all responses to it can be used to provide a benchmark, or by filtering that benchmark to provide sector-specific comparisons.

Enterprise Feedback Management providers like Vovici are probably more aware than most MR software suppliers that their products will appeal to both the corporate user wishing to do their own research, and the research agency – and the platform lends itself to collaborative working between client and supplier. For example, the community portal and interfaces with corporate data sources could all be under the responsibility of the corporate client, while the creation of actual surveys and the preparation and publishing of results can be contracted out to one or more research suppliers, using the same platform.

This is a vast system with massive potential, which in its very design reveals some research-literate minds were behind it. Users I spoke to report that that the community functionality is stable, reliable and relatively easy to learn, and has enabled users to standardise and systematise their research and harmonise measures across very large enterprises. Perhaps the product’s greatest strength is in its ability to integrate with CRM systems and other business intelligence sources, making research more relevant and mainstream within the corporate enterprise.

A version of this review first appeared in Research, the magazine of the Market Research Society, November 2008, Issue 509