The latest news from the meaning blog

 

Qi from Manthan Services reviewed

In Brief

Qi

Manthan Services, India
Date of review: August 2012

What it does

Online platform for creating advanced dashboards based on survey which delivers to the end user an online environment for data exploration, review and collaboration.

Our ratings

Score 3 out of 5Ease of use

Score 4 out of 5Compatibility with other software

Score 4 out of 5 Value for money

Cost

SaaS with annual subscription based on volumes. Example cost $8,000 for up to 5 projects (approx. 5,000 cases and 250 variables) with discounts available for higher volumes.

Pros

  • Very comprehensive offering
  • Understands the specifics of market research data
  • Focus on collaboration and knowledge sharing
  • Takes care of any complex web- and database programming

Cons

  • Works on IE8 and IE9 but some formatting experienced on other browsers
  • Online documentation/help is fairly basic
  • Set-up requires some skill

In Depth

Dashboards tend to be among the most advanced and also the most treacherous of deliverables for research companies to provide. Tucked away at the end of an RFP, an innocuous-sounding request for “dashboard-style reporting for managers and team leaders across the enterprise, with drill-down capabilities for self-service problem solving” will almost certainly mean something vastly more sprawling and costly to provide than anyone imagined.

Dashboard delivery can be a trap for the unwary. Many an online dashboard has become the constantly-leaking plughole in the project budget through which profits keep draining away.

What makes them difficult to control is they are usually tackled as custom developments, built using tools developed for corporate database systems and business intelligence (BI) tools. Any custom development is both costly and unpredictable and research companies often don’t have the skills in-house to manage a software development project effectively. Worse than that, survey data is difficult to handle with these BI tools. They aren’t designed to function smoothly with monthly waves of data, with new questions added or weighting or percentages that need to add to a constant respondent base. It’s not just a matter of generating the number of records returned from a SQL query.

Manthan Services, an India-based developer, noticed the opportunity to build on the dashboard and business information systems it was providing corporate customers and developed a research-friendly package called Qi (as in “chi” or energy). An online platform for creating advanced dashboards based on survey data, Qi delivers an online environment for data exploration, review and collaboration. It is a tool for building dashboards and an environment in which end-users can then access those dashboards, share, collaborate and even, if allowed to, create their own analyses and dashboards.

It is very smart software that aims to find the middle ground between typical BI dashboard tools like SAP Crystal Dashboard Design (the new name for Xcelsius) and Tableau, where the possibilities are infinite, given enough time and money, and the fairly restrictive kinds of online dashboard creation capabilities found in some of the more up-to-date MR analysis tools. If you really want to produce any kind of dashboard, or have a client that is highly prescriptive about presentation, then you may find Qi is just not flexible enough.

On the other hand, you may be able to use the horizons as a useful limiting factor in what you do provide to your client, as it is likely to do 99 percent of what they need – just not necessarily in the first way they thought of it. For the real advantage of using this product is that you really can produce portals packed with data with relatively little effort and no programming expertise required. Furthermore, when you add new waves of data, all of their derivative reports will be updated too.

There are also built-in modules within the Qi environment to set up different kinds of dashboards or portals for certain applications. There is one for employee research, for example, and another for mystery shopping, with reporting at an individual case level. In addition, there are models provided for performance management, scorecarding and benchmarking. There is also a tool for building an organization hierarchy and this can then ensure each user is given the relevant view of the data when they log in. These can be tied to “group filters” which reflect the organization’s hierarchical structure in the actual data that get displayed.

There is an integrated alerts publisher and a user’s portals can be configured with an alerts area or tab. You then define the exceptions or thresholds where alerts should be generated. These are then recalculated for each individual user’s view of the data so they are only alerted on what is relevant to them.

Elegant concepts

There are some very elegant concepts at the heart of Qi which help to give your work shape. Everything you create is based on one of three “assets” based on data: charts, dashboards and tables. Dashboards come in a variety of shapes with placeholders for you to populate with charts or tables. There is also the concept of a “portlet,” which can house a report, an alert, a chart, favorites or messages. You can then arrange your portlets into pages or publish them on their own.

There is a reasonable though not especially exotic selection of charts – pretty much what you might find in Excel. There are, however, some nice multidimensional bubble charts.

Behind the scenes is a SQL Server database. It can be loaded with survey data using the survey metadata provided by either SPSS or Triple-S. If you want to work with other kinds of data – which is possible – you may need to get help from Manthan Services in setting up an appropriate database schema, however, and also help with the database load process.

A particular snare to be found in many RFPs asking for dashboards is the request for drill-down capabilities. There is often an assumption that deciding what to drill down to is a trivial, automatic choice. It is not – there is often more than one level of detail a user is likely to want to see when a particular KPI turns red or a trend chart shows a worrying dip. In Qi, you have two tools to satisfy this: a drill-down tool that lets the user trace the antecedents or components of any item of data and a drill-across tool which lets you move up and across in your hierarchy of reporting.

End users are provided with a lot of options out of the box to personalize their dashboards – they can create favorites, apply sticky notes, customize the view of the data, create their own portlets (if you allow this) and republish or share these with others. It can make for a highly collaborative environment both within the enterprise, and equally, between enterprise and research agency.

Overall, this is an industrial-strength platform for research companies to use to create portals and dashboard systems with a dizzying array of functionality to pick from. The documentation could be made a lot more comprehensive – it is cryptic in places and tends to gloss over some quite advanced capabilities. I also experienced some issues viewing the portals I was given access to on any browser on IE8 or IE9, though Manthan claims it works with different browsers and tablets.

Same set of tools

Max Zeller is head of the retail insights division for a large global research company in Europe. (His name has been changed at the request of his employer.) His division introduced a white-label version of Qi last year, which it presents to its customers as one of its own branded services. “Many of our clients today require online reporting,” he says. “As a global company we wanted to offer the same set of tools to all clients and also leverage on the one investment across all our companies and for most of our studies. We also wanted something that you could implement quite quickly locally, to create portals and dashboards, which did not require any programming or special skills to run it. Also we wanted a tool that both researchers and users could modify and even create their own views or dashboards for themselves.

“We looked at many different products but eventually chose one from Manthan Services. On all criteria they were on top and they understood market research, which was very important.”

Though the software is very extensive, with quite a lot to learn, he says, in practice his firm’s research and DP teams have found it well within their capabilities to deploy it. “The people in contact with the client – the project managers supported by DP staff – do the technical and setup work. You need someone in the team that champions the product who can translate the requirements of the client in terms of how the software is going to work. Then it can be more junior DP people who do the implementation, because it is all menu-driven – which gives them a new opportunity as well.”

Zeller estimates that setting up a new portal for a client demonstration, comprising 25 different charts and allowing different levels of access, can be achieved in a day or so by his local teams – a pace that was new for the company. “Before this we had to go though IT and the process was not just longer but so much more expensive. It would have taken several days to a week with what we had before. We need to be as lean, as quick and as close to the client as possible – and that’s exactly what we have here. You can give the specs from the client directly to the team – you don’t really have to translate the requirements into a technical specification and that is what saves the time and delay.”

Zeller strongly advises allowing adequate time to learn to use the software, however. “This is not something you can jump into in an hour – it does take two intensive days of training. But overall, I think the trade-off between functionality and ease of use is good. Once you are accustomed to the software it is easy and productive to use.”

He also stresses that everyone, especially those setting client expectations, must be aware that this is a packaged solution. In other words, not all client requests may be achievable. “[When speaking with clients] you need to be aware of what you can and can’t do. Even though it is very flexible, it is working to a standardized framework. There are many things you find you have not thought of first and when you try, you discover there is a way to do it. But it is not fully customizable so there are some areas you cannot change.”

However, in these cost-conscious times, some imposed limits can be an advantage, as Zeller points out: “It is very difficult for research companies to earn money from these portals if what you are doing fully customized.”

Overall, he concludes, “We are quite happy with this software – and I am working with people who have a lot of experience. We think it is a good solution.”

A version of this review first appeared in Quirk’s Market Research Review, January 2013 (p. 28)

Beyond PowerPoint

Detail from Tim Macer's ASC 2010 presentation

While detractors have been denouncing PowerPoint as a vehicle for presenting research results for several years now, MR seems to be as wedded to it now as at any time in the past. It’s a topic Tim Macer, MD of meaning ltd explored at the  ASC conference in London “Putting the Pizzazz into Research: renewing the rules of engagement“, in the event’s closing talk “It doesn’t have to be PowerPoint”. In the presentation, Tim observed that PowerPoint is a provided as a client deliverable on 54% of research projects, and examined some of the problems that critics have identified in the use of PowerPoint. He then offered six “antidotes to PowerPoint.

Tim introduced these by explaining: “These are my subjective pick of software products I have seen over the past year where I’ve seen providers offering something that is useful and different in this area. ” Tim’s presentation was given with the aid of  one of the featured tools – Prezi – a new and quite ground-breaking presentation tool.

Infotools Viewers and Consoles reviewed

In Brief

What it does

A collection of graphical data delivery tools, based around a single interactive and highly responsive  for integrating streams of research and non-research data. May be used as an alternative to dashboard reporting.

Supplier

Infotools

Our ratings

Score 5 out of 5Ease of use

Score 4.5 out of 5Compatibility with other software

Score 3 out of 5Value for money

Cost

Per project. Viewers, from £1000 for a one-off project, £2000 annually for trackers. Unlimited users, includes Espri data analysis tool. Consoles: from £20,000.

Pros

  • Integrates diverse data streams into a single interactive view
  • Compatible with any web browser
  • Compelling style of presentation makes complex data highly accessible

Cons

  • Architecture an interface places limits on the number of variables or dimensions that can be included
  • No tool to build databases: only in conjunction with Infotools’ data import services
  • Expensive for small-scale uses.

In Depth

Simplicity is the hallmark of good design, whether it is for a mobile phone, a website or a piece of software. It is something we are surprisingly haphazard at achieving in market research. Compared to other kinds of corporate data, survey data tends to be complex, and the analysis and reports that come out of the other end all too often reinforce rather than reduce complexity by stripping it back to the essential. For New-Zealand based MR software provider Infotools, the aim of a new range of data reporting tools, which they call simply Viewers, is to pare back on clutter, and bring radical simplicity to data analysis.

In this initial release, there are four different Viewers to choose from, InfoPlot, InfoSwitch, InfoTrend and InfoWorld.  Each follows the same principles and uses a single chart, accompanied by a small amount of tabular data, to present a related series of measures.  These would typically be brands or product categories. Each Viewer is aimed at presenting particular kinds of data. InfoSwitch, for example, presents brand switching behaviour. They are all minimalist Web 2.0 products – users are hardly aware they are using software. Instead, they interact with a single webpage containing a chart, sometimes a few figures too and some lists of hyperlinks to other items.
The clean design approach means there are no legends with the charts. Each bar, line or pie slice always has its own label, and the software determines that the labels are always visible, even on quite crowded charts.  For instance, in the InfoTrend viewer, mouse over a trend line, and it will highlight it, and the others will shade out. Click on the trend line, and it will split it at that point, and redraw the chart to show trends before and after that point.

All across the range of tools, the charts respond intuitively to pointing, clicking and dragging, in ways that avoid the need for any kinds of explicit controls. There are no lists of properties, no menu of options, no clutter: all the control is ceded to the actual data being viewed.  Other items are listed on screen to allow you to vary the view. These are organised into are measures and demographics. Measures could be from the same survey, e.g. brand awareness, affinity or association, but very powerfully, they could be from other sources: marketing spend, retail performance and so on, all scaled to the same axis. You just pick the ones you want to see together.
Other viewers offer different views. InfoWorld presents data plotted over a world map, and maps covering other geographic regions are also feasible. InfoPlot brings to live two-dimensional plots or perceptual maps, with the same highly intuitive approach to encourage experimentation.

Brand managers could get very interested in InfoSwitch – if data are available on brand loyalty, previous brands or alternative brands used. This shows each brand as a sphere orbiting other brands, and the strength of the links between them, with all the Viewer possibilities of moving items around, filtering them and focusing on subgroups, to really understand the interplay of brands, as consumers divide or shift their loyalties.
The Viewers offer great flexibility in deployment. You could build them into an intranet, publish them on a website or you could email links to recipients for them to download a package they can run locally on their laptop without for a permanent internet connection. The technology is built on Microsoft’s excellent Silverlight technology ¬– something we are going to be seeing much more of, in software applications for MR. It will run on any browser and any platform.

Infotools also offer what they call Consoles, which are bespoke data delivery portals based on one or more customised Viewers – they are deliberately avoiding calling these dashboards, though the result is definitely dashboard-like in its approach.

Is there a catch? Well, there are potentially two. For many, a deterrent is that both Viewers and Consoles are not offered as standalone software – they are tied into Infotools’ service offer. Only Infotools can build the underlying database and populate your Viewer with your data. However, the database is in the same structure as for its full-blown Espri cross-tab and analysis tool. They will throw in a copy of this for free if you are using a Viewer. This has the advantage that the numbers in published data and any subsequent ad hoc requests are likely to agree, as they come from the same data source.

The other catch is the constraint imposed by the interface on the number of variables or dimensions you can include. There is space for around 70 items, but as this includes all measures and all the categories of any demographics, you could struggle to accommodate all you need from a sizeable tracker. However, even here, this discipline could be an advantage, as it forces you to focus on the essential.  After all, some do say that less is more.

Customer viewpoint: Othman El Ouazzani, The Coca-Cola Export Corp., Casablanca, Morocco

The Coca-Cola Export Corporation, North & West Africa Business Unit has implemented an Infotools Console based around the new Viewer technology. Knowledge & Insights Manager Othman El Ouazzani, based in Casablanca, Morocco, worked closely with Infotools during the implementation, and introduced the new console to around 25 regional marketing managers and national sales managers.
The console brings together data from a wide range of sources, merging sales data, retail audit data and consumer tracking data from different fieldwork suppliers, media data and even data on weather and temperature.
“All these streams of data can be very difficult to look at or grasp in an in,” Othman explains.  “So the idea was how to integrate all of this data into one platform – so as to make it easy for the user to see not all of it, but the  most important parts of it so they can make sense of what is happening with the business and the with the brands.”

Infotools took the team through a process of identifying the data and what the users needed to see. “We had to limit ourselves to the most important things, which was difficult, but it was also a valuable activity. This process took us about 3 months, from inception to the first run of the Console.”

The Console was designed to meet the needs of different users within the company – providing high-level views for the marketing unit, as well as individual country views for the different national managers.
“The success we have had lies in the fact that we are able to integrate all of this information within one interface. We can chart sales data with retail data, brand equity data plus media data. You can see all of them, all at once, on the same chart. This makes it very easy to navigate your way through all of this information and intuitively drill down if you wished so.

“It does not require any training – it is so self-evident. If you are an Internet user, it is exactly like using the Internet, except that it does not take you to a new page  each time, it just displays everything on the same page”.
Othman advises anyone considering this approach to “make it simple for the users from day one – don’t try to complicate it. You have to be very picky on what dimensions to include in each matrix. Recognise you cannot satisfy everyone’s needs with one tool. The user eventually finds that all most important information is there, and they can refer to the databases directly if they need more.

He concludes: “One of our biggest constraints is time. This allows us to have a very quick view of the current situation – how the brands are doing, the volumes and so on. It is quite easy to use and it saves you quite a lot of time.  Before, it would have been hours and hours of work to do this, but now it is available in just the click of the mouse. “

A version of this review first appeared in Research, the magazine of the Market Research Society, January 2010, Issue 523

DataDynamic reviewed

In Brief

What it does

Desktop or web-based tabulation and charting tools for researchers or end-users with an integrated script-based data-processing module for data specialists. It can also be used to build data portals and dashboard reporting systems.

Supplier

Intellex Dynamic Reporting

Our ratings

Score 3.5 out of 5Ease of use

Score 4 out of 5Compatibility with other software

Score 4.5 out of 5Value for money

Cost

In euro (€): Offline tool €1100 per user per annum. Online: €1500 set-up fee, €1800 annual fee, plus €225 charge per person and per project.

Pros

  • Easy import from SPSS .sav files or Triple-S
  • Extendable gallery of output styles for both tabs and graphs
  • Powerful editing and data preparation workflow
  • Advanced machine-learning-based coding module for verbatim responses

Cons

  • Restricted filtering within online and desktop tools
  • Very limited range of statistics
  • Some limits on dynamic links to PowerPoint
  • No specific support for multi-language studies; interface is English only

In Depth

DataDynamic is a new arrival on the MR software stage. But is there space for yet another tab program? And is there anything significantly different about this one? Actually, there is on both fronts. While a majority of MR analysis tools effectively take their cue from the Quantum/Quanvert model, with the online tool existing as an add-on stage to a data processing activity, and the end-user working on a closed database of results, DataDynamic takes the more open SPSS as its muse.

It is often overlooked just how many researchers around the world use SPSS to do all their analysis on their quantitative surveys. On the whole, SPSS does a decent job for the market researcher wanting to analyse their own data, but it has its downsides. There is a steepish learning curve and the problem of picking your way though a host of options that are either rarely or never used. It is also a struggle to produce report-ready output for Word reports or PowerPoint briefings and summaries. But SPSS does make the raw data readily available to the researcher for them to work on and even edit. As a means of distributing data to other users this can also be a liability.

DataDynamic can work as a desktop tool, like SPSS, or as an online, web browser-based tool like SPSS MR Tables or Confirmit’s Reportal. The desktop tool (but not the online tool) also carries with it a complete interactive suite for coding and editing your data, which is aimed at the researcher as much as the data-processing specialist. It gives this product appeal to those researchers who are or like to be self-contained in their data processing capabilities. Better still, it provides them with the means to publish results to clients in a variety of ways appropriate to the needs of different audiences for research data.

For those that just want a dashboard with a few KPIs every month, there is a disarmingly simple process to create and publish these as web reports for clients to access securely in a data portal. These offer a single scrolling page of side-by-side tables, charts and commentaries, which can then be refreshed automatically as each subsequent wave of data is added. Users can look and copy data into their own reports, but cannot change it.

For those who want to dig into the data, their online log-on can be made to unlock access to the online cross-tab and charting capabilities. These are more or less identical to the capabilities of the desktop cross-tab and charting tool which is the core of DataDynamic. In either version it uses a familiar drag and drop technique to allow you to build cross-tabs from a structured list of questions. It is quick to put together tables, and there are all the options you would expect there to vary percentages, rank answers in order and apply or remove filtering, weighting or presentation options. Strangely, it only offers one significance test at present – the greatly abused t-test, which can risk being a safety blanket lined with asbestos.

On the other hand it contains a marvellous tool for creating your own target groups or profiles, such as those derived from segmentation models (though you would need to use something else to produce segmentations). It also scores two out of three for weighting: you can apply respondent weighting, and you can apply projections to a population total. However, you can only calculate simple arithmetic weights – there is no iterative model for creating so-called rim or target weights. Filtering too, is an oddity. It is quick and easy to apply any answer as a filter and combine answers from the same question, but it assumes you would always want to ‘or’ answers from the same question or ‘and’ answers from different questions. And if you create a target group, you cannot apply this as a filter – which would be handy.

Several of these restrictions can be overcome by using scripting. A powerful hidden feature of DataDynamic is the Visual Basic scripts that drive it. End users are unaware that these are being created as they compose their tables and charts, but they can be captured and edited, or folded into larger scripts to automate report production. It is akin to syntax in SPSS.

Other strong points include clear, attractive charting capabilities, based on the Microsoft Office charting engine, with user-definable template galleries; a surprisingly sophisticated suite for coding data, which even includes a trainable coding engine which will then automatically code similar datasets on the basis of examples you have provided before, and a great range of selective and cumulative imports from either SPSS or Triple-S data.

There are several other areas where more depth of functionality is needed. There is currently no real support for presenting or publishing results in more than one language for users to select their preferred language, for instance, and there some difficulties in publishing dynamic reports with charts that will refresh automatically, due to oddities in the Microsoft charting engine.

What I find most tantalising is that Intellex have used this platform to build a number of bespoke enterprise dashboard and drill-down reporting systems. At the moment, DataDynamic as shipped does not have all the tools needed to create your own enterprise feedback system, particularly in the area of user and data permissions. But enterprise reporting is something Intellex are planning to develop further and, if so, they could possibly be the first to market with a dashboard or EFM product that will work with research data from any source.

Customer Viewpoint: Yumi Stamet, Intelligence Group, Rotterdam

Intelligence Group is a research and consultancy firm based in Rotterdam in the Netherlands, specialising in the employment and recruitment research. It publishes, quarterly, a rolling two-year survey of the Dutch labour market comprising some 32,000 interviews to a wide range of commercial and public sector clients. It is a substantial survey with a large number of variables which it now distributes very effectively using the offline version of DataDynamic. Yumi Stamet, Operations Manager, explains: “We have been providing the data to the customers for a number of years using another software package, but we were not very happy with the way this software forced us to work, so we wanted to find a new way to process and deliver our data. The software had to be quicker and better – what we were using before was very slow – and of course, it had to be more user friendly”.

An important obstacle to overcome was the processing of the data, prior to it being ready for analysis and distribution, including coding a large amount of unstructured data and also applying weighting to balance the data.

“Intellex were very helpful in brainstorming on how it could be made easier and faster,” Yumi continues. “When we got the data into DataDynamic, they were able to help us to automate the process with scripts.”

As the new data was released using DataDynamic, Yumi was concerned whether clients familiar with the previous software would warm to it. “When they saw it they were very enthusiastic, particularly because they were now able to run a complete analysis of the dataset in a matter of minutes. It is a lot faster, and it looks a lot better too. Our clients also liked being able to make their own charts – and many other things are easier too, like sorting items, and above everything else the ability to create target groups very easily. Creating a target group just takes a few clicks, and it is easy to go back and refine it if it is not exactly what you want. A big advantage to me is that the software is very stable. With other software programs they can tend to crash if you ask to much of them, but you seldom see that with DataDynamic.”

To any company considering using DataDynamic, she advises: “It is very important you have at least one person learn how to do scripting as it will save you a lot of time. You don’t need to be a programmer, but it should be someone familiar with the dataset and somewhat IT-minded. If you look at all the time we have saved – with the preparation, I think we have gone from almost 24 hours to 12, and most of that time is because of the coding we have to do. With producing the reports, we have gone from a day’s work to just a couple hours.”

A version of this review first appeared in Research, the magazine of the Market Research Society, February 2008, Issue 500