The latest news from the meaning blog

 

Warp reviewed

In Brief

What it does

Web-based cross-tabular and charting suite for researchers and clients, which extends the capabilities of Quanvert and works with Quantum/Quanvert databases.

Supplier

Warp Online

Our ratings

Score 5 out of 5Ease of use

Score 2 out of 5Compatibility with other software

Score 3 out of 5Value for money

Cost

Per month: hosted solutions from $1500 (£750) per user annually; purchase from $90,000 (£45,000) for a 10-concurrent user licence, plus 20% annual maintenance.

Pros

  • Lets you distribute existing reports, define your own or both
  • Real-time access to one central database ensures that users work with the latest data
  • By using Flash, provides a very Windows-like drag-and-drop user experience
  • Good intuitive support for hierarchical data

Cons

  • Only works with data prepared in Quantum and Quanvert
  • Charting in Flash only, which is not always compatible across platforms or media
  • Cannot work offline

In Depth

While travelling at warp speeds beyond that of light was the core fantasy of Star Trek, travelling forward at any speed remains the fantasy of users and companies with a major investment in data processed through Quantum and Quanvert. Still seen by many as the gold standard of MR data analysis, both programs have been on a trajectory to nowhere since acquisition by SPSS in 1998, thereby missing out completely on the internet revolution.

The newer, web-friendly products on the market rarely approach the depth of functionality to handle tricky analyses such as hierarchical data, re-weighting, rolling averages, indexing or creating composites such as Key Performance Indicators. Warp, a totally web-based analysis, reporting and presentation tool, does all this, and will do so directly from Quantum or Quanvert outputs in the form of a Quanvert .PKD file. The deliberate aim of San Francisco-based Warp Online is to create a worm-hole through which Quanvert users can travel instantly from an era when browsers were something only to be found in a public library, to today’s world where distance is no longer a barrier to information.

Indeed, to anyone familiar with Quanvert, Warp is an easy transition to make, as it still contains the same concepts of variables, tables and table axes. There is a separate web-based system administrators’ interface which allows for users and groups to be set up and for the interface to be customised for different clients, both in look-and-feel and in the options they may select – according to the extent to which you trust their skills. It is here you load in the Quanvert datafiles and then set permissions as to who is allowed to see them in their lists of available projects. This process can also be automated to a large extent, for continuous projects. It will also make a lot of helpful auto-adjustments when new waves of data are added, and there are changes to resolve.

For the end user, the interface is deceptively simple. The program is highly context-sensitive, so it tends only to display those capabilities or options which are relevant at that time. As you dig down, some very sophisticated capabilities emerge. Variables are selected simply by dragging and dropping – even though it is all web-based – as it makes extensive use of Flash to provide a very comfortable and productive working environment that really feels like desktop software. Being completely web-based, there is no software to install, no files to distribute, and it means everyone is always working on the latest set of data. Warp will only work with Microsoft IE6 (or IE7 with a special patch applied), which may disappoint some users, but IE6 is the preferred browser for most corporate users, and Warp promise to support IE7 once there is demand for it. However, this is not a program that will suit you if you need to work for long periods with no Internet connection.

With every table you produce you can choose to save, or not. By letting you choose, you can keep your reports folder uncluttered, which can be a problem with many reporting tools. This is also how you create a new variable or filter – any combination of rows or columns on a saved table can then be used to create a filter or a variable, and the top of the table can be saved as a breakdown or banner. These and most other options are available from a context sensitive right-click menu, which also reinforces the desktop feel of the software. Any variable or filter you create goes into a ‘My Variables’ folder. You can also apply filters or breakdowns globally from the right-click menu to existing tables, or make this the default for any subsequent reports you create.

Reports can also be graphs, which are produced using Warp’s own charting facility using Flash. This does not limit you to the rather pedestrian charts that tend to come out of Excel or PowerPoint – you can even create animated charts to show actual growth or change. However, some version of MS Office struggle to accept the Shockwave files produced by Warp, which could cause your presentation to unravel before a client, if you are not using your own laptop.

Also within very easy reach of the non-technical user is the ability to create new weighting schemes, to work with hierarchical data, and to perform significance testing.

All the way through, the program demonstrates a profound understanding of what the user needs to do. The program will also work out what is the appropriate weighting scheme to add when working on different levels of data, and restrict your choice of breakdown variables to those which are appropriate, given the questions already placed in the table.

Warp actually holds all project data in one highly optimised database. This means that end users can also share items that they have created with other users, such as filters or new variables. The optimised database also means that tables are crunched very fast.

Ultimately, the success of Warp will depend on how it lets users break free of the Quantum paradigm, and for that, it needs to be able to handle other types of metadata than Quanvert. For the moment, to coin a phrase, that remains the final frontier.

Customer viewpoint – Dave Bostock, MarketTools, USA

MarketTools, a US-based research and technology provider specialising in online research, uses Warp to deliver online analysis to its clients. Dave Bostock Director and Business Architect at Market Tools, explains: “We conduct large trackers for clients, where we have to go back repeatedly to the same respondents to see how opinions shift over time and how that impacts the perception of brand and market share. We’ve looked at a number of third-party reporting tools to deliver this type of data set and we have not found any that can compete with the speed, usability and power of Warp.

This application actually gives the MarketTools researchers and our clients the ability to generate their own analysis and conduct deeper dives into the data without having to go through the traditional table specification and creation stage. This has always been the big drawback in providing additional insightful analysis. This type of solution gives an analyst the flexibility to view data, query results and run reports via their desktop. Our clients are extremely happy with the solution.

Other programs require a lot of specialised or stylised work at the set up stage. Warp allows us to take our normal production process using Quantum and, with a quick conversion into Quanvert clients can access the project data remotely. Clients access Warp via secure login. One key advantage that Warp provides us with is the ability to publish a project to a client user group from one dataset, with different nuances for each user – filtering which reports, charts and variables are seen by each user for example. The drag and drop interface is really important in making it an easy, engaging and enjoyable experience for the user”.

“Our mission is to provide our clients with on-demand research solutions, and Warp helps us accomplish this. The research business, like the internet overall, is becoming more and more driven by consumers – Warp is a tool that empowers our clients. Reactions [from them] have been very positive.”

Dave is also impressed with Warp’s advanced analytical and table manipulation capabilities, which is delivered in simple steps allowing users to take full advantage of them. “I like the ease of use of the hierarchical data functionality, and the ability to apply pre-determined weights on the fly. Profiling respondents very quickly is also a huge benefit. There are several things I have not yet seen in any other tool, such as Warp’s very handy column differentiator test, or how you can create share tables by easily adding a numeric variable to the increment area on the screen. Our clients need to feel empowered, and Warp certainly gives them that. It is a huge win for us.”

A version of this review first appeared in Research, the magazine of the Market Research Society, August 2007, Issue 495

SPSS Dimensions Desktop Reporter 4.1 reviewed

In Brief

What it does

Windows-based cross-tabular reporting program for end-users.

Supplier

SPSS An IBM Company

Our ratings

Score 4.5 out of 5Ease of use

Score 4 out of 5Compatibility with other software

Score 2.5 out of 5Value for money

Cost

$1,500 (approx £750) annually for single user annual licence. Perpetual licence $2,450 (approx £1225) for a single user, plus 20% annually for support and updates. Volume discounts are offered.

Pros

  • Powerful, but still simple to use
  • Supports hierarchical data
  • Outputs in Excel and PowerPoint can be auto-refreshed from live data link
  • Will work directly on Quanvert databases

Cons

Cannot colour-code, or highlight variances or significant valuesGetting data in can be a challenge (in the current version)

Some performance issues with large datasets in XML or SQL

In Depth

There are some old programs that refuse to die. Even the original authors of Quanvert must be taken aback by its astonishing longevity. So must SPSS, its current owners, who had lined up mrTables, its new online cross-tab tool, to be a Quanvert killer. But the convenience of running an application at full tilt on your desktop or laptop continued to give Quanvert the upper hand for many users. Despite a gnarly old interface, Quanvert did many things that were hard, if not impossible to do in mrTables.

Now, SPSS has launched a second potential Quanvert-buster on the market. It is called Desktop Reporter, thus wearing its USP on its sleeve – and it is clear that this time SPSS are serious about providing the critical mass of functionality that should entice users away from old Q for ever.

Like mrTables, it is a Dimensions product, which means that it sits on top of SPSS’ tiered architecture of a standardised Data Model and another model relating to the creation of tables called the Tables Object Model, or TOM. The immediate benefits to end-users of this are not always easy to see, but it does make it a lot easer to integrate Dimensions software with other applications, or even use these products as a springboard to create your own analytical systems, information portals and so on.

Indeed, SPSS has tended to take an engineer’s rather than a designer’s approach to much of its Dimensions line. The user experience was often disjointed, leaving you with the feeling that it was not as easy as it should be, or could be in some of its rivals’ offerings. Desktop Reporter looks to me like a break with the recent past – it’s elegant, sophisticated, sassy and, best of all, highly intuitive.

The main screen has the now typical column on the left where selections take place, a large pane on the right, where the action happens, and buttons and controls on top to effect actions, change options and so on. Right-clicking always seems to bring up a sensible menu of options relevant to what you are doing, and these often duplicate buttons, menu options and keyboard shortcuts, so providing power users with many ways to skip through producing tables

Some users like to handcraft all their analysis, question by question. A table is simply created by dragging and dropping items on to it. A separate tab in the main window lets you set up filters using parallel techniques.

But it is just as easy to throw all of your questions into the pot, and let Desktop Reporter produce one tab for each one, as a total or with a standard break, if you prefer.

Existing tables can be used as templates for new tables, to speed up the definition. Stats and sig. tests are also easily applied, and instead of overwhelming users with options (most of which 99% of users will never use), more obscure options are hidden away, but accessible generally from a “More” button. Your default output can be a cross tab or a chart, or both; charts and tables are easily posted into Excel or PowerPoint, and both can retain links to the data for automated refreshing when more data arrives.

It even offers some multivariate analysis for the statistically challenged – letting you present a table of ‘difference attributes’ which lets you throw together any number of variables, and it will rank the combinations of answers to show those where the disparity is greatest at the top. You can alter the parameters to show affinity too.

There is much more that the program does very well – sensible defaults, right mouse button menus and tight navigation mean that whether you are exploring your data or finessing your output, a couple of mouse clicks is usually sufficient to get you where you want to be.

Quanvert users are likely to take to the program straight away. It appears to do much of what Quanvert did, with the benefit of a modern graphical user interface. Even hierarchical data are supported. Some Quanvert terminology, like Axes and Levels has come over too, which may be a puzzle to other users.

There are some constraints in the version I saw, however. The whole business of getting data in is a work-in-progress, as Jeff Thompson, our interviewee, points out. SPSS promise that version 4.5, due out this month, will remedy much of this. It probably will not remedy some of the other constraints that the Tables Object Model imposes on what tables look like, so that some of the manipulation features (often scripted in Quantum) are unachievable in this, and are likely to remain so.

An opportunity missed is to update significance testing. A faithful reproduction of the letter-code sig test tables from Quanvert will satisfy some users, but leave many more baffled – those who are used to looking at colour-coded tables based on exceptions. Unlike most of its rivals, this is something Desktop Reporter won’t let you do.

Despite its version 4 number, this is effectively the first version of Desktop Reporter (the number reflects the generation of the Dimensions suite) and, as a first version, it is most impressive. Will it turn into a real Quantum-buster we must wait and see what the loyal Quanvert users have to say.

Customer viewpoint: Jeff Thompson, Kantar, Austin TX

Kantar Operations provides operational support for Kantar Group research companies, and is an early adopter of SPSS Desktop Reporter. Jeff Thompson, Director of Research Technology, based in Austin, Texas, describes his experiences.

We are at the beginning of deployment, so it is not on everyone’s desktop yet – we have only used it with a few select teams. But we have now made it our preferred tool for delivering SPSS Dimensions-based output. We are already using it to work on Quanvert data. We did a really thorough gap analysis between this tool, Quanvert and some other in-house tools, and we found it really did do everything we wanted it to. There are a some limitations in this version, but 4.5, which is due anytime now, will probably address a number of these.

We have found that the ease of use of Desktop Reporter is so much greater than that of Quanvert. The user interface design is so much better. What we are seeing, overall, is that the number of things that people can do themselves is increased, so the number of requests to DP is diminished.

“For instance, it is so much easier for users to create nets, to create and edit filters and they can derive new variables very easily. These are often things that users had to come back to DP team for changes. It has some interesting new ways of looking at data too. It allows you to select a whole set of variables and run a report, and look for things that are statistically significant. But for us, the real advantage is the ability it provides for researchers to do things on their own and not have to come back to the operational teams to script new variables or tables.

“We are in very early stages and have only used it with a few select teams. They have been impressed, but these people are the ‘early adopter’ types who are often able to get the most out of new tools. We have yet to see if users as a whole will use it in the same manner.

In the past, in the Dimensions side of SPSS, there have been some weak user interfaces. The improvement in this over mrTables is enormous – it stands head and shoulders above anything the Dimensions team have produced to date. We have long felt they had good engines behind their software, but the interfaces were poorly, or perhaps quickly thought out. This interface seems to have been done really well, and I am really hopeful that we are seeing the start of a new age in their development – one where they put more thought into the user interface design of their tools.

The one awkward area is still there is not an ideal data store. You can easily hit Quanvert data but you cannot easily write out Quanvert; you can use XML data, but it does not perform well, or you can use SQL server, which performs quite well, but it just isn’t portable. The tools around the whole distribution of SQL server databases are lacking somewhat. We are expecting this to go away with 4.5.”

A version of this review first appeared in Research, the magazine of the Market Research Society, June 2007, Issue 493

Instant Intelligence reviewed

In Brief

What it does

A delightfully simple online cross-tab and topline reporting tool which works with survey data from most standard MR data collection tools, and also offers an optional integrated web-based data capture service from paper.

Supplier

Data Liberation, UK

Our ratings

Score 4 out of 5Ease of use

Score 5 out of 5Compatibility with other software

Score 4.5 out of 5Value for money

Cost

Entry level for analysis: £3600 per year, includes 1 admin user, 4 report writers and 10 viewers. Extra report writers £50 per month; viewers £10 per month. Scanning from 2p per duplex page; free scanner for committed volumes.

Pros

  • Easy import via triple-S or SPSS for most MR web or CATI data collection tools
  • Very quick to learn – majors on simple, straightforward analysis
  • Very clever scanning solution, if you need to capture data from paper
  • Gives research buyers an independent alternative to using agency’s embedded analysis tools

Cons

  • Filtering options rather too limited
  • Cannot create new variables, regroup variables or create a composite breakdown
  • Analysis is queued rather than done in real time, so there can be a wait for each table

In Depth

It has long been my belief that the greatest opportunities that the web provides for research are not in data collection, but in the downstream activities. One firm I have been watching for a while that takes this to heart is Data Liberation. They developed a web-based scanning or data capture tool for paper questionnaires two years ago, and last year added a simple online analytical tool. Now, the two have been integrated and launched at the 2006 Insight Show in November. If scanning is not your thing, this still provides a great web-enabled way to analyse data from any source.

Web-based scanning is not the oxymoron it might seem to be in Data Liberation’s hands. As a DIY user, you can design your own paper questionnaire very easily using Excel on your desktop. It provides the grid — you tend to work with very narrow columns to give you better layout control, and by adding texts of various size and hue then selectively adding borders you can create great looking questionnaires. Tickboxes you create using the box character in the Wingdings font.

It is not the most obvious way to design a paper questionnaire, but once you see how it is done, it is ingenious. In some ways, it is better than using Word as you always know exactly where everything will be, to the pixel – hence its attraction as a robust way to define scannable forms. It handles multi-page and double sided documents too, but if you decide to change your pagination, then it can involve a lot of manual editing as there is no way to flow the content as you can in Word.

The document can then be printed on your office laser and copied in bulk. What is usually the nasty part — defining the scanning template — is done by logging into your account on the website and uploading your Excel document. You then view a bitmapped image of your forms on screen, and use a smart mark-up tool to point out the regions where questions and answers appear. It makes intelligent guesses about the questions and answers, snapping to rows and columns of boxes, and letting you confirm or guide it to the right place by dragging on screen. Again, it is a painless process.

Existing or ‘legacy’ documents can be handled by sending them to Data Liberation who, as a service, will create a matching Excel template. You also send them the paper to scan — though if you are doing a lot of scanning, they will provide you with your own scanner connected over the Internet, to their site.

Speaking to several of their customers, it is clear that this very unconventional approach actually yields highly accurate results — as good as anything else on the market, it seems, and capable of handling very high volumes as well as small one-off jobs.

The part of Instant Intelligence you are likely to spend most time in, though, is the analysis tool. Unlike most analysis tools, there is very little setup involved in bringing data in for analysis. As an administrator, you can import all of the variables and texts when you start from a triple-s or an SPSS file, which many packages will output. If you only have an Excel or CSV file, you can still work with this, but will have more setting up to do. Variables can be grouped into a hierarchy of folders or sections, which makes navigating around very large projects much easier. You can also define users and to some extent configure permissions, such as allowing them to view reports, or perform analysis and select the projects they may view either by group or individual.

The analytical capabilities are basic: cross-tabs, frequencies and percentages. To me, it seemed to be a slightly hollow centre after all sorts of tantalising goodies on the outside, but it does most of the things that most end-users want. It does not go very far with filtering, and options to combine variables are too limited to be useful. It also incorporates a topline report which does the job but without much grace in its output style. These are all things that are likely to change in future versions.

The tool could struggle to live up to its instant credentials if you hit the server at a busy time: analyses are queued in the background and not done in real time, so there can be a wait before your output is displayed. This is not uncommon in ASP solutions, though to be fair, when I was testing it, the response was more than acceptable.

But just as this product is versatile on the way in, it is also versatile on the way out and dovetails seamlessly into Excel and PowerPoint. Overall, this is one of the most imaginatively different software products I have seen in years. With very little extra development, it could be a stunner.

Customer Perspective: Global luxury goods company, London

A global luxury goods company uses Instant Intelligence for an annual in-store survey of customers — a massive global project that spans 30 countries. A major attraction was the system’s fusing of paper and web approaches, with automated data capture.

“The survey runs for a month in store and questionnaires are then sent back to London for scanning” explains Laura Simmonds, global market researcher at the company’s head office in London. Scanning is carried out by Data Liberation, who also verify and clean the data, then post the results directly into Instant Intelligence, for Laura and her team to analyse.

“If we did not have the data entry scanned in this way, we would have no way of doing this. If we had to key all data in manually it could take all year!”

Some of the data originates from an online survey, the data is then easily imported into Instant Intelligence and merged with the other data, ready for analysis. It is really only the data interrogation and analysis part of the suite that is visible to users. However, online access means the company can access the survey results around the world.

“The nice thing about Instant Intelligence is that it is so easy to use and train others on,” Laura observes. “For our purposes, it performs all the cross-tabs and you can safely let other people pull information off for themselves.

“Other databases that I have used are often more complex and usually have to be managed centrally because there are so many factors to consider to ensure that correct information is taken from the database. These databases take some time to be trained on to become a competent user, compared to Instant Intelligence, which can be taught in an about one hour.”

She concludes: “Instant Intelligence is perfect for our needs, it provides a reliable, user-friendly database at a relatively low cost.”

Names have been changed in this article at the company’s request. A version of this review first appeared in Research, the magazine of the Market Research Society, January 2007, Issue 488