Főoldali vendégtartalmak

Quant Essentials: Research Design

RW Connect - 2017. October 16., Monday
 Research Design

Kevin Gray discusses the principals of quantitative research design.

Continue reading Quant Essentials: Research Design at RW Connect.

Empathy vs. Incentives?

RW Connect - 2017. October 13., Friday
Empathy vs. Incentives?

By Cristoph Welter

This year at the Best of ESOMAR-Germany meetup, we measured the pulse of the industry at what seems to be one of the most hotly contested metrics these days: Trust! In four talks that scrutinized the topic from different perspectives, we talked about surveys and all manner of interview techniques, about participants, about clients and about the wider public – so how is research doing, and how can we regain some of the trust that we seem to be losing?

Nikki Lavoie of Mindspark asked two simple questions in the course of her talk: Imagine somebody asks you if you’re up for having lunch with them for an hour, or alternatively, that they offer you 10 Euros for having lunch with them for an hour. The latter is the mode we routinely bring our participants into. “What is happening to our participants in that moment” Nikki quizzed the audience. In fact, we switch them into economic actors, weighing costs and benefits… is that what we want? Do we really want participants that measure their time expenditure against the money we offer? This was the central argument in Nikki’s criticism of our reliance on extrinsic motivation through monetary incentives.

The talk raised quite some controversial discussion amongst the audience – it seems Nikki hit a nerve there. And let’s make one thing clear: She doesn’t advocate that we should all scrap monetary incentives now, but we shouldn’t rely on them exclusively, as it will get us into an ever aggravating price spiral. Rather, we should think about how to engage with participants using empathy – before we actually meet them or engage them in a survey. How can we invite them to participate in a manner that will trigger their intrinsic motivation, make them feel valued and appreciated right from the get-go?

A large-scale survey on the public perception of research carried out by Research Now with the support of ESOMAR, in fact tapped into a matching challenge. Liam Corcoran of Research Now presented key findings from the study in the UK, USA and Germany. In order to gain a broad picture of how people relate to research, they included a variety of different channels, from CATI across online panels, social media panels, and also Google Consumer Surveys. A somewhat worrying key finding for Germany: People tend to try and complete surveys as quickly as they can. And here we are again: economic logic – price vs. time spent – utility maximizing actors.

What can we do? Liam pointed towards a potential crucial issue: people often don’t have a clear idea what a survey is for, what is the purpose, the Why. And what will happen with the data – where do they feed into? If people don’t know why they are participating, they might not put a lot of effort into their participation, and they might not trust the industry that is behind these surveys a lot. So, in conclusion, Liam alerted us to the need for the research industry to engage the public more, to involve participants more, and eventually also to involve the clients even more – education on the potential and the limits of research.

Kim Smouter, Head of Public Affairs and Professionals Standards at ESOMAR, presented some of the recent activities on behalf of ESOMAR which aim in exactly this direction: Raising the profile and reputation of research with the wider public, and crucially, with political actors. In a data driven world, political actors at the EU level need to understand the positive impact that research makes for society, and they need to understand how their actions affect the potential and limits of our industry. Of course, this requires each and every one of us to be mindful of our own ethical responsibility, so that we can establish ethical transparency and good conduct along the whole supply chain.

Finally, then, we asked: What advice and input can the next generation of researchers give us? So we had invited Emma Kirk from JoinTheDots, winner of last year’s Young ESOMAR Society Competition at Congress in New Orleans. Emma reminded us of our responsibility to stay up to date and engage audiences with timely ways of communication. Enter the Emoji. It seems so simple to say that we should all use Emojis in research, almost a no-brainer – but Emma stressed that it’s a valuable, but by no means easy feat. There is still a lot of education to be done on the fact that Emoji in fact do constitute a veritable communication system, and something of a universal language. The tricky bit however is that the same visuals do carry culturally distinct meanings. And to complicate things further, depending on device and OS, the same Emojis might end up looking quite differently. And of course, in order to use them in scales of any kind, we need solid validation groundwork. Despite these challenges, Emma called out to the industry to continue promoting the use of Emoji – after all, speaking the language of the people we try to engage is one of the most basic conditions for trust to emerge in the first place.

Empathy vs. Incentives?

Empathy vs. Incentives? 1











Christoph Welter is Managing Director at Point-Blank International and ESOMAR Representative for Germany. 

Continue reading Empathy vs. Incentives? at RW Connect.

AQR Award Finalists Announced

RW Connect - 2017. October 12., Thursday
AQR Award Finalists Announced

AQR Award Finalists Announced

ESOMAR’s Director General Finn Raben was honoured to have been invited to join the judging panel for the prestigious AQR Prosper Riley-Smith award. Following extensive deliberation and debate, the jury are delighted to announce the Finalists below, in no particular order:

ECB Cricket Unleashed: Gurpinder Sidhu and Emma Bennett, Sky UK Ltd

Shift the Mix: How cultural intelligence helped pets around the world: Oliver Sweet, Ipsos MORI

The Apprentice: The Real Winner in the Boardroom: Rhiannon Price and Daniel Tralman, Northstar Research and James Livingstone, Jaguar Landrover

The 3 Finalists are invited to join the AQR table at the MRS Awards Dinner.

The Winner, who receives the ‘Prosper’ and a prize of £1,000, will be announced and presented at the MRS Awards Dinner at Supernova in Embankment Gardens London on Monday, 4 December 2017.

The AQR would also like to thank very much this year’s distinguished Judging Panel from across the industry for their valuable time and support of this Award.

Richard Drury – Walgreens Boots Alliance, Vice Chair of AURA
Jane Frost, CBE – CEO of the MRS
Helen James – npower Business, Chair of AURA
Sarah Jenkins – Magenta Research, AQR Board
Tom Kerr – Treasurer of AURA
Sarah Newman – Director of the APG
Finn Raben – Director General of ESOMAR
Peter Totman – Jigsaw Research, AQR Board

Many congratulations to the three Finalists and best of luck!

Continue reading AQR Award Finalists Announced at RW Connect.

The German Election in the polls.

RW Connect - 2017. October 12., Thursday
The German Election in the polls.

Excellent Exit Polls – very revealing pre-election polls

By Richard Hilmer

In the run-up to the German election there was a vivid debate about how reliable polls would be. The background of this discussion, of course, was the impression the German public had that the polls in the British Brexit decision and the American presidential election had been altogether incorrect and misled the public[1]. The main reproach was that polls underestimated populistic in the British and US population.

This too negative impression was reinforced by the 2016 German regional elections where the right-wing party AfD in particular had been underestimated in the pre-election and some exit polls. In the regional elections of 2017 it became clear that all polling institutes in Germany had successfully adjusted their instruments. The polls in the 2017 regional elections in Schleswig-Holstein and Nordrhein-Westfalen were quite accurate, including the results of the AfD. The only fly in the ointment: in the Saarland election all polls significantly underestimated the CDU and showed a neck-and-neck race between CDU and the SPD, but in the election the CDU was more than 11 points ahead. This kept the discussion alive about polling accuracy and whether polls would get the results (or at least the range of the parties) correct in the national election in Germany of September 24th.

The German election is over and we can say that the polling institutes did quite a good job. The exit polls were excellent as they have been in recent national elections. The cumulated deviance for the six relevant parties[2] amounted to 2,5 (Forschungsgruppe Wahlen and Forsa) to 2,8 points (Infratest dimap). That means an error of less than 0,5 points per party.

[1] The arguments that in case of Brexit all polls showed that it would be a close race and that in case of the US-election at least the national poll showed correctly a 2-3 percent lead of Clinton did not had great effects on that impression.

[2] CDU/CSU, SPD, Linke, Grüne, FDP and AfD with a share of 95 percent of the votes

The German Election in the polls.

So on election´s eve the audience was very accurately informed about the outcome of the elections — just as in previous elections. But this time it wasn’t anything at all like a matter of course. For the first time a right wing party managed to get into the Bundestag and because the Liberals also celebrated a comeback, the German parliament (for the first time ever) consists of six parliamentary parties – if you count CDU and CSU as one.

Volatility is not so much a problem for exit polls – because they deal with decisions already made – but it can be all the more so for pre-election polls. This is especially true for Germany because we had been used to a quite stable party constellation, more stable than in most of our neighbor countries. The German polling institutes, as in 2013, had to deal with an amount of extremely high volatility never seen in Germany before (Pedersen Index 30,0). The legislative period was split into two completely different parts. Through August 2015 there was a high degree of political stability with almost no change in party preference in the polls. This situation changed profoundly with the European refugee crisis which brought heavy instability in Germany´s political landscape. For the AfD, which almost had disappeared from the political landscape, this new topic was an elixir of life. The party grew in the polls from 3 to 15 percent. In the regional elections in 2016 it became clear that this was a serious challenge for the established political parties in Germany. The share of votes the AfD won in those elections was between 12 and 25 percent and the AfD is now represented in almost all regional parliaments today.

In the beginning of 2017 there was another unprecedented shift in the party landscape. After the nomination of Martin Schulz as front-runner of the Social Democrats – he also took over the leadership of the party from Sigmar Gabriel – the numbers for the SPD jumped within just a month in all polls from 20-21 up to 31-32 percent – and this was just the shift that was published. In the raw data of Forschungsgruppe Wahlen – the only institute that publishes its raw data – the SPD doubled its share from 21 to 42 percent – and even the older generation of pollsters hadn´t experienced a shift like this before. Just as quickly the astonishing rise of the SPD was followed by the decline. Two months before the election the situation seemed to be stabilized again with CDU/CSU close to its result from 2013, SPD close to its starting point in January and AfD support cut in half from 15 to 7-8 percent. But the decision-making process of the voters had still not come to an end. In the final stages of the election campaign both parties of the great coalition of CDU/CSU and SPD lost support significantly to the advantage particularly of AfD and FDP: a minus of 13,8 points means the highest loss for a German government ever.

The German Election in the polls. 1

Did the last published polls reflect this development correctly? On the whole yes. As table 3 shows the cumulated deviance over nine institutes amounted to 9.2 points. This deviance was significantly higher than in the elections of 2013 (6,3) and 2009 (5,7) but not as high as in the 2005 election (11,8). Closest to the election result came INSA with a deviation of 6,4 points, while the highest deviation was YouGov with 12,6 points. And what is particularly important for the credibility of polls: Seven polling institutes correctly identified the top 3 parties, and the two leading institutes – Infratest dimap and Forschungsgruppe Wahlen – even had the order of all six parties correct. The polls also mirrored the decline of CDU/CSU and SPD and the growing support for the AfD and the FDP more or less correctly, even if the decline of the CDU/CSU had been underestimated by all institutes and the rise of the AfD by most institutes. The biggest deviance was that of the CDU/CSU, ranging from 1.1 (INSA) up to 4.1 (Infratest dimap and GMS) and an average over all institutes of 2,98 points –only in the 2005 election was the deviance for a party higher (6,38) and that time it was also for the CDU/CSU. An interesting side effect: the greatest discrepancies between polls and results in 2017 in Germany referred to the biggest party by far, the CDU/CSU. But the differences went in different directions. In the Saarland election the CDU was heavily underestimated, in the national election the CDU and CSU were overestimated significantly.

As in the presidential election in the USA there were some German polls concerning the outcome of the national election in states – same method, same sample size as those for the nation. And as in the United States the regional results were far less accurate than those on the national level. So several polls in Bavaria – one of them conducted and published in the week of the election – predicted the CSU at 47-48 percent. In the end the CSU received 38.8 percent. The deviation over six parties amounted to more than 20 points! The big difference with the USA: in Germany the composition of the national parliament is based on the national, not the state results. Therefore this bias was not so crucial.

The differences in the accuracy of the polls were not a matter of methodology. Seven institutes used CATI and two online interviews, amongst them INSA with the highest congruence with the real result and YouGov with the highest deviance. The poor polls in Bavaria also used different approaches. One institute used CATI, the other online interviews.

Other approaches of predicting the outcome of the election were not very convincing, which was favorable for polling. One of those models was published by YouGov just two weeks before the national election – a new approach YouGov successfully tested in Great Britain in the last national election. It was based on an online-panel approach combined with selected secondary data. The YouGov model predicted 25 percent for the SPD instead of the 20,5 percent it received, and 7 percent for the FDP instead of 10,7 percent. The cumulated deviation of 14.6 was significantly over that of the normal polls.

The second prediction model was developed by Helmut Norpoth and Thomas Gschwend. It is based on a correlation of some polling results (popularity of the chancellor, longtime support of the governing parties) and traces of wear on the government over the time. It does not deliver figures for single parties but the probable outcome for the existing government and the most promising alternative coalitions. In the past the predictions of this model came quite close to the real results but this time the model was not very useful. It delivered only the probability of two coalition options. For a coalition between CDU/CSU and the FDP the model predicted 49,4 percent and for a coalition of CDU/CSU and the Greens the prediction was 49,3. In the election CDU/CSU and FDP together reached just 43,6 while CDU/CSU only received just 41,8 – the outcome was way off a realistic majority for both of the options.

Conclusion: The exit polls were excellent as they always have been in the last decades – not least due to a strong competition between TV-stations and polling institutes in Germany. And the pre-election polls in Germany were quite decent and adequate taking into account the high volatility. Differing from the 2016 regional elections the institutes succeeded in getting the results of the new right-wing party AfD correct. Apparently they derived the right conclusions from the notorious underestimation of the AfD in those contests. This was the main reason why there was no public discussion after the election about the reliability of polls in the Germany.

By Richard Hilmer, Policy matters

Continue reading The German Election in the polls. at RW Connect.

A New Age for Research

RW Connect - 2017. October 11., Wednesday
A New Age for Research

A month on from Congress, we look at the hard hitting message still resonating around what's next for research.

Continue reading A New Age for Research at RW Connect.