General Online Research Conference

4-6 March 2013
Baden-Wuerttemberg Cooperative State University Mannheim


Conference Agenda

Overview and details of the sessions of this conference. Please select a date or room to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).

Session Overview
A2: Incentives in online panels
Time: Tuesday, 05/Mar/2013: 10:30 - 11:30
Session Chair: Mario Callegaro, Google


Rewards - Money for Nothing?

Peter John Cape, Peter Martin

Survey Sampling Germany GmbH, Germany

Relevance & Research Question: Rewards are ubiquitous in online research. In online access panels the type and value of rewards may not be known to the client. It is generally believed that rewards boost response rates. However response rates in online research are simply poor, with or without rewards. The value to the online access panel supplier is economic not methodological. With increased price compression in online sampling it is timely to consider the methodological implications of manipulating reward levels downwards.

Methods & Data: 20 treatments were undertaken; 5 reward levels by 4 interview lengths. Two cases of zero rewards were considered – one, the reward level is simply absent, the other clearly stated 0 points would be given. Matched samples of 2000 per cell were invited to the study. The body of the questionnaire was a battery to measure altruism (Rushton et al), a series of statements relating to attitudes to survey taking and rewards along with perception questions of the rewards on offer.

Results: The results demonstrated only marginal gains in response resulted from substantial increases in rewards levels. E.g. doubling the incentive from $5 to $10 increased response by a mere 14%. This is partly due to the invisibility of rewards. Only 60% thought they knew how much reward they were getting and, of these, only 60% were correct in their estimate. At the same time sample profiles, in terms of distribution of levels of altruism, were unaffected by rewards or interview length. Whilst this holds out promise for reducing rewards spend without impacting response or sample psychographics it does leave as an open question the problem of expectation of rewards – which is being set at the recruitment stage. We may then “get away” with reducing rewards in the short term, it may have a catastrophic long term impact unless we find new ways of recruiting respondents. That itself may have much more far reaching consequences for sample compositions.

Added Value: This paper enables researchers to make informed choices about rewards; not only the impact on response rates but also sample composition over and above demographics.
Cape-Rewards - Money for Nothing-115.pdf

Effects of incentive reduction after a series of higher incentive waves in a probability-based online panel.

Bella Struminskaya, Lars Kaczmirek, Ines Schaurer, Wolfgang Bandilla

GESIS - Leibniz Institute for the Social Sciences, Germany

Relevance & research question: Positive effects of incentives in longitudinal surveys for initial participation are well-documented in the literature. Introducing or increasing an incentive amount on panels has also received scholarly attention. However, increased usage of incentives raises concerns about creating respondents' expectations of being paid for survey participation. Our research goal is to answer the question whether respondents form incentive expectations after having participated in several monthly surveys or whether they are motivated by their experience within the panel. The latter would allow saving on incentives in later waves.

Data and method: We analyze data from the offline-recruited probability-based online panel conducted by GESIS. Starting in February 2011, respondents were surveyed every month for 8 months. The panel implemented differential incentives: the amount was varied experimentally during recruitment, comprising groups of 0, 2, 5 and 10 EUR. Additionally, we tested the effect of a one-time 20 EUR bonus payment for responding to 8 questionnaires in some groups. At the end of the 8th online survey we requested an agreement for further participation. Those respondents, who agreed, received an invitation to the 9th survey, for which the incentive was changed to 2 EUR (increase for 0-EUR-group, no change for 2 EUR group and a decrease for 5- and 10-EUR groups).

Results: First results indicate that incentive manipulations do not affect completion rates of the wave where incentive reduction or increase was implemented. The previous response history is one of the most significant predictors for participation in the 9th wave. Measures of survey enjoyment from the previous wave can only partly predict longitudinal response.

Added value: We report reassuring findings that respondents are not affected by possible expectations of a specific incentive. This implication may be of interest if faced with budget constraints, which make increase of incentives not affordable or even force to consider a reduction of future incentives. As large-scale panel surveys generally do not employ incentive manipulations for ethical or logistical reasons, our results can have a practical value in panel management strategies.

Timing of Nonparticipation in an Online Panel: The effect of incentive strategies

Salima Douhou, Annette Scherpenzeel

CentERdata, Netherlands, The

Relevance & Research Question: Nonresponse in online panel surveys is problematic since it may lead to a bias. An important measure to secure respondent cooperation and the quality of responses is the use of monetary incentives. The purpose of this paper is to find out which incentive strategy is efficient for long term participation of respondents. Efficiency implies both low recruitment costs combined with high response rate after entrance in the panel.

Methods & Data: An experiment was carried out in the LISS panel (Longitudinal Internet Studies for the Social Sciences, an online panel based on a true probability sample of households) in 2007 to determine the optimal recruitment strategy for a new online household panel. The monetary incentives varied during the recruitment. The incentives were either promised or prepaid and the amount varied (10, 20 or 50 euros). More than 500 respondents were randomly selected in the different incentive conditions. This paper takes a different approach to model the time-to-event of nonparticipation: survival analysis. The event in this case is nonparticipation. This method has two important advantages: 1) incorporates the timing of the event and 2) allows for censoring. This research will provide new evidence on the timing of nonparticipation and the influence of different incentive strategies on this timing.

Results: A pilot study was performed to evaluate the effect of the incentive strategies on the recruitment of respondents for the online panel. The highest response rate was found for the lowest prepaid incentive. Section incomplete, see remarks.

Added Value: The willingness of respondents to participate for a long term in the panel for different incentive strategies is an important topic in the literature on survey nonresponse. The innovative aspects of this study are as follows. First, the recruitment incentives are investigated to determine which strategy is optimal for both recruitment and retention of respondents for a longer term. Second, this paper will use a different method of analysis in order to look at the timing of nonparticipation in relation to incentive strategies. This helps us to define an efficient incentive strategy for an online panel.

Contact and Legal Notice · Contact Address:
Conference: GOR 13
Conference Software - ConfTool Pro 2.6.49+TC
© 2001 - 2012 by H. Weinreich, Hamburg, Germany