seaturtle.org : MTN : ARCHIVES : Sign In

This contrast between approaches to biological questions and to social or policy questions is found in all global efforts to protect biodiversity, not just those efforts focused on marine turtle conservation. Our understanding of the biological aspects of ecosystem conservation rests, at least in part, on well-designed empirical studies. In contrast, our understanding of the way in which policy interventions can stem species loss and ecosystem degradation rests primarily on case study narratives and field projects that are not designed to answer the question, “Are our conservation interventions working better than no interventions at all?”

When it comes to evaluating the effectiveness of its interventions, the field of ecosystem protection and biodiversity conservation lags behind most other policy fields (e.g., poverty reduction, job training, criminal rehabilitation, public health). However, there is no reason that the same sophisticated approach that conservation biologists and practitioners apply to testing biological questions cannot be applied to testing questions of conservation effectiveness. In fact, there is a large field, called “program evaluation,” from which marine turtle
scientists and conservation practitioners can draw to accomplish such tests (Rossi et al. 2004). The field of program evaluation uses experimental and quasi-experimental methods that, in the context of conservation, can help answer the question, “What is the effect of X on the conservation outcome of interest?” The “X” might be public education, technology transfers, regulatory restrictions, or changes in the larger economic or cultural realms. The conservation outcome of interest” might be changes in sea turtle populations, changes in turtle by-catch, or changes in the fishing, hunting or consumption practices of coastal populations.

I am not referring to analyses of whether circle hooks lead to less sea turtle interactions than J hooks, or if new methods of returning captured turtles from fishing boats reduce physical damage. Such experiments are common and important, but do not evaluate which interventions generate conservation gains in the field. I am also not referring to the kind of “monitoring and evaluation” that has been promoted for over a decade in the conservation arena (Margoluis & Salafsky 1998). In these exercises, overburdened and undertrained field staff are expected to collect data on a variety of “indicators” and somehow these data are expected to lead to improvements in conservation interventions.

I am instead referring to a small number of carefully controlled ex ante or ex post empirical evaluations that, for example, will tell us whether training fisherman to use a new technology leads to fewer turtle deaths than in areas where fisherman are not trained to use new technologies. The absence of these kinds of program evaluations is an important reason why we have little idea about what works and what does not work in protecting biodiversity.

The Millennium Ecosystem Assessment was published in March 2005. While the biological chapters are rife with data and empirical studies, the Policy Responses chapter lists as one of its “Main Findings” the following: “Few well-designed empirical analyses assess even the most common biodiversity conservation measures.” We do not know how effective different regulations are. We do not even know if the 32-year old US Endangered Species Act is effective. There have been no well-designed empirical analyses of its effectiveness, which means that proponents and opponents simply use anecdotes and spurious correlations to present their cases. We do not know if conservation education works and to what degree – we know it can increase awareness, but there is little evidence that it can change behavior among large numbers of people, particularly those whose behavior we most need to change. Although I am an enthusiastic proponent of economic incentive programs (Ferraro & Kiss 2002, 2003; Ferraro 2001), I have few data to support my contention that such programs can generate net conservation gains. Although I admit to being a neophyte in the field of marine turtle conservation, my impression from the 25th Annual Symposium is that the situation here is no different from that in other areas of biodiversity conservation (in fact, Matthew Godfrey has brought to my attention that I am not the first to call for the use of experimental methods to evaluate the effectiveness of turtle conservation strategies; see, for example, Mrosovsky 1983, 2000).

I earlier posed the question, “Are our conservation interventions working better than no interventions at all?” Answering the question is difficult because we wish to comment on a counterfactual event: we cannot observe how people who receive an intervention would have behaved without the intervention. A lot of conservation practitioners believe, of course, that their interventions are more effective than nothing, but as development practitioners have been discovering over the last 8 years as they increasingly use program evaluation methods, there are a lot of initiatives that are no better than no intervention at all (Duflo & Kremer 2003).

Consider a concrete example from the related field of water conservation. In Florida, a water conservation education campaign was designed to reduce water use on residential landscaping (Mulville-Friel & Anderson 1996). Communities that received educational materials decreased their water use by nearly 40 percent, which seems to imply that educational campaign was a huge success. However, inspection of water use among non-participating communities revealed a different story: nonparticipating communities had reduced their water use by more than 30 percent during the same time period. High rainfall was observed in Florida during the study period. Thus, the program accounts for less than 10 percent of the reduction in the participants’ water use.

When evaluating the effect of a conservation intervention, the analyst must worry about confounding effects – effects that are cotemporaneous with the intervention and mask the intervention’s effect. Examples of confounding effects include historical trends, other unrelated programs or policies, and unobserved environmental and social characteristics. As in all scientific research, confounding effects are controlled for through baselines, covariates and controls (Ferraro & Pattanayak 2005). Baselines measure pre-intervention conditions and behaviors, and thus control for initial conditions that may affect measures of program effectiveness. Covariates are observable factors that are likely to influence the outcome measure; these factors may be socio-economic, biophysical, economic or institutional. Controls are individuals, communities or areas that do not experience the intervention but are otherwise similar (on average). Only by comparing sites or individuals with an intervention and those without can a convincing case be made for the intervention’s effectiveness. Even if one cannot obtain measures on covariates or baselines, one should at least construct a control group.

Two potential problems deserve mention because of their widespread, and apparently not well understood, effects on the ability of conservation scientists and practitioners to make inferences about program effectiveness: (1) selection bias and (2) the Hawthorne effect. Selection bias occurs when characteristics that influence the outcome variable also influence the probability of participating in the program. For example, the local citizens who tend to be the first to volunteer to collaborate with outside conservationists are also typically those who have a conservation ethic or have low opportunity costs from conservation activities. Citizens with conservation ethics or low opportunity costs are less likely to damage the environment in the absence of any intervention (note also that local collaborators typically receive benefits from collaboration – wages, in-kind transfers, prestige – thereby further lowering their opportunity costs of conservation). Thus, the postintervention behavior of these individuals does not provide a useful measure of program effectiveness.

Another example illustrates the potential for bias in the other direction. Say someone wants to evaluate the effectiveness of community development projects on conservation outcomes by examining the correlation between desired conservation outcomes and the presence of development investments (e.g., Struhsaker et al. 2005). They may find zero or negative correlation, not because the development projects have no effect, but simply because such projects are not randomly assigned across the landscape. They are located in areas with the most pressure and therefore comparing the average outcome of areas with projects to the average outcome of areas without projects yields a biased (downwards) estimate of the effectiveness of community development projects on conservation
outcomes. The potential for selection bias points to the need for carefully chosen controls in program evaluation.

The Hawthorne effect is a well-known artefact where individual behaviors are altered because the subjects know they are being studied. The original Hawthorne effect concerned temporary improvements in a production process that were caused by the obtrusive observation of that process. In health, such effects are observed when targets change their behavior to please outsider observers, but when the outsiders are gone or have simply overstayed their welcome, the behavior reverts. In conservation, such effects are observed when communities know that the outside observer is interested in seeing hunting or habitat destruction decline and thus attempt to help the outside observer achieve the desired objective, at least in the short run when opportunity costs are small. In current evaluations of conservation interventions, a lot of people confuse short-term Hawthorne effects with success.

How can researchers avoid the pitfalls inherent in empirical evaluations of conservation interventions and draw reliable inferences about causal effects? The best approach would be to implement a field experiment in which an intervention is randomly assigned across individuals, communities or regions (Burtless 1995; Greenberg et al. 2003). If these entities are subject to an intervention at random, there should be no systematic differences in the characteristics that affect the outcome variable across control and treatment groups. Potential confounders are balanced across intervention and control groups and therefore any differences in the outcomes between the two groups can be attributed to the intervention.

Randomized experiments are an important component of policy design in fields such as poverty assistance, criminal rehabilitation, public education, and public health. In contrast, randomized experiments are not used in the field of environmental policy. Their absence, however, does not derive from an inherent difference between environmental policy and other policy fields. As an example, consider how a randomized design could allow practitioners to determine the effectiveness of a conservation education campaign. The practitioners first randomly assign education treatments across communities/regions. They then monitor conservation outcomes such as turtle nesting, meat or eggs collected or sold in markets, near-shore species abundance, or other measures of conservation effect (e.g., by-catch or strandings) in areas that receive the treatment and areas that do not. Thus, the approach requires that evaluation be built into the design of the original program and that data be collected on both treated and control communities, which can be expensive. However, this is what we do if we want to know whether a vaccine or public health campaign is effective, and program evaluation research in other fields has demonstrated that this technique can teach us a lot about behavioral responses to policy interventions. When randomized experiments are infeasible (as they often will be; Heckman & Smith 1995), one can draw from the large literature on quasi-experimental methods that can, ex post, disentangle program effects from other effects on the outcome variable of interest (Shadish et al. 2002). These methods include (1) matching methods, which select controls conditional on observable characteristics, (2) instrumental variable methods, which use additional sources of variability that are correlated with the intervention but uncorrelated with the outcome variable, and (3) ‘natural’ experiments, which exploit rapid, isolated changes in one aspect of the environment that allow one to make inferences about the effect of a conservation intervention (e.g., an oil price shock or natural disaster in a country that drastically reduces fishing effort may allow practitioners to evaluate the effect of reduced fishing effort on turtle strandings). For non-technical overviews, see Baker (2000) and Ravallion (2003). Ravallion (2001) provides an intermediate-level discussion of the key ideas.

Does this discussion of the need for program evaluation in conservation mean that all sea turtle biologists and field practitioners should train themselves in program evaluation methodology? No. Instead, they should familiarize themselves with the basic concepts and then seek collaborators from the social sciences who, when convinced that sea turtle conservation is an intellectually challenging and personally gratifying field in which to work, can ensure that we learn something from our investments.

Direct Payments for Sea Turtle Conservation

While attending the 25th Annual Symposium, I also overheard much discussion about offering “incentives” to achieve conservation goals. Examples of such incentives included “alternative livelihoods” for residents whose current activities detrimentally affect turtle populations, subsidies for fisherman to adopt new technologies, and payments to people who identify and protect turtle nests. In the hope that marine turtle scientists and practitioners will think more clearly about such incentives and avoid the mistakes made in the 1990s by terrestrial initiatives that attempted to use incentives, I wish to highlight current economic thinking about incentives.

My impression is that the evolution of strategies among marine turtle conservation strategies has paralleled the evolution of conservation strategies across the globe. The initial stage emphasizes regulation (rule and force) and moral suasion (convincing people through education that conservation is the “right” thing to do, whether it be for moral, aesthetic, ethical, religious, or scientific reasons or for the public good). After some time, regulation and moral suasion are deemed ineffective or insufficient. Conservation practitioners and policymakers thus increasingly focus on using economic incentives as a complement or substitute to regulatory and persuasive approaches.

These economic incentives generally fall into two classes: indirect and direct. One can divide indirect incentives into two categories. The first is what I call ‘conservation by distraction,’ where conservation practitioners attempt to redirect capital, labor and consumption away from activities that deplete species. This category includes interventions such as supporting the creation of “alternative livelihoods” (a phrase I read frequently in documents outlining strategies to protect turtles in low-income nations). It also includes attempts to encourage people to use substitute products. In terrestrial ecosystems, there is no empirical evidence that such indirect interventions work – and there is some evidence that they can exacerbate threats to biodiversity (by, for example, relaxing capital constraints on resource exploitation).

The second category is the valorization of species in situ, where one encourages commercial activities that produce conservation as a joint product. Examples of such interventions include promoting eco-tourism or the sustainable harvest of wildlife – both are perceived as increasing the incentives for local residents to protect wildlife for private gain. Although there is empirical evidence that the revenues from eco-tourism and wildlife harvest in some areas can be substantial, there is little empirical evidence that such initiatives lead to changes in behavior required to achieve conservation objectives (Kiss 2004; Wunder 2000). Such interventions may be good ways to raise money or awareness for conservation, but they are generally poor vehicles for achieving conservation outcomes. An alternative to indirect approaches to conservation is to pay for conservation performance directly (Ferraro & Kiss 2002). In this approach, host-country and international actors make periodic, conditional payments to individuals or groups that supply services of ecological value (intact ecosystems, targeted wildlife). For every hectare of rain forest protected or every turtle nest that successfully generates hatchlings, the protector is paid $X. The basic idea is that biodiversity is a valuable good and its protection is a valuable use of resources. Thus we should pay for it just like we pay for any other valuable good.

The most well known activities in this category are the terrestrial habitat conservation contracting initiatives of high- and low-income nations. These initiatives include the agro-environmental programs of high-income nations (OECD 1997, Claassen et al. 2001), the ecosystem payment initiatives of non-governmental organizations (e.g., Delta Waterfowl Foundation’s adopt-a-Pot-Hole program and Environmental Defense Fund’s Stewardship Fund), Costa Rica’s Program of Payments for Environmental Services (Programa de
Pago de Servicios Ambientales), conservation leases for wildlife migration corridors in Kenya, conservation concessions on forest tracts in Guyana, and South Africa and American Samoa’s “contractual national parks,” which are leased from communities. Although less common, there has been experimentation with payments for the protection of specific species, based on abundance or reproductive success, including attempts to pay landowners for occupied wolf dens (Defenders of Wildlife) and to pay communities for snow leopard and snow leopard prey abundance (International Snow Leopard Trust).

In sea turtle conservation, there are few direct payment initiatives, but there has been experimentation in several nations, including Kenya, Tanzania and the Solomon Islands, where outside agents make payments to local agents conditional on nest protection, hatchling reproduction or ensuring that turtles are not killed as by-catch (a payment example in Suriname from 30 years ago is described in Schulz 1975).

A lot of marine turtle conservation strategy plans talk about the need for compensation (e.g., the Bellagio Blueprint for Action on Pacific Sea Turtles). Direct payments are a form of compensation. Unlike direct payments, however, most forms of compensation are not tied to conservation performance in a credible and excludable way (e.g., building schools or health clinics for communities near endangered ecosystems). Thus compensation is generally a poor conservation tool (Ferraro & Kramer 1997).

Are any of these direct payment initiatives working? I cannot say for reasons to which I alluded earlier: conservation programs are rarely implemented in a way that you can learn about their effectiveness. As one its “Main Findings,” the Responses chapter of the Millennium Ecosystem Assessment finds that “[d]irect incentives for biodiversity conservation usually work better than indirect incentives.” Although there are good theoretical reasons to believe this conclusion is warranted (Ferraro & Kiss 2002), there are absolutely no data that support the conclusion.

Can such payments work in the context of marine turtle conservation? Without carefully controlled experimentation, we will not be able to say. On paper, the advantages of direct payment interventions are: (1) forging unambiguous links between human well-being, human actions and species conservation; (2) focusing practitioner resources on a small set of critical parameters: namely, institutions, which are critical for all conservation initiatives; (3) encouraging more efficient use of scarce funds – more conservation outcome per dollar spent; and (4) permitting precise targeting and adaptation across space and time.

There are, of course, disadvantages associated with direct payments. One of the obvious ones is that direct payments require sustained financial commitment – there is no prospect of shortrun investments generating long-run returns. Other disadvantages include requirements for conservationists to be precise about the desired conservation objectives (in order to strike a contract, results must be measurable), the inability of residents in remote areas without markets to turn cash or in-kind benefits into the commodities they need for survival, and the potential that by making biodiversity more valuable, one can exacerbate power differentials and corruption.

In the case of marine turtles, there are other constraints on the success of direct payments. Property rights for nests, eggs or migrating turtles are difficult to establish. I should emphasize, however, that the “rights” to which I am referring are only to receive benefits in return for conservation performance. We’re not talking about ownership over the species. The transaction costs for an individual or community to protect an infrequently seen, migratory species can be also quite high (i.e., the opportunity cost of protecting sea turtles is greater than the direct value lost). Furthermore, sea turtle conservation is a weakest-link production technology – you can secure 90% of the relevant life-cycle habitat, but if you miss a critical breeding area or juvenile feeding area, you may not experience success. Finally, monitoring performance can be difficult, although with technology advances, monitoring is becoming easier. Although one should not dismiss these constraints, one should also recognize that they are not insurmountable nor are they specific to only direct payment turtle conservation initiatives. Marine turtle conservation practitioners would do well to experiment with different forms of direct payments in a way that will allow the conservation community to learn about the effectiveness of these payments in helping marine turtle populations recover globally.

For More Information

Further reading on program evaluation can be found on-line at William Trochim’s (2001) The Research Methods Knowledge Base, 2nd edition (<http://www.socialresearchmethods.net/kb/index.htm>) and the World Bank’s Impact Evaluation Web Site (<http://www.worldbank.org/poverty/impact/index.htm>, then click on Impact Evaluation) Further reading on direct payments can be found on-line at Ferraro’s home page: <http://epp.gsu.edu/pferraro/special/special.htm> and <http://epp.gsu.edu/pferraro/publications>. On the latter page, easy to read articles include the short essay by Ferraro and Kiss (2002) and the longer essay by Ferraro and Simpson (2005).

BAKER, J.L. 2000.Evaluating the Impact of Development Projects on Poverty: Handbook for Practitioners, World Bank. Available on-line at <http://web.worldbank.org/>

BURTLESS, G. 1995. The Case for Randomized Field Trials in Economic and Policy Research. Journal of Economic Perspectives 9: 63–84.

CLAASSEN, R., L. HANSEN, M. PETERS, V. BRENEMAN, M. WEINBERG, A. CATTANEO, P. FEATHER, D. GADSBY, D. HELLERSTEIN, J. HOPKINS, P. JOHNSTON, M. MOREHART & M. SMITH. 2001. Agri-environmental Policy at the Crossroads: Guideposts on a Changing Landscape. Agricultural Economic Report no. 794. Economic Research Service, United States Department of Agriculture, Washington, DC.

DUFLO, E. & M. KREMER. 2003. Use of Randomization in the Evaluation of Development Effectiveness.” In Proceedings of the Conference on Evaluating Development Effectiveness, July 15-16, 2003, World Bank Operations Evaluation Department (OED): Washington, D.C.

FERRARO, P.J. 2001. Global habitat protection: limitations of development interventions and a role for conservation performance payments. Conservation Biology 15: 990-1000.

FERRARO, P.J. & A. KISS. 2003. Will direct payments help biodiversity? Response. Science 299: 1981-1982.

FERRARO, P.J. & A. KISS. 2002. Getting what you paid for: Direct payments as an alternative investment for conserving biodiversity. Science 268: 1718-1719.

FERRARO, P.J. & R.A. KRAMER. 1997. Compensation and Economic Incentives: Reducing pressures on protected areas. In Last Stand: protected areas and the defense of tropical biodiversity, R. Kramer, C. van Schaik & J. Johnson (eds.). New York: Oxford University Press, pp. 187-211.

FERRARO, P.J. & S. PATTANAYAK. 2005. Money for Nothing? A Call for Evidence-based Practice in Biodiversity Conservation. Environmental Policy Program Working Paper. Georgia State University, Andrew Young School of Policy Studies, Atlanta, GA.

FERRARO, P.J. & R.D. SIMPSON. 2005. Protecting forests and biodiversity: are investments in eco-friendly production activities the best way to protect endangered ecosystems and enhance rural livelihoods? Forests, Trees and Livelihoods 15: 167-181.

GREENBERG, D., D. LINKSZ & M. MANDELL. 2003. Social Experimentation and Public Policymaking. Urban Institute Press, Washington, DC.

HECKMAN, J.J. & J.A. SMITH. 1995. Assessing the case for social experiments. Journal of Economic Perspectives 9: 85–110.

KISS, A. 2004. Is community-based ecotourism a good use of biodiversity conservation funds? Trends in Ecology and Evolution 19: 232-237.

MARGOLUIS, R. & N. SALAFSKY. 1998. Measures of Success: Designing, managing, and monitoring conservation and development projects. Island Press, Washington, DC.

MROSOVSKY, N. 1983. Conserving Sea Turtles. British Herpetological Society, London.

MROSOVSKY, N. 2000. Sustainable Use of Hawksbill Turtles: Contemporary Issues in Conservation. Key Centre for Tropical Wildlife Management. Northern Territory University, Darwin.

MULVILLE-FRIEL, D. & D.L. ANDERSON 1996. Justifying water conservation through evaluations. Florida Water Resources Journal: 18-20.

OECD. 1997. The Environmental Effects of Agricultural Land Diversion Schemes. OECD, Paris, France.

RAVALLION, M. 2003. Assessing the Poverty Impact of an Assigned Program. In: Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. Chapter 5. Available on-line at <http://poverty.worldbank.org/library/view/12928>

RAVALLION, M. 2001. The mystery of the vanishing benefits: An Introduction to Impact Evaluation. The World Bank Economic Review 15: 115-140.

ROSSI, P.H., M.W. LIPSEY & HOWARD E. FREEMAN. 2004. Evaluation: a Systematic Approach. 7th Edition. Sage Publications, Thousand Oaks, CA.

SCHULZ, J.P. 1975. Sea turtles nesting in Surinam. Zoologische Verhandelingen 143: 1-143

SHADISH, W.R., T.D. COOK & D.T. CAMPBELL. 2002. Experimental and Quasi-experimental Designs for Generalized Causal Inferences. Houghton Mifflin, Boston, MA.

STRUHSAKER, T.T., P.J. STRUHSAKER & K.S. SIEX. 2005. Conserving Africa’s rain forests: Problems in protected areas in possible solutions. Biological Conservation 123: 45-54.

WUNDER, S. 2000. Ecotourism and Economic Incentives: an Empirical Approach. Ecological Economics 32: 465-479.