Ethics Review

I almost forgot to mention that I did this before contacting stakeholders….. but I did so here is a post about it.

Any researcher working with people needs to go through their school’s Ethics Review Board process. There are good historical reasons for such a process, even though the review process in-itself can’t protect against all ethical challenges you might face (you need to deeply think about how you view your ethics as a scientists as well). Interestingly interdisciplinary environmental research has a lot of ethical implications that are not always explicitly addressed in US and Canadian Ethics Review Boards.

The application process can seem like a formality (especially when you are doing research about a topic that isn’t inherently dangerous or personal) but can actually help you structure your ideas very clearly. Don’t forget to do this before you need to go out and collect your data with people (it can take quite a while to get your application “just right” and get approval). If you don’t, you could face serious problems went trying to graduate or publish your work later. I only have two small pieces of advice as university websites and other blogs have a lot of info on the process.

1. Ask for examples: get your advisor and labmates to provide you with successful applications you can use as templates. There is no reason to reinvent the wheel with format.

2. Go to info sessions and “office hours” for internal ethics review board specialists: My school offered both these options and basically allowed me to make sure I was doing everything right because submitting and thus shorten my review and revision time with the board. Take advantage of your university resources!

IMG_2453

McGill University, to represent the university structure present for ethics review and thus data collection and field work.

Advertisements

Choosing how to contact people, how to administer a survey and collect the data [part 1]

There are many ways to get people to answer survey questions, and many ways to collect the information they give you. Here is a summary of how I chose contact people and how to collect the data I needed through surveys:

Contacting (recruiting) people to take the survey:

There were three main ways to contact people 1) by email, 2) by phone, and 3) in person. Because I needed to interview more than one type of actor in the system (from a city wastewater manager to an individual backyard gardener), from the start I knew there probably would not be only one standard way to contact people. I based my initial list of actors to contact on city reports about urban agriculture and food and waste management and worked with the contact information that was publicly available (a lot of the time it was or a phone number or an email but not both).

My survey is quantitative and my project is a little hard to really explain in two lines so I like the email contact option because then you can really give them all the information they need and they can process it at their own pace. Email correspondence also means you have a record of your interaction with the person of interest which can be helpful. However, its much easier to dismiss an email than a phone call so the response rate seems to be lower.  Thus, although I favor email, if I don’t get an email back within a month of initial contact I think I am going to do call-backs on the phone.

Issue that needed to be addressed: I realized that because I was sending all the info up-front, the survey looked really big and time consuming (even though one can fill it out in 15-20 minutes not problem if they know the information they need to discuss). I thus started to add a note about this in the emails.

For a lot of the municipality-based actors, there are no publicly available email addresses. In these cases I came up with a basic script to present the project and ask them if they wanted to participate or could refer me to the person who would be best suited to discuss the information I wanted to obtain. Phone calls were the necessary entry-point but often an email needed to be sent after the phone call so that the actor could look at something concrete.

Issue that needed to be addressed: After one pretty negative phone call I realized that I was perhaps doing a good enough job explaining the project but not necessarily making the actor feel valued or showing them why it was important to them. I thus changed the speech and the email to try and make it more clear that I value their unique and expert knowledge and that I would share a report about my findings once I was done.

Ultimately the most successful recruitment is usually though your own contacts. Over the past year I have tried to build-up contacts in the Montréal UA community and thus could ask some of them directly (by email, by phone, or in person) to fill out the survey with me, or refer me to other people who could do it with me. Both of my field assistants (which I will talk about later) also have good contacts in the Montreal community. Having mutual trust is really key in getting people to take time out of their busy schedules to help you with a survey. I also systematically ask people I survey about other people I could contact, which isn’t as close of a trust relationship than existing contacts, but much better than a “cold call”.

Changing the plan: focusing on just one city

My thesis chapter outline and thus my data collection plans have changed quite a bit after giving a lengthy presentation to my lab group about my research plan for my comparative UA and P cycling thesis chapter, a presentation to my thesis committee, and many discussions with my advisor and other colleagues in the field.

I knew from the start that my initial plan to compare six cities with new data on UA was an ambitious one and that I might need to adapt as we went along. After careful consideration it became clear that trying to plan for more than one field season in more than one locations what not optimal. I had been working on protocols, collaborations, and survey designs that were in deferent languages and that all had to be adapted to the particular realities on the ground. For example the type of UA practices that are applicable in a multiple choice question in Montreal are simply not the same as in Ghana.

In order to focus my (and everyone I work with) energy, I am going to start by characterizing UA and P cycling in one city by doing the field work this summer, and then consider comparisons and collaborations once that is done.  In the end I think this is probably the best approach as it’s a clear goal that will result in thesis chapter. I have not given up the idea of cross-city comparisons. I am working on the literature portion of such a comparison with a wonderful groups of academics at the moment, and if my field work goes well in Montreal I will try and apply it to other cities for comparison.

100_0596

View of the Montreal Botanical Garden Greenhouse from the top of the Olympic Stadium.

What should I actually be measuring: local context

I have been focused mostly on the P budget but through my survey and literature review I need to also look at local context.

955_0171 IMG_0414

Example of two very different biophysical contexts between in British Columbia, Canada (top) and New South Wales Australia (bottom). Photo Credit: Genevieve Metson and Carissa Taylor

I want to better understand the role of local biophysical, social, political and economic context of a city really to better understand what type and extent of UA (and P management options in general) are best suited for particular locations. But what should I actually be considering as “local” context? I think the idea of nested scales makes identification of variables (or drivers) for local context difficult. Existing waste management infrastructure in a city doesn’t happen in a vacuum, its actually dependent on may processes happening at larger scales, including national policies, the state of technology, global economic trade status ect. Its really the same thing with biophysical properties like precipitation or growing season length, but at least these a little easier to define.

Within the four categories of context (biophysical, social, political, and economic) I have made two subclasses: 1) variables that are systematically collected and/or available at the city level or regional scale that, based on literature in ecology, agriculture, or urban planning, could influence UA and P cycling, 2) variables that stakeholders (people) within the UA or food and waste management sectors believe influence how they make decisions that then impact UA and P cycling.

In the first category I included variables like (these are just examples, there are many more possibilities in each category):

Biophysical: annual precipitation, average temperate and annual variation

Social: food security measures, demographic make-up, employment type and amount (in particular agricultural training)

Political: existing laws on land-use, fertilizer and amendment use

Economic: type of waste collection infrastructure, water management system, income mean and range, market availability

In the second category I planned to ask interview questions about why they manage inputs, production, and outputs the way they do, and what they view as social norms, policies and regulations, technologies, and other factors that facilitate or constrain the practices they currently or would like to do.

I was hoping there would be some overlap between the variables mentioned in interviews and what I had identified in the literature. Ultimately, the second category is measuring something different from the first. It is measuring the perception of facilitator and barriers to certain practices, which is in itself a variable (perception that is). Ultimately peoples’ motivations and perception of the system are important factors in understanding how to change a system. However, considering the time and expertise it takes to properly design and analyze interview data, I probably over reached and I would have never been able to give these aspects of the research questions the time they need.

Developing survey questions

IMG_3146

printed copies of my survey in French and English to give out to participants

I know what information I need to calculate P flows in the city, but asking questions that actually get the information I need is really a separate topic in and of itself. What seems straight-forward to me, in my PhD student context and love of P cycling, isn’t necessarily as clear in the context of a waste manager, a market manager, or a farmer/gardener. Apparently there is an art of developing survey questions, and I am giving it a try.

For every P flow I want to calculate I try and break the flow into components that make sense to the person I will ask. That is, I might be looking for one value for the P coming into the city, but if I am interviewing a gardener who works on his own little plot he will not know about the city scale, and possibly not about P specifically. I need to break the question down to the type of inputs he might use in his garden that contain P, and ask him about how much he uses in a year and where he gets these inputs from.

Every time I write up and rewrite a version of the survey (which is different for every type of actor in the system I want information from) I ask myself:

Am I asking the question that will give you the answer I am are looking for?

In order to test if indeed the answer to the question above is yes, I took the following steps:

0: Review literature on surveys, and articles that used surveys for data collecting and look at them.

1. Doing a pilot survey to see what happens:

I gave my survey to friends and family through email (and 2 in person) who garden and asked them to fill out my survey. This exercise gave me the opportunity to see where people didn’t answer at all (because they didn’t understand or it was “too hard” and where they didn’t answer in the way I expected or needed them to (i.e. not putting the information I was really asking for). It also allowed me to see if how the “delivery” method of the survey worked (online vs in person). I was lucky enough to have a friend with a master’s in social psychologist who has done a lot survey work in my pilot group. She was thus able to give me some specific design suggestions about things in my survey that were not working the way I wanted them to.

2. Make changes to the survey based on feedback: Here is a list of the big changes I made to my survey after the pilot study.

-Put more hierarchy in my questions by asking my “big” questions organized in categories (general information, inputs, production, consumption, ect) but asking very specific sub-questions.

-Minimize open-ended questions by creating pre-determined and organized answer choices with multiple choices, drop down menus, and fill in the blank tables. I always allowed for an “other” category, but giving choices increases your chances of response as the person taking the survey doesn’t need to concentrate and remember as much. The one big draw-back is that it makes the survey look very long, even though it really isn’t (only 8 questions).

-Add prompts for each section of questions that explains the goal of the questions (giving context for the survey respondent and also allowing them to “switch gears” between sections)

-Add definitions or examples for any terms in the questions or answer-choices that might be (or at least I saw in the pilot) interpreted more than one way or might be technical terms.

4.Get survey reviewed by peers, and experts in your study system:

Its good to make the survey answerable, but it also needs to be collecting data that we can use as scientists and to answer research questions. I got my advisor, field assistants, collaborators, labmates, and colleagues in social science labs to read through the survey. One example of feedback at this stage was that adding some of more general (non-quantitative) questions could ease the respondent in and then he/she is in the right mind space to answer quantitative questions. My collaborators at another university have more experience in the urban agriculture field in Montreal so their feedback and comments were extremely important to validate questions and survey design.

5.Match each survey question to a P flow I want to quantify in the city :

This step allowed me to see if I had any questions (and possible answers) that were not directly relevant to the data I needed to quantify flows or if some flows did not have an associated survey question (or other possible data source). It was important to note that some questions were there to determine “local context” and allow for general data collection on which to make assumptions if the respondent did not provide quantitative data (e.g., “check all the inputs you use on this list” instead of just “what quantity of the following inputs do you use”).

6.Create equations (with units and data sources) for each flow to be quantified and check it against the list of survey questions:

Step 6 is in some ways the opposite of step 5 and gave me the opportunity to really make sure I was collecting all the data I needed to calculate my flows.

7. Test survey with a “real” respondent in the field:

Once my collaborators and I were ready to start collecting data we chose an actor we knew would be responsive to taking the survey and had a lot of the information we were trying to collect already documented. After filling-in the survey with them, we realized that we needed to sightly simplify the level of detail of our answer choices as if this respondent did not have this detailed data it was very very unlikely others would have it. We realized we could not ask for input data for each type of garden but rather for all gardens managed by the actor or by site they managed.

Once the survey is created, one still needs to contact respondents and administer the survey. There are many options for both of these steps and I will discuss some of them in my next post.

Collaborations and the Titanic

image credit to Anthony Kosner post on Forbes.com

My friend Sandra shared some wise words a few weeks ago that I want to share with you all.  I was in the midst of a “mini-crisis” about how I needed to take a step back on this urban P and urban agriculture work because I moved into methods too fast and wasn’t doing things right and needed a new-and-improved focused game plan. She said:

“Sometimes research is like navigating the goddamn titanic. You’ve pulled a teeny bit too much to the right and its fucked up your trajectory. So now you’re going to tug a little to the left and you’ll wind up going straight in no time. And to continue on with the metaphor, collaboration are the icebergs (hehehehe) you can go past them and take some pictures but you don’t want to drive straight at that shit cuz it’ll fuck you up.”

Ok, she was helping me thorough a tough moment and obviously both of us value collaborations as we are both looking at sustainability and we know inter- and even trans-disciplinary collaborations are necessary to answer a lot of the questions we are both interested in. That said she makes a good point. Collaborations may be wonderful but sometimes, if they are not done properly, they can take you off course. There is a really delicate balance in trying to maximize synergies for data collection when you are studying the same system, but also ensuring that you both get the information you need to answer your respective research questions.

This balance doesn’t apply to all types of collaboration. I have had great collaborations where we co-create the research questions and this duality between data collection and research questions does not exist. I am talking here about collaborations that are mostly focused on data but which will also hopefully result in some great comparative, or larger system, co-publications.

At this particular junction in my work I think collaborations for this project are important because:

1. I do not want to over burden actors in the UA community as they often get the same requests over and over and in the end it makes them unwilling (and with reason) to keep helping researchers that are not giving anything back to them.

2. I do not want to duplicate things that have already been done. (Granted this is actually a very good thing to do, but with time and budget constraints it is hard to justify if the main research question is not about measuring the validity of that past work.)

3. I can’t do all the collection myself in every single city for every relevant actor in the urban P cycle and UA sector. Collaboration could thus allow me to cover more ground and better answer my questions.

4. My research questions might be focused but my interests and the relevancy of my work intersect with many other researchers objectives and findings. Bringing them together could really increase our understanding of agricultural systems, urban ecosystems, ect.

The issue is, and I have actually mentioned it before with regards to using existing literature values, differing research questions create results and data collection that may appear similar to your own data needs, but in the end are not. This isn’t making me shy away from collaboration. On the contrary!  Collaboration is actually a tool that forces me to be more critical and iterative in my survey development and my research design in general. In fact having many perspectives on the same survey may allow me to correct flaws I would not have seen alone. There is upfront time spent forging the collaboration but hopefully time saved having help collecting or analyzing data and publishing great papers. Still one needs to weigh the benefits in each circumstance.

I need to make sure that I am not setting my goals on an iceberg and rather that I am navigating, with tiny pushes left and right, to an interesting and robust thesis chapter.

Selecting methods [part 5]

“Primary” field data collection:

From a purely SFA and natural science perspective I might say (even though I am aware this not entirely true) this is the gold standard. Collecting your own data specifically for your research questions is in many ways ideal if you have no time or money constraints. (Note that it doesn’t mean not using other methods because you probably still want to verity your results).

Keys points I take into consideration for SFA:

  • Particularly useful to collect “place-specific” P content measurements. (P content records might exist for wastewater management. But for flows operating within the city (i.e., smaller than standardized government record keeping) on-site measurement could be great (e.g., compost, runoff from UA sites, and soils)
  • Would require a substantial amount of man-power to measure flows of food, UA practices and production. Even if we only did a sub-sample, we do not have a good idea of within city variability (although it is probably high) so targeting sample size and representative sample groups is more difficult. Same problem with P content sampling.

These are not problems within themselves, they just make field data collection perhaps beyond the scope of a 6 city comparison.

Keys points I look into consideration for local context:

  •  Some of the same pros and cons as for SFA but for different variables (e.g., rain, temperature, ect).
  • For some of the more social factors observation or participant-observation (of people and practices but I think also documents) might be useful but time consuming. Such “field” data-collection can tell you something different that asking why people do what they do.

Based on these criteria (“selecting methods” posts parts 2,3,4, and 5) I am opting to use literature review, surveys, and review of official government and organization reports (I put this separate from literature review as the objective of the publications are different). By using all three of these data sources I hope to get the most accurate data possible, and when possible triangulate bias by looking at the same P flow or context factor with more than one data source. Next I will write about my experience developing survey questions, which is KEY to quantifying P in UA and thus to my whole thesis chapter.

labmates doing field work in Monteregie, Qc.

Labmates doing field work in Monteregie, Qc.