Discipline and publish

[rivkat]

I’ve been thinking a lot about this fan/sex survey thing going around. I found the kink_bingo mods’ response compelling, but I have a couple of concerns to add.

First, institutional review boards (IRBs) are bad. [ETA: I'm leaving this in for accuracy, but it really was an overstatement that I should have avoided.  I still believe IRBs are not generally fit for purpose, but that doesn't make every IRB bad, especially when discipline-specific standards are applied.] They’re bad for social science and they’re bad for ethical thought, too often substituting medical models for the best, most informed debate within a particular discipline. Not everyone is subject to an obligation to do no harm (think historians: should they care about the sensibilities of the Kennedy family?), and IRBs often don’t know what the relevant risks of harm from a particular course of research even are. I occasionally need reminding of this bit of trivia: the Stanford Prison Experiment—a core example of what IRBs are supposed to avoid—was IRB-approved. It was stopped because (one of) the experimenters recognized a violation of their own professional ethics.

One could persuasively argue that reporting the netporn survey authors to the IRB is a viable strategic move, attempting to enlist a more powerful entity against people who have more social/educational capital than pseudonymous fans are perceived to have. I still think this is a mistake, because IRBs use the very same totalizing, naturalizing models of personhood, risk, and harm that the original kink_bingo response so powerfully deconstructed. Fans can assert the entitlement to be disciplined only on their own terms, and I think we should, but the IRB move goes against that commitment.

Second, I think vidders’ recent debates about visibility are relevant here. Fanwork creators are visible: Hollywood knows, lawyers know, the Simpsons writers know, teachers know. This means that nonfannish (and non-antifannish) academics are going to follow their noses to us once in a while, no matter what. We can and probably should put greater trust in our homegrown academics (a set that overlaps with, but is not identical to, fan theorists) than in outsiders, and we can definitely decide not to help educate outsiders—or to do so. None of us individually has any obligation either way, nor do we have a community obligation to explain ourselves or to stay silent.

But beyond reaction to this survey, I suggest we should think about the benefits of visibility as well as the risks. Social scientists, like the proverbial drunk under the lamppost looking for his keys, like to go where the data are. If—and it’s a huge, difficult if—visibility dispels the stereotypes that fannish pleasures don’t exist/are somehow flawed/etc., then that’s a good result, and it won’t happen without some kind of visibility. Francesca Coppa has made the point that vidders do well in the art world and the legal advocacy community because vidders are organized, articulate, and show up to talk about their interests. When you become a prototype, social policy starts being made around you. (Or against you. This isn’t inevitable or riskless.)

Any subcultural group that asserts the entitlement to define its own worth, I suspect, ends up with its liberals (me) and its radicals (kink_bingo mods, I think, but that’s my characterization). Because I think it’s inevitable, I’m more interested in talking about strategic moves and self-understanding than in convincing anyone to do anything—other than, here, rethinking the deployment of the IRB.

Comments on LJ

Comments

[jadelennox]

I absolutely won't argue that IRBs don't need some reforming, because they do. I agree with you that some of them have models of what constitutes harm that are either (depending on circumstance) overly strict or overly lax for non-medical disciplines. But I disagree with you that they don't serve any purpose at all. For example, in this case, any IRB worth its salt would have insisted on a clear disclosure statement about privacy and about long-term preservation of data. Moreover, they would have looked at some of those questions and insisted that at least be a token attempt to turn away minors.

Certainly IRBs are, like the rest of the world, having to adapt to new models of doing research on the Internet, but I think that those very simple additions (disclosure statements, attempts to prevent minors from reporting information about their potentially illegal activities) would have actually made a big difference.

I find your critique of the IRB interesting because I actually think it reflects very much what went wrong in this particular instance. Discussing the incident with my colleagues we came to the conclusion that because these guys aren't social scientists, they have no idea that the social sciences have their own view of what constitutes "harm", and they went by the medical model of "it's a survey, how could it possibly do anybody any harm". As flawed as IRBs are, they still would have caught some of the medical-focused mistakes these guys made.

(incidentally, I find your statement about historians of course not being required to do no harm really thought-provoking, not because I disagree -- I don't! -- but because I've known many archivists (professionally inextricable from historians) who act as if they have an ethical responsibility to do no harm, although that is not part of the archivists code of ethics as accepted by the SAA.)

*blinks slowly*

[elke_tanzer]

I am, as always, learning so much from fandom... I have never previously encountered the SAA, and now I'm reading http://www.archivists.org/about/history.asp.

Re: *blinks slowly*

[jadelennox]

it's a really interesting organization. A fairly hidebound field, adapting to a profession changing at warp speed, and a professional organization that's trying to help with various levels of success and being part of the problem itself.

Re: *blinks slowly*

[elspethdixon]

If you're super-interested, you might want to check out ARMA (American Records Management Association) and the Association of Canadian Archivists (http://www.archivists.ca/), too. jadelennox is right -- the archival profession's been changing incredibly quickly and in profound ways with the digital age (even if you're only looking at the practical aspects, digital formats require completely different preservation techniques), and there are all kinds of issues of intellectual property, authenticity, privacy, etc. that are being re-evaluated. In light of the actual topic at hand -- one of the issues with state archives is that appraisal (the decision process re: the value of documents and what information needs to be preserved) is not as objective and neutral a process as people would like to think. Certain kinds of records and certain kinds of record creators are more likely to be dismissed or preserved than others, and simply being in the position to make those decisions gives people a very real kind of cultural power in a historical big picture sense.

[rivkat]

I agree that these guys went wrong by attempting to do social science they weren't trained to do. IRBs are at most accidental places to get that sort of training (my IRB training was entirely medical, which has been the standard training model for years; a few places are developing discipline-specific models, but these guys would have had to have known enough to get trained in a different discipline in the first place). What they needed was a source of disciplinary ethics. Many IRBs are nurses & doctors, so they are extremely unlikely to catch the problems here.

I disagree that surveys should, in general, be thought to offer a potential to cause harm sufficient to trigger IRB review. In fact, the government says that surveys are generally exempt from its IRB requirements, though lots of institutions have covered them through mission creep. Now, absolutely privacy and data security should have been part of the process. But that's so little of what IRBs do, and they do so much that's counterproductive, that I feel compelled to argue against their involvement.

[jadelennox]

Interesting. At my institution, there are two different IRBs, one for the medical people and one for everybody else, so the medical model is absolutely not used for the non-medical research.

[(Anonymous)]

I'm part of a discipline-specific social science IRB, and all I could think through this whole clusterfuck was that people like this are why the rest of us can't have nice things. Seriously. I know IRB review is usually excessive (I'm a fan of that blog) and I'm the primary voice on my IRB for moderation and common sense. But all that red tape is there because apparently common sense isn't enough for some people - this was so far beyond the pale I seriously couldn't believe it was happening - and I wasn't the only one, looking at how many people thought they were trolls or engaging in identity theft.

[meara]

As someone who does medical research, I have to agree with jadelennox that I find your assertion that "IRBs are bad" a bit black-and-white. Certainly there are times when I am puzzled by the variety of answers I get from various IRB/ECs. And I can certainly see how the medical vs. social research fields require very different things from an IRB, and depending on what sort/level of IRB you are dealing with, the people involved may not have the required knowledge. Or may be dealing with their own biases, politics, etc. Human nature.

But that said...I don't think they are bad in total! Having a committee that oversees the ethics of research? That is supposed to protect human subjects? Having had to take courses repeatedly about the reasons there are IRBs, and the Declaration of Helsinki, Good Clinical Practices, etc...

[rivkat]

Ethical standards are vital, and oversight is a good idea. But IRBs are a terrible implementation of that good idea, and specifically they are terrible for social science because they have expanded based on the medical model even as they have encompassed more and more nonmedical research. My IRB training was all about medical ethics, and I'm a lawyer. There are many places where IRBs comprised of doctors and nurses oversee all research, and they just can't do it. I need dialogue on the ethics of participant observation and pseudonymity, not hypothermia experiments. Discipline-specific review, or, as the author of the IRBlog linked in the main post suggests, ethical bibliographies as a required part of research proposals, would be much better ways of getting researchers to think about their ethical obligations given their particular research aims.

However...

[dakiwiboid]

will these people GET any discipline-specific review? Are they thinking about their ethical obligations? What I've seen of this "study" and their plans for the book make me suspect that neither of these things are true.

Re: However...

[rivkat]

Well, they're certainly getting an ethical education now. Although IRB approval might (or might not) shut down this survey, it also empirically does awful things to legitimate research. In a world where IRBs have often tried to take over ethical evaluation, there's no perfect solution, but yelling at them and trying to get their department involved are better, in my opinion.

[laurashapiro]

Thanks for this.

[robin_arede]

Fair disclosure: I am a lit/creative writing person with recent self-training and experience in internet research and IRB issues.

I agree: IRBs can be bad (there are some good sites I read about the problems of what happens when models developed for one or two disciplines are applied to others). They're also often bad in that they're only justification is keeping lawsuits from happening: I actually hold myself to a higher ethical standard than my university IRB (and I am an IRB member).

NO IRB person I know about would claim that an historian cannot write things that might make a Kennedy upset (public figures, disciplinary values). Nobody is going to tell a current student of mine that she cannot analyze the public documents put out by the Barbara Bush foundation in her analysis of literacy rhetoric because Babs might feel bad. That's just bizarre, completely a straw man argument.

Each university sets up its own IRB because the federal government mandates it. IRBs can be horrific (we had an IRB chair years ago who refused to allow ANY qualitative research whatsoever--the qualitative people cheered when he was replaced). But that's true of any university entity (or ANY group of people). Just because some politicians do bad things is not grounds for claiming all politicians are bad (though these days...)

And not all IRBs are the same, do the same thing, or work the same way.
They can and do recognize the validity of other professional standards and models (the way oral historians have in fact created their own standards, ditto journalists, etc.). I know because my campus one does, and I doubt we're the only ones out there (odds are the bigger name the school, the more pressure to confirm there is).

I am not against scholarship on fandom: I DO scholarship on fandom. I am not the biggest fan of anonymous surveys, but I can live with them when they're well done. The issue is not "no scholarship," but the need for ethical, well informed, useful scholarship (and useful to more than just the academics involved).

And don't forget this group isn't doing scholarhip: they're doing a pop book on what "netporn" teaches "them" about the human brain.

[rivkat]

It is awesome to hear about people of good faith involved in the IRB process, especially non-medical people who recognize disciplinary variation. And yet historians and folklorists and others are routinely told that they can't do what their disciplines tell them to do. The question is whether the IRB is a good way to get researchers to adopt appropriate ethics, and in practice the answer is too often no; the IRBlog I linked to in the post is full of horror stories. The question is whether the costs of centralized review stemming from the medical model are worth the benefits, and with disciplines engaging in their own ethical debates I come down on the side of "no."

Incidentally, the federal government does not mandate IRB coverage of significant chunks of university research; that's mission creep.

And I should be clear that I think this is a bad book, but I don't think pop scholarship is necessarily not scholarship.

[crypto]

I wish I had some deep thoughts to contribute to the discussion, but mostly I just agree with you. Then again, you had me at the subject line.

[brownbetty]

Last word on Ogi Ogas and ilk is that BU sez they aren't associated with, and they really wish he'd stop using their name. So it doesn't seem likely that the IRB will end up being part of this situation.

But, I really think you are correct that what we ought to see more of is good fandom-study driving out bad, FUBU, so to speak, and I live what I've seen of it so far, and I'd like to see more. I think if more fan-academics conducted more studies openly, too, fans would start to see what an ethically designed study looks like and be more suspicious of studies that don't resemble that. ("Wait, where's your statement on data retention?")

[rivkat]

Very interesting news. Incidentally, Virginia has a law purporting to require all people engaged in human subjects research to go through an IRB, but those laws are probably unconstitutional and as far as I know there's never been any enforcement.

I'm more used to journalistic enquiries not framed as surveys (making them arguably much more scientific than this one). No one ever asks journalists about their data retention policies!

[rivkat]

Oh, hey, do you have a link for BU's response handy? I have been lost in the wilds of this debate and my personal IRB enthusiast would like to know.

[brownbetty]

Not directly from BU, but [Bad username or site: deadlychameleon" @ livejournal.com] reports her(?) experience, via bossymarmelade.

[rivkat]

thank you!

[brownbetty]

Woops, html fail, but at least the link works. *facepalm*

[brooklynmili.livejournal.com]

Here via MetaFandom, hi.

I'm fascinated by this post, because I just came back from talking with Peregrine Schwartz-Shea (http://www.ipia.utah.edu/faculty/affiliate_faculty/schwartz_shea.html) at a reception, and we chatted briefly about her new work on IRBs. (Some of it's been published --I have the cite lying around if you haven't seen it and would want to.) One of the arguments she uses that the IRB model assumes that subjects come to the researcher, which means that researchers who go find their subjects are in a totally different positions, and the set of ethical questions raised are different. Of course, that's the problem here--the bad!survey!folks went to find some subjects, and did so in a way that was utter and complete fail, and that an IRB might not have been able to prevent. (But the IRB might have been able to look at that survey and say WTMFF? Because, oy.)

I'm with you in the not-the-IRB-but-yes-ethical-overview category. I'm also guilty of being one of those interpretivist researchers who avoids the IRB via magic tricks with hands. After six months of working on my IRB application for ten minutes, and then yelling at it about how it's all wrong and stuffing it back in its metaphorical drawer until my dissertation advisor yelled at me about it again, I finally reread the section on "exempt research," and wrote a compelling, yet bullshit, paragraph on how ethnography was covered under the "observing subjects in their everyday environments" clause, and therefore I only needed permission to do interviews, which is the smallest possible subset of my research. And they approved it without question. But, you know what? I spent a ton of time working up ideas with myself and my advisor about the ethics of being an ethnographer, particularly a majority-community ethnographer working in a marginalized community. That was where the work happened. The IRB couldn't have helped me with that--but I would have loved to have sat down with a hypothetical mirror universe IRB that could have.

I also happen to think that a more complicated set of questions about research ethics provides a slightly different take on the "outsiders shouldn't study us" question. Because, yes, we can't help but be studied. But I turn that around, as I do in my own work, and reframe it as "outsiders who want to study Others, especially others who are marginalized or have a sense of marginalization, need to gain permission from the community they study in a complex way, which includes engaging with the actual categories that the community uses to think about the issue under study, and should have an eye to providing some sort of recursive benefit to the studied community." And this is not just the discourse-theorist ethnographer talking; I think this applies to research with human subjects across the board.

Anyway--fascinating, and sorry for going all tl;dr at you. Also, your title: best evar.

[rivkat]

Thanks--I've seen descriptions of her work but haven't read it myself. The work you describe--essentially an ethical bibliography as part of the research itself--seems like a much more promising solution.

I generally agree with your last point, but I'd sound some notes of caution: (1) The key mistake that occurred with IRBs was to treat things like ethnography as human subjects research, when at most non-medical lab studies (behavioral psych) really fit the model. Is the language "human subjects" with its medicalized distance even a proper fit for ethnography, oral history, sociology, etc.? (2) Defining recursive benefit is really hard, and awareness is really all we can ask. Suppose you conclude in all good faith that (to make up an example) the Amish suck -- in only the most Kantian sense, if at all, can your book be considered to benefit the Amish, though you might be able to claim that you're honoring the interests of some subgroup over another. This may be clearest with biography. (On the other side, some communities may consider "advancing outside knowledge about and thus sensitivity to this community" not a benefit to them at all -- what you and I think are recursive benefits may only be so by our terms, not theirs.)

I'm glad you liked the title!