A Review of Reviews
In less than a week or so those who submitted research papers to the AEJMC conference should find out if their papers were accepted, which got me to thinking about the research review process. As much as we try to make journalism and mass communication research as scientific as possible, the review process remains one of the most arbitrary and subjective processes imaginable.
To me, the biggest problem is a lack of consistency/standards. There's nothing an author can point to and say, "OK, now I understand what the rules are." Every journal is different and judges to its own standards. I submitted a paper to a journal one time and it was returned with high marks, but one reviewer wanted a revision because the theory used was inappropriate. I took out the theory and resubmitted, and it was then rejected because it didn't have theory! No wonder authors get frustrated.
There's also the issue of bias, which is present any time human beings are involved in a process of this sort. Sometimes I wonder who's looking at the paper, what kind of mood he/she is in, is there any professional jealousy involved, etc. To be sure, most of the reviewers seem dedicated and professional. But it also seems that with every journal there is some small cadre who's job is always to say no and be extremely unpleasant about it.
A couple of years ago a researcher at the University of Texas (whose name escapes me at present) sent out a survey asking for feedback on some of these issues. I filled out the survey, but have never seen the results anywhere. If anyone knows where they are I would be interested in seeing them.
In the end, the research review process is a lot like the Nielsen television ratings system--it has a lot of flaws, but for the moment its the best, and only, system we've got.
To me, the biggest problem is a lack of consistency/standards. There's nothing an author can point to and say, "OK, now I understand what the rules are." Every journal is different and judges to its own standards. I submitted a paper to a journal one time and it was returned with high marks, but one reviewer wanted a revision because the theory used was inappropriate. I took out the theory and resubmitted, and it was then rejected because it didn't have theory! No wonder authors get frustrated.
There's also the issue of bias, which is present any time human beings are involved in a process of this sort. Sometimes I wonder who's looking at the paper, what kind of mood he/she is in, is there any professional jealousy involved, etc. To be sure, most of the reviewers seem dedicated and professional. But it also seems that with every journal there is some small cadre who's job is always to say no and be extremely unpleasant about it.
A couple of years ago a researcher at the University of Texas (whose name escapes me at present) sent out a survey asking for feedback on some of these issues. I filled out the survey, but have never seen the results anywhere. If anyone knows where they are I would be interested in seeing them.
In the end, the research review process is a lot like the Nielsen television ratings system--it has a lot of flaws, but for the moment its the best, and only, system we've got.
posted by Brad Schultz at 8:02 AM
0 Comments:
Post a Comment
<< Home