Peer Review [Was: RE: Review process (was: [jox] New Draft CFP)]
- From: "Mathieu O'Neil" <mathieu.oneil anu.edu.au>
- Date: Thu, 20 Aug 2009 13:56:38 +1000
Hi all
OK, I've had a bit of a think about the review process. In particular I read
the article referenced by Graham a few weeks ago:
"Reinventing academic publishing online. Part I: Rigor, relevance and
practice" by Brian Whitworth and Rob Friedman.
First Monday, Volume 14, Number 8 - 3 August 2009
http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/viewArticl
e/2609/2248
It is really excellent and paints a good picture of how the academic
publishing system works right now. In particular, the extraordinary power
that editors of renowned journals have because publishing with them is THE
barometer of quality. Here is a quote:
"In competitive economics scarcity reflects demand, so high journal
rejection rates become quality indicators. This creates a self?reinforcing
system, where exclusive journals that reject more attract more, as their
exclusivity makes them more attractive. When journal ?impact factor? is
number of citations divided by number of publications, publishing many
papers dilutes a journal?s citation ranking (Lamp, et al., 2007). When
exclusivity is based on rigor, avoiding faults becomes more important than
new ideas. Wrongly accepting a paper with a fault gives reputation
consequences, while wrongly rejecting a useful paper leaves no evidence, as
it doesn?t see the light of day."
So they advocate opening up the review process (reviews would become visible
so reviewers would get some recognition) while maintaining its blindness
(the identity of the author is secret during the review process). However
how this would work will only be revealed in Part II - when will this come
out? Next month perhaps.
They advocate fighting the exclusive property rights of commercial
publishers by setting up open-access journals.
They also mention journals with a choice between heavy (three reviews) or
light (one review) editorial processes.
Now, this is a complex issue. For example, I have published very few
articles in peer-reviewed journals. Pretty close to zero. I am working on
two papers right now. They are important to me, the result of several years
of thinking. I know I could (perhaps should) do the noble thing and send one
or both to an Internet-based journal. But deep down I feel that I would like
the validation that only an established journal can give. When I have
published one or two papers like that - showing that I can do it - then I
would submit to open-access journals. I think a lot of emerging academics
think like that - or they can't afford not to think like that if they want
to get a job in this super-competitive environment. I have a job, so I am
more motivated by professional pride. I am trying to be honest as possible
to advance the discussion. What does this mean for CSPP?
In a nutshelll: we need some incentives for people to publish with us.
Here are some random thoughts.
1- We could encourage people to submit papers that have already been
published in a closed journal - that way they would reach a new audience and
get more feedback.
2- For this we need good reviewers and good comments. There could be a
system where open discussion on a list leads to editorial comments being
appended to papers. Not sure. I don?t think we should rely on "anyone can
comment" to do this job - no-one may comment or comments may be mediocre.
3- There needs to be some clear guidelins for an open comment process:
-- closed editorial list / closed registration process?
-- deadlines for comments to be made?
4- It is clear that different review processes could be useful. We need to
define precisely the different review processes: blind or not, open or not,
etc.
5- The second part of the paper cited above may have some interesting
leads...
6- In conclusion: we eventually need to get some more people on the
editorial board to help advance how the review process works. We will need
some input from the people we will be approaching to work with us. So we
need to progress the rest of the "charter" so we can start approaching
people.
Cheers,
Mathieu
-----Original Message-----
From: owner-journal oekonux.org [owner-journal oekonux.org] On Behalf
Of Stefan Merten
Sent: Monday, August 17, 2009 6:42 PM
To: journal oekonux.org
Cc: Stefan Merten
Subject: Review process (was: [jox] New Draft CFP)
Hi Mathieu, Athina, all!
Last week (11 days ago) Mathieu O'Neil wrote:
d) Regarding peer review
I suggested the following: for research papers, authors can request a
traditional double blind review. But following this process, research papers
(as all other submissions) will be collectively discussed on the list.
I think it is an interesting idea to have different processes.
However, I'm not sure about the consequences. What do others think?
For those not too deep into the traditional process: Could you please
explain what are the features of the traditional double blind review?
Last week (10 days ago) Athina Karatzogianni wrote:
About d, I think it would be prudent to think about the implications of
discussing papers openly on a list. perhaps people will be much less
critical of a work once it is openly discussed.
That was a concern mentioned before. If this point is important then
it would indeed impact the quality of the journal. This would be bad.
Who would be able to see
this discussion?
It depends. Oekonux lists are usually published on the site but we can
also have a non-archived list. On such a mailing list a discussion
would be open among the editorial board but closed to the public.
Also there can be exchange based on personal e-mail. However, I'd find
it bad for transparency if regular personal e-mail exchange would
occur unless it is between persons who are working closely together on
a particular task - such as reviewing a contribution. To prevent this
I'd rather suggest a second, non-archived mailing list.
what if one of us wanted to publish a paper, would we look
at the reviewers comments while they were formulating them?
What's wrong with this?
I think some
thought should be paid there. The tradition is to have 2-3 blind reviewers
for a paper.
See above. Can you please explain what "blind" means exactly?
I dont see and please explain to me how when ten people have a
long discussion over an email list, quality and speed improve. I think it
will be quite the opposite.
IMHO this depends much on the culture of such a list. I know most of
the persons on this list personally and most for quite some time now
and I don't think that there will be unnecessary discussion.
Anyway I understood that there will be explicitly assigned reviewers
for each contribution - 2-3 sounds good to me. They are responsible to
review the particular contribution and alone for reasons of lack of
time people will probably trust the judgement of the reviewers.
Blind reviewing most of the time works in favor
of the author. Discussing between us endlessly a paper [unless it is
highly
controversial and only after it has been blindly reviewed] I think will be
a
waste of time and effort.
Endless is discussion is not very probable IMHO. If a contribution is
too controversial it simply will not be included. That would at least
mean an orientation in consensus in the editorial board (where
consensus means that nobody *has to* object).
Grüße
Stefan
______________________________
http://www.oekonux.org/journal
______________________________
http://www.oekonux.org/journal