Re: [jox] Re: Multi-rating mode of evaluation / Updating papers
- From: Christian Siefkes <christian siefkes.net>
- Date: Sun, 22 Nov 2009 20:24:30 -0500
Hi all,
Stefan Meretz wrote:
The journal might contain strong pov articles, which is good. One aim of
the Journal must be to support a "thinking against the mainstream". If
peer production is really a new thing, then theorzing around this topic
will and has to be new and unfamiliar. -- Will the expert committee work
in this fashion? Or are unfamiliar povs are rated out, because the pov
is not shared? (this does say anything against the persons listed which
I don't know).
if a public rating system is used, POV and other controversial articles will
generally end up with an average rating, since some reviewers like them,
while others don't. Hence it might indeed be better to stick with a binary
"publish / don't publish" decision from the journal editors / reviewing
committee, and let the readers do the ratings. Of course, reader ratings
will have the same effect of punishing controversial items, but at least
reader ratings don't sound quite as "official" as committee ratings.
(Incidentally, I noted that effect with the IMDB: excellent, but unusual
movies often get an average rating -- typically, high ratings do indeed
indicate that a movie is good, but average or poor rating don't give much
reliable information about the movie.)
Of course, internally the reviewers will probably use some kind of rating
system, since peer review usually does, and if we want transparency we can't
hide these ratings from the readers -- but at least I wouldn't show them by
default, but place them somewhere in the background information about the
article (reader has to click on "reviewing process / reviewer feedback" or
something like that).
Best regards
Christian
--
|------- Dr. Christian Siefkes ------- christian siefkes.net -------
| Homepage: http://www.siefkes.net/ | Blog: http://www.keimform.de/
| Peer Production Everywhere: http://peerconomy.org/wiki/
|---------------------------------- OpenPGP Key ID: 0x346452D8 --
A misleading benchmark test can accomplish in minutes what years of good
engineering can never do.
-- Dilbert