Message 00086 [Homepage] [Navigation]
Thread: joxT00000 Message: 70/176 L17 [In date index] [In thread index]
[First in Thread] [Last in Thread] [Date Next] [Date Prev]
[Next in Thread] [Prev in Thread] [Next Thread] [Prev Thread]

Re: [jox] Re: Peer Review [Was: RE: Review process



[Converted from multipart/alternative]

[1 text/plain]
Hi Graham

Following on from your earlier comment - I was not suggesting that people submit exact same papers but (as you said I think) modified versions, the benefit being that they then reach a wider audience.

Regarding the comment below - first, thanks for looking at the other journals! Good idea. I think First Monday is an obvious example of a journal that spans the academic / non-academic gap, but I have no idea how their review system works. What I was getting at in my message is that in the traditional process there is a lot of free labour that is lost, that of the reviewers - its a benefit for them to have their work recognised and publicised. We need good reviewers.

Another issue is time - it's very discouraging when you get a response after a long time (12 months+ in some cases)... some journals you pay a small fee and you get a response with reviews in 3 months - I'm not suggesting that at all btw just pointing out that is a plus for submitters - but once again you need a fairly deep reviewing bench...

I think we are probably going to have a multiple-choice system - my concern being that one option not be perceived as more scientific / legitimate than the other(s).

Finally while I see this venture as having experimental value (ie the democratic / open collaboration etc etc) I dont see that as the sole end or an end in itself - i.e. if it generates interest and positive developments it should not be summarily executed! Wait and see I guess.. 

cheers
Mathieu

----- Original Message -----
From: graham <graham theseamans.net>
Date: Thursday, August 20, 2009 11:14 pm
Subject: [jox] Re: Peer Review [Was: RE: Review process
To: journal oekonux.org

Following on from Mathieu and Athina's comments:

This is all getting rather binary: high prestige model non-transparent
blind reviews versus transparency and potential flame wars and spam.

We aren't the first to be creating a kind of parallel universe 
to the
academic journals; it's something the old left have done for 
years, the
newer left are doing now, and we're not the only ones 
concentrating on
the commons one way or another. Clearly the old left approach is not
relevant (review consists of deciding if the party line is being
followed). So I thought I'd have a look at some of the newer 
ones. It's
suprisingly hard to see what the process is:

commoner.org.uk (Massimo de Angelis and friends) - can't find 
how to
submit an article.

multitudes.samizdat.net
http://multitudes.samizdat.net/Submission-and-evaluation
process modelled on academic one

freesoftwaremagazine.com (more of a magazine than a journal)
Individual editor reviews 
http://www.freesoftwaremagazine.com/write_for_us
www.metamute.org/content/magazine (IMO very mixed quality of articles)
can't find review process

www.posseweb.net (negri and friends)
can't find how to submit an article

http://turbulence.org.uk
Current CFP: http://turbulence.org.uk/2009/06/turbulence-call-
for-art/
Can't find review process

[Nearly all these have a print version; given how easy it is to 
do POD I
think we should consider this fairly early on (maybe talk to openmute
www.openmute.org and using their pod setup?)]


So the review processes seem to split into two: academic style, and
informal non-transparent 'an editor does it'. I haven't found anyone
deliberately doing a transparent version.

So how about we be the ones to do it, but as an experiment? We offer
blind review as the default route, but authors willing to be 
part of the
experiment submit their article for open commenting and feedback
instead. The final version of each article is flagged as to 
which review
process was used, and after a period (three issues?) we wrap up with
some conclusions on the merits of each system and assessment of the
quality they generate.

The big downside to this is it's likely to involve double the 
work for
us :-(

Graham
PS. Another thought: anyone have experience of spip
(http://www.spip.net) ? It was originally created to manage web 
journals..



Mathieu O'Neil wrote:

2- For this we need good reviewers and good comments. There 
could be a
system where open discussion on a list leads to editorial 
comments being
appended to papers. Not sure. I don’t think we should rely on 
"anyone can
comment" to do this job - no-one may comment or comments may 
be mediocre.

3- There needs to be some clear guidelins for an open comment 
process:> -- closed editorial list / closed registration process?
-- deadlines for comments to be made?

4- It is clear that different review processes could be 
useful. We need to
define precisely the different review processes: blind or not, 
open or not,
etc.

5- The second part of the paper cited above may have some 
interesting> leads...

6- In conclusion: we eventually need to get some more people 
on the
editorial board to help advance how the review process works. 
We will need
some input from the people we will be approaching to work with 
us. So we
need to progress the rest of the "charter" so we can start 
approaching> people.

Cheers,
Mathieu


-----Original Message-----
From: owner-journal oekonux.org [owner-
journal oekonux.org] On Behalf
Of Stefan Merten
Sent: Monday, August 17, 2009 6:42 PM
To: journal oekonux.org
Cc: Stefan Merten
Subject: Review process (was: [jox] New Draft CFP)

Hi Mathieu, Athina, all!

Last week (11 days ago) Mathieu O'Neil wrote:
d) Regarding peer review

I suggested the following: for research papers, authors can 
request a
traditional double blind review. But following this process, 
research papers
(as all other submissions) will be collectively discussed on 
the list.

I think it is an interesting idea to have different processes.
However, I'm not sure about the consequences. What do others think?

For those not too deep into the traditional process: Could you 
please> explain what are the features of the traditional double 
blind review?

Last week (10 days ago) Athina Karatzogianni wrote:
About d, I think it would be prudent to think about the 
implications of
discussing papers openly on a list. perhaps people will be 
much less
critical of a work once it is openly discussed.

That was a concern mentioned before. If this point is 
important then
it would indeed impact the quality of the journal. This would 
be bad.

Who would be able to see
this discussion?

It depends. Oekonux lists are usually published on the site 
but we can
also have a non-archived list. On such a mailing list a discussion
would be open among the editorial board but closed to the public.

Also there can be exchange based on personal e-mail. However, 
I'd find
it bad for transparency if regular personal e-mail exchange would
occur unless it is between persons who are working closely 
together on
a particular task - such as reviewing a contribution. To 
prevent this
I'd rather suggest a second, non-archived mailing list.

what if one of us wanted to publish a paper, would we look
at the reviewers comments while they were formulating them?

What's wrong with this?

I think some
thought should be paid there. The tradition is to have 2-3 
blind reviewers
for a paper.

See above. Can you please explain what "blind" means exactly?

I dont see and please explain to me how when ten people have a
long discussion over an email list, quality and speed 
improve. I think it
will be quite the opposite.

IMHO this depends much on the culture of such a list. I know 
most of
the persons on this list personally and most for quite some 
time now
and I don't think that there will be unnecessary discussion.

Anyway I understood that there will be explicitly assigned reviewers
for each contribution - 2-3 sounds good to me. They are 
responsible to
review the particular contribution and alone for reasons of 
lack of
time people will probably trust the judgement of the reviewers.

Blind reviewing most of the time works in favor
of the author. Discussing between us endlessly a paper 
[unless it is
highly
controversial and only after it has been blindly reviewed] I 
think will be
a
waste of time and effort.

Endless is discussion is not very probable IMHO. If a 
contribution is
too controversial it simply will not be included. That would 
at least
mean an orientation in consensus in the editorial board (where
consensus means that nobody *has to* object).


                                        	Grüße

                                        	Stefan
______________________________
http://www.oekonux.org/journal

______________________________
http://www.oekonux.org/journal

______________________________
http://www.oekonux.org/journal

****
Dr Mathieu O'Neil
Adjunct Research Fellow
Australian Demographic and Social Research Institute
College of Arts and Social Science
The Australian National University

E-mail: mathieu.oneil anu.edu.au
Tel.: (61 02) 61 25 38 00
Web: http://adsri.anu.edu.au/people/visitors/mathieu.php
Mail: Coombs Building, 9
Canberra, ACT 0200 - AUSTRALIA





[2 text/html]
______________________________
http://www.oekonux.org/journal



Thread: joxT00000 Message: 70/176 L17 [In date index] [In thread index]
Message 00086 [Homepage] [Navigation]