Volume 1, No. 1, Art. 30 – January 2000
What Does Peer Reviewing Mean for FQS
Gwen Peniche & Jarg Bergold
Table of Contents
1. The Potential of Peer Reviewing
2. Barriers in the Review Process
3. A Re-evaluation of the Peer Concept: Questions which a Review Model Must Answer
4. Proposals for a Multi-step Review Model in FQS
4.1 Criteria for reviewing
4.2 The reviewers
4.3 Steps in the review process
4.3.1 Pre-selection of manuscripts and selection of reviewers
4.3.2 Constructive criticism as a part of the multi-step review concept
4.3.3 The author's decision
4.3.4 Further review process
4.3.5 Publishing a contribution and (upon agreement) the steps of the review process
FQS wishes to set an innovative example in all aspects of the journal, one of which includes the evaluation and selection of submitted papers. One of the most widely used selection and evaluation methods for scientific journals is peer reviewing. Many of the problems involved in this process were recently discussed in the Cyberconference "Peer Review in the Social Sciences" from May 26 to June 15, 1999. The following contribution will open by briefly presenting some of the aspects of reviewing that were discussed there. Since it does not make sense for FQS to uncritically use traditional evaluation and selection procedures, this contribution will, in closing, present our first preliminary review-model based on these considerations. It is open for discussion and revision. We gladly welcome all suggestions on improvement and further development at all times. We also see the Internet—with its availability of space, wide accessibility and the quick and efficient methods useful for communication, review and publication—as an opportunity to discover and to try out new possibilities for the peer reviewing process. [1]
1. The Potential of Peer Reviewing
Peer reviewing seems to offer an ideal forum for productive analysis and feedback which could lead to a high quality paper ready for publishing. Both sides—the reviewer and the author—could ideally profit from the interaction: The reviewer could expand or deepen his/her knowledge in a research field, be surprised by innovative research or thought or see his/her beliefs substantiated by a sound empirical study. The profit for the author would be pointers in polishing, revising or correcting the manuscript. The paper which is finally published would have undergone a noticeable improvement in quality. Indirectly, the author could gain new insights e.g. in planning possible follow-up studies. Furthermore, if this process were made more or less public, the (in some cases rejected) author would also be protected from unfairness or intellectual property theft. Even the reader could, if the information concerning the review process were made available, learn from pointers which had been given. The Internet also opens—with the availability of space, wide accessibility and quick and efficient methods of communication, review and publication—new possibilities for the peer reviewing process. [2]
2. Barriers in the Review Process
Barriers which hinder a constructive peer reviewing can be found both in the general academic arena and its "laws" as well as in the specific minds of involved persons. These could be possible biases towards topics, styles, research (sub-) cultures, gender, status, "correct" method usage, etc. which could all potentially influence the review process. A review process which is grounded in or (even unwillingly) supports such personal and institutional biases is not of interest to FQS. We are aware that, even given FQS' ideals of openness/transparency and fairness, the realization of these ideals is only possible if we continually invest our time in details. This is especially necessary when one considers that many different (and perhaps conflicting) disciplinary and national perspectives will be coming together in FQS. [3]
3. A Re-evaluation of the Peer Concept: Questions which a Review Model Must Answer
Unlike the source and implicated meaning of the peer concept, academia is inherently hierarchical and is not marked by a certain equality, except perhaps that all hold an academic degree of one sort or another. If one follows the peer metaphor for the reviewing process in scientific journals, then certain problems arise: How can one presumably "equal" peer "reject" another peers ideas? Who exactly is a peer? (Little consideration, e.g., has been given to taking reviewers who work in the practical field rather than theoretical, although these persons could, based on their experience, also be able to assess the usefulness of an academic publication from a very interesting and relevant standpoint.) Specifically, in view of the previously mentioned considerations, the following questions arise and should be considered when designing a peer review model for FQS:
Should reviewing be conceptualized as a one-step or multi-step process?
Based on what criteria, with what methods should peer-reviewers be selected?
What kind of guidelines should be given to what kind of reviewers and how strictly must these be followed?
How can constructive criticism and constructive feedback be supported in the review process? How can the author be protected in the review process?
What possibilities exist for evaluating the quality of the review process?
Should everyone follow the same model, even when the involved persons are not from the same disciplines? How can different perspectives be adequately acknowledged in the evaluation process? [4]
FQS, as an interdisciplinary journal, needs to address and finally agree upon the issues of choice of reviewers, guidelines and criteria, steps belonging to the review process and possibilities for improving the quality and control of reviewing. [5]
4. Proposals for a Multi-step Review Model in FQS
The following review procedure, which we designed for FQS, is to be taken as a proposal- a working model that is open for discussion, revision, and expansion. We welcome all continually incoming comments, criticisms and recommendations. [6]
There is certainly no perfect or infallible way of peer reviewing. Too many aspects are involved. Nonetheless, we are committed to creating a review model that strives towards fairness, openness/transparency and usefulness/practicability for all involved persons. We consider the relationship inherent in the review-model to be potentially reciprocal and flexible. We value this kind of communication between equal peers rather than one which develops from a static, hierarchical or intimidating/intimidated relationship. Constructive reviewing—the giving and taking of productive feedback—is not an easy or perfectly natural process. Nevertheless, if successful, it can be a learning process for all involved persons. [7]
The following general guidelines should be applied in peer reviewing for FQS:
The procedure should fulfill the fundamental principles of qualitative research. Thus, it should be above all comprehensible and transparent for the reader. We believe that the Internet will allow us to develop such an understandable procedure.
A further fundamental principle of qualitative research should also be considered: The protection of the involved persons. If we are interested in developing a trusting relationship between authors, reviewers and editors, then it is necessary that especially the authors do not have to fear embarrassment.
FQS wishes to support (cooperative) work in the field of qualitative social research. Thus, the review process must be designed so that all authors have the opportunity to receive support in their contextual and methodological work through the review.
What makes FQS different is an academic cooperation that strives to go beyond the boundaries and perspectives of individual disciples and nationalities. This makes it necessary to have every submitted paper evaluated by at least two disciplinary perspectives. Beyond that, whenever possible, reviewers of differing national origins should be involved. [8]
Generally, members of the FQS Editorial Board should be available. Beyond that, we would like to closely follow an APA recommendation (www.apa.org/journals/underrep.html) which invites members from scientific groups which are (not yet) represented in the Editorial Board, to become involved in the review process. Manuscript reviewers are vital to the publication process. As a reviewer, you would gain valuable experience in publishing. The FQS Board is particularly interested in encouraging members of underrepresented groups to participate more in this process. [9]
If you are interested in reviewing manuscripts, please write to Katja Mruck. Please note the following important points:
To be selected as a reviewer, you must have published articles in peer-reviewed journals. The experience of publishing provides a reviewer with the basis for preparing a thorough, objective review.
To select the appropriate reviewers for each manuscript, the editor needs detailed information. Please include with your letter your vita. In your letter, please describe your area of expertise. Be as specific as possible.
Reviewing a manuscript takes time. If you are selected to review a manuscript, be prepared to invest the necessary time to evaluate the manuscript thoroughly. [10]
4.3 Steps in the review process
Generally, we prefer an open review rather than a blind evaluation since we are assuming that a different kind of review culture can only develop when reviewers and authors are known to one another. A problem with this kind of open review could be that prominent or powerful authors receive special treatment. By making the process open/public, this would become more difficult and noticeable. That is why we—when the involved persons agree—will try to make the review process transparent and open by making the steps of the review process and the editing available on the Internet. [11]
4.3.1 Pre-selection of manuscripts and selection of reviewers
For volumes with an emphasis
In the case of volumes with an emphasis, the respective editor decides on which kinds of contributions to include. Additionally, those who are interested in a specific volume can directly send their contributions to the responsible editor. Contributions sent to the FQS editors will be re-routed to the volume editor. The editors of the volume are also to function as (main) reviewers. They are to stay in contact with the authors and the FQS editors. When a contribution is received, a reviewer from a different discipline is selected/agreed upon by the volume editor and the FQS editors. [12]
For other individual contributions
Authors are to e-mail an abstract to the Editor-in-Chief Katja Mruck. This is to determine whether or not the article is within the scope of FQS. If it is, the Editor-in-Chief asks the author to e-mail the full text. If it is not within the given scope, the abstract is rejected. Upon receiving the full text, the editors select two reviewers that stem from different disciplines. The person whose own discipline is closer to the manuscripts, is the responsible reviewer who stays in contact with the author and the editors. [13]
4.3.2 Constructive criticism as a part of the multi-step review concept
The reviewers evaluate the contribution by pointing out the specific strengths, possibilities for improvement or by encouraging the author to re-conceptualize entirely. As FQS progresses in time, detailed standards will develop and will be made available to the reviewers. Well tested standards are, among others, "Treatment of Relevant Literature", "Appropriateness of Length", "Clarity of Objectives", "Conceptual Rigor", "Methodological Rigor", "Coherence of Organization", "Defined Conclusion". But since scientists from many different disciplines are participating in FQS, we do not want to immediately set too many guidelines. Instead, we would like to postpone our suggestions until we have all gained more experience and can mutually work out suggested criteria for "good scientific research". [14]
If both reviewers have accepted an article, then it is included; if both reject it, it is not included. If differing opinions exist, then suggestions for improvement coming from the positive evaluator are shown to the rejecting reviewer. If the rejecting reviewer cannot accept the suggestions, then a third reviewer must be called upon. [15]
The author decides whether or not he/she accepts/rejects the suggestions and revises the manuscript accordingly. [16]
The revised manuscript is once again sent to the responsible reviewer and he/she then assesses the changes and, if necessary, again calls upon the second reviewer. She/he recommends the acceptance, a new review-round (when large discrepancies between the reviewers exist, then, if necessary, through an additional reviewer) or the rejection of the submitted paper. [17]
4.3.5 Publishing a contribution and (upon agreement) the steps of the review process
In the case of acceptance, the article is published in a volume. At the same time, the reader should be given the opportunity to re-trace the entire review process. If both the author and the reviewer agree, then the responsible reviewer makes the steps involved in the specific review process available in the Internet. At the end of the article, a link will give access to the different versions of the contribution and to the respective comments from the reviewers. This will make the review process public and transparent and will allow other potential authors to get to know the standards and criteria which the journal uses. [18]
In the case of non-acceptance, the author's request, a link at the end of the volume can give the interested reader access to the submitted article and its review process. In order to protect the author, this should only occur upon the author's specific request. [19]
Gwen PENICHE
Peniche, Gwen & Bergold, Jarg (2000). What Does Peer Reviewing Mean for FQS [19 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 1(1), Art. 30, http://nbn-resolving.de/urn:nbn:de:0114-fqs0001305.
Revised 3/2007
Creative Commons Attribution 4.0 International License