Abstract: This paper is about using evidence from previous authors; literature reviews. This has already been done, for example by WEBSTER and WATSON (2002) in the journal MISQ and LATOUR (1987) in "Science in Action," but the former used the root metaphor of the previous literature being objective facts (truthful sign posts) towards empirics while LATOUR used the root metaphor of references ganging up on the reader to persuade. This paper will provide a middle road on the same topic using the root metaphor of the courtroom. Revealing and justifying alternative root metaphors is central to interpretive research. Therefore, presentation of these different interpretations of the same topic (literature reviews) provides a unique opportunity to appreciate "seeking interpretations" as a research methodology. Readers are advised to read these two other interpretations as well as this paper, noting how a different root metaphor can lead to very different appreciation of a situation. Therefore, and more specifically, this paper will argue for social inquiry researchers to adopt the root metaphor of considering literature reviews to be the calling upon expert witnesses to provide supporting or counter evidence justifying their paper's conclusion. It will present and demonstrate the courtroom as a root metaphor.

One day, an otherwise very competent mid-career masters' student came to my office to discuss his research thesis. He started the conversation by placing a list of abstracts very proudly on my desk, announcing that he had completed his literature review. He was pleased because his newly found computer skills had enabled him to complete what he saw as an academic ritual in only three hours. He was studying "scenarios for organizational planning," so had drafted a list of relevant key words and entered them in to the library's electronic databases. As a result, he now had nearly 70 good quality abstracts sorted into date order. He asked what to do next. [1]

I replied (as I often do), "What is your argument¹⁾?" which stopped him smiling. But after a moment's reflection, he smiled again and said, "What do you think my argument is?" I answered, "That scenarios could usefully assist corporate planning." "Yes, right." So I said, "Is there any evidence in this list of abstracts to support or refute that?" He replied that he did not know as he had not yet read the abstracts. [2]

This paper will argue that a literature review is the supporting evidence for a thesis (argument) by treating previous authors as "experts" and/or witnesses. I am assuming that research is an argumentative process. A research thesis (article, report) is a reasoned argument written to provide the reader with supporting evidence for its conclusions. I am not assuming a research report is a journal of the author's emerging self-discovery during the research experience. [3]

This paper will draw on the courtroom process as a metaphor where each article is seen as a potential expert or witness that can be brought forward to support or counter the main argument. The tension or limit in the metaphor that needs to be highlighted is that courts are often "clumsy." It is the intent of their process, designed to justify interpretations of a physical event, which is being alluded to. MASSY (1996) identifies other possible metaphors for references. Examples include "as currency" to buy credibility or using the other meaning of the word, references are expected to be "up to date." Previous literature can be thought of as "building blocks" with which to build an argument or as a foundation. Literature as a mirror uses the analogy of seeing oneself in context; as a lens it may help a researcher focus; as a signpost it points to where new research is required. LAKOFF (1992) divides all metaphors into object and motion, which he argues reflects the ability of human senses to either detect objects or movement (relative location). This can be used with these literature review metaphors, where previous articles can be thought of as objective facts to be discovered. The doing or changing of relative location can be thought of as using them to justify (argumentation) a position. So, for example, a metaphor of past literature could be the courtroom. This metaphor is also attractive as it aligns with the word "thesis" as meaning "argument." Under this courtroom metaphor any empirics would be supporting or refuting evidence from primary (direct) experience, in contrast with the literature (references) which would be experience from other authors. Moreover, the courtroom metaphor is thought useful because it at least makes an attempt to counter power and appeal to authority issues. Ideally the rules of evidence, like the use of the literature, should be as far as is reasonable independent of a defendant's wealth or position; appeals to authority also need to be justified. Further application of the courtroom²⁾ metaphor to literature reviews will be explored. [4]

First, in the tradition of argument, some working definitions of words in the argument of this paper or words anticipated to be problematic in the discussion below are presented. In his "Summary of the Rules of Evidence" for new courtroom lawyers, DiCARLO (2002) provides the following explanation: "An 'expert' ... is one who is allowed to give testimony in the form of an opinion or conclusion. Other witnesses generally may testify only as to that which they have perceived (e.g., seen, touched, heard, felt, smelled) ..." [5]

"If scientific, technical or other specialized knowledge will assist ... to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training or education, may testify thereto in the form of an opinion or otherwise." [6]

Literature (books, articles, reports) includes experts whose expertise has to be justified and witnesses who can simply report their direct experiences. The difference can become blurred, but what we need to avoid is dogma, opinions not based on justified reasoning. "Literature" is used to mean any published materials, a copy of which is available. The word, "review," is used in the sense of to "look over" or "survey." Here the "look over" is to see whether it provides anything of relevance to a researcher's argument. This provides my lens (concern, or context) on the literature. [7]

Why is this paper being written? Because it is part of a larger project by the author aimed at convincing fellow researchers that research (inquiry) is best perceived and communicated as an argumentative process. This is particularly relevant for research into the introduction of social situations as they are very complex and involve extensive connectivity—meaning that context is everything. [8]

Moreover, the publishing of WEBSTER and WATSON'S (2002) very "one-truth" genre on how to undertake a literature review "properly" with no reflection on their own perspective or that of others such as MASSY (1996) or LATOUR (1987) could be accused of not being "enlightening." This paper and as with MASSY, LATOUR and WEBSTER and WATSON, are on the same "object under study" (the literature) but use very different root metaphors (perspectives, worldview). MASSY draws on the now extensive metaphor literature and LATOUR argues that the use of references in scientific papers can be seen to come close to the unacceptable appeal to authority in the form of the author ganging up with unsuspecting previous authors to unduly persuade the reader by force of numbers. WEBSTER and WATSON appear to use an objective-facts root metaphor where previous literature is seen as indisputable facts or signposts that locate a paper's empirics; the dominant objectivist epistemology. This perspective leads to apparently rational advice to authors in well-respected knowledge gatekeepers such as MISQ³⁾. On November 1, 2002, its website, the section on "MISQ Manuscript Guidelines", had a subsection that read:

Although writing style is generally up to the author(s), the following guidelines should be noted:

I argue that the second dot point, on face value, is problematic for those who believe that humans can have numerous perfectly valid interpretations of the same physical event. The reason for the advice in dot point two is believed to be that the editors are trying to make large paragraphs of a literature review more readable. This is a perfectly reasonable if you use the root metaphor of the previous literature as being objective facts that can be separated from who "discovered" them. Signposts do not reveal "who said" that this is the right way to market. However, using the root metaphor of the literature being the opinions of arguing expert witnesses, MISQ's advice to separate the opinion from who gives it, is very dangerous. In the second dot point, who are "Smith and Jones", what is their expertise, why is their opinion worth considering? One reason the courts insist on establishing experts' credentials before asking their opinion is that the jury is reminded that the expert's opinion is just that, their interpretation (perspective) not an "objective fact⁴⁾." This is reinforced by the practice of barristers calling upon numerous "experts" to demonstrate disagreement. [10]

(The ease of e-talk facilities these days tempted me to ask the ISWORLD listserve participants for their thoughts on referencing. I was seeking interpretations not measuring samples and the medium only really allows a few sentences of explanation. While not central to this paper, those interested in an edited sample of the responses should look to the appendix). [11]

This paper is arguing for a literature review to be seen as evidence in support of the argument, not in the bullying way of ganging up on the author identified by LATOUR (1987). Rather, it is suggesting that, when dealing with people and justified interpretations of physical events, it is very important to maintain the link between who said what and in what context. In which case, expertise is a very important part. When reporting on the physical world using an interpretation (KUHNian paradigm) aiming for universal laws, such as Newtonian physics, there is less need to remind the reader who said what and in what context. COHEN (1994), author of an extensive history of the scientific method, argues that this led to what he calls the "lazy" experimental write up where it was only necessary to locate your experiment in relation to previous experiments. While good argument is universally accepted as the cornerstone of science, COHEN (1994) suggests that "writing up experiments" developed shorthand of its own. It was encouraged at a time when writing was time consuming and expensive and when research was focused on a few issues central to the Newtonian paradigm. So research reports were communications to a very small, very select audience of scientists who were familiar with each other's research and the previous research. [12]

With research that seeks people interpretations (perceptions, ideas), where, in place of universals, there are personal interpretations or perspectives, it is necessary to be rigorous about not separating a claim from the author. For example, I can study Australia from a race, geographical location or IT policy perspective. The range of possible perspectives means the interpretive researcher needs to spend some time constructing the alternative perspective. This means that statements cannot be treated as independent of who said them. Most readers would accept that this is also the case in the justice system. Who said what and when is very important. A witness statement needs the witness's name attached. [13]

Does a literature review need to include everything ever written about a topic? I think the guiding principle here must be how useful a reference will be to assist your argument. If there are three people who make the same point then one reference may be enough. However, the writer may feel that pointing out that three experts have concluded the same thing adds to the weight of the evidence. Put another way, it seems unreasonable to criticize a literature review because one reference is missing if the point has been made without using that particular reference. The literature review is not just a complete list of other authors. [14]

This brings us to the difference between a chronological list of abstracts, an annotated bibliography and a literature review. A list of abstracts seems clear enough. There is very likely no common theme or argument running through all the abstracts. The test of the usefulness of an abstract is relevance to the report's argument. Indeed, having an upfront argument gives the researcher the "eyes" to see the relevance of a report for her particular research. The reference paper may contain a good counter argument, so will alert the researcher as to what she needs to argue in order to convince that particular author. An annotated bibliography is interpreted to mean "notes" made by the researcher about other authors' papers. If these notes are made with a specific argument in mind, and thus a specific audience, then the annotations may well be usefully elaborated in the literature review. A literature review is the use of previous researchers to provide evidence in support of the thesis. [15]

A more practical issue that literature reviewers face is whether the best approach is to first map out the evidence sought and then go looking in the literature. Alternatively, should the literature be accumulated first and then be mapped into supporting evidence? While it is expected that a recursive relationship will occur between the evidence map and the available literature, it is an issue worth a few words. Again the analogy with a courtroom lawyer is drawn upon. A barrister may want a solid eyewitness, a clear alibi and so on but what she has is a given set of actual evidence supplied by the police and miscellaneous witnesses, expert or otherwise. Nothing is gained by wishing for evidence that a reasonable search indicates is not available. Likewise, researchers may prefer listing the available evidence (literature) and then mapping out how they want to present it to the reader. [16]

If a reference is evidence, then it will need to be subjected to some kind of test to determine if it is of a reasonable standard to be convincing to the audience. Most readers will have a passing familiarity with at least two "rules of evidence." The first would have been presented at high school under the title "fallacies of argument." Examples include ad hominem and "thin-edge-of-the-wedge" arguments. These point out poor reasoning practices. WALTON (1998) has written extensively about these "rules" trying to emphasize that it is dangerous to try to make universal rules about reasoning. Rather, each piece of reasoning has to be judged by its context. Attacking the "man" (not the argument), for example, may be justified if the argument uses his experience as part of its justification. [17]

The second exposure to "rules of evidence" most readers will have experienced is that communicated by the science teaching in chemistry or physics. Examples include the need for impartibility, repeatability, universal truths, exact measurement and an absence of bias. These are accepted as perfectly valid ideals for supporting arguments about the behavior of gasses under temperature and pressure changes but turn out to be of limited use in a courtroom. If measurements are available, courts will use them but avoiding bias, seeking universal laws of human behavior and asking for exact re-runs of events would be impractical and inappropriate. [18]

Therefore, a related, but much more publicly tested set of rules of evidence has been developed to settle complex and serious human affairs as played out in courtrooms. These rules, over many centuries, have developed some rigor as indicated by Federal Rule 702 cited in the definition section above. It is not being suggested that their approach is ideal, nor that these rules are all directly applicable in social research. However, they may contribute to critically appraising references used to support social research arguments. [19]

The starting point of these rules is to decide what evidence the courts will accept or not as "admissible." The basic prerequisites of admissibility are relevance, materiality, and competence. Mapping these across to referencing in a literature review, the "relevance" of a reference is something that can be easily identified provided the argument is clearly stated at the outset. A failure to explain relevance can prompt questions like, "Why have you mentioned this reference?" Where the relevance is not immediately apparent, the author is required to make the connection explicit. For example, if the argument were that "Australia needs an improved IT policy," then the relevance of references that uncritically comment on Singapore's policy would need to be explained. [20]

Slightly different from relevance is "immateriality." References may be relevant (on the topic) but not have any practical effect or significant weight in making the argument convincing. So a reference that contains the opinions of the mayor of a small rural Australian town on IT policy may not convince the reader as being material to the argument that all Australia needs a revised IT policy. [21]

References need to be competent (reliable). This touches upon academics' favorite topic of evidence collection method(ologie)s. When a reference is drawn upon, it is incumbent upon the researcher to ensure she fully understands the methodology used in the reference. Further, that it produced reasonably competent conclusions. Put in terms of a reference being used to report a perception, the author needs to fully appreciate both the background of the person in the reference and the details of his or her perception (to listen carefully to the reference). [22]

After laying down the rules for what is admissible evidence, the courts have found it useful to distinguish between types of evidence. This can again be mapped across to referencing. Four types are usually identified: real, demonstrative, documentary, and testimonial. [23]

Providing what courts call "real" evidence within academic articles is going to be rare. Real meaning the original copy, the actual words and so on. Academic articles are typically copies, where interviewees cannot directly be seen talking about direct experiences. Only copies of contracts are possible, physical artifacts cannot be attached and measures are only recorded. Hyperlinks to authenticated recorded audio and video files and photographs come close to being "real" but these are still "silent witnesses" because the witness cannot be questioned. The contents of academic articles would normally be considered poor evidence in a courtroom. The academic is protected from "being there." [24]

The closest to real evidence in academic articles is "demonstrative" evidence. Examples include quotes, maps, photographs and measurements. This is an area where the quality of evidence provided in some research theses could be improved. If interviews (testimonials) are undertaken, then the reader might be given details of the questions asked and how they were asked. Presenting quotations from respondents rather than an author's summary seems preferable. When statistics are used, the raw form of the data should be made available (maybe via a web link) for readers to recalculate any results. In these ways, the importance of supporting evidence is also emphasized. [25]

"Documentary" evidence is again hard to produce in academic articles. Copies or excerpts might be photocopied, but care can be taken to fully record what is being presented. Indeed, a reference may itself be considered documentary evidence. A copy of which is not provided to the reader, rather only a reference to enable a copy to be obtained. The issues surrounding the quality of documentary evidence raise, for academics, more general issues of authenticity. For example, a quote or reference cannot be presented and then the author overwrite it by saying something like, "Ah, but what she meant to say was ..." Nor can a reference be from an old draft of an article simply to save the effort of referencing the full-bodied final version. When using a reference, authors may want to make clear the authenticity of evidence being drawn upon as part of the literature review. [26]

The use of interviewees' opinions and quotes as reflection of their experience as managers and community members are being increasingly used in social research. This is particularly true of research that is trying to collect a variety of interpretations (ideas). The reason for this sort of research is that researchers cannot observe from the experience of others, especially after the event. The "testimonial" evidence of witnesses is required, however such evidence may be unreliable. When using a reference or any form of observation from others, care needs to be taken about the competence of the witness. In general, a witness needs to meet four requirements: she must be in a position to be able to tell the truth, she must have personal knowledge (through her own senses), she must remember what she perceived and must be able to communicate what she perceived. [27]

There are some interpretive researchers who promote relativism by allowing witnesses to state an opinion that is regarded as not open to criticism. At the other extreme, some objectivism researchers feel opinions are not valid evidence, being concerned that untrained observers (which sounds a little elitist) may lie or be mistaken. The courts, and I, take the middle road. People's observations (experiences) either of one incident or over their lifetime, are considered useful evidence but extreme care has to be taken to cross-examine (explore) what they say and why. [28]

The same is true of the authors of a reference. Using an article as a reference needs to start by careful homework on the journal and the authors. If the journal is double blind peer refereed, has editors that could be considered experts and generally has a good reputation for quality, then acceptance for publication may be considered as making the authors "experts" in the topic of their article. Otherwise some justification for treating the authors as experts needs to be sought. First impressions of the quality of the article may be indicated by its rigor or its referencing may suffice. Else the careers of the authors may be used as the basis of giving them expert status. [29]

Whatever justification is used, it is necessary to establish whether or not the authors are to be treated as experts and to determine their area of expertise. If their expertise is not established, their opinion may not be convincing. The courts only allow non-experts to give statements about what they saw, heard, felt, tasted or smelled. In court, attempts are made to exclude non-experts from explicitly expressing opinions or drawing conclusions. Non-expert academic authors may still make a useful reference if they are reporting on some empirical work they have personally undertaken. [30]

However, this distinction between "observation" and opinion is not always clear. Many witnesses find it impossible to communicate observations without including some form of judgments or opinions. Some leniency has to be given in allowing non-expert witnesses to give explanations of others' perceptions. Examples include giving comments on identity, appearance, sanity, mood, opinions of measurements (distance or speed), sobriety, ownership and identification of handwriting. All of which point to the difficulty of distinguishing opinion from observation. The same is true of academic authors. However, if the authors are giving their opinion about some social situation based, not on specific empirical research, but rather "long experience" then the value of this "long experience" needs to be justified. [31]

Validation of the opinions expressed in a reference is as important as empirical methodology. It is an important part of the homework of a reference user. Beyond expertise, authors of references should also be evaluated for the quality of their opinions. Have they held the same opinion over time and how much justification is offered in support of their opinion? Further, do the authors compare their opinion with others, do they openly address the counter opinions, are they free from bias, prejudice, inconsistency and publication opportunism? [32]

Most readers will be familiar with the court practice of not accepting hearsay from a witness. "He said, she said," is not acceptable. Hearsay evidence comes out in references as "Smith as cited in Jones." One author is reporting a reference used by another without having seen it first hand. The general rule being; do not reference an article unless you have read it yourself. In practice, there are exceptions. A common one being with citing philosopher's work such as, "Ulrich (1983) interprets Habermas to say ...," this may be appropriate if the work of HABERMAS requires translation and a thorough appreciation of a specialized area. [33]

I have argued that the justice system rules of evidence could be, and need to be, mapped extensively onto academic referencing. However, the point has now been made and the analogy can stop. By way of example of what is clearly my interpretation of how referencing could be improved, below is a paragraph from an article written by KLEIN and MYERS (1999). It is their definition of interpretive research. MYERS is an editor of MISQ which is constantly ranked very highly as a source of good research by I.S. academics. They use referencing in this excerpt so it provides a useful example of present best practice for interpretive researchers. My main concern is still author(ity), how do we know the expertise of the witness cited?

"IS research can be classified as interpretive if it is assumed that our knowledge of reality is gained only through social constructions such a language, consciousness, shared meanings, documents, tools, and other artifacts. Interpretive research does not predefine dependent and independent variables, but focuses on the complexity of human sense making as the situation emerges (Kaplan and Maxwell 1994); it attempts to understand phenomena through the meanings that people assign to them (Boland 1985, 1991; Deetz 1996; Orlikowski and Baroudi 1991). Interpretive methods of research in IS are "aimed at producing an understanding of the context of the information system, and the process whereby the information system influences and is influenced by the context" (Walsham 1993, pp. 4-5). Examples of an interpretive approach to qualitative research include Boland's (1991) and Walsham's (1993) work." (KLINE & MYERS, 1999, p.69) [34]

With the "(Kaplan and Maxwell 1994) and (Boland 1985, 1991; Deetz 1996; Orlikowski and Baroudi 199)" references it is unclear to me what exactly these people actually, saw, said or did. Is the sentence, "Interpretive research does not predefine dependent and independent variables, but focuses on the complexity of human sense making as the situation emerges" now an objective, unquestionable fact or is this still the opinion of KAPLAN and MAXWELL. If it is their opinion, then I have the right to ask (with all respect) who they are and why I should treat them as an authority. With the "(Walsham 1993, pp. 4-5)" reference at least we know what he said, but again on what basis should I treat him as an expert witness, rather than simply as just a witness. My interpretation of interpretive research is that its researchers believe in multiple (equally reasonable) interpretations of physical events and seek to provide explanation and justification of those interpretations. If part of that justification is an appeal to authority, then the basis of that authority needs to be established. The last reference, "Boland's (1991) and Walsham's (1993)," seems reasonable as it merely cites them as examples (observations). However, the whole sentence has to be read as the opinion of the article authors, whose authority is not explained. [35]

Of course, short cuts are necessary for effective communications. However, I worry that social inquiry research will not be useful unless we lift our practices with respect to appeals to authority, especially if we are to look for advice increasingly from interpretive research and the methodological styles of the critical social theorists. [36]

It might be informative to reflect on my use of references in this paper. If I use LATOUR's perspective back on my use of references in this paper, then my gang, against you the reader, only has a membership of eight. Three of these (LATOUR 1987, WATSON and WEBSTER 2002) I am using to convince you that the topic of this paper is worthy. Non of these know that I am soliciting their help ex-post asynchronously and I suspect Webster and Watson will either not thank me or think I am simply not a properly trained scientist (which I was). LATOUR also points out that I will be honored to be cited in any future articles as a gang member used to persuade their reader because citation is an honor; a performance measure in academia. Indeed, I dream, rather vainly and overly optimistically, that I will get cited more than WEBSTER and WATSON as the motivation for writing this paper was to provide newcomers with an improved and explicit metaphor for designing a literature review. WEBSTER and WATSON's approach hides the designing root metaphor. [37]

If I used WEBSTER and WATSON'S advice on how to use literature, then this paper would not have been written; partly because I have no empirics to prove some point but partly because they have already explained very thoroughly the one truth of the matter already. Literature is not research. The "opinion and chat" that makes up the body of this article is not a proper literature review (by author or by topic) and the responses listed in the appendix, which I saw as an attempt to solicit justified perspectives, does not follow the rituals of objective research so has not produced knowledge. WEBSTER and WATSON, I suspect would say that it is merely a poorly sampled collection of hearsay and opinions, from which nothing can be proved or generalized. The researched "molecules" (subjects) should not be encouraged to reason. [38]

I hope ALVESSON and SKOLDBERG would approve of my use of references in this paper, although I suspect they would say that apart from the courtroom metaphor there is little new in my paper. It might be useful to students. I have used only those authors that gave me an interesting insight (perspective). So, to some extent I have used references, apart from LATOUR and WEBSTER and WATSON, only to acknowledge where I sourced an idea. This is consistent with my writing in the style of saying of "Smith argues ..." rather than merely "dumping" their name at the end of a statement of apparent indisputable fact (WEBSTER and WATSON 2002) No attempt has been made to be exhaustive by listing all those ever published as discussion on literature reviews. Examples excluded, as they did not appear to provide a fresh justified insight, include the discussions in the Journal of Accounting Literature. [39]

Last, I feel some tension in not taking DiCARLOS' advice about distinguishing expert witnesses from ordinary witnesses. I have spent no time outlining the expertise of any of my gang of eight. LATOUR appears to be the only one who has done any formal research and I suspect WEBSTER and WATSON would not think that was scientifically valid. Nor have I explained why else my gang might be an authority except to say implicitly they managed to jump or side step the bureaucratic hoops of being published in peer-reviewed journals. WEBSTER and WATSON's paper was an editor's note, it is not clear if it has to pass the normal rigors of positivist review. My excuse for not presenting any authors' credentials is that I am not saying, "X is true because Smith said so." I hoped I was merely letting the reader know where I sourced an interesting idea and implicitly where they might look if they want to read more on the topic. LATOUR (1987) suspects that most of you will not. [40]

This paper is about literature reviews. The author is concerned that some interpretive researchers may not have explicitly conceptualized the important difference between literature reviews in support of experiments on the physical world compared to research that seeks interpretations. For the latter, this paper has argued that the literature be seen as a source of expert witnesses that can be used as per courtroom argument, that is, as evidence in support of an argument. The literature being a supply of authors who can be viewed as expert witnesses due to their research findings or other experiences. Some of the rules of evidence from the justice system were used to illustrate issues such as expertise, authority and hearsay in an attempt to draw analogies with the use of referencing. [41]

ALVESSON and SKOLDBERG (2000) argue in their chapter on the mythological issues surrounding critical social theory, which these researchers have long drawn on the humanities tradition of referencing rather than the scientific one. The important distinction they make is whether research is about justifying interesting insights or reporting precise measurements. For the justified insights research, the literature and observations are persuasive tools of the author. However, in order to be persuasive, rules of evidence need to be established. [42]

Acceptable rules of evidence is an issue the justice system, with its huge powers over people's lives, has agonized over for thousands of years. They have designed a system of human inquiry that is in response to "wicked" social interactions rather than precise measurement of experimenter's physical contrivances. Interpretive researchers may have a lot in common with the court's inquiry methodology. Therefore, they may want to reflect on how the courts would view a literature review. [43]

At a different level of abstraction, this paper has provided an example of how different social objects can be seen from different perspectives (root metaphors, worldviews, interpretations). WEBSTER and WATSON (2002) appear to have discussed literature reviews using the root metaphor of literature as objects that can be separated from their creator. LATOUR (1987) on the other hand, uses a very contrasting root metaphor of references as being a means of ganging up on a lonely reader so as to be persuasive. This paper used the courtroom root metaphor where who says what and why is central. One way to understand human inquiry is as a search for these perspectives that are often totally hidden even from the researcher. Revealing these hidden assumptions can help us understand how what appears to be well-reasoned advice can be so much at odds with someone else's well-reasoned advice. The intent of this revealing process is not to find the one correct interpretation but rather to learn from appreciating the perceptions of others. [44]

At the end of the paper some attempt was made to reflect on how this paper had itself used references. Reflection needs a perspective, so rather than repeat the courtroom one, those authors that I refer to were used as the perspective. [45]

I wrote to the ISWORLD Listserver asking for comments about referencing. So, needing to be brief I wrote:

I ask because I suspect the first assumes, it is now a fact (objective knowledge) and is inappropriately vague about what the past author and what s/he did. Whereas, the second assumes that all that should be reported is an expert-witness' interpretation (subjective knowledge?).

"Checkland (2000), after three decades of research, concludes that soft systems methodology is effective."

I don't think the difference is as clear-cut as you suggest, especially if the phrase "after three decades of research" is removed from the second version, or added to the first—although that would affect your argument, I concede.

There are other points, as well—the first one could be seen as implying "..and I agree with it", whereas the second may imply "...but I (or others) still disagree". On the other hand, the second one does give greater credence that the author has actually read the reference cited, rather than relying on it at second- (or third-, or fourth-) hand. Some universities or journals advocate different styles, too, and even that may have an effect.

OK, I'll come off the fence; of the two you offer, as written, I prefer the second.

I prefer the second, but then again, I also prefer writing in the first person. I agree that the first style of citation tends to disguise the fact that there are particular individuals making the claims.

I like the second one. It relates the finding to the source; and does not make an absolute statement which will be misleading if not incorrect. I think social science findings should always be reported in this manner—rather than making giant leaps of faith.

I usually decide which one based on my own confidence (thru rsch validation) in the past author's expertise and on what degree it supports my own present rsch.

"Some have found soft systems methodology to be effective (Checkland 2000, +1 or more other author)."

I don't think that the citation styles that you referred to in your ... posting have much to do with positivist or interpretive research cultures. That is an artifact of the way you created the examples and the culture of writing for some journals. I use both styles and I think that any skillful writer should do so.

"Rockart and Crescenzi (51) recommended initial interviews with five key executives and 10 other managers, followed by a focusing workshop to fully identify the CSFs."

The other style is good for providing evidence to support ones own arguments, e.g., "The critical success factors concept is widely accepted for use in ISP. Indeed, it appears to anchor contemporary thought about ISP to the extent that most published research on ISP refers to CSF, e.g., (33, 45, 53), and, in practice, ISP generally incorporates CSF in one way or another, e.g., (2, 19)."

... I agree with your view that there is a glimmer of an epistemological issue. I'd probably use a form of discourse analysis to explore it more fully. I also think it is important to criticise the modernist 'fact building' approach implicit in the dump method—yet another bias against post-positivists as much as anything else.

Lamp (2001) found no evidence for the distinction while Metcalfe (2001) and Nguyen (2001) using an experiental and interpretivist approach over a period of five minutes found that postivist's dump and intepretivist's waffle. Obviously both sides will use dumping and waffling but is there a significant difference.

What you call the "dumping style" and I'll call the "I'm saying something style" leads to concise, readable text. A little bit of the "somebody else is saying" style goes a long way. Its overuse leads to verbose, evasive writing, where the author doesn't really assert anything. "Metcalfe (2000) concludes that the sky is blue," says little more than "The sky is blue (Metcalfe 2000)," but it adds words and the author avoids committing himself to whether Metcalfe is right. XX never said that you can't use the "somebody else is saying" style, but he discourages its heavy use, as do most journal editors.

I use this (dumping) form when I have a number of references which support the statement, or when the reference has collected a number of views.

"Evaluations at Deakin (Thompson 1994) and other educational institutions around the world indicated the benefits of enhanced educational and social dialogue through increased interaction between students, and the opportunity to work collaboratively to build knowledge and understanding of course content (Harasim 1989; Mason and Kaye 1990; Bates 1991, D'Souza 1991; Kaye 1991; Boston 1992; Rich 1992; Lewis and Hedegaard 1993, Hiltz 1994; Mason and Kaye 1994)."

"Coloured Petri Nets (CP-nets) were proposed by Jensen (1981) as a more elegant way of dealing with some technical problems of PrT-nets."

My answer is—it depends. If Checkland's work is well known, authoritative, and respected, then the positivist method is fine. On the other hand, if Checkland's work was published in two conference proceedings in relatively obscure conferences and in the Journal of Poorly-Circulated Systems Research, then I would incline toward an interpretive citation.

I don't think either method has an inherent "better" or "more appropriate" structure—the question is which method conveys the purpose of the citation to the reader.

In general, my use of either style depends on how I feel about the matter I'm citing. If I accept it as a fact, or acknowledge that it is generally accepted as a fact, I will use the positivist style. If, however, I want to note that the evidence for the matter I'm citing is based primarily on one or a few specific studies, I tend to use the interpretive approach.

I also use the interpretive approach when I think it is relevant to give some details about the study since, as in your example, the method is more natural for this kind of usage. The corollary of this is that I sometimes use the positivist approach when I want to list citations, but I'm feeling too lazy at the moment to give details about why I cited the reference.

Personally I prefer the latter method, where "context" of some type is provided. This is not an issue of Positivist vs. interpretivist perspective or culture. Many good positivist authors provide a written context. I have reviewed far too many papers, of both research types, were statements are made as factual with no support or evidence at all ( be it positivist of interpretivist).

All statements should have references and all references should provide some context; unless they are the authors original assertions, arguments, conclusions, etc.. and then they should give some context about the fact that they are making an assertion, or stating an axiom as part of a theory, etc... For example if the cited authors made an assertion, you should say "J and Y asserted that..", if they concluded, you say that, etc.

Too many authors simply "Dump" a name and date (as you put it), and provide no evidence or argument through a reference with a meaningful context. Credibility is the real issue here. If as a reader I know that Checkland has been studying this for 30 years and concluded that it is useful, that gives me some idea of the context of the reference.

This is similar to using a survey without validating it for reliability and construct validity, wherein your results are uninterpretable. The same is true here. If you just state as a fact that "Soft systems methodology has been found effective (Checkland 2000)." I may not be convinced as I have no referent of the evidence that the cited author has any credibility (unless I know who it is and have read extensively, etc..) I think that good arguments require carefully crafted statements that build the case through evidence, much as good research instruments are validated before they are used.

You should use the one that is most supportable. Even in interpretive writing, it is always best to state things in the way that is farthest from invention and closest to reporting. This may come down to semantics, otherwise known as word choice. Eg. "The subjects were afraid" vs. "The participants expressed confusion (quote participant).

This is a very interesting question that has made me think of the way that I approach this. I favour the later for the reason that (in this case) in Checkland's opinion soft systems are effective. In taking an interpretivist perspective I suppose it would be important to disclose the experience of the author, however I think that when a reader reads this statement he would be at least vaguely familar with Checkland—who he is, what hI1 contribution is to IS etc... In another case where the author is less well known I would suggest that the reader would analyse the experience of the quoted author through the quality of the reference—type of journal / conference etc.. If this is not of much use, the reader would need to drill down and get the original reference if it is pivotal to his own concerns.

I think rather than focusing on the author perhaps it would be more expeditious to concentrate on how he/she came to the conclusion that he/she did and weather that conclusion is justified. In the case of the what to write within the later research, I would want to explain what Checkland meant by effective and how the quote fits in with my specific argument.

1) The word "argument" is being used here to mean the "one line conclusion," which will need to be supported by evidence. <back>

2) The court system is not standardized around the world. For example, in Britain and the US the judge is to ensure procedure and is not expected to ask questions of witnesses or call witnesses. In France the judge does these things in a air of balanced inquiry. In a US court the lawyers for the defense or prosecution only present their side of the argument in an adversarial style, the judge does not mediate. In an academic paper, winning the reviewer over may be best achieved by presenting all the supportive and counter evidence and then reasoning the balance. <back>

3) Of course, my intention is not to undermine the excellent efforts of MISQ, but merely to use their public instructions as a counter point to bring out my argument. <back>

4) A "fact" being defined as something that both parties agree is not in dispute at the present time. <back>

Alvesson, Mats & Skoldberg, Kaj (2000). Reflexive Methodology. Thousand Oaks, Calif: Sage.

Cohen, H. Floris (1994). The Scientific Revolution. Chicago: University of Chicago Press.

Klein, Heinz K. & Myers, Michael D. (1999). A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems. MIS Quarterly, 23(1), 67-95.

Lakoff, George (1979). The Contemporary Theory of Metaphor. In Andrew Ortony (Ed.), Metaphor and Thought (pp.202-251). Cambridge, Cambridge University Press.

Latour, Bruno (1987). Science in Action. Cambridge, MA: Harvard University Press.

Walton, Douglas (1998). The New Dialectic. Toronto: University of Toronto Press.

Webster, Jane & Watson, Richard (2002). Analyzing the Past to Prepare for the Future: Writing a Literature Review. MIS Quarterly, 26(2) xiii-xxiii.

Mike METCALFE's research interest is problem definition (including research as a problem) using argumentative inquiry, which he defines to include critical systems and perspectival thinking.

Metcalfe, Mike (2003). Author(ity): The Literature Review as Expert Witnesses [45 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 4(1), Art. 18, http://nbn-resolving.de/urn:nbn:de:0114-fqs0301187.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License