Abstract: “Analyze a Published Research Study” invites students to examine a published study's research methods to learn not only what a research report says, but also how the research was designed, carried out, and communicated. While this writing assignment was originally designed for an undergraduate course on research practices in literacy and composition, it may be used with both undergraduate and graduate students and may be appropriate for courses across the disciplines in which students study methods of scholarship. The primary goal of this assignment is to use writing as a mode of learning how to read scholarly research.
“Analyze a Published Research Study” is a collaborative writing-to-learn assignment situated in an upper-division course in research methods titled “Researching Writing and Literacy.” This course, capped at 25 students, is one of three required core courses in the writing major, including “Rhetoric and Civic Engagement” and “Professional Lives and Literacy Practices,” which students typically take in no particular order. The purpose of “Researching Writing and Literacy” is to introduce undergraduate majors in an independent writing program to common research practices in the field and to provide opportunities for them to design a qualitative research study of their own. Sharlene Nagey Hesse-Biber’s (2016) The Practice of Qualitative Research serves as the course’s principle text.
Given that, historically, research practices in the discipline of writing studies have varied widely, a persistent line of inquiry involves asking the following questions: what counts as research in literacy and composition?; what do different research paradigms afford and/or constrain?; and what do different methods of scholarship reveal about evolving and sometimes competing values, assumptions, and beliefs concerning the production of disciplinary knowledge? While students read a variety of studies from a host of perspectives throughout the course, replicable, aggregable, and data-supported (RAD) research principles govern the projects students themselves design. In his analysis, Haswell (2005) defines RAD research this way:
RAD scholarship is a best effort inquiry into the actualities of a situation, inquiry that is explicitly enough systematized in sampling, execution, and analysis to be replicated; exactly enough circumscribed to be extended; and factually enough supported to be verified. RAD scholarship may or may not use statistics. It includes a case study of one student when the participant’s background is defined, observation procedure and data analysis are specified, and participant’s behavior is recorded to the point that someone else could conduct a comparable study to validate, qualify, and perhaps add to the first study. It would also include an institutional writing-across-the-curriculum survey where courses sampled, questions asked, answers tabulated, and teacher practices observed are reported so precisely that other people at other institutions could conduct similar surveys and be able to compare the data meaningfully. (p. 201)
Haswell’s purpose is not merely to define RAD research, but, more importantly, to call for more rigorous and systematic research practices in rhetoric and composition. As a long-time writing center administrator, I have witnessed the recent turn in writing center studies toward more RAD research. Persuaded by its value, not just for writing centers, but also for the wider discipline, I make it the focus of my course on research methods. In keeping with our undergraduate curriculum, which also favors RAD research, this emphasis serves students well in laying the groundwork for designing their own studies in other courses. It is especially useful for students who go on to write an honors thesis or who pursue further study in our graduate program.
Because reading research studies can be challenging for novices, and because so much research published in literacy and composition is not RAD research, students need guidance in how to make sense of scholarship. The primary learning objective of the course is to design a qualitative research study that exemplifies RAD principles. Examining published studies as models is one way to teach these principles. While a more advanced assignment later in the curriculum might take up more complex matters, such as statistical procedures, the goals of this assignment are more general and are appropriate to an introduction to research methods.
As in many other college courses, much of what students write is based on a text they have been assigned to read. Because newcomers often struggle with complex academic texts written for an audience of experts, I designed “Analyze a Published Research Study” in order to scaffold students’ reading and writing in the discipline. In this writing-to-learn assignment, students work in small groups to apply a RAD heuristic to an article, design a related in-class activity, and write a 3-5-page analytical response paper.
While this assignment could be completed individually, a key feature was to make it a small-group assignment. My assumption in designing this assignment was that if students could practice reading and writing about difficult research studies collaboratively, then they might also learn from each other a wider repertoire of strategies to navigate challenging texts and to communicate their understanding in writing.
This assignment emerges from three key assumptions about effective disciplinary reading: 1) readers use specific strategies, 2) they internalize those strategies through repeated practice, and 3) they are sometimes conscious of them. Therefore, the assignment offers instruction in particular strategies, opportunities to re-apply them, and speaking and writing exercises that encourage meta-awareness. Reading specialists Beck, McKeown, Sandor, and Kucan (1996) and Fisher and Frey (2004) recommend a variety of general strategies for improving comprehension. While such strategies are useful, their value is limited. Griswold (2006) points out that different kinds of texts read for different purposes require different reading strategies, and thus different pedagogical interventions (p. 67). Given that much research in literacy and composition is not RAD and that a course goal is for students to produce a project that demonstrates RAD methods, I needed to design a reading intervention that would at once privilege RAD research while at the same time provide a heuristic for making sense of scholarship that falls outside its parameters.
To build scaffolding for reading, I started with the idea that, regardless of the research paradigm, I wanted students to test each study they read throughout the semester against the principles of RAD research and to consider to what extent the research does—or does not—conform, and with what consequences for the creation of disciplinary knowledge. This approach would allow us to explore both the advantages and limitations of various methods of scholarship, while returning again and again to test the usefulness of a RAD perspective. Reading for RAD features would let us consider the value of alternative approaches and permit us to see which ones a RAD paradigm may occlude.
I was inspired to develop this assignment by Driscoll and Perdue’s (2012) “Theory, Lore, and More: An Analysis of RAD Research” in The Writing Center Journal, 1980-2009. The authors analyze writing center research to determine to what extent articles published in The Writing Center Journal exemplify RAD requirements. They identify seven characteristics of RAD research: background and significance, study design and data collection, selection of participants and/or texts, method of analysis, presentation of results, discussion and implications, limitations and future work. Driscoll and Perdue offer a reading rubric of sorts. They understood at the outset that, historically, most of the research published in The Writing Center Journal, as in the wider field of literacy and composition, has not been RAD, so they created a heuristic through which to read any study, which allowed them to assess the degree to which RAD principles were evident. Because their heuristic includes the very features my students later include in their own qualitative research designs in my class, Driscoll and Perdue’s reading rubric provides guidance in how to read published scholarship, demonstrates what to look for, exactly, from a research methodology perspective, and reinforces the genre features students in “Researching Writing and Literacy” are expected to demonstrate in their own work. In order to help students start to understand the insider discourse and conventions of researchers in the discipline so that they can later design quality research studies of their own, this assignment comes early in the semester. Other major assignments that build from it include an annotated bibliography, a literature review, a research question, and a research proposal.
To develop students’ reading practices early on, I position this assignment at the beginning of the course, with small groups presenting their studies from time to time throughout the semester. To provide some practice in reading published research studies before students take up their own studies, I implement this assignment by modeling it myself, using Driscoll and Perdue’s study as an example. After all, if the authors advocate for more RAD research, then we should expect their own article to model its principals.
This assignment is an example of what Elbow (1997) terms a “low stakes” writing assignment. Because this is a relatively small assignment, focused on only one sample study, students working in groups of three to four can easily complete it in a short time without significant effort, though working collaboratively does take more time than working individually.
This assignment has three key components: 1) a written analysis of the selected study that applies Driscoll and Perdue’s (2012) reading rubric; 2) the design and implementation of a meaningful in-class activity to prompt writing and talking about the study; and 3) individually written reflections on what the student learned. Typically, I offer a list of studies to choose from, deliberately selecting scholarship from disparate perspectives that illustrate methodologies highlighted throughout the course. Alternatively, groups may select their own focal study for analysis, so long as they make a case for its relevance to the course.
Having designed many small group assignments and witnessed lots of collaborative group projects go awry in the writing center, I have learned to caution students not to divide this task into parts to be completed individually. Instead, the goal is to prompt students to read and write together, to learn from each other how they read and write, and to practice reading with a more focused, deliberate sense of purpose. In this way, I emphasize Tierney and Pearson’s (1983) idea that reading is a “composing process,” in which readers develop “mental models.” Tierney and Pearson argue that reading is like writing, an act of composing, involving “planning,” “drafting,” “aligning,” “revising,” and “monitoring” (pp. 568-78). Emphasizing reading as not merely a receptive activity, but also a productive one, they assert that reading mirrors composing. This idea is particularly apt for a writing course. What is more, making this a small group assignment shifts reading-as-composing from an individual process to a shared one, in which students read and make meaning jointly, while simultaneously externalizing their reading processes for one another to witness and to learn from.
Students have been especially engaged by completing this assignment together in small groups, creating lively writing activities, interactive quizzes, and games related to their focal studies. Groups guide classmates to draw connections to key concepts and research methods described in the textbook, and to other research studies included on the syllabus. Again and again, student-led discussions have been the most energetic classes of the semester. Key to ensuring the success of groups, I meet with each group at least a week before they lead class discussion. The purpose is to hear group members detail their understanding of the study’s research methods and to transform their ideas into effective plans for leading class activities. Because students are not experienced classroom teachers, this is where they benefit from intervention. They need help to design engaging activities and to anticipate how best to structure and manage class time. As I often remind groups, their purpose is not to lecture at their classmates about what they think of a particular study, but, instead, to engage the class in taking part in meaningful activities around key ideas or especially noteworthy research methods, design, and presentation choices.
The written analysis portion of this assignment invites students to develop an argument about the extent to which their selected study does—or does not—exemplify the seven requirements of RAD research, as Driscoll and Perdue describe them. These analyses are most interesting when the study does not adhere entirely to RAD principles. This leads students to wonder why and to speculate about the possible consequences. In doing so, they begin to understand how values, assumptions, and beliefs about writing research shift over time in the discipline. They also see that different venues have different expectations in terms of research methods and ways of communicating to specific audiences. For example, one recent group chose an unpublished dissertation as its focal study. Rather than read the entire dissertation, to make the task manageable, they narrowed their attention to the chapter about methods. In their analysis, they wondered aloud why the methods were so much more detailed than in much shorter published journal articles. Initially, the group was put off. But, over time, they came to see this difference in terms of genre expectations. An emergent researcher, writing to showcase her knowledge for a dissertation committee, had a different purpose and audience than a more established scholar writing for readers of a specific journal. As this instance illustrates, if an aspect of RAD research is missing, or if it seems excessive, or if it defies expectation in some way, then the written analysis offers a platform for students to offer alternatives, to imagine how the study might be different, if it were revised to adhere more strictly to Driscoll and Perdue’s (2012) rubric.
Initially, I designed this assignment with the expectation that groups would first write their analysis and then design and lead class discussion. But students suggested an important revision. Groups recognized that composing a draft analysis was essential to laying the groundwork for the class activity because they first needed to demonstrate their understanding of the study in writing. But often the talk that resulted from class discussion led to new insights and understandings, which made groups want to revise their writing. They asked for the opportunity to submit their final drafts after class discussion. Because this is a writing course, encouraging revision only makes sense. It provides groups the opportunity to reflect on the usefulness of their in-class activities and to showcase newly emergent thinking that results from learning in cooperation with classmates. Revision invites groups to highlight their learning explicitly, prompting metacognitive awareness about both reading and writing strategies and reinforcing the writing-to-learn aims of this assignment.
Reflections, composed individually by each group member, reinforce this metacognitive awareness by prompting students to consider not only what they have learned and thought about in terms of reading practices, but also how they have worked together with peers, what their contributions have added, and how the group negotiated and managed conflicts.
If different kinds of texts require different reading strategies, and thus different pedagogical interventions, then Driscoll and Perdue’s (2012) rubric serves my purpose for reading in this particular course. One limitation, however, is that it narrows and focuses reading to the seven features Driscoll and Perdue identify. Their heuristic may not be right for other purposes. This structured reading may ignore or occlude other important aspects of a study. Likewise, similar to other strategies recommended by reading specialists, if applied too prescriptively, reading for these characteristics alone may foster a rigid, mechanistic approach to reading.
As a writing center administrator, I am particularly interested in writing center research and the turn in recent years, thanks to scholars such as Driscoll and Perdue (2012), toward RAD research. But not all of my students, more broadly in writing and rhetoric, share my interest. With that in mind, I work to emphasize that writing center research is not our primary focus; rather, Driscoll and Perdue’s study is just one sample we examine, amid a range of scholarship in the larger field of literacy and composition, which explains and models RAD features.
Although making this a collaborative assignment was important, given my course aims, students could also complete this assignment individually. Likewise, it might be adapted to examine more than one research study, perhaps to compare multiple studies, or to develop an annotated bibliography.
While I initially designed this assignment for undergraduate majors in writing and rhetoric, I have since used it with similar success in a graduate research methods course too. Because many M.A. students come to us with backgrounds in English and literary studies, they have never before read and analyzed research studies in literacy and composition. Like their undergraduate peers, novice graduate students benefit from guidance in learning the insider discourse and conversations of researchers in the discipline.
Finally, while the course this assignment was designed for is focused narrowly on qualitative research in literacy and composition, the assignment might be appropriate for analyzing research across the disciplines. Although Haswell (2005) decries the paucity of RAD research in writing studies, he points out that, as a method of scholarship, RAD research is “currently healthy and supported by every other academic discipline in the world” (p. 200). Hesse-Biber (2016), the author of the textbook I use in my course, is a professor of sociology. This assignment would be a great fit for the social sciences, in particular.
In a course devoted to research methods, students use the model studies selected for this assignment to launch their own qualitative research designs. Later, they return to apply the RAD heuristic again when providing peer feedback on their research proposals. Throughout the semester, students refer back, from time to time, to their group’s study and those of other groups, comparing and contrasting, as we examine varied research methods over the course of the semester. Prompting students to read and write with a more focused, deliberate sense of purpose, this writing-to-learn assignment scaffolds students’ understanding of disciplinary discourse and conventions so that they can later design effective research studies of their own. In this way, “Analyze a Published Research Study” is one way to help students develop as more effective and confident consumers and creators of disciplinary scholarship.
See the Supplementary Files for this article at thepromptjournal.com for a PDF facsimile of the original formatting of this assignment.
This assignment includes 3 components: 1) A 3-5-page written analysis, composed collaboratively, 2) Planning and implementation of a class activity and discussion about your selected study’s research methods, 3) An individual written reflection. Your group should turn in your final written analysis no later than one week after your in-class activity. This delay will allow you time to incorporate new insights that emerge from class discussion into your paper.
We will study a variety of different kinds of published studies to learn not just what they say about their subjects, but, more importantly for our purposes, how research studies are designed, carried out, and communicated to the discipline. We’ll look to other studies as models you may follow in designing your own research.
For newcomers, reading research in literacy and composition may be a challenge initially, because it is typically composed for an audience of experts, not novices. This assignment is designed to help you learn to read complex research reports by providing specific criteria for evaluating studies.
As a writing major, you will read lots of research across courses in the curriculum. Likewise, you will have multiple opportunities to design and carry out your own research. This assignment is intended to help you to read with greater purpose and understanding, not only in this course, but also in other courses as well. It offers a heuristic for reading that you may find useful as you gather and study secondary sources that will inform your own research project in this course.
The purpose of this assignment is to use writing to assess the design, methods, and reporting of a published research study.
Together with a small group of classmates, write an analysis of the study you select.
Then design a plan for leading an activity-based class discussion. One purpose of this activity is to prompt your classmates to draw connections between your study and other reading from the course, including research practices described in your textbook, The Practice of Qualitative Research.
Together with your group, schedule a meeting with your instructor at least a week before your in-class activity to discuss your plans. In this meeting we will collaborate to design your activity.
Don’t make the mistake of dividing up this group assignment into individual tasks. Everyone in the group is equally responsible for contributing to every part of the assignment. The goal is to develop your insights collaboratively. You can expect to meet together with all group members present multiple times in order to complete this assignment successfully. A shared Google Doc is recommended to facilitate your work.
Your written analysis should address the seven areas of concern detailed in Driscoll and Perdue’s (2012) “Theory, Lore, and More: An Analysis of RAD Research in The Writing Center Journal, 1980-2009.” Follow the model of Driscoll and Perdue, detailed on pages 21-23, to analyze your focal study in order to make an argument about the extent to which it either does—or doesn’t—exemplify the requirements of RAD research:
Study Design & Data Collection: In what ways does the researcher identify research questions or hypotheses, describe data collection, justify methodological choices, and explain how those choices relate to study objectives? In what ways does the study address and control for bias? Is the design sufficiently detailed so that the study could be reliably replicated in a new context?
Selection of Participants and/or Texts: In what ways does the researcher describe and justify the selection and/or sampling of participants and/or texts?
Method of Analysis: In what ways does the researcher describe the method of analyzing data? In what ways does it fit the study’s design? How are categories of analysis and operational definitions presented and described? What evidence does the study provide that analysis was done in a systematic manner? Could the method of analysis be reliably replicated in a new context?
Presentation of Results: In what ways does the researcher present the results? In what ways are they supplemented with graphics, excerpts from texts, interviews, or other evidence? In what ways are results presented to separate them from discussion/opinions of the researcher?
Discussion & Implications: In what ways does the researcher describe how the results of the research contribute to the field’s understanding of the issue and how the current study informs, complicates, or extends previous work? How does the researcher address implications and broader applications of the results?
Limitations & Future Work: In what ways does the researcher give suggestions for future work that they or others may pursue relating to the study results? How does the researcher acknowledge biases and study limitations?
As you’ll see, much good and influential research in our discipline isn’t RAD. You may find, then, that the study you select doesn’t match the criteria set forth by Driscoll and Perdue. Here are some additional questions to consider as you develop your analysis:
How do the authors of this study understand what counts as research in literacy and composition?
What does the particular research paradigm afford and/or constrain?
What does this research methodology reveal about evolving and sometimes competing values, assumptions, and beliefs concerning the production of disciplinary knowledge? With what consequences?
After leading class discussion about your study, what new and different understandings emerged as a result?
The audience for this assignment includes your classmate and instructor. You’ll assign the class to read your study in advance of leading class discussion about it. But you should assume that your classmates will have a range of familiarity: Some will have read with care. Others may not have read at all. With that in mind, you’ll need to design a class activity that will engage all of your classmates and help them to learn something new about research methods in our discipline, regardless of their level of preparation.
The written portion of this assignment should follow either MLA or APA conventions.
The activity portion of this assignment may take many forms. Be creative. Avoid lecturing at us. Instead, design an activity, including writing-to-learn, which engages the class in conversation.
Written reflection: Each group member should turn in a brief written reflection, which discusses your individual contributions to the group and how your group worked together. Explain something new you learned or thought about as a result of reading this study together as a group and presenting it to the class.
Your group will receive two grades for this assignment, one for the collaborative written analysis and individual reflections, another for the in-class activity and discussion.
Beck, I. L., McKeown, M. G., Sandora, C., Kucan, L., & Worthy, J. (1996). Questioning the author: A yearlong classroom implementation to engage students with text. The Elementary School Journal, 96(4), 385–414.
Driscoll, D. L., & Perdue, S. W. (2012). Theory, lore, and more: An analysis of RAD research in The Writing Center Journal, 1980 - 2009. The Writing Center Journal, 32(2), 11–39.
Elbow, P. (1997). High stakes and low stakes in assigning and responding to writing. In M. D. Sorcinelli & P. Elbow (Eds.), Writing to Learn: Strategies for Assigning and Responding to Writing Across the Disciplines (1st ed., pp. 5–15). San Francisco, CA: Jossey-Bass.
Fisher, D., & Frey, N. (2004). Improving adolescent literacy: Strategies at work (1st ed.). Saddle River, NJ: Prentice Hall.
Griswold, W. G. (2006). Postsecondary reading: What writing center tutors need to know. Journal of College Reading and Learning, 37(1), 59–70.
Haswell, R. H. (2005). NCTE/CCCC’s recent war on scholarship. Written Communication, 22(2), 198–223. http://doi.org/10.1177/0741088305275367
Hesse-Biber, S. N. (2016). The practice of qualitative research (3rd ed.). Los Angeles, CA: SAGE.
Tierney, R. J., & Pearson, P. D. (1983). Toward a composing model of reading. Language Arts, 60(5), 568–580.