School librarians making global connections: Conjecture mapping and researcher practitioner partnerships
Peer reviewed article
Dr Barbara Schultz-Jones (University of North Texas), Dr Marcia Mardis (Florida State University) and Dr Sue Kimmel (Old Dominion University, Virginia) share their research on a way to support evidence building of causal relationships between the school library and a teacher librarian’s contribution to student outcomes.
Writers’ note: We often look to our Australian colleagues for research they’ve conducted and experiences they are willing to share. We thought our recent research project could be of interest as we explored causal evidence of practice.
Editor’s note: This research is presented for Australian teacher librarians to consider a research project from overseas and how it may impact how we develop evidence of practice.
This research describes the origins of the American Association of School Librarians (AASL) Causality School Librarians and Student Success (CLASS) work and the high-level findings of the initial research phase. Strategies are shared for operationalising further research on those high-level findings through conjecture maps that guide researcher-practitioner partnerships.
School librarians or teacher librarians as they are known in Australia and elsewhere, are guided by the school library guidelines of the International Federation of Library Associations (IFLA). These guidelines position school librarians as unique contributors to the learning environment through their own work, as well as through their collaborative work with classroom teachers.
The American Association of School Librarians (AASL) initiated a project called Causality School Librarians and Student Success, or CLASS. The initial phase of the CLASS project was a 2014 convening of researchers that culminated in a white paper in which AASL laid out a ten-year research agenda. In the CLASS II phase, three research teams from Florida State University, Old Dominion University and the University of North Texas set out to accomplish two pieces of research. To answer the research question “What causal relationships between school-based malleable factors and student learning are present in published research?” we performed a large-scale meta-synthesis of effective classroom teacher practices documented by high quality causal research. Then, we examined those practices to develop theories regarding which ones fit best with the work of school librarians, and provided testable designs employing the findings from our aggregation. These resulted in theoretical conjectures to test through small-scale pilot studies. These pilot studies allowed us to refine our theories and understand how classroom practices translate to a school library environment.
The aggregation of high quality causal research was extensive. Each team conducted independent searches. Florida State concentrated on the What Works Clearinghouse where the most stringent, quantitative research is stored, and examined articles across the educational spectrum. Old Dominion conducted a widespread search of EBSCOhost with a focus on educational best practices and subtopics, and examined Hattie's Visible Learning (2009) analysis of 800 syntheses, focused on all learning. The University of North Texas searched the Scopus database with the keywords “school librar*” + the relevant terms identified during the broad Scopus search with keywords “caus* AND school* AND/OR learn* AND/OR achiev*”. Collectively, this resulted in 1,598 studies as our starting point and these were then winnowed down to 310 studies that met the U.S. Department of Education’s highest standards for evidence, as identified in the Every Student Succeeds Act, a federal omnibus of education legislation for primary and secondary schools. There are four levels of evidence ranging from demonstrates a rationale, through promising which may include well-designed qualitative research all the way up to moderate and strong levels, which include quasi experimental and experimental research. The 310 research studies illustrated the two levels of moderate and strong. From those 310 studies a number of effective practices surfaced that school librarians could implement and probably are implementing. Many of these approaches are ones that would be expected by school librarians to use when they engage with learners, however, they haven’t been studied in terms of whether they have a causal relationship with student learning outcomes in the school library context.
As we proceeded with our work, federal education policy began shifting. The following quote from Mark Schneider, Director of the Institute for Educational Sciences (IES), the research arm of the U.S. Department of Education, indicates that IES is now shifting from documenting what is effective to implementing what is effective and this shift in emphasis fits well with the CLASS II work:
In years past, IES has spent much of its budget and energies identifying what works for whom under what circumstances. But that’s only part of our job.
Just as important: We need to figure out the best channels to get that information into the hands of teachers, so that more students have teachers who are using the most effective, evidence-based methods. (2018)
One of those promising methods to identify and implement effective teaching and learning practices is researcher-practitioner partnerships (RPPs).
In this section we share some of our findings employing conjecture mapping, a tool employed by Research-Practice Partnerships and a useful and promising tool for our next step.
Rather than serial experiments, RPPs allow educators to work alongside researchers to test, examine, and refine improvement incrementally and cyclically along the lines of design-based implementation research. These long-term partnerships allow for deep engagement in the practices and with the participants (Penuel & Gallagher, 2017). RPPs are a strong model for beginning to investigate causal phenomena in learning in school libraries because RPPs are flexible and tailored to the local environment. The role of the educator and researcher are transparent in this model and they're accounted for, and the focus remains on understanding the changes in teaching and learning practices that improve outcomes as well as on the evidence generated by these implementations. Importantly, along the way, learners are able to continue their normal practices and benefit throughout the process.
Researchers and practitioners begin with the theory or conjecture, then collect data, analyse the results and adjust the intervention for subsequent implementations and, throughout the initial conjectures, refine as well, adding to the transferability of results. The RPP model has many benefits including considering student success beyond test scores and leading to the question, how do we create a plan for operationalising RPP? We begin with a simplified version of a conjecture map to illustrate the components and application of this technique.
Learning scientists interested in RPP’s often employ Design-based research (DBR), an iterative process of testing and refining both theory and practices. Sandoval (2014) proposes that conjecture mapping provides a means of simultaneously evaluating a design and building a theory in order to uncover “causal mechanisms of effective learning environments” (p. 20). In synch with design based research, conjecture mapping allows for iterative cycles of conjecture and field testing that provide the researcher with the opportunity to elaborate and build out the initial conjecture and map based on findings and local contexts. The attention to context is especially key for educational research where contexts are understood to impact learning outcomes.
Sandoval (2014) asserts that any learning environment inherently expresses theoretical hypotheses about the learning that occurs in that environment. Those of us interested in research into libraries as learning environments must therefore pay attention to the kinds of learning that are made possible by our designs not only of the facility or the materials in our collections but of the types of social and discursive practices in these spaces. Conjecture mapping requires that we make those theoretical hypotheses explicit and allows for predictions that can be empirically tested regarding learning outcomes in the environment. In practice, conjecture mapping often leads to revisions to initial conjectures and maps in order to conduct further testing and refinement of theory. Lee, Recker, and Philips (2018) provide an overview of how their conjectures and map changed as they examined the development of STEM makerspaces in rural school and public libraries.
Components of conjecture mapping
The conjecture map starts with a high-level conjecture. An example might be “school librarians impact student achievement in reading through the provision of a variety of texts.” This is a familiar conjecture to school library research as the object of numerous correlational studies beginning with Gaver (1961) and followed by a wealth of studies known as the State or Impact Studies (Scholastic, 2016). Conjecture mapping is highly structured with particular elements often referenced as a “grammar” of conjecture mapping and depicted in Figure 1: design conjectures, mediating processes, and outcomes. These elements serve initially as a thinking tool asking the researcher to surface theories or assumptions about what is happening in a design for learning to produce the desired outcomes. Figure 1 depicts the components with a preliminary example provided for illustrative purposes.
Conjecture: Students with access to a school library staffed with a professional school librarian read more and read better.
- Tools and materials: Collections
- Participant structures: Access to collections
- Discursive practices: Talk about books
- Observable interactions
- Librarian talks with students about books and reading
- Students have access to collection
- Students choose reading materials
- Unlimited check-out.
- Artifacts: Collection of materials curated by school librarian to meet student needs and interests.
- Students read more (as measured by circulation, observation, self-reporting)
- Students read better (as measured by assessments and teacher observation).
Figure 1. Draft conjecture map
Sandoval (2014) suggests reading the map as “‘if learners engage in this activity (task + participant) structure with these tools, through this discursive practice, then this mediating process will emerge’” (p. 24) and “‘if this mediating process occurs it will lead to this outcome’” (p. 24). The design conjecture attends to the tools and materials, task and participant structures, and discursive practices in the learning setting. Another way to think about this would be what will participants be doing, with what tools or materials, and what kinds of talk or discourse will they engage in? These features of the design ask library researchers to extend beyond the facility and collections of materials provided by the library to think about the people in the setting: their roles, activities, and talk. From Figure 1, we originally jumped to thinking about collections and access to the collection as necessary to promote learning to read, but the conjecture map caused us to wonder about what discursive practices might need to be in place in the library. An obvious choice from our experience was that librarians often talk with readers about books and what they are reading. Booktalks, for example, might be a formal way that librarians talk about books but the conversations through read-alouds, among the shelves, or at the circulation desk were other places we might find discourse about books and reading. The conjecture map drew our attention to these practices as potentially salient to a theory of how librarians and libraries contribute to the development of readers.
Mediating processes are those artifacts or interactions enabled by the structures of a design that we theorise will lead to the desired outcomes. Thinking through this component of the conjecture map requires us to think more deeply about the way our proposed design for learning: access to a collection and talk with the librarian about books and reading function to develop students as readers. At this phase we are also thinking about what we can observe: artifacts and interactions. This component therefore provides guidance about what types of data we might collect to test our conjectures. What protocols will we develop for observations, and what kinds of work samples or other artifacts might we collect? These mediating processes are by definition in the middle between our designs and the outcomes we hope to measure and are connected by theory to those other components. Sandoval distinguishes these connecting theories as “design conjectures” and “theoretical conjectures.” These connecting conjectures might develop from early empirical trials or from the extant research literature. In the early stages, they may be preliminary and uncertain. For example in the library if we wanted to say that a collection of easy to read books leads to improved reading outcomes for a group of students we would need to document some mediating processes or artifacts such as students selecting those books to read. Our illustrative example includes interactions between a student and the school librarian and between the student and the collection. An artifact we might include would be statistics about the size and nature of the collection. We also propose some measures of the desired outcome that students will “read more and read better.”
Findings applied to a conjecture map
In our synthesis of the studies in the CLASS II Aggregation we have numerous studies related to reading. In this section we present these findings as evidence that might enlarge and focus the preliminary conjectures we provided above regarding the contributions to reading achievement of a centralised library collection staffed by a professionally certified school librarian.
To begin, we focused our conjecture on new readers from lower socioeconomic status (SES) homes, because our findings from educational research suggested that the impacts of effective teachers (Konstantopoulos, 2009; Nye et al., 2004), access to books (Allington et al, 2010), and early interventions (Assel et al, 2007; May et al., 2014; Pinnell et al., 1994) have a greater impact on those students from lower SES. In the design for learning we added details about the kinds of materials and access provided to new readers. The research indicated readers need access to a wide selection of reading materials (Allington et al., 2010; Fisher et al., 2001) and practice in reading real and engaging texts (Stevens & Durkin 1992; Vadasy et al., 2005) that includes new vocabulary (Swanborn & de Glopper, 1999) and with predictable or decodable texts (Box & Aldridge,1993; Jenkins et al., 2004). A professionally trained school librarian is clearly in a position to select and provide access to a wide selection of these kinds of materials. But the research suggests providing a collection of books is not sufficient (McGill-Franzen, 1999). Talk about books and reading is important to reading comprehension as students construct meaning about what they are reading as they develop into stronger readers (Fisher et al., 2001; Pinnell et al., 1994; Rosenshine & Meister, 1994; Saunders & Goldenberg, 1999; Stevens & Durkin, 1992; Wasik & Bond, 2001; Whitehurst et al., 1994). Conjecture mapping led us as school library researchers to wonder about the kinds of discursive practices that might be important in our designs for learning and the CLASS II research provided empirical support for a focus on targeted talk with readers about books and reading. A revised map informed by our findings is included in Figure 2.
Conjecture: New readers from lower SES who are provided frequent access to a library with a professionally curated collection of books, along with opportunities to talk about what they are reading, read more and read better.
- Tools and materials: Collection includes a wide variety of real and engaging texts selected to include new vocabulary and predictable and decodable texts for new readers.
- Participant structures: New readers have daily access to the collection and the librarian.
- Discursive practices: Students have opportunities to talk with the librarian about books and reading.
- Observable interactions
- Conversations between individual students and the librarian about books and reading.
- New readers are provided the opportunity to visit the library at least daily to select books from the collection.
- New readers are not restricted in the number of books they may borrow.
- Artifacts: Circulation record of the students.
- New readers find more books to choose, read more, and read better.
Figure 2. Revised conjecture map
As researchers, we see in this conjecture map several contrasts with common practice and a map for designing research to test these conjectures. Our high level conjecture translates to a hypothesis that might be tested with a matched quasi-experimental design:
Kindergarteners from lower SES who are provided daily access to a library with a professionally curated collection of books along with opportunities to talk about what they are reading, read more and read better than similar kindergarten students who are provided with “business as usual” access to the collection and opportunities to talk with the librarian.
However, we might also start with an exploration of the intervention the conjecture map suggests. The CLASS II research also provided numerous studies that suggested the importance of training educators to implement an intervention. In this case, we see that both school librarians and teachers would need to be introduced to a new practice of allowing kindergarten students daily access to the library with unlimited check-out. The research provides a rationale and educators would need to buy into these practices. We might pilot this intervention and observe what design or mediating processes might need to be built into the map. Such an iterative cycle of testing and refining is exactly what conjecture mapping provides and is illustrated in the work of Lee, Recker, and Philips (2018) who quickly discovered there were practical realities for the librarians in their study that they had not anticipated with their early conjectures.
CLASS II research finds these researcher-practitioner partnerships and using conjecture maps as a productive way to start to build theory, build out theory and build in form or context and the complexity that we know exists in educational settings.
References and further reading
Allington, R. L., McGill-Franzen, A., Camilli, G., Williams, L., Graff, J., Zeig, J., et al. (2010). Addressing summer reading setback among economically disadvantaged elementary students. Reading Psychology, 31, 411-427.
American Association of School Librarians [AASL]. (2014, December). Causality: School librarians and student success (CLASS). American Association of School Librarians National Research Forum white paper. Chicago, IL: American Library Association.
Assel, M. A., Landry, S. H., Swank, P. R., & Gunnewig, S. (2007). An evaluation of curriculum, setting, and mentoring on the performance of children enrolled in pre-kindergarten. Reading and Writing, 20(5), 463-494.
Box, J. A., & Aldridge, J. (1993). Shared reading experiences and Head Start Children's concepts about print and story structure. Perceptual and Motor Skills, 77(3), 929-930.
Fisher, D., Lapp, D., & Flood, J. (2001). The effects of access to print through the use of community libraries on the reading performance of elementary students. Reading Improvement, 38(4), 175-183.
Gaver, M. V. (1961). Effectiveness of centralized library service in elementary schools (phase I). Library Quarterly, 31, 254–256.
Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York: Routledge.
Jenkins, J.R., Peyton, J.A., Sanders, E.A. and Vadasy, P.F. (2004). Effects of reading decodable texts in supplemental first-grade tutoring. Scientific Studies of Reading, 8(1), 53-86.
Konstantopoulos, S. (2009). Effects of teachers on minority and disadvantaged students' achievement in the early grades. The Elementary School Journal, 110(1), 92-113.
Lee, V. R., Recker, M., & Phillips, A. L. (2018). Conjecture mapping the library: Iterative refinements toward supporting maker learning activities in small community spaces. In J. Kay & R. Luckin (Eds.), Rethinking learning in the digital age: Making the learning sciences count, 13th International Conference of the Learning Sciences (ICLS) 2018 (Vol. 1, pp. 320-327). London, UK: ISLS.
May, H., Gray, A., Gillespie, J. N., Sirinides, P., Sam, C., Goldsworthy, H., Armijo, M., & Tognatta, N. (2013). Evaluation of the i3 scale-up of Reading Recovery year one report, 2011-12. Philadelphia, PA: Consortium for Policy Research in Education.
McGill-Franzen, A., Allington, R.L., Yokoi, L. and Brooks, G. (1999). Putting books in the classroom seems necessary but not sufficient. Journal of Educational Research, 93(2),67-74.
Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26(3), 237-257.
Penuel, W. R. & Gallagher, D. J. (2017). Creating research-practice partnerships in education. Cambridge, MA: Harvard Education Press.
Pinnell, G. S., Lyons, C. A., DeFord, D. E., Bryk, A. S., & Seltzer, M. (1994). Comparing instructional models for the literacy education of high-risk first graders. Reading Research Quarterly, 29(1), 8-39.
Rosenshine, Barak and Meister, Carla (1994). Reciprocal teaching: A review of the research. Review of Educational Research, 64(4), 479-530.
Sandoval, W. (2014). Conjecture mapping: An approach to systematic educational design research. The Journal of the Learning Sciences, 23, 18-36.
Saunders, W. M., & Goldenberg, C. (1999). Effects of instructional conversations and literature logs on limited-and fluent-English-proficient students' story comprehension and thematic understanding. The Elementary School Journal, 99(4), 277-301.
Schneider, M. (2018). Message from IES Director: How to make education research relevant to teachers. Institute of Education Sciences.
Scholastic. (2016). School libraries work: A compendium of research supporting the effectiveness of school libraries. New York: Scholastic Library Publishing.
Stevens, R. J., & Durkin, S. (1992). Using student team reading and student team writing in middle schools: Two evaluations. Part I. Student team reading: A cooperative learning approach to middle school reading instruction (pp. 1-10). Baltimore, MD: Center for Research on Effective Schooling for Disadvantaged Students.
Swanborn, M. S., & De Glopper, K. (1999). Incidental word learning while reading: A meta-analysis. Review of educational research, 69(3), 261-285.
Vadasy, P. F., Sanders, E. A., & Peyton, J. A. (2005). Relative effectiveness of reading practice or word-level instruction in supplemental tutoring: How text matters. Journal of Learning Disabilities, 38(4), 364-380.
Wasik, B. A., & Bond, M. A. (2001). Beyond the pages of a book: Interactive book reading and language development in preschool classrooms. Journal of Educational Psychology, 93(2), 243-250.
Whitehurst, G. J., Arnold, D. S., Epstein, J. N., Angell, A. L., Smith, M., & Fischel, J. E. (1994). A picture book reading intervention in day care and home for children from low-income families. Developmental Psychology, 30(5), 679-689.
Acknowledgement: This research is supported in part by the American Association of School Librarians through IMLS grants RE 62-13-0212-13 and RE-00-15-0114-15.
How to cite this article – Schultz-Jones, B., Mardis, M. & Kimmel, S. (2020). School librarians making global connections: Conjecture mapping and researcher practitioner partnerships. Scan, 39(8).