These activities can be inspired by heterogeneous learning theories; a graph models the integration of heterogeneous activities into a coherent pedagogical scenario. The edges of the graph connect activities. They represent the two-fold relationship between activities—how they relate to each other from a pedagogical and from an operational viewpoint.
From the operational viewpoint, edges are associated with operators that transform the data structures produced during a learning activity into the data structures needed to run the next activity. A sequence of operators constitutes a workflow.
From the pedagogical viewpoint, an edge describes why an activity is necessary for the next activity; it can, for instance, be a cognitive prerequisite, a motivational trick, an advanced organizer, or an organizational constraint. The edges are classified in a library of 28 pedagogical ideas. The extent to which one activity is necessary for the next one is encompassed in the weight of an edge. The transition between two activities is stored as a matrix; the cell (m,n) of a transition matrix stores the probability that a learner in cognitive state m will evolve to state n in the next activity. I propose a list of 28 states that have a specific meaning in education. The transition matrix can be summarized by a parameter that constitutes the edge weight; an edge between two activities has a heavy weight if learner performance in one activity is highly predictive of success in a connected activity. The graph also constitutes a probabilistic network that allows predicting the future state of a learner.
When the pedagogical scenario is running, the actual state of the learner can be inferred not only from his past activities but also from his current behavior and from the activities of others. Learner modeling combines these 3 sources of information and is hence represented as a cube.
This book does not propose a learning theory. It describes how rich learning activities, often designed for small classes, can be scaled up for use with thousands of participants, as in MOOCs. It also describes how a pedagogical scenario can be adapted to the level of participants or repaired on the fly when problems occur. The graph describes how the scenario can be modified, stretched, cut, and extended. Orchestration refers to the real-time management of pedagogical scenarios to ensure the maximum the satisfaction of many constraints.
Orchestration Graphs
The modeling language proposed in this book aims to describe every kind of pedagogical scenario. By “pedagogical scenario,” I mean any sequence of learning activities, also referred to as a lesson plan, a didactic sequence, or a script. A pedagogical scenario will be modeled as a graph with specific properties. Describing a lesson plan as a graph may seem pointlessly complex as most lessons, as well as most existing MOOCs, are built as a linear sequence of activities. However, pedagogical scenarios based on a richer structure also exist, and, with orchestration graphs, I aim to capture these richer scenarios. Therefore, I will extract the workflow that underlies these scenarios; that is, the (often invisible) sequence of data transformations between activities. I hypothesize that formalizing the workflow of a pedagogical scenario will enable our community to scale up learning activities that may initially appear as non-scalable.
A model is like a lens; it constitutes a specific way of looking at reality, emphasizing some elements, variables, or phenomena. The proposed modeling language pays particular attention to the way educational scenarios concretely unfold with time, what happens practically during the course, and what needs to be repaired. The term “orchestration graph” embeds this pragmatic viewpoint: “orchestration” refers to the real-time management of learning activities (Dillenbourg, 2013) such as launching activities, managing time, circulating data, and adapting the scenario on the fly. The term “orches- tration” is not the optimal metaphor, because orchestration differs from conducting an orchestra. But, intuitively, it is true that there is a touch of the maestro in the performance of a talented teacher enthusiastically conducting rich class activities, running across multiple planes (individual, team, and class activities), with or without computers. How does an elementary school teacher adapt the current activity if two students missed the previous activity or if a crane comes into view in front of the classroom window and starts to operate? How does a university lecturer react when he notices that he is losing the attention of his audience? How does a MOOC teacher cope with thousands of complaints that the last video was incomprehensible? Of course, the adaptation of instruction is not a new topic, but this term usually refers to the process of adapting instruction to individual learner needs. The notion of orchestration is broader. It includes the selection of the most appropriate activities for the learners, but it also extends to very practical aspects of classroom management such as managing time (“I have 5 minutes left—it’s too late to start a new chapter”), managing space (“My classroom is too small to move chairs while forming teams”), assessment constraints (“I have to issue individual grades because the school does not accept team grades”), energy constraints (“I will use peer grading because the number of assignments to grade is too high”), and safety constraints (“The students cannot explore the city on their own, so each team will investigate architectural aspects of the same street”). Managing a lesson or a MOOC requires continual regulation—monitoring the learners’ activity, adapting some activities, or even modifying the scenario. For this reason, this book follows a kind of systemic viewpoint on the educational ecosystem, in which the scenario is a species that constantly needs to adapt to the constraints of its environment in order to “work well.” As a consequence, the proposed modeling language has to capture the flexibility of a pedagogical scenario, that is, the features that make the orchestration process easy or difficult.
There is a contradiction between these two paragraphs: the first one claims that scale requires automated workflows, while the second explains that orchestration requires flexibility. This tension will be present throughout this book—a “flexible workflow” is an oxymoron, like a deafening silence. This tension constitutes an interesting challenge for computer science: how to modify, skip, and reorder the various steps of data processing without breaking data consistency?
To learn more : By modeling pedagogical scenarios as directed geometrical graphs and proposing an associated modeling language, this book describes how rich learning activities, often designed for small classes, can be scaled up for use with thousands of participants. With the vertices of these graphs representing learning activities and the edges capturing the pedagogical relationship between activities, individual, team, and class-wide activities are integrated into a consistent whole. The workflow mechanisms modeled in the graphs enable the construction of scenarios that are richer than those currently implemented in MOOCs. The cognitive states of learners in two consecutive activities feed a transition matrix, which encapsulates the probability of succeeding in the second activity, based on success in the former. This transition matrix is summarized by a numerical value, which is used as the weight of the edge. This pedagogical framework is connected to stochastic models, with the goal of making learning analytics more appealing for data scientists. However, the proposed modeling language is not only useful in learning technologies, it also allows researchers in learning sciences to formally describe the structure of any lesson, from an elementary school lesson with 20 students to an online course with 20,000 participants.
Extract from Orchestration Graphs - Modeling Scalable Education
By Pierre Dillenbourg
Published by EPFL PRESS