Research Statement Benjamin D. Nye, Ph.D. Research Assistant Professor, University of Memphis

Research Focus My research program explores answers to the question: “How can we improve how societies distribute knowledge and fill gaps in expertise?” As expertise has become increasingly specialized, society’s capacity to build expertise has been overwhelmed. This problem ranges from the one-third of college students who fail to graduate [2] to university departments where the volume of relevant articles makes it nearly impossible to read them all [28]. To approach these issues, I study both the technology and the social ties that promote learning. I am currently working on three main subquestions: 1. Service-Oriented Intelligent Architectures: How can we design intelligent systems that follow service-oriented principles to encourage innovation and reduce development costs? 2. Social Learning: How can we design technologies that promote social learning and strengthen social relationships that encourage sustainable, life-long learning? 3. Social Systems: How can we design high-impact learning technologies that are viable in low-resource and cross-cultural contexts, such as the low-income communities and the developing world? My long-term vision is to unify these lines of research and develop universally-accessible and rapidlyprototyped learning environments that leverage and strengthen social ties across learning contexts. My research trajectory has provided distinctive insights into these problems, based on my contributions to intelligent systems, models of social learning, and models of social systems. On the topic of intelligent systems, I have created service-oriented architectures for conversational intelligent tutoring systems (ITS) [14, 13] and simulation-based learning environments [18, 27]. On the topic of social learning, my thesis work on cognitive models for memes explored socio-cultural mechanisms that underpin the spread of knowledge and new behaviors [18, 9]. On the third topic, I am researching features of ITS that make them suitable for different social and cultural contexts [10, 8, 19, 12]. As I study these topics, I engage with potential collaborators and funding agencies. Preparing grant proposals is an essential spark for new collaborations and lends new perspectives to my research. Over the last year, I co-authored competitive grant proposals for six different programs: the Advanced Distributed Learning program (ADL), Army Research Lab (ARL), Federal Emergency Management Agency (FEMA), Institute of Education Sciences (IES), National Institutes of Health (NIH), and Minerva Initiative. Two of these proposals were funded and I am supporting faculty for an additional three funded proposals (one by ONR, the Office of Naval Research, and two by NSF, the National Science Foundation). Historically, I have received support and presented to program officers from ONR, ADL, ARL, DARPA (Defense Advanced Research Projects Agency), and AFOSR (Air Force Office of Scientific Research). I am also an ad-hoc reviewer for the NSF Cyberlearning Program, an advisor and book editor for the ARL Generalized Intelligent Framework for Tutoring (GIFT) advisory committee on ITS, and a member of the Advanced Distributed Learning (ADL) Competency Alignment working group. As such, my background puts me in a strong position to approach this research. The following sections summarize how I conceptualize and am working on each area.

Service-Oriented Architectures for Intelligent Systems Core Problem: Intelligent systems are typically monolithic, which threatens re-use and interoperability. Across the field of intelligent systems, “application islands” (i.e., monolithic designs) remain a common design pattern [21]. These approaches are incompatible with the emerging ecosystem of web services;

components need to be shareable and embeddable. I am working to overcome this issue for ITS by developing a service-oriented framework for prototyping and composing ITS. At a low level, this seems like an intractable problem: ITS have very different designs. However, different ITS mostly share the same high-level behaviors [29], forming a de-facto ontology that can be used for service communication. The fundamental problem is that ITS split up their functionality in different ways. For example, one system may store their hints in a pedagogy module, but another may store them in a domain knowledge module. Traditional object-oriented and web-service patterns do not handle this well, because module API’s are not the same across systems. While components can be manually connected (explicit service composition), the semantic meaning of each module’s information already constrains which services need that information. As such, ad-hoc routing configurations are at best redundant and at worst incoherent. To address this, I created a new service-oriented framework for tutoring systems [7, 13]. This framework uses semantic messaging, where the content of a message determines which services receive it and how they process it. The topography and module boundaries of the architecture (e.g., servers, clients, cross-domain HTML frames, service API’s) are abstracted away by a network of gateway nodes that route messages [14]. This capability for plug-and-play composition makes it unique among ITS, even compared to contemporary service-oriented ITS architectures such as ARL’s GIFT system.

Shareable Knowledge Objects for Service-Oriented ITS (Ongoing Work) I am currently the project manager and lead developer for the Shareable Knowledge Objects ITS in the ONR STEM ITS Grand Challenge [14]. On this project, my framework integrates AutoTutor natural language tutoring dialogs into the ALEKS commercial math tutor, delivering over 400 tutoring dialogs on Algebra. Each service only communicates directly with a local gateway node, making it trivial to move or swap out services for dialog engines, semantic analysis, and animated agents. I also lead five faculty and mentor six graduate students on this project. These students were trained and involved in educational data mining to model student knowledge based on tutorial dialogs [15], developing new semantic analysis techniques [3], and building novel authoring tools for conversational dialogs [20]. Evaluations are ongoing.

Open Source Intelligent Tutoring Service Architecture (Next Steps) I am revising the Shareable Knowledge Objects core framework for release as an open-source generalizable ITS architecture. Due to the interdisciplinary skill set needed to build an effective ITS [16], composing ITS services should greatly accelerate development and reduce cost for ITS. Secondary goals are to build adaptors to integrate with other open projects, including GIFT, the ADL xAPI standard for Learning Record Stores, and the Pittsburgh Science of Learning Center DataShop repository for ITS and learning data.

Insights from Socio-Cognitive Agents and Training Games (Background) My insights behind the Shareable Knowledge Object framework came from my graduate work. At the University of Pennsylvania, I developed descriptive artificial intelligence: agents that emulate how humans make decisions, including their values, emotions, and limitations. This work had two related technical challenges: 1) composing cognitive agents out of empirically-supported cognitive components and 2) integrating cognitive agents into larger simulations and 3D training environments. For the PMFServ cognitive architecture [26], I developed a plug-in architecture for registering cognitive components, model composition to build an agent’s unified cognition out of components, and a dependency system where components bind to other mandatory or optional components. A model-driven architecture was a good match for PMFServ because each sub-model was designed based on specific theoretical assumptions. This system was used by all later PMFServ-based systems, including the NonKin training simulation described in Autonomous Agents and Multi-Agent Systems [27]. Composing ITS modules for simulation-based training had different challenges. I helped develop two

counter-insurgency training simulations: NonKin Village [27], on cultural competencies, and Attack the Network (AtN), on district stability [25, 11]. While an ITS adds pedagogically-relevant semantics to raw data streams and events, training information is divided across modules that are largely based on pragmatic concerns (e.g., efficiency, network topography). To address this, I designed the Complex Environment Assessment and Training System (CEATS) [11]. CEATS integrated and communicated data using a configurable ontology that mapped metrics (i.e., raw data) to semantic data. One limitation was that top-down ontologies generalize poorly, because standards are difficult to agree on. This influenced my work on the Shareable Knowledge Objects framework, which does not require an explicit ontology but instead allows a folksonomy: messages must conform to a specific semantic message format, but services can define the values for fields. This message format combines elements from the Foundation for Intelligent Physical Agents (FIPA) agent communication language and ADL learning experience (xAPI) messaging standard.

Identifying Effective ITS Features (Background) To understand ITS designs, I also synthesize empirical findings to identify common features. For the Cambridge Handbook on Multimedia Learning, I outlined archetypes of ITS (problem-based, simulation-based, conversation-based, etc.) and the empirical evidence supporting key features [12]. Next, in the International Journal of AI in Education, I analyzed natural language ITS related to AutoTutor [13] to identify successful tutoring strategies, semantic analysis, and roles for pedagogical agents. I am also an upcoming volume editor for the Generalized Intelligent Framework for Tutoring (GIFT) advisory committee, which is publishing a ten-year book series of design recommendations for ITS [19].

Modeling and Supporting Social Learning Core Problem: Social ties support learning, but these are weaker in online learning environments. Social bonds keep students invested in their coursework and participation. Online environments lose much of this social infrastructure. In the next generation of learning environments, we need to provide better support for relationships between different stakeholders (e.g., students, teachers, parents). These relationships may be persistent (e.g., in-person social networks) or temporary (e.g., students sharing a course). Learners also build familiarity and relationships with technology, such as pedagogical agents or application affordances (e.g., Apple interfaces). Adaptive mechanisms are needed to strengthen and build on these social ties to foster feelings of belonging, competition, or academic motivation. My dissertation focused on cognitive mechanisms and social ties that promote social learning. This research integrated social psychology mechanisms, social network models, and empirical analysis of how social ties impact learning. In this work, I found significant gaps in both social network and social psychology literature with respect to studying the combined impact of multiple levels of social ties (e.g., authority, in-groups, liking). As such, in my recent work I am collaborating with experimental psychologists to study the impact of technology on social tie formation and social learning.

Pedagogical Agents in Learning Technologies (Ongoing Work) Pedagogical agents offer one mechanism to build social ties with learners. Existing research shows that lower-knowledge learners benefit most from interacting with animated agents. However, these benefits may also be moderated by perceptions about learning or the domain content. In an experiment during Fall 2014, I am studying how students perceive and relate to animated tutoring agents as a function of their views about learning and mathematics. This work is a first step in a larger line of inquiry to examine mechanisms that build educationally-productive social ties in online learning.

Data Mining Tutor.com Sessions (Ongoing Work) As part of an active ADL grant I co-authored, I am also data mining human-to-human online tutoring sessions to find effective tutoring strategies. This project involves data mining approximately 250,000 Algebra and Physics tutoring sessions from Tutor.com. Over a thousand sessions have been hand-coded by expert tutors for speech acts and dialog modes (e.g., scaffolding) and machine learning of the full corpus is in progress [4]. This analysis has implications about how and when ITS should use instructional strategies, such as building rapport.

Social Sharing of ITS Modules (Next Steps) My next step in this line of research is to transition my service-oriented ITS architecture to a mobile application designed to share ITS content modules between users. This application will support mobile-to-mobile transfer of learning objects, using Bluetooth protocols. This capability will enable a study to contrast face-toface sharing against sharing learning objects online. It will also enable experiments that study the combined impact of different social ties on social learning. A grant to support this work is in preparation.

Memetic Model of Affordance Learning (Background) In my dissertation work, I studied social learning of affordances based on a novel meme-transmission model [5]. Affordance learning occurs when an agent learns the possibility of performing an action [17]. I modeled affordance learning as a meme, where Information Theory modeled transmission through the environment and Socio-Cognitive Theory modeled processing by a human actor [22, 1]. To design computational agents capable of social affordance learning, I operationalized and programmed more than twenty social science theories on attention, social influence, novelty, and memory. This model was tested in two simulation experiments, published in Presence [18] and Computational and Mathematical Organization Theory [9]. One experiment examined the spread of pro-government and anti-government affordances in the Hamariyah NonKin village, which is based on US Marine Corps data for an archetypal Iraqi village [18]. I applied clustering analysis to study how social networks and cultural factors impacted when each agent learned about and performed each new action (if ever). This analysis demonstrated that new behaviors spread plausibly, meaning that the social learning and memes (e.g., fads, tactics) might increase the social realism of agents in learning environments. A second experiment applied validation tests to a simulation of the Stanford Prison Experiment, a well-known psychology study where subjects randomly assigned as “guards” abused “prisoner” subjects [9]. While designing this experiment, I was struck by the lack of research that studied multiple (e.g., three or more) social influence factors at the same time. This led me to explore collaborations to measure social learning.

Models of Social Systems: Effective Features and Barriers to Technology Core Problem: What features of social systems impact effectiveness of learning technologies? My final area of interest is the adoption and sustainability of educational technology. It is no secret that the field is littered with systems that produced significant learning gains, but were never again used by learners. I have an extensive background in modeling and simulating social systems, including my thesis work modeling the adoption patterns for new behaviors. Historically, I have combined applied mathematics (probabilistic models, game theory, systems dynamics) and generative simulations (agent-based models, cognitive models, Monte Carlo sampling) to model socio-political instability and breakdowns in social services such as education and health care. More recently, I am applying these skills to consider ITS adoption.

Modeling Barriers to Intelligent Tutoring Systems (Ongoing Work) To examine the intersection of ITS and social systems, I conducted a systematic mapping study on barriers to ITS adoption. This study analyzed 1054 papers published between 2009 and 2012, focusing on general

barriers to adoption [8] and also barriers that are distinct to developing countries (forthcoming in the International Journal of AI in Education) [10]. This research revealed that ITS designs should place greater focus on the needs of teachers [8], that smartphones and multi-user laptops are critical ITS platforms for the developing world [10], and that different developing world educational contexts (classroom, institutional, and informal) have very different educational technology needs [6].

Modeling Conflict and the Dynamics of Access to Public Goods (Background) My research on education is informed by my work modeling social systems, particularly my work on access to public goods in systems with disparate stakeholders, such as education and health. From studying conflict in Iraq, Afghanistan, and elsewhere, poor education is a significant second-order force for conflict, due to its impact on social mobility. Lack of social mobility worsens poverty and conflict. My earlier work includes FactionSim [24], which modeled group membership and factional interaction, and its successor, StateSim [25], which modeled country and regional stability. StateSim simulations showed persistent under-investment in education, due to its long time horizon, as well as disenfranchisement of less-influential factions as a function of ingroup biases for groups in power. This implies that grassroots educational interventions are particularly important for developing countries, which often have severe problems with corruption. While my major focus is now education, I am still involved in studying conflict and public goods. I co-authored a recently-awarded Minerva grant where I am modeling differences between leaders’ threats and bluffs, based on constellations of the structural context and leaders’ linguistic features.

Summary This a pivotal period for educational technology where my combination of skills is particularly essential. As adoption and investment have tripled in educational technology over the last decade, new opportunities for intelligent learning environments have emerged: 1. Individualized Adaptation: Large-scale blended and online courses (e.g., MOOC’s) have increased our need to adapt to students (e.g., ITS) and analyze outcomes (i.e., educational data mining). 2. Social Adaptation: Educational technologies need to leverage and strengthen social ties, which are still not well-understood in online learning or even in face-to-face interactions. 3. Interoperability and Scalability: The rise of web-based educational technology and service-oriented computing have made interoperability a central concern for next-generation educational technology. On these topics, I am a leader and mentor in a team research environment, with tight integration between researchers, graduate students, and professional software developers. I have expertise building sociocognitive agents, simulation-based learning environments, and conversational intelligent tutoring systems [23, 27, 7, 14, 13]. My expertise also covers modeling social learning [9, 18] and studying social systems [24, 25], particularly those with educational implications [19, 8, 6]. Given the pressing needs in the field and my skill set, I am positioned to build a robust research program. The goal of this program is to improve long-term quality of life. High-quality education needs to be pervasive in society, as it is essential for economic opportunities, informed health decisions, and social mobility.

References [1] BANDURA , A. Social Foundations of Thought and Action: A Social Cognitive Theory. Prentice-Hall, Englewood Cliffs, NJ, 1986. [2] B OWEN , W., C HINGOS , M., AND M C P HERSON , M. Crossing the Finish Line: Completing College at America’s Public Universities. Princeton University Press, 2009. [3] H U , X., N YE , B. D., G AO , C., H UANG , X., X IE , J., AND S HUBECK , K. Semantic representation analysis: A general framework for individualized, domain-specific and context-sensitive semantic processing. In Human-Computer Interaction International (HCII) 2014. Foundations of Augmented Cognition. Springer, Cham, Switzerland, 2014, pp. 35–46.

[4] M ORRISON , D. M., N YE , B., AND H U , X. Where in the data stream are we?: Analyzing the flow of text in dialoguebased systems for learning. In Design Recommendations for Intelligent Tutoring Systems: Volume 2: Adaptive Instructional Strategies and Tactics, R. A. Sottilare, X. Hu, H. Holden, and K. Brawner, Eds. U.S. Army Research Laboratory, 2014, pp. 217–223. [5] N YE , B. D. Modeling Memes: A Memetic View of Affordance Learning. PhD thesis, University of Pennsylvania, 2011. [6] N YE , B. D. Comparing paradigms for AIED in ICT4D: Classroom, institutional, and informal. In Artificial Intelligence in Education (AIED) 2013 Workshop on Cross-Cultural Differences and Learning Technologies for the Developing World (LT4D) (2013), I. Z. Ivon Arroyo and B. P. Woolf, Eds., CEUR, pp. 1–8. [7] N YE , B. D. Integrating GIFT and AutoTutor with Sharable Knowledge Objects (SKO). In Artificial Intelligence in Education (AIED) 2013 Workshop on the Generalized Intelligent Framework for Tutoring (GIFT) (2013), R. A. Sottilare and H. K. Holden, Eds., CEUR, pp. 54–61. [8] N YE , B. D. Barriers to ITS adoption: A systematic mapping study. In Intelligent Tutoring Systems (ITS) 2014. Springer, Berlin, 2014, pp. 583–590. [9] N YE , B. D. Cognitive modeling of socially transmitted affordances: A computational model of behavioral adoption tested against archival data from the Stanford Prison Experiment. Computational and Mathematical Organization Theory 20, 3 (2014), 302–337. [10] N YE , B. D. Intelligent tutoring systems by and for the developing world: A review of trends and approaches for educational technology in a global context. International Journal of Artificial Intelligence in Education (In Press). [11] N YE , B. D., B HARATHY, G. K., S ILVERMAN , B. G., AND E KSIN , C. Simulation-based training of ill-defined social domains: The complex environment assessment and tutoring system (CEATS). In Intelligent Tutoring Systems (ITS) 2012 (New York, NY, 2012), Springer, pp. 642–644. [12] N YE , B. D., G RAESSER , A. C., AND H U , X. Multimedia learning in intelligent tutoring systems. In Multimedia Learning (3rd Ed.), R. E. Mayer, Ed. Cambridge University Press, Cambridge, UK, 2014, pp. 705–728. [13] N YE , B. D., G RAESSER , A. C., AND H U , X. AutoTutor and Family: A review of 17 years of science and math tutoring. International Journal of Artificial Intelligence in Education (In Press). [14] N YE , B. D., G RAESSER , A. C., AND H U , X. AutoTutor in the cloud: A service-oriented paradigm for an interoperable natural-language ITS. Journal of Advanced Distributed Learning Technology (In Press). [15] N YE , B. D., H AJEER , M., F ORSYTH , C., S AMEI , B., H U , X., AND M ILLIS , K. Exploring real-time student models based on natural-language tutoring sessions: A look at the relative importance of predictors. In Educational Data Mining (EDM) 2014. 2014, pp. 253–256. [16] N YE , B. D., R AHMAN , M. F., YANG , M., H AYS , P., C AI , Z., G RAESSER , A. C., AND H U , X. A tutoring page markup suite for integrating shareable knowledge objects (SKO) with HTML. In Intelligent Tutoring Systems (ITS) 2014 Workshop on Authoring Tools. In Press. [17] N YE , B. D., AND S ILVERMAN , B. G. Affordance(s). In Encyclopedia of the Sciences of Learning, N. M. Seel, Ed. Springer, New York, NY, 2012, pp. 179–183. [18] N YE , B. D., AND S ILVERMAN , B. G. Social learning and adoption of new behavior in a virtual agent society. Presence 22, 2 (2013), 110–140. [19] N YE , B. D., S OTTILARE , R. A., R AGUSA , C., AND H OFFMAN , M. Defining instructional challenges, strategies, and tactics for adaptive intelligent tutoring systems. In Design Recommendations for Intelligent Tutoring Systems: Volume 2: Adaptive Instructional Strategies and Tactics, R. A. Sottilare, X. Hu, H. Holden, and K. Brawner, Eds. U.S. Army Research Laboratory, 2014, pp. xv–xxvi. [20] N YE , B. D., YANG , M., H AYS , P., S ILVA -L UGO , R., C AI , Z., R AHMAN , M. F., H U , X., form-based authoring of natural language tutoring trialogs. In GIFTSym2. In Press.

AND

G RAESSER , A. C. Rapid,

[21] ROSCHELLE , J., AND K APUT, J. Educational software architecture and systemic impact: The promise of component software. Journal of Educational Computing Research 14, 3 (1996), 217–228. [22] S HANNON , C. E. A mathematical theory of communication. Key Papers in the Development of Information Theory (1948). [23] S ILVERMAN , B. G., B HARATHY, G. K., J OHNS , M., E IDELSON , R. J., S MITH , T. E., AND N YE , B. D. Socio-cultural games for training and analysis. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans 37, 6 (2007), 1113–1130. [24] S ILVERMAN , B. G., B HARATHY, G. K., N YE , B. D., AND E IDELSON , R. J. Modeling factions for “Effects based operations”: Part I Leader and follower behaviors. Computational and Mathematical Organization Theory 13, 4 (2007), 379–406.

[25] S ILVERMAN , B. G., B HARATHY, G. K., N YE , B. D., K IM , G. J., RODDY, M., AND P OE , M. M&S methodologies: A systems approach to the social sciences. In Modeling and Simulation Fundamentals: Theoretical Underpinnings and Practical Domains, J. A. Sokolowski and C. M. Banks, Eds. Wiley and Sons, Hoboken, NJ, 2010, pp. 227–270. [26] S ILVERMAN , B. G., J OHNS , M., C ORNWELL , J. B., AND O’B RIEN , K. Human behavior models for agents in simulators and games: Part I: Enabling science with PMFserv. Presence: Teleoperators and Virtual Environments 15, 2 (2006), 139–162. [27] S ILVERMAN , B. G., P IETROCOLA , D., N YE , B. D., W EYER , N., O SIN , O., J OHNSON , D., AND W EAVER , R. Rich sociocognitive agents for immersive training environments – the case of NonKin Village. Autonomous Agents and Multi-Agent Systems 24, 2 (2012), 312–343. [28] T IGHT, M. Working in separate silos? what citation patterns reveal about higher education research internationally. Higher Education (2014). [29] VANLEHN , K. The behavior of tutoring systems. International Journal of Artificial Intelligence in Education 16, 3 (2006), 227–265.