A report for the UK’s TLRP Technology Enhanced Learning – Artificial Intelligence in Education Theme. May 2011.
Authors: Joshua Underwood and Rosemary Luckin, The London Knowledge Lab.
This is the first in a series of reports on Artificial Intelligence in Education. The series also includes:
This report is one of the outputs from (the AIED theme of the Technology Enhanced Learning (TEL) research programme). This initial version of the text will be subject to revisions based upon feedback from readers. The Artificial Intelligence in Education (AIED) theme within the Personalisation strand of TEL is concerned with exploring the ways in which the work conducted under TEL within and across projects can contribute to the (inter)discipline of AIED.
Education and AIED
AIED stands for Artificial Intelligence in Education, but is AIED research having an impact in mainstream Education? Ten years ago Cumming and McDougal noted AIED could “scarcely claim to be in Education” (2000, p.197). However, during the last decade AIED has made great progress in moving out of labs into real world large-scale deployments that are having significant impacts (see Table 1). Furthermore, tremendous technological advances, for example in mobile systems and social networks, have been made and technologies, powerful enough to support AI techniques such as user modelling and speech recognition are now pervasive throughout much of society and our daily lives. The ubiquitous technology-rich settings that can enable widespread uptake of AIED for more effective formal and informal learning are now in place, and while developing robust solutions for AI problems is still challenging, such as speech recognition that works well with continuous speech in noisy environments and with young learners, existing AI technologies offer new opportunities for technology-enhanced learning.
However, other barriers to adoption and the everyday use of AIED systems remain. There are many notable technological successes (see Table 1) and yet these technologies are not fully exploited for educational purposes (Woolf, 2010) particularly within mainstream Education. Cumming and McDougal observed that as more and “more of learners’ time will be spent with technology (this) will bring to the fore in Education questions of how to design computer-based learning resources that are effective, while retaining allegiance to Education’s rich conceptualisations of learning.” They went on to claim that “(t)hese are precisely the issues on which… …AIED claims insight, so Education should discover that it needs AIED!” (2000, p.203).
Ten years after the publication of Cumming and McDougal’s paper the relevance of AIED research for today’s teaching and learning challenges is not fully recognised in mainstream Education. It may be that the AIED community has so far failed to successfully communicate with educators and policy makers or provide the kinds of evidence and support required for wider adoption. We believe AIED has the potential to make a much broader contribution to Education than it currently is. AIED techniques can respond to today’s educational challenges by delivering more flexible and inclusive, personalised, effective and engaging learning experiences throughout lifetimes and across formal and informal settings. However, realising this potential requires a greater effort to communicate the relevance of AIED research and to support its uptake within the wider learning community and by other stakeholders in Education. It also requires that the AIED community direct greater attention to understanding and communicating the manner in which AIED technologies can be integrated into educational settings, for example through recognising and developing a clear role for practitioners in relation to these technologies. Here, we describe what AIED is, what it has to offer and why it is relevant now and for the future of Education.
What is AIED?
AIED is interdisciplinary research “at the frontiers of computer science, education and psychology’. It promotes rigorous research and development of interactive and adaptive learning environments for learners of all ages, across all domains” (International AIED Society, 2010).
The nature of the research conducted under the heading of AIED has developed and evolved over the past 25 years and the community has become a broad church concerned with supporting the learning and teaching process in situ and in real time. AIED research addresses learning wherever it might occur, including in formal classroom settings as well as outside the classroom, both to support formal education and for broader, lifelong learning. AIED researchers are paying increasing attention to the affective and social as well as the intellectual aspects of learning with very active research being conducted to investigate collaboration, meta-cognition, self-regulation, motivation, and emotions, as well as the more traditional areas of scaffolding and intelligent tutoring; and the new possibilities afforded by data mining techniques.
AIED research is driven by educational problems and is as much about a way of doing research as about technology development. Theoretically grounded research is supported by systematic empirical evaluation that informs further theory development. The AIED community is actively exploring the ways in which learning and teaching can benefit from new and cutting edge technology, particularly drawing on research in Artificial Intelligence (AI).
What is the AI in AIED for?
Artificial Intelligence is concerned with: creating computational models of human faculties (e.g. speaking, learning, walking, and playing), enabling systems to replicate common-sense tasks (e.g. understanding language, recognising visual scenes, summarising text) and otherwise reproducing intelligent behaviour (Russell and Norvig, 1995). Intelligent behaviour may be usefully defined as acting rationally; that is doing the right thing given the available information, taking actions expected to maximise goal achievement (Russell and Norvig, 1995), which in the case of AIED means acting to maximise learning both in the short and long term – where learning is increasingly understood in a very broad way including the development of personality, a sense of self-esteem and self-efficacy among other forms of development. A key motivation for using AI techniques in the development of Technology Enhanced Learning is to support the development of systems that do the right things, or help teachers and learners to do the right things to maximise learning. This involves understanding and modelling learners, teachers, effective pedagogies and learners’ contexts. AIED systems are adaptive, offering support dynamically by responding appropriately to changing and incomplete information about learning objectives, learners, collaborators and context.
Why do Teaching, Learning and Education need AIED?
One of AIED’s scientific goals is to “…make computationally precise and explicit forms of educational, psychological and social knowledge which are often left implicit” (Self, 1999, p. 1). Such precise models of learning may become AIED’s greatest contribution to Education (Cumming & McDougal, 2000). Modelling is a central concept in AIED and increasingly rich models are being developed. Such models take into account characteristics of learners and teachers, collaborative, social, affective and meta-cognitive aspects of learning, the settings learning takes place in, and the learner’s context (e.g. Luckin, 2010). Baker (2000) identified three main roles for AIED models:
- Model as scientific tool. A model is used as a means for understanding or predicting some aspect of an educational situation.
- Model as component. A computational model is used as a system component enabling a learning environment to respond adaptively to user or other input.
- Model as basis for design. A model of an educational process, with its attendant theory, guides the design of technology-enhanced learning.
In the intervening decade a fourth key role has emerged:
4. Open Models as prompts for learner and/or teacher reflection and action: Computational models, usually of learner activity and knowledge, are made inspectable by and possibly opened for learners and/or teachers to edit. Such open models can prompt users to reflect on their learning and support meta-cognitive activity (Dimitrova, McCalla & Bull. 2007).
Models of these types and the adaptive systems that employ them have great utility for Education research and practice both in terms of developing our understanding of educational situations and learning and in delivering more efficient, personalised and contextualised support for learning and teaching. Furthermore, deployed technology enhanced-learning environments can be used to gather data efficiently and on a large scale, both to test and refine our understanding of learning and teaching, and in order to provide evidence for the effect of these systems on learning. Evaluation has been a consistent and increasingly important theme in AIED research (see Underwood & Luckin, 2011 for an exploration of themes in AIED research). There is now considerable work in evaluating AIED systems, particularly in well-defined domains, that are applied on a global scale with hundreds of thousands of students in classroom or university settings (see Table 1 for examples) (Dimitrova, 2010).
Table 1 Example ‘Mainstream’ Intelligent Learning Environments
For Learning Foreign Culture & Language
Alelo’s Tactical Language and Culture Training System uses a virtual game based environment and interactive lessons to provide foreign language and culture training. TLCTS employs AI techniques to process learners’ speech, engage in dialogue and evaluate performance and has been used by more than 40,000 learners worldwide with independent evaluations showing significant gains in learners’ knowledge of language and culture and greater self-confidence in communicative ability (Johnson & Valente, 20009). See http://www.alelo.com/ for more information.
For Learning Maths
Carnegie Learning’s Cognitive Tutors use AI techniques to provide learners of Maths with individualized attention and tailored material based on continual assessments (Carnegie Learning, Applying Cognitive Science to Education). Cognitive Tutors aim to act like human tutors constantly monitoring learner actions and guiding learners towards correct solutions, providing help on demand and in response to common mistakes and giving meaningful feedback to students on their acquisition of skills (Carnegie Learning, The Cognitive TutorTM: Successful Application of Cognitive Science). Cognitive Tutors are used in many schools in the US and elsewhere and several evaluations of Cognitive Tutors have been conducted (see Carnegie Learning, 2010). Evaluations have demonstrated that Cognitive Tutors can improve problem solving and critical thinking skills (Koedinger, Anderson, Hadley & Mark, 1997), improve performance on exams (Sarkis, 2004), improve student attitudes to mathematics (Morgan & Ritter, 2002), and show strong results for disadvantaged populations (Sarkis, 2004). See http://www.carnegielearning.com for more information.
Wayang Outpost is an intelligent tutoring system that helps learners prepare for maths tests and helps teachers in their assessment of students’ strengths. Wayang can provide interactive hints leading to the solution for a problem. As the student progresses through problems the system adjusts instruction using individualized strategies that are effective for each student. An evaluation of Wayang (Beal, Walles, Arroyo, Woolf, 2007) shows significant improvements on pre to post-tests and suggests the greatest benefits are for weaker students and those who make most use of the multimedia help features. For more information and to register to try the system out see http://wayangoutpost.com/
ActiveMath is an adaptive learning environment for Mathematics that applies AI techniques to automatically assemble individualised courses. ActiveMath can generate courses adapted to the learner’s curriculum, language and field of study, as well as to her cognitive and educational needs and preferences such as learning goals, preferred style of presentation, goal-competencies, and mastery-level (Melis & Siekmann, 2004). ActiveMath includes interactive exercises that can provide feedback and hints of different kinds in response to learner input. The ActiveMath system has been used and evaluated in classrooms and universities in various European countries for several years (see http://www.activemath.org/Software/Evaluation/). A Europe-wide formative and summative evaluation investigated usability and learners’ opinions of automatically generated courses; results indicated that learners appreciated the generated courses, felt these were personalized and that the generated courses helped learners to find their own way of learning (Ullrich & Melis, 2010). For more information about the ActiveMath system, research and access to a demonstration version see ActiveMath.org.
For Learning Physics
Andes is an intelligent homework helper for Physics. Students enter steps in solving a problem, such as drawing vectors, drawing coordinate systems, defining variables and writing equations and Andes provides feedback after each step (VanLehn et al, 2005). Andes encourages learners to use good problem solving strategies, provides immediate feedback on learner input and offers different kinds of instructional assistance depending on the kinds of error learners make. Andes has been used successfully since 2000 in the US Naval Academy and is in use elsewhere at college and high school level (see http://www.andestutor.org for more information). Evaluations in real classrooms over five years show that Andes is significantly more effective than doing pencil and paper homework and at a low cost, with students spending no extra time doing homework, and with no need for teachers to revise their classes in order to obtain these benefits (VanLehn et al, 2005). The Andes Physics Tutor is in use on an Open Free Physics course provided through the Open Learning Initiative.
For Learning Programming and Database skills
SQLTutor provides adaptive individualized instruction that helps learners’ master key concepts in database courses using student and pedagogical models. SQLTutor has been in large-scale use with several thousand users (Mitrovic et al., 2006), evaluated on numerous occasions and refined for more than a decade (see SQLTutor Evaluations). Evaluations of SQLTutor have demonstrated the need for feedback to be personalized to individual students’ needs (Martin & Mitrovic, 2006) the value of both negative and positive feedback, as opposed to only negative feedback, with students receiving both forms of feedback requiring significantly less time to solve the same number of problems, in fewer attempts and learning the same number of concepts as students in the control group (Barrow, Mitrovic, Ohlsson, & Grimley, 2008). SQLTutor is one of a number of constraint-based Intelligent Tutoring Systems (ITSs) produced by the Intelligent Computer Tutoring Group (ICTG) at University of Canterbury (New Zealand). These ITSs have proven effective not only in controlled studies but also in real classrooms, and some of them have been commercialized (Mitrovic et al, 2009). SQLTutor and other adaptive tutors for database skills are available through Addison-Wesley’s Database Place. ICTG are also working towards making it easier for teachers and domain experts to develop ITSs. ASPIRE (Authoring Software Platform for Intelligent Resources in Education) assists users in developing and delivering online constraint-based tutors and is freely available to all New Zealand Government-owned Tertiary Institutions. For more information about Intelligent Tutors developed by ICTG see http://www.cosc.canterbury.ac.nz/tanja.mitrovic/projects.html
ELM-ART: Episodic Learner Model – The Adaptive Remote Tutor
ELM-ART is an intelligent interactive system that supports learning to programme in LISP. “ELM-ART provides all learning material online in the form of an adaptive interactive textbook… …ELM-ART provides adaptive navigation support, course sequencing, individualized diagnosis of student solutions, and example-based problem-solving support.” (Weber & Brusilovsky, 2001, p.351). Provision of the system online was found to greatly contribute to flexibility and efficiency of learning with students accessing the system from both home and university locations, with many students completing the course in very short periods of time and achieving very good results in the final programming task (Weber & Brusilovsky, 2001). One AIED approach employed in ELM-ART is adaptive link annotation. Adaptive annotations augment hyperlinks with personalised hints that can help guide learners to the most personally appropriate learning content at any given moment. Adaptive annotation has been adopted by many systems and “(e)mpirical studies of adaptive annotation in the educational context have demonstrated that it can help students to acquire knowledge faster, improve learning outcomes… (and) …significantly increase student motivation to work with non-mandatory educational content” (Brusilovsky, Sosnovsky & Yudelson, 2006, p.51). ELM-ART has been used over many years by hundreds of students to support delivery of a university course. You can try out ELM-ART at http://art2.ph-freiburg.de/Lisp-Course
Knowledge Sea II is a mixed corpus C programming resource that bridges the gap between closed corpus materials in the form of lecture notes and open-corpus materials in the form of links to online resources for C programming. Knowledge Sea II helps users navigate from lectures to relevant online tutorials by providing links to online material related to search keywords. Search is adapted to the user by taking into account both the past interactions of the individual user and the user’s group (other learners). KnowledgeSea prompts learners to access material related to the user’s search by providing traffic and annotation based social navigation support. Social navigation support is realised by marking links to material with icons and colour codes that indicate the amount of traffic (time spent reading the linked material by other learners) and positive and negative individual and group annotations of the linked material (Brusilovsky, Farzan, & Ahn, 2006). Evaluations of KnowledgeSea II show that pages automatically predicted as important for a learner were actually rated as important by students and that the adaptive link annotations successfully influenced learner behaviour, with learners preferentially accessing more highly ranked pages and those with link annotations that indicate higher traffic (Brusilovsky, Farzan, & Ahn, 2006). For more about KnowledgeSea see http://www.sis.pitt.edu/~paws/system_knowledgesea2.htm. You can register to try the system at http://adapt2.sis.pitt.edu/cbum/.
AIED, Productivity, Personalisation, Inclusion and Flexibility of Learning
“In looking to TEL for improvements in productivity we need to look for ways of:
- Improving the quality of teaching in order to improve the quality of learner achievement against the learner’s time
- Increasing the number of learners achieving quality outcomes against teacher time
- Reducing the amount of teacher or learner time needed for learner achievement”
Personalisation improves the use of learner time by enabling learners to work at their own pace, receive targeted feedback, and be supported in their learning without relying on teacher presence (TLRP-TEL Programme, 2010). AIED systems can deliver such learner control but they may also employ models of learners and pedagogic strategies to seek to engage learners and push them when necessary. Enabling systems to deliver such personalisation is a key driver of much AIED and User Modelling research. Throughout the last decade various intelligent techniques that contribute to the personalisation of learning have been developed and empirically shown to be effective (Dimitrova, 2010); these include:
- modelling the learner’s cognitive states to provide individualised learning (VanLehn, 2006);
- using tutoring dialogues, even with shallow natural language processing, to deepen learning experiences (Litman, 2009);
- using open learner models to promote reflection and self-awareness (Bull & Kay, 2007; Dimitrova, McCalla & Bull, 2007; Mitrovic & Martin, 2007);
- adopting meta-cognitive scaffolding to increase learner motivation and engagement (du Boulay, Rebolledo Mendez, Luckin & Martinez Miron, 2007; Harris, Bonnett, Luckin, Yuill & Avramides, 2009).
Such AIED systems can now be deployed online and to personal and portable devices within and beyond formal educational settings and consequently can also contribute to both the flexibility of learning and to greater inclusion.
Flexibility is a way of improving the use of both learner and teacher time, face-to-face teaching can be replaced by online teaching, individual learning and group work (TLRP-TEL Programme, 2010). Many AIED systems are now web-based (see Table 1) and AIED researchers are exploring the use of mobile devices to deliver adaptive materials for more flexible anytime anywhere learning. Social and collaborative aspects of learning are increasingly important themes in AIED research and systems that monitor group work and provide effective intelligent support for collaboration, both at a distance and face-to-face, are being developed (e.g. Upton & Kay, 2009).
Inclusion is a way of increasing the number of learners attaining a particular level: attracting disaffected learners through more engaging forms of learning; providing additional help for learners with special needs; and motivating learners who cannot attend school (TLRP-TEL Programme, 2010). Motivation and Affect have been major themes in AIED research throughout the last decade (see Underwood & Luckin, 2011). There is substantial on-going AIED research into the use of games to deliver more engaging learning experiences (Johnson, 2010) and much AIED research employing novel user interfaces (e.g. natural language, speech and gesture recognition, eye-tracking and physiological and other sensors), which offer opportunities to engage learners with widely differing needs. Some systems (e.g. CognitiveTutors) have demonstrated strong results for disadvantaged populations (Sarkis, 2004). AIED researchers are also concerned to develop methods that enable learners, including children (Good & Robertson, 2006) and those with specific needs (Porayska-Pomsta, Bernardini & Rajendran, 2009), to participate in the design of systems that meet these users’ particular requirements. Other current and emerging themes in AIED research (e.g. adaptive support for inquiry learning, exploratory learning, lifelong learning and learning in ill-defined domains – see Underwood & Luckin, 2011) will also contribute to greater inclusion and flexibility of learning in the near future.
Grand Challenges for future AIED
AIED research is driven by the need to respond to educational challenges and to develop and exploit new technologies in pursuit of solutions to these challenges. Several Grand Challenges for future technology enhanced learning have been identified over the last decade. In the UK Grand Challenges for Computing include Learning for Life (Taylor et al, 2008). This challenge identifies requirements for future systems: to model learners and support dynamic evaluation and assessment, to develop facilities that support learning outside formal educational settings over a learner’s lifetime, to encourage and support creativity and problem solving and to ensure that learning for life is a viable option for all through enhanced accessibility and inclusion. In the US Grand Research Challenges in Information Systems identifies the need to “provide a teacher for every learner” asserting that in the future “individually tailored learner-centred tutoring will enable people to more fully realize their potential”(Computing Research Association, 2003, pp. 2). More recently the National Academy of Engineering (2010) identifies the need to “advance personalized learning” as a Grand Engineering Challenge recognising that new and emerging technologies offer the potential for instruction to be “individualized based on learning styles, speeds, and interests to make learning more reliable”. These are amongst the key challenges that AIED responds to.
Following on from the identification of such challenges a recent NSF funded report, GROE: A Roadmap for Educational Technology (Woolf, 2010), considers the role educational technology needs to play in advancing education over the next 30 years. The report draws on findings from several workshops, led by a large team of international experts. These workshops focussed on identifying promising technologies with which to address seven key educational challenges described as: “personalizing education, assessing student learning, supporting social learning, diminishing boundaries, developing alternative teaching strategies, enhancing the role of stakeholders, and addressing policy changes” (Woolf, 2010 pp.6). The report goes on to identify seven particularly promising areas of technology research and development that offer opportunities to satisfy the educational challenges identified: user modelling, mobile tools, networking tools, serious games, intelligent environments, educational data mining, and rich interfaces. User Modelling and Intelligent Environments are central to AIED research and the other technologies identified are major themes in AIED research over the last decade (see Underwood & Luckin, 2011). Visions for next generation technology-enhanced learning integrate expected advances in these areas of research to deliver solutions to the educational challenges identified earlier.
What will next generation AIED learning environments be like?
In this section we summarize visions of next generation learning environments, as described in the GROE report (Woolf, 2010), in order to highlight the expected role of AIED research.
Intelligent teaching and learning environments, incorporating sophisticated user models, will provide flexible and adaptive assistance, personalised to individual learners needs. Such, assistance will be domain independent and will include support for “soft skills, such as creativity, critical thinking, communication, collaboration, information literacy, and self-direction, and will be open-ended and exploratory in nature, allowing learners to question and enhance their understanding about areas of knowledge in which they are motivated to learn” (Woolf, 2010 p58). Personalised support and feedback will be available to learners across subjects and across formal and informal settings and throughout their lifetimes. Open user models will prompt learners to reflect on their learning and how they learn. Such systems, accessible ubiquitously through mobile and distributed rich interfaces, will improve flexibility and inclusion and help dissolve the boundaries between sites of learning and connect learning across subjects (Woolf, 2010).
Future intelligent teaching and learning environments may be self-learning, using machine-learning techniques to learn about students and improve their own performance by evaluating how they are used and associated with learning outcomes. However, such environments will not aim to replace teachers but rather work with teachers both being informed by teachers’ input and informing teachers’ decisions and actions. These environments will take into account the needs and interests of learners and will employ novel interfaces and techniques from games to deliver highly engaging and accessible learning experiences. Next generation learning environments will augment the real world with interactive representations that support learning through interaction with tangible physical world and mixed reality interfaces while making perceptible, phenomena that are too large, too small, too quick and too slow to observe in the real environment (Woolf, 2010). Simulations and micro-worlds will support learning through exploration and will provide appropriate and personalised feedback. These environments will enable learners to move seamlessly between real and virtual worlds and will span formal and informal activities and will better connect learning across these worlds (Woolf, 2010).
Intelligent teaching and learning environments will also use networking tools to better support social and collaborative learning, introducing suitable collaborators, guiding learners towards effective collaboration and helping teachers to monitor and support group work. Such networking tools “will facilitate individuals to learn within communities, communities to construct knowledge, and communities to learn from one another” (Woolf, 2010 p 51.). In these communities, learner roles will be more fluid, with teachers often acting as facilitators and opportunities for learners to participate as producers and teachers as appropriate to their knowledge and interests. Ubiquitous access to such communities will contribute to diminishing boundaries between formal and informal learning (Woolf, 2010).
Educational data-mining techniques will be used to discover patterns in the vast amounts of data that become available from such integrated intelligent learning environments and will support identification of success factors and problems (Woolf, 2010). These analyses will inform and guide stakeholders, including teachers, parents and policy makers. Teachers will be supported with easily accessed, more accurate and timely information and analysis of individual and group learning. This information will support teachers in making strategic decisions and providing appropriate guidance, and in the continuous assessment of learning. Appropriate information may also be shared with parents enabling them to provide additional help and motivation (Woolf, 2010). Educational data-mining, an emerging discipline concerned with developing methods to explore data from educational settings and better understand students and the settings they learn in, will reveal new challenges and opportunities for the AIED community. The rich data captured by these systems will provide researchers with new opportunities to evaluate models of learning and develop theory (Woolf, 2010).
AIED research offers some solutions to today’s educational challenges and has the potential to deliver more flexible and inclusive, personalised, effective and engaging learning experiences throughout lifetimes and across formal and informal settings. Over the last decade AIED research has made substantial progress in demonstrating this potential by moving out of labs into large-scale deployments that are having impacts in real-world settings as illustrated by the systems described in Table 1. The technological infrastructure required to deliver AIED learning experiences is increasingly ubiquitous. Current research in AIED aims to develop more flexible systems that will increase access to effective, personalised and engaging, anytime, anywhere learning throughout lifetimes across the full range of knowledge domains and skills and employing varied pedagogic approaches. Realising this potential will certainly involve overcoming technical obstacles, but mainstreaming AIED into Education will also require much more. It will require the successful communication of the value of AIED research and systems. In particular, the role that AIED systems can play within the broader educational settings of their use and with respect to the other resources available to learners, such as teachers, peers and the physical features of the environment, must be more clearly explained. Cumming and McDougal (2000) suggested one key reason AIED had not been taken seriously in Education ten years ago was the use of insufficiently rich models of learning. In the intervening decade one of the main focuses of AIED has been to develop much richer models of learning, learners, teachers and, to a lesser extent learners’ contexts. However, AIED research has typically been published in specialist journals and conference proceedings, which have not been sufficiently visible beyond the community and are only now becoming more visible in educational research resources (e.g. ERIC; Education Resources Information Center – http://www.eric.ed.gov. ERIC is progressively indexing the International Journal of Artificial Intelligence in Education and at 31/01/2011 listed a total of 11 records, this compares to 1120 records from Computers & Education on the same date). The AIED community needs to better explain to Education the nature of the models used and the value of these and the systems developed using them.
Even within AIED it is difficult and time consuming to keep track of, integrate and synthesise work happening in all the specialist areas, let alone combine components into working systems. The AIED and ITS conferences do provide “opportunities for the cross-fertilization of approaches, techniques and ideas from the many areas that make up this interdisciplinary research field, including: agent technologies, artificial intelligence, computer science, cognitive and learning sciences, education, educational technologies, game design, psychology, philosophy, sociology, anthropology, linguistics, and the many domain-specific areas for which AIED systems have been designed, deployed and evaluated” (AIED Conference, 2011). However, this increasing multi-disciplinarity is both a great strength of AIED and a massive challenge. Communication across disciplines is notoriously difficult and integration of approaches and conceptual frameworks even more so (Conole et al, 2010). AIED needs to communicate successfully both within the field and beyond, particularly with mainstream Education. The need to support research that “harnesses and integrates knowledge across multiple disciplines to create a common groundwork of conceptualization, experimentation and explanation that anchor new lines of thinking and inquiry towards a deeper understanding of learning” (United States National Science Foundation – Science of Learning Centers) is recognised in the NSF Science of Learning Centers program, with The Pittsburgh Science of Learning Center (PSLC) aiming to enhance scientific understanding of robust learning in real educational settings by bringing together basic and applied research and supporting field-based experimentation with AIED systems. New technologies offer opportunities to support such communication and interdisciplinary research (e.g. see PSLC Robust Learning Theoretical Framework Wiki and work in the TLRP Technology Enhanced Research theme) The need for such resources is recognised within the AIED community, Woolf (2009) suggests “we need: cadres of bibliographies, suites of project inventories, component exchange communities and global networks of test beds for intelligent learning environments”. These resources would be helpful both within the discipline and beyond, facilitating access to specialist AIED research and easier, faster development of learning environments that incorporate intelligent components. However, substantial work is required to specify requirements for such resources and this in itself will require collaboration between AIED researchers, the wider Education community and other stakeholders.
Many thanks to our Advisory Committee: Vania Dimitrova, Judy Kay, Paul Brna, Kaska Porayska-Pomsta, Chee Kit Looi, for their invaluable input to this report.
Baker, M. (2000). The roles of models in Artificial Intelligence and Education research: a prospective view. International Journal of Artificial Intelligence in Education, 11, pp. 122-143.
Barrow, D., Mitrovic, A., Ohlsson, S., Grimley, M. (2008). Assessing the impact of positive feedback in constraint-based ITSs. B. Woolf et al. (eds) Proceedings of the 9th Int. Conf. ITS 2008, LCNS 5091, Springer-Verlag, pp. 250-259.
Beal, C. R., Walles, R., Arroyo, I., and Woolf, B. P. (2007). On-line tutoring for math achievement testing: A controlled evaluation. Journal of Interactive Online Learning, 6 (1), pp. 43-55.
Bull, S. & Kay, J. (2007). Student Models that Invite the Learner In: The SMILI Open Learner Modelling Framework, International Journal of Artificial Intelligence in Education, 17(2), pp. 89-120.
Brusilovsky, P., Farzan, R., and Ahn, J-w. (2006). Layered Evaluation of Adaptive Search. In: R. W. White, G. Muresan and G. Marchionini (eds.) Proceedings of Workshop on Evaluating Exploratory Search Systems at SIGIR 2006, Seattle, USA, August 10, 2006, pp. 11-13. Available online at http://www.sis.pitt.edu/~peterb/papers/EESS2006_CRC.pdf
Brusilovsky, P., Sosnovsky, S., and Yudelson, M. (2006) Addictive links: The motivational value of adaptive link annotation in educational hypermedia. In: V. Wade, H. Ashman and B. Smyth (eds.) Proceedings of 4th International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH’2006), Dublin, Ireland, June 21-23, 2006, Springer Verlag, pp. 51-60.
Carnegie Learning, Applying Cognitive Science to Education. Available at http://www.carnegielearning.com/research/whitepapers/
Carnegie Learning, Successful Application of Cognitive Science. Available at http://www.carnegielearning.com/research/whitepapers/
Carnegie Learning (2010). Cognitive Tutor Effectiveness – Executive Summary. Available at http://www.carnegielearning.com/static/web_docs/2010_Cognitive_Tutor_Effectiveness.pdf
Computing Research Association (2003). Grand Research Challenges in Information Systems Available at http://www.cra.org/uploads/documents/resources/rissues/gc.systems_.pdf
Conole, G., Scanlon, E., Mundin, P., and Farrow, R. (2010). Interdisciplinary research – Findings from the Technology Enhanced Learning Research Programme. TLRP, UK. Available 19/01/2010 at http://www.tlrp.org/docs/TELInterdisciplinarity.pdf
Cumming, G., and McDougal, A. (2000). Mainstreaming AIED into Education? International Journal of Artificial Intelligence in Education, 11, pp. 197-207.
Dimitrova, V (2010). ImREAL project: http://www.imreal-project.eu
Dimitrova, V., McCalla, G., and Bull, S. (2007). Open Learner Models: Future Research Directions – Special Issue of the IJAIED (Part 2). International Journal of Artificial Intelligence in Education, 17, 3, pp. 217-226.
du Boulay, B., Rebolledo-Mendez, G., Luckin, R., Martinez-Miron, E.A., and Harris, A. (2007). Motivationally Intelligent Systems: Diagnosis and feedback. In R. Luckin, K. R. Koedinger & J. Greer (Eds.), AIED 2007, IOS Press. Frontiers in Artificial Intelligence and Applications, 158, pp. 563-565.
Good, J., & Robertson, J. (2006). CARSS: A Framework for Learner-Centred Design with Children. International Journal of Artificial Intelligence in Education, 16 (4) pp. 381-413.
Harris, A., Bonnett, V., Luckin, R., Yuill, N., Avramides, K. (2009). Scaffolding effective help-seeking behaviour in mastery and performance oriented learners. In V. Dimitrova, R. Mizoguchi, B. du Boulay, A. C. Graesser (Eds.), AIED 2009, IOS Press. Frontiers in Artificial Intelligence and Applications, 200, pp. 425-432.
International AIED Society (2010). About the society. At http://iaied.org/about/ checked 31/01/11
Johnson, W.L. (2010). Serious Use of a Serious Game for Language Learning. International Journal of Artificial Intelligence in Education, 20 (2) pp. 175-195.
Johnson, W.L., Valente. A. (2009) Tactical Language and Culture Training Systems: Using AI to Teach Foreign Languages and Cultures. AI Magazine, 30 (2) pp. 72-83. Available online at http://www.aaai.org/ojs/index.php/aimagazine/article/view/2240
Koedinger, K. R., Anderson, J. R., Hadley, W. H., and Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30‐43.
Litman, D. (2009). The Evolution of Natural Language Processing in AIED: Successes and Challenges. Panel on the Evolution of AIED @ AIED09, July, 2009, Brighton, UK
Luckin, R. (2010). Re-designing Learning Contexts: Technology-rich, Learner-centred Ecologies. Routledge, London.
Martin, B., Mitrovic, A. The effect of adapting feedback generality in ITS. In V. Wade, H. Ashman, and B. Smyth (Eds.): AH 2006, LNCS 4018, pp. 192-202, 2006.
Melis, E., & Siekmann, J.H. (2004). ActiveMath: An Intelligent Tutoring System for Mathematics. In L. Rutkowski, J.H. Siekmann, R. Tadeusiewicz, L.A. Zadeh (Eds.): Artificial Intelligence and Soft Computing – ICAISC 2004, 7th International Conference, Zakopane, Poland, June 7-11, 2004, Proceedings. LNCS 3070, pp. 91-101, Springer.
Mitrovic, A. & Martin, B. (2007). Evaluating the Effect of Open Student Models on Self-Assessment. International Journal of Artificial Intelligence in Education, 17(2), pp.121-144.
Mitrovic, A. & the ICTG team, (2006). Large-Scale Deployment of three intelligent web-based database tutors. Journal of Computing and Information Technology, 14 (4), pp. 275-281.
Mitrovic, A., Martin, B. Suraweera, P., Zakharov, K., Milik, N., Holland, J., McGuigan, N. (2009) ASPIRE: An Authoring System and Deployment Environment for Constraint-Based Tutors. International Journal of Artificial Intelligence in Education, 19,(2), pp. 155-188.
Morgan, P., & Ritter, S. (2002). An experimental study of the effects of Cognitive Tutor Algebra I on student knowledge and attitude. Available at http://www.carnegielearning.com/web_docs/morgan_ritter_2002.pdf
National Academy of Engineering (2010). Grand Challenges for Engineering – Advance Personalise Learning. Available at http://www.engineeringchallenges.org/cms/8996/9127.aspx
Porayska-Pomsta, K. Bernardini S., Rajendran, G. (2009) Embodiment as a means for Scaffolding Young Children’s Social Skill Acquisition. In Proceedings of the Workshop on Children and Embodied Interaction: Seeking Common Ground, 8th International Conference on Interaction Design and Children (IDC’09), Como, Italy, 2009
Russell, S., & Norvig, P. (1995). Artificial Intelligence: A Modern Approach. Prentice-Hall, Saddle River, NJ.
Sarkis, H. (2004). Cognitive Tutor Algebra 1 Program Evaluation: Miami‐Dade County Public Schools. Lighthouse Point, FL: The Reliability Group.
Self, J. (1999). The defining characteristics of intelligent tutoring systems research: ITSs care, precisely. International Journal of Artificial Intelligence in Education, 10, pp. 350-364.
Taylor, J., Greenwood, Wood, W., Rae, J., Rico, M. (2008) A Grand Challenge for Computing: Learning for Life. Available at http://kn.open.ac.uk/public/workspace.cfm?wpid=6042
TLRP-TEL Programme (2010). Productivity: Achieving higher quality and more effective learning in affordable and acceptable ways. Available at http://www.tel.ac.uk/productivity/productivity-achieving-higher-quality-and-more-effective-learning-in-affordable-and-acceptable-ways/
Ullrich, C., & Melis, E. (2010). Complex Course Generation Adapted to Pedagogical Scenarios and its Evaluation. Educational Technology & Society, 13 (2), pp. 102–115.
Underwood, & Luckin, R., (2011). Themes and trends in AIED research, 2000 to 2010. A report for the TLRP. Available from http://www.tel.ac.uk/personalisation/artificial-intelligence-in-education/themes-and-trends-in-aied-research-2000-to-2010/
Upton, K., & Kay, J. (2009). Narcissus: Group and Individual Models to Support Small Group Work. User Modeling, Adaption, and Personalisation. LNCS, Vol. 5535/2009, pp. 54-65.
VanLehn, K. (2006). The Behavior of Tutoring Systems. International Journal of Artificial Intelligence in Education, 16(3), pp. 227-265.
VanLehn, K., Lynch, C., Schultz, K., Shapiro, J. A., Shelby, R. H., Taylor, L., Treacy, D., Weinstein, A., and Wintersgill, M. (2005). The Andes physics tutoring system: Lessons learned. International Journal of Artificial Intelligence and Education, 15(3), pp. 147-204.
Weber, G., & Brusilovsky, P. (2001) ELM-ART: An Adaptive Versatile System for Web-based Instruction. International Journal of Artificial Intelligence in Education, 12, pp. 351-384.
Woolf, B. (2010). A Roadmap for Education Technology. GROE Available online at http://www.cra.org/ccc/docs/groe/GROE%20Roadmap%20for%20Education%20Technology%20Final%20Report.pdf checked 31/01/11
Woolf, B. (2009). AIED Grand Challenges. Panel on the Evolution of AIED @ AIED09, July, 2009, Brighton, UK