Intelligent Tutoring Technologies for Ill-Defined Problems and Ill-Defined Domains

(Deadline Extended April 20th 2010)


At the 10th International Conference on Intelligent Tutoring Systems (ITS 2010) in Pittsburgh.


Intelligent tutoring systems, and intelligent learning environments support learning in a variety of domains from basic math and physics to legal argument, and hypothesis generation. These latter domains are ill-defined referring to a broad range of cognitively complex skills and requiring solvers to structure or recharacterize them in order to solve problems or address open questions. Ill-defined domains are very challenging and have been relatively unexplored in the intelligent learning community. They usually require novel learning environments which use non-didactic methods, such as Socratic instruction, peer-supported exploration, simulation and/or exploratory learning methods, or informal learning techniques in collaborative settings

Ill-defined domains such as negotiation, intercultural competence, and argument are increasingly important in educational settings. As a result, interest in ill-defined domains has grown in recent years with many researchers seeking to develop systems that support both structured problem solving and open-ended recharacterization. Ill-defined problems and ill-defined domains however pose a number of challenges. These include:

  • Defining viable computational models for open-ended exploration coupled intertwined with appropriate meta-cognitive scaffolding;

  • Developing systems that may assess and respond to fully novel solutions relying on unanticipated background knowledge;

  • Constraining students to productive behavior in otherwise underspecified domains;

  • Effective provision of feedback when the problem-solving model is not definitive and the task at hand is ill-defined;

  • Structuring of learning experiences in the absence of a clear problem, strategy, and answer;

  • Developing user models that accommodate the uncertainty, dynamicity, and multiple perspectives of ill-defined domains;

  • Designing interfaces that can guide learners to productive interactions without artificially constraining their work.

These challenges must be faced in order to develop effective tutoring systems in these attractive, open, and important arenas. A stimulating series of workshops has been held at ITS 2006, AIED 2007, and ITS 2008. Each workshop brought together a range of researchers focusing on domains as diverse as database design and diagnostic imaging. The work they presented ranged from nascent system designs to robust systems with a solid user base. While the domains and problems addressed differed from system to system, many of the techniques were shared allowing for fruitful cross-pollination.

Due to the success of those workshops and the growing interest in extending intelligent tutoring systems and learning environments to address ill-defined domains we feel a workshop at ITS 2010 is warranted. This event will allow researchers from prior workshops to share their lessons learned while allowing new developers to explore the this dynamic area.

Call for Papers:

We invite work at all stages of development, including particularly innovative approaches in their early phases. Full research papers (up to 8 pages) and demonstrations (up to 4 pages, describing an application or other work to be demonstrated live at the workshop) are welcome for submission.

Paper topics may include but are not limited to:

  • Model Development: Production of formal or informal models of ill-defined domains, constraints or characteristics of such domains or important subdomains.

  • Teaching Strategies: Development of teaching strategies for ill-defined problems and ill-defined domains, for example, Socratic, peer-guided, or exploratory strategies.

  • Metacognition and Skill-Transfer: Identification of essential skills for ill-defined problems and domains and the transfer of skills across domains and problems.

  • Assessment: Development of student and tutor assessment strategies for ill-defined domains. These may include, for example, qualitative assessments and peer-review.

  • Feedback: Identification of feedback and guidance strategies for ill-defined domains. These may include, for example, Socratic (question-based) methods or related-problem transfer.

  • Exploratory Systems: Development of intelligent tutoring systems for open-ended domains. These may include, for example, user-driven “exploration models”, simulations, and constructivist approaches.

  • Representation: Free form text is often the most appropriate representation for problems and answers in ill-defined domains; ITSs in these areas need to accommodate and yet guide this free description.

The topics can be approached from different perspectives: theoretical, systems engineering, application oriented, case study, system evaluation, etc.

Submission Instructions

  • 8:30-9:45

    • Intro (15min)
      1. Collin Lynch: Welcome to IllDef2010

    • 2 papers (30 ea 20/10)
      1. Kevin D. Ashley: "Borderline Cases of Ill-definedness and How Different Definitions Deal with Them"

      2. Paula Durlach: "The First Report is Always Wrong, and Other Ill-Defined Aspects of the Army Battle Captain Domain"

  • 10:15-12:30

    • 3 papers (30 ea 20/10)
      1. Nancy Green: "Towards Intelligent Learning Environments for Scientific Argumentation"

      2. Lydia Lau: "What is the Real Problem: Using Corpus Data to Tailor a Community Environment for Dissertation Writing"

      3. Manolis Mavrikis: "Layered Learner Modelling in ill-defined domains: conceptual model and architecture in MiGen."

    • Impulse Statement (15 min.)

        Prof. David Herring: University of Pittsburgh's School of Law Aspects of instruction in Legal Reading and Writing.

    • Breakout Sessions. (30 min) Small group discussions (may continue over lunch) with each group developing a set of points for post-lunch reporting. Topics covered:
      1. The role of assessment metrics in ill-defined domains.
      2. Possible avenues for assessment methods.
      3. What technologies they would like to develop.
  • 2pm-3:45

    • 3 papers (30 ea 20/10)
      1. Art Graesser: "Using a Quantitative Model of Participation in a Community of Practice to Direct Automated Mentoring in an Ill-Formed Domain"

      2. Matthew Hays: "The Evolution of Assessment: Learning about Culture from a Serious Game"

      3. Art Graesser: "Comments of Journalism Mentors on News Stories: Classification and Epistemic Status of Mentor Contributions"

  • 4:15-6pm

    • Short Paper (15 min 10/5)
      1. Richard Gluga: "Is Five Enough? Modeling Learning Progression in Ill-Defined Domains at Tertiary Level"

    • Report Session: (45 min)

      Each breakout group presents its results in a 5 minute statement (based on a poster prepared in the morning session)

    • Closing Panel: (60 min)

      Each member of the organizing committee + additional invitees presents for no more than 10 minutes each addressing two points:

      1. Something they especially liked in the workshop presentations and wanted to see more of in the future (and why)
      2. Something they were surprised to see missing in the presented research (at this workshop, or even more broadly in the research on ITSs for ill-defined domains so far) and think should be addressed in future research (and why)

      Panelists: Vincent Aleven, Amy Ogan, Sergio Gutierrez and Hameedullah Kazi

Workshop Organizers

  • Collin Lynch University of Pittsburgh, United States.
  • Dr. Kevin Ashley University of Pittsburgh, United States.
  • Prof Tanja Mitrovic University of Canterbury, New Zealand.
  • Dr. Vania Dimitrova University of Leeds, United Kingdom.
  • Dr. Niels Pinkwart Technische Universität Clausthal, Clausthal-Zellerfeld, Germany.
  • Dr. Vincent Aleven Carnegie Mellon University, United States.

Program Committee

  • Vincent Aleven, Carnegie Mellon University, United States.
  • Kevin D. Ashley, University of Pittsburgh, United States.
  • Vania Dimitrova, University of Leeds United Kingdom.
  • Declan Dagger, Trinity College Dublin, Ireland.
  • Paula Durlach, Army Research Institute, United States.
  • Matthew Easterday, Carnegie Mellon University United States.
  • Nikos Karacapilidis, University of Patras, Greece.
  • Lydia Lau, University of Leeds, United Kingdom.
  • Collin Lynch, University of Pittsburgh, United States.
  • George Magoulas, London Knowledge Lab, United Kingdom.
  • Moffat Mathews, University of Canterbury, New Zealand.
  • Tanja Mitrovic, University of Canterbury New Zealand.
  • Amy Ogan, Carnegie Mellon University, United States.
  • Niels Pinkwart, Technische Universität Clausthal, Clausthal-Zellerfeld, Germany.
  • Amali Weerasinghe, University of Canterbury, New Zealand.

Register for the 21st International Conference on Intelligent Tutoring Systems.