Research paper

ICER 2026

Steering AI Tutors Through System Prompts:
A Crossover Study on Self-Regulated Learning
and Cognitive Engagement Scaffolds in CS1

12 August 2026, Uppsala

Maximilian Barth*, Sverrir Thorgeirsson*, Khashayar Etemadi, Juho Leinonen, Carlos Cotrini, and Zhendong Su

ETH Zurich
Steering AI Tutors Through System Prompts
12.08.26
Context
ICER 2026
1/16

Code Expert

Code Expert programming homework interface
  • Used across many ETH courses
  • Programming homework
  • Bonus points for exams
  • Require considerable effort
ETH Zurich
The starting point
12.08.26
Problem
ICER 2026
2/16

Helpful is not always pedagogically helpful

Student asking an AI assistant for help with an exercise
AI assistant returning a complete solution
Code submission passing all tests
ETH Zurich
The pedagogical tension
12.08.26
Motivation
ICER 2026
3/16

Why create an AI tutor for students?

  • On-demand support at scale
  • Always available
  • Lower barriers to seeking help
  • Support without simply giving answers
  • Access without expensive subscriptions
OpenAI logo
Anthropic logo
Gemini logo
ETH Zurich
Purpose-built tutoring
12.08.26
Conditions
ICER 2026
4/16

Can established learning theory fit inside a system prompt?

1.Baseline

  • Guide without giving away the answer
  • Brief and professional
  • Use only small code examples

2.Zimmerman's cyclical self-regulated learning

Forethought
Performance
Self-reflection

3.Chi's ICAP cognitive engagement framework

InteractiveConstructiveActivePassive
ETH Zurich
Three tutor conditions
12.08.26
Interface
ICER 2026
5/16

Tutors in a familiar environment

Code Expert interface
Ente tutor interface beside a programming exercise
Code ExpertEnte
ETH Zurich
Embedded in coursework
12.08.26
Live walkthrough
ICER 2026
6/16

Demo

Yellow duck mascot
ETH Zurich
Demo
12.08.26
System
ICER 2026
7/16

About the system

Requirements:

  • Hosted at ETH
  • Swiss inference provider
  • Zero data retention

Qwen3235B-A22B-Instruct-2507

Ente yellow duck mascot
ETH Zurich
Private by design
12.08.26
Prompt design
ICER 2026
8/16

The prompts

Baseline tutoring instructions
Optional pedagogical strategy
Static context
Chat history
Dynamic context
User question
System prompt
Context
ETH Zurich
Prompt assembly
12.08.26
Coding
ICER 2026
9/16

Qualitative coding

2blinded coders
1,983messages coded
80.7%agreement
ICAP distribution
AgentStudentsMessagesPassiveActiveConstructive
Baseline4958668.8%23.5%7.7%
Chi4879066.8%19.7%13.5%
Zimmerman4860760.0%25.8%14.3%
ETH Zurich
Behavioral evidence
12.08.26
Study design
ICER 2026
10/16

A six-week crossover study

  • CS1 for engineers at ETH in C++
  • 1,500+ students
  • Recursion, DSA, OOP, memory management
  • Authentic coursework
W1Ex 1Ex 2
W2Ex 3Ex 4
W3Ex 5Ex 6
W4Ex 7Ex 8
W5Ex 9Ex 10
W6Ex 11Ex 12
ETH Zurich
Authentic coursework
12.08.26
Study scale
ICER 2026
11/16

Some stats

Weekly active users in the tutor conditions and university platform
87%consent rate
>49,175messages
>2,552 htime on task
>26,989code runs
ETH Zurich
Study scale
12.08.26
Measures
ICER 2026
12/16

Preregistration

Tutor response preference

  • Thumbs-up / thumbs-down reactions
  • Like ratio

Conceptual learning

  • Exercise-specific conceptual MCQ
  • Quiz success

Usability

  • UMUX-Lite: 2 items, 7-point scale

Cognitive load

  • Paas mental effort: 1 item, 9-point scale
Survey
ETH Zurich
Preregistered outcomes
12.08.26
Primary result
ICER 2026
13/16

No preregistered outcome differed significantly

Grouped bars with 95% confidence intervals comparing Baseline, Zimmerman, and Chi across Like Ratio, Quiz Success, Usability, and Cognitive Load
Four outcome measures by tutor condition with confidence intervals
Mixed-effects model results for preregistered outcomes
ETH Zurich
Preregistered outcomes
12.08.26
Exploratory result
ICER 2026
14/16

Theory-guided tutors elicited more engagement

ETH Zurich
Behavior changed
12.08.26
Takeaways
ICER 2026
15/16

Summary

Platform

  • Built for existing courses and large-scale studies
  • Battle-tested in a CS1 course at ETH
ETH Zurich
Implications
12.08.26
Discussion
ICER 2026
16/16

Questions?

Maximilian Barth