This position is filled.

We have an opening for a PhD student to work on automatic grading and tutoring in online courses. In particular, we are interested in automatic feedback and assessment techniques that we can apply in programming education. The results of the research will be applied and evaluated in several courses at TU Delft and other universities.

We are looking for a candidate with a Master’s degree (or equivalent) in computer science or a related discipline, a passion for online education, and a broad interest including systems programming, compilers, language engineering, programming environments, web programming, machine learning, interaction design, education research, and teaching.

Applications will be accepted until the position is filled.

PositionPhD Student
AreaOnline Education
EmployerTU Delft
Contact PersonEelco Visser
PostedFebruary 1, 2015

Note: We also accept applications for postdoc positions in this area.

TU Delft

Delft University of Technology is the leading University of Technology in the Netherlands with a global reputation in the fields of science, engineering & design. TU Delft’s mission is to make a significant contribution towards a sustainable society for the twenty-first century by conducting ground-breaking scientific and technological research which is acknowledged as world-class, by training scientists and engineers with a genuine commitment to society and by helping to translate knowledge into technological innovations and activity with both economic and social value.

Open & Online Education

TU Delft has been a forerunner in the field of Open & Online Education since 2007, and is one of the sustaining members of the OpenCourseWare Consortium and since 2013 charter member of the EdX Consortium. In March 2014, the board of the TU Delft approved a 2-year innovation program to accelerate the development of open & online education. The goal is to create a Delft Extension School that will bundle all open & online courses for a global population of life long learners.

Research in Open & Online Education

In connection with TU Delft's goal to be a top provider of open & online education, it has also the ambition to be known for its top research in the area of open & online education. With (massive) online education becoming more mainstream it is essential to investigate the quality, effectiveness and interplay of such offerings to support teachers and course participants. Important questions to answer include: Who benefits from a MOOC or online course? How much do the course participants learn? Which strategies improve the learning experience of the participants? Are MOOCs less or more effective than a traditional classroom-based course? How can MOOCs be integrated into campus-based learning? How can traditional assessment and feedback approaches be automated and translated to a massive scale? Given TU Delft's focus on engineering, these questions are specifically relevant in the context of learning and teaching in engineering-related topics. To answer these questions, TU Delft has strategically chosen to extend its research activities in the area of open & online education.

Job description & requirements

The research in this position will focus on methods and techniques for the design and implementation of automatic assessment of and feedback on assignments in online courses. An assessment should reflect a student’s understanding of the course material, feedback should provide the student with detailed explanations of an assessment, and tutoring should help a student improve that understanding. This requires a thorough analysis of the student’s work. In traditional education environments, this analysis is carried out directly by instructors who can apply the amazing pattern matching and cognitive faculties of the human brain. However, this approach does not scale to the numbers of students of MOOCs, and is already challenged by large regular courses at universities. Automating assessment, feedback, and tutoring requires that we formalize the ‘assessment function’ that is applied by instructors.

The objective of this research is to develop a framework that allows instructors to design assignments together with a function that analyzes the answer given by students to produce a grade with feedback to explain the grade, and hints for further study. We think such a framework can be especially successful for courses in which the answers to assignments can be formally described, including programming, mathematics, and many engineering courses. A particular challenge is to analyze properties of assignments beyond correctness; how can ‘good design’ be formalized? In order for instructors to be effective, the framework should provide high-level abstractions for constructing assessment functions for particular classes of assignments. Furthermore, assessment functions should be scalable, such that they can be applied to massive numbers of assignments in real time in order to provide rapid feedback to students. The framework should be safe and secure, in particular in the face of the execution of student generated code, and guarantee that student data is not corrupted or stolen. We will investigate advanced assessment functions that do not just grade assignments, but also tutor students using interactive instructions about problem solving strategies. The research will consist of a combination of conceptual design of learning environments and hands-on engineering to construct a working online learning environment and test it in practice in real courses. We will work together with the instructors of these courses to understand their approach to assessment and design techniques for formalizing these.

As a researcher you will become part of the Open & Online Education team and you will be hosted, supported, and supervised by the Software Engineering Research Group. In particular, you will work in the context of the Software Language Design and Engineering group of Prof. Eelco Visser, who has constructed the WebLab online learning environment, which will provide the basis for the experiments in this project.

For the position of researcher Learning Automatic Assessment & Feedback we are searching for a researcher with:

  • a solid background in computer science or a related discipline,
  • a passion for online education, and a broad interest including systems programming, compilers, language engineering, programming environments, web programming, interaction design, education research, and teaching,
  • demonstrable experience in several of these topics,
  • proficient in spoken and written English,
  • the ability to collaborate in inter-disciplinary research, and
  • the academic skills to be part of a research team aiming for high impact.

Conditions of employment

The successful candidate will be granted a temporary employment contract with the university for a period of 4 years (48 months), with an evaluation at the end of the first year. Within the contract period, the candidate is expected to produce a dissertation (PhD thesis) of sufficient quality for him/her to be awarded a PhD.

TU Delft offers an attractive benefits package, including a flexible work week, and the option of assembling a customized compensation and benefits package (the 'IKA'). Salary and benefits are in accordance with the Collective Labour Agreement for Dutch Universities. Salary grows from 2125 EUR gross in the first year to 2717 EUR gross in the fourth year (2015).

As a PhD candidate you will be enrolled in the TU Delft Graduate School. TU Delft Graduate School provides an inspiring research environment; an excellent team of supervisors, academic staff and a mentor; and a Doctoral Education Programme aimed at developing your transferable, discipline-related and research skills. Please visit for more information.

Information and application

For more information about this position, please contact Eelco Visser (phone: +31 15 27 87088, e-mail: To apply, fill in the form.

Applications will be accepted until the position is filled.