Khadijah Jilani (Kashef Janjua)
Discipline: Graduate School of Education
Project Summary
1. Master’s Dissertation (MA in Language Testing, Lancaster University, awarded 2013)
Here Is the Picture. Describe Everything You Can See: A Comparison of Intended and Observed Production of Language Functions in Part Three of the English Test for Aviation
This dissertation looked at Part 3 of the English Test for Aviation (ETA). The ETA, which was jointly managed by Oxford Aviation Academy and Emery-Roberts, had been designed to assess the English proficiency level of pilots. Part 3 of the ETA took the form of an oral proficiency interview. This study investigated to what extent there was a match between the language functions produced by test takers and the language functions that the test designers intended the prompts to elicit. It found that, in the cases of test takers who achieved a minimum Operational Level proficiency (as defined by the International Civil Aviation Organisation), 87.6 % of the desired language functions were produced. It also looked at the cases in which language functions were not produced successfully and suggested some reasons why this may have occurred.
This study also investigated whether there was a relationship between production of intended language functions in Part 3 of the ETA, and the overall proficiency level awarded to candidates. It concluded that there was a strong to very strong positive correlation between these two variables.
Finally, it considered whether the production of any of the elicited functions might be a criterial feature of Operational Level proficiency. It concluded that further research using larger sample sizes was needed before any definitive claims could be made in this area.
During the course of this study, the observation checklist was shown to be a reliable instrument for use in validation studies of Part 3 of the ETA.
2. Doctorate Pre-thesis Study - Perspectives on Professionalism (EdD (TESOL), University of Exeter)
An Investigation into Whether Changes Made to the Classroom Observation Procedure at a University Foundation Year Programme Had Reduced the Stress Experienced by EFL Teachers
This small-scale empirical enquiry explored whether changes made to the classroom observation procedure at a university foundation year programme had reduced the stress experienced by the EFL teachers who worked there. The findings of the focus group discussion with six teachers showed that the changes made (in the form of separating evaluative observations from professional development observations and emphasizing the mentoring and coaching functions of the latter) had the desired effect of reducing the feelings of stress perceived by the teachers. Teachers also felt that observations conducted for professional development purposes rather than for purely evaluative purposes were less stressful and more beneficial as a tool for personal improvement and learning.
The focus group also investigated what the various sources of stress were for teachers undergoing different types of classroom observations. The findings illustrated that teachers felt stressed if they thought that the observation was high-stakes, that is, if they thought it would affect their prospects of contract renewal.
The findings of this study also revealed that teachers felt stressed if they were asked to produce a detailed lesson plan as part of the observation process because it represented a script that had to be memorized and they were stressed by the artificiality that produced.
All teachers who participated in the focus group thought that the personality of the observer and the observee affected the degree of stress experienced. They also agreed that having two conferring observers present during a high-stakes observation was reassuring.
3. Doctorate Pre-thesis Study – TESOL Classrooms and Pedagogy (EdD (TESOL), University of Exeter)
Students’ and Teachers’ Perceptions of an ‘Assessment by Learning Event’ in a Foundation Year English Programme in Riyadh
This report details a small-scale study I carried out in the English Department of a university foundation year programme in Riyadh. The study explored students’ and teachers’ perceptions of an assessment tool that I designed three years ago while I was working at this particular institution and which is still being used, as originally designed.
The assessment tool, known as an Assessment by Learning Event (ABLE), relies on a mixture of summative and formative assessment techniques. Its objective is to provide evidence of student learning in a low-stakes environment while simultaneously fostering the reinforcement of previous learning and the development of new skills.
The study showed perceptions of teachers and students towards the Assessment by Learning Event to be favourable. Students expressed their enjoyment of taking part in it and their preference of it as a form of assessment over an examination. Both teachers and students thought it was an effective pedagogical tool and a fair assessment instrument.
The study highlighted areas that need further investigation such as the relationship between fair mark allocation, fear of losing marks and the facilitation of deep learning. Based on the results of this study, the ABLE appears to have succeeded in two of these areas, but let the students down in the third, namely the fear of losing marks. It is a dilemma that needs to be tackled if students are to be offered a better alternative to the high-stakes, anxiety-laden testing of our times.
