A considerable number of studies have been conducted concerning academic writing modes; however, only few attempts have so far been made at assessment of other writing modes such as narrative and descriptive essays (NDE). The purpose of this paper is to investigate the reliability of NDE writing measures, using the multi-faceted Rasch measurement model.

Each of the 60 NDE essays collected from 30 high school learners was scored by four raters, and the essays were evaluated using a rating scale consisting of eight components (Overall Impression, Content, Organization, Cohesion & Coherence, Creativity & Originality, Grammar, Vocabulary, Mechanics). The analyses were conducted using FACETS in order to examine whether NDE writing can be measured in unidimensional scale and examine the extent to which rater’s severity is consistent.

The results indicated that NDE measurement using the prepared scale is reliable and inter-rater correlations among the raters were relatively high. However, there were significant differences among rater’s severity and rater-component interactions. It was, thereby, shown that some raters scored some components harshly or leniently. The paper suggests that modifications of wording in the scale descriptors and increases in awareness of rater role would be key elements in a more reliable NDE assessment.