Writing Proficiency Initiative: A Brief History

 

In 1995, the Montana Board of Regents approved a policy requiring that the University System "adopt a uniform assessment tool to be used in determining if students or prospective students have the basic proficiencies in math and English to provide them a reasonable chance of success in postsecondary education." In February of 1998, Deputy Commissioner Stuart Knapp convened a small group of college and high school composition instructors to explore transitions for students from high school English classes to college freshman composition.

That work led to the formation of a Composition Standards Committee, appointed jointly by Superintendent Nancy Keenan and Commissioner Dick Crofts, which first met in December 1998. During its first year, the committee studied new K-12 Writing Content Standards, high school transcripts, freshman composition syllabi, and commercially available tests. To meet the Regents' 1995 charge, Commissioner Crofts and Deputy Commissioner Joyce Scott envisioned "a statewide authentic assessment of high school seniors to be scheduled at regular intervals at our institutions of higher education." In 1999, a subcommittee reported on other states writing assessments and the possibility of a state writing assessment in which high schools would team with campuses.

In 1999-2000, OPI published a leaflet entitled "Looking Ahead to College Composition;" a subcommittee evaluated sets of high school and college freshman essays to discuss alignment issues; and plans were made to initiate a field test, since none of the readily-available tests met criteria set by the committee. ACT was selected to assist because their rubric and prompts could be adapted to fit Montana content standards and practices and allowed for design work and scoring in Montana.

Committee Chair Beverly Ann Chin conveyed the committee's final report to the Board of Regents in May of 2000 and in July the Board approved the recommendation to initiate a three-year field test. Since then, over 11,000 tests have been administered and scored by 312 professors and teachers from 117 high schools. In 2004, another 6,000 essays will be written and scored by approximately 120 teachers from 93 schools. In addition, training of trainers will give about 50 teachers the skills to train scorers and conduct scoring sessions in their own regions, building capacity and ownership at the district level.

In 2000, the committee and ACT envisioned a 2004 test designed in Montana and administered by ACT, with students paying directly for the test and ACT reporting scores. However, in the summer of 2002, both SAT and ACT announced that they would be adding writing samples to their national tests in 2005. The writing assessment approved by the Board of Regents for a three-year field test was no longer available from ACT. As a result, the committee began to explore ways to continue the use of an assessment that would meet the needs and criteria identified in 1998 and continue to provide the professional development and K-16 dialogue that the field test inspired.

The Montana Writing Assessment has several advantages: 1) the scoring rubric is aligned to K-12 Performance Standards; 2) prompts, timing, and mode can be selected to align with curriculum; 3) inter-rater reliability has been established; 4) data provides specific feedback to students, teachers, and schools; 5) professional development is an integral part of the scoring process; 6) pre-service teachers have an opportunity to assess student writing; 7) teachers and college instructors from different schools and levels collaborate; 8) testing specifications and reporting can be responsive to changing needs; and 9) all students in a school (at junior or senior level) can be tested. In addition, there are disadvantages with a Montana Writing Assessment: 1) state agencies and/or schools assume administrative costs; 2) oversight from OCHE is needed to maintain quality and collaboration; 3) K-12 budgets may be strained to participate in training/scoring; and 4) time is taken from high school instructional time (one period to test students, two days for teachers to score).

The use of SAT or ACT writing scores also has advantages: 1) students assume all costs; 2) scores can be used for out-of-state colleges and universities; 3) validity and reliability concerns are addressed by testing companies; and 4) Saturday test days do not intrude upon instructional time. SAT or ACT writing samples will also have certain disadvantages: 1) the brevity of 30-minute writing sample produces significantly lower scores; 2) the writing sample score must be anchored with multiple choice questions; 3) students will have no choice of prompts or the option to use word-processing; 4) the prompts, rubric, timing or changes in test parameters cannot be controlled in state; and 5) Montana teachers and professors will have limited involvement.

The Chief Academic Officers were asked for input on November 19, 2003. They are concerned about the level of preparation of incoming students, as well as the issue of articulation between two and four-year programs. They also expressed frustration about the quality of writing demonstrated by their students at the junior level. However, some are opposed to: 1) funding programs that do not directly benefit their campuses; 2) funding a writing assessment when there will be two available for which students could pay; and 3) over-testing students who take both the ACT and a Montana test. At the same time, the expressed the desire to create a seamless system: from high school to college; from two-year to four-year; and at graduation. They have an interest in exploring the possibility of purchasing the ACT for all students and considering the Montana Writing Assessment for multiple uses, including placement

The P-20 Committee of the Board of Education has set as a goal to ensure alignment of standards, curriculum and assessment systems across all levels. In addition, they are concerned about teacher quality and professional development. Admissions standards can serve as an important link between K-12 and higher education, particularly if they engage teachers, pre-service teachers and college instructors in conversations about standards, curriculum and instruction, as does the Montana Writing Assessment Teachers who completed survey instruments recognize the alignment issue and how scoring helps them teach:

  • I am please that OCHE and OPI are encouraging this. Student writing needs to be more stressed in all academic areas.
  • When I first showed up yesterday morning I thought I was in over my head (junior high teacher). I am glad I stayed because it emphasized to me that teachers of all levels should be exposed to this technique (scoring) to aid their students futures.
  • I think scoring definitely helps make us better teachers of writing.
  • I've learned the value of assessment and what to teach to reach high standards.
  • No wonder we are having so much difficulty with college level writing!
  • Every year a person teachers, the more useful sessions like this are. They encourage sharing of ideas and best practices.
  • I learned about areas of weakness that I can more adequately address in the classroom.
  • We're on the right track!

With the "training of trainers" session scheduled for February 23-24, 2004, as many as 50 teachers and professors will learn to teach others how to score essays reliably. They will practice these skills, with assistance from experienced trainers, in the 2004 training and scoring sessions. These steps will help build the capacity to sustain a writing assessment and allow more teachers of writing and pre-service teachers to realize the benefits past scorers describe.