READ | TEACHING @ | EVALUATING STUDENT LEARNING
designing | grading | learningPrimary Trait Analysis
One method of analytic scoring is Primary Trait Analysis (PTA). The primary traits of a piece of student work are the instructor-defined criteria which are necessary for the successful completion of the task. Unlike holistic scoring which lumps these together, looks at them simultaneously, and then assigns one grade, PTA scores each trait separately. This allows an instructor to systematically compare student performance for each trait as well as to assign grades to individual students. Instructors can then identify strengths and problems in their preparation of students for their task, in test or activity design, and in student performance. PTA is a way to take what we already do—record grades—and translate that process into both a very effective strategy for responding to student work and an assessment device (Walvoord & McCarthy, 1990).
Advantages of PTA for assessment include (1) using information that is already available, (2) bringing to consciousness the mostly subconscious processes that go into recording grades, and (3) looking at performance strengths and weaknesses in individual pieces of an assignment, course, or curriculum. This section illustrates how PTA works.
Each teaching professor has a view of what he or she wants students to accomplish. The view, even if it is an unconscious one, pictures ideal student achievements at the end of a particular class, a unit of instruction, or an entire curriculum. At the end of an assignment or course, students who achieve the goals and “look like” the ideal tend to get A’s; those who look a bit less like the ideal get B’s, and so on. Because students (and professors) are not perfect, achievement of goals is usually uneven. Students may excel in one area and be merely adequate in another. Nevertheless, most instructors record a single, holistic grade that tends to sum the student’s performance and provide an overall judgment of merit. Primary Trait Analysis (PTA) does not yield a single, holistic grade. Instead, it reveals parts. The example below outlines the process of scoring a paper using PTA.
Primary Trait Analysis: An Example
Course objective: Students are required to “understand” scientific conclusion and process. The scientific paper is a visible representation of scientific understanding.Primary traits: The components of the assignment are recognized as primary traits (essential or central components of the discipline) to be learned by the student. These could include:
- Introduction section should contain history and context plus the testable hypothesis.
- Materials and Methods section should contain general approach plus specific equipment and procedures.
- Results section should contain observations upon which conclusion is based.
- And so on, including correct format, references, mechanics, and grammar.
The instructor constructs rubrics representing the level of achievement for each primary trait. For example:
Introduction—history:
- 4 points: History well researched. Major contributions presented with discrimination and balance. Controversies outlined and weighed.
- 3 points: History adequately outlined. Role of major contributions recognized. Relative merit of conflicting opinions somewhat unclear.
- 2 points: Historical outline present. Contextual development and relative merit of contributions unrecognized or ragged. Presentation of conflicting ideas absent.
- 1 point: Historical outline absent or garbled. Contributions listed as in a diary; consideration of merit absent. Notions of conflicting ideas ignored.
Introduction—hypothesis:
- 4 points: Hypothesis clearly recognized or well crafted and elegantly stated in testable form. Hypothesis cleverly embedded in context.
- 3 points: Hypothesis recognized or well stated. Contextual connections evident.
- 2 points: Hypothesis detectable but may not be stated in testable form. Contextual connections tenuous.
- 1 point: Hypothesis undetectable or garbled so as to violate scientific principles. Context absent or ignored.
Materials and Methods—procedures:
- 4 points: Procedures clear, need no interpretation. Appropriate details present.
- 3 points: Procedures easily interpreted. Relevant information dominates.
- 2 points: Procedures unclear but interpretable. Irrelevant information interferes.
- 1 point: Procedures scrambled. Irrelevant information predominates. Reads more like a bad diary. Students write paper in scientific report format.
In the example, the instructor reads Papers #1, 2, 3,… but assigns point values to various parts according to rubrics. By adding points horizontally, the instructor arrives at point values for the two primary traits found in the Introduction (18 and 11), the two primary traits found in the Materials and Methods section (16 and 20), and so on as shown in the right hand column labeled Assessment.
The crucial point is that if the instructor compares grades only, he or she would be unlikely to uncover the following penetrating insight: regardless of their grades, students are having difficulty learning how to phrase or interpret a scientific hypothesis (Intro. IB). By comparing assessments of primary traits, instructors have integrated assessment information to make their curriculum visible.
In addition, PTA can be used to objectively compare multiple sections of the same class.
Paper #1 |
Paper #2 |
Paper #3 |
Paper #4 |
Paper #5 |
Paper #6 |
Assessment |
|
Intro. IA |
3 |
4 |
3 |
3 |
3 |
2 |
18 |
Intro. IB |
1 |
3 |
2 |
2 |
2 |
1 |
11 |
M&M IIA |
2 |
3 |
2 |
3 |
3 |
3 |
16 |
M&M IIB |
3 |
4 |
3 |
4 |
3 |
3 |
20 |
Etc. |
3 |
3 |
3 |
4 |
3 |
2 |
18 |
Grade |
12 |
17 |
13 |
16 |
14 |
11 |
A few notes about Primary Trait Analysis:
- Primary traits may be identical to goals or learning objectives.
- If the sample is large, multiple readings of the same papers are unnecessary.
- Assessment makes learning visible to dispel assumptions, guesses, and rumors.
- PTA makes teaching (and grading) conscious. It uses information already (unconsciously) available.
- PTA, as illustrated, will allow the instructor to determine what students have learned, but not the relative contributions of curriculum structure, teaching styles, learning styles, effort, student study habits, etc.
 previous page |
