1. PROSPECTIVE STUDENTS
2. CURRENT STUDENTS
3. FACULTY & STAFF
4. ALUMNI
5. COMMUNITY & FRIENDS

1. CAMPUS LINKS
   1. Inside UNCG
   2. Academics
   3. Administration
   4. Admissions
   5. Arts & Entertainment
   6. Athletics
   7. Continuing Education
      & Outreach
   8. Employment
   9. Giving to UNCG
   10. International Programs
   11. Libraries
   12. Online Courses & Degrees
   13. Research
   14. Technology
   15. University News

UNCG Home » UTLC » Faculty Resources and Programs » Instructional Development Services

Collected Comments on Peer Observation

James Eison, Ph.D. Director
Center for Teaching Enhancement
University of South Florida
4202 East Fowler Ave, SVC 1088
Tampa FL 33620-6912
(813) 974-2576 fax (813) 974-5620
eison@acomp.usf.edu

Wrote:
I too have found it especially helpful to

(1) ensure that the terms “Peer observation” (involving such processes as conducting an actual classroom visit) and “peer evaluation” (involving the judging of a colleague’s activities, accomplishments, materials) are not used thoughtlessly and synonomously. For example, I once taught at an institution in which faculty “swapped written materials like syllabi, tests, and cv’s” prior to asking the colleague to complete a “peer evaluation form” (two completed forms were required annually). No classroom visits were involved or required.

(2) clarifying the different views that faculty hold regarding what should and should not be observed and/or commented upon when conducting classroom visits and then attempting to reach consensus on the differences. For example, at another instution in my past, if my memory serves me correctly, untenured faculty were required to have two “classroom observations” done by tenured peers each semester for their first three years and one classroom observation done annually by the department chair. Prior to conducting a workshop for tenured faculty (i.e., observers) in one college, I surveyed all the tenured faculty to ask what they felt peer observers should and should not look at and report. Views differed all over the place: some faculty said it is very important to look at the colleague’s presentation of course content while others said don’t look at “what is taught;” some said look at the effectiveness of the instructor’s specific teaching strategies while others said don’t pay attention to how the person chooses to teach. The specific diffferences identified in the little survey covered, in my humble opinion, both issues of considereable importance (such as the two I’ve already noted) and other differences of opinion regarding issues of less instructional significance (e.g., manner of dress).

(3) Mike’s side note regarding peer evaluations being “a popularity contest” reminded me that at the first institution peer evaluation truly was synonomous to “adding a numerical constant to a statistical procedure for virtually everyone's ratings.” That is, faculty exchanged ratings of “5” (on a five-point scale) for the peer evaluations that were done by “experienced faculty.” While faculty never complained, as Mike suggests about “peer inflation,” problems and interpersonal friction (of varying degrees) inevitably arouse under one set of circummstances -- when “new faculty” who did not know how the “system worked” (i.e., through faulty informal mentoring) gave a colleague in the department or division a more candid and truthful evaluation of his or her accomplishments (i.e., any rating less than 5).

Some other notions were described in a piece I wrote several year’s back (Thanks Erin Porter for remembering); “Designing Effective Peer Observation Programs,” Journal of Staff, Program, & Organization Development Vol 6, No 2, Summer 1988, pp. 51 - 59.

In short, I’d say that among the challenges faculty, administrators and faculty developers confront skillful peer observation and peer evaluation are among the most difficult processes to design and implement. They requires clear thinking, careful planning, and campus-sensitive implimentation I say this from the perspectives first of a faculty member giving and receiving such ratings, then from the perspective of a reader of the research literature, and more recently from the perspective of a faculty developer.

Cynthia Gray
Director of Instructional Development
Wayne State College
cgray@wscgate.wsc.edu

Wrote:
A model I have found successful is the Small Group Instructional Diagnosis (SGID), by Nancy Diamond in an earlier POD publication “A handbook for new practioners” Stillwater, OK: New Forums Press, 1988, 89-94, because it combines feedback from students given through your peer who acts as a consultant in helping to interpret the results. Although it is not a direct observation of each others teaching, I find it more dynamic because it assumes my students also have something to contribute to my teaching process. I use it at mid-term as somewhat of a grounded theory approach to adjust my teaching and course activties. The triangualtion of the students, my peer, and my own responses to the information has made for some encouraging results in the more formal course evals at the end of each term. It’s a strength-based model, providing just enough of a structure to insure that trust and respect accompany the process. It allows for written and verbal feedback--the results can also be taken back to the students to affirm that you heard them and are willing to make adjustments to the teaching and learning process thereby modeling reflective teaching and collaboration.

Mike Theall
theall.michael@uis.edu
U of IL at Springfield
Ctr for Tchng & Lrng
460 Brookens
Springfield, IL 62794-9243
217-786-7157

Wrote:
Just a note to reinforce the importance of distinguishing between peer observation for feedback/improvement and peer evaluation for personnel decision-making. The references Erin and other have cited are all valuable, particularly those which guide formative evaluation and stress the importance of well defined process, clear understanding of the purpose, agreement among all parties about what will happen and what will result, valid instrumentation, and particularly, an understanding of the criteria which will be used to define terms like “effective” or “good” teaching. It is probably a mistake to assume that all faculty know enough about pedagogy to be able to accurately assess others’ teaching. The input of ID/FD people can be very valuable in this context and a major contribution to good process. As members of formative teams, we can add a dimension of clarity and utility which might otherwise be missing.

The cautions about peer evaluation for summative purposes (like the recommendations in Arreola’s “Develolping a comprehensive evaluation system”) are well worth stressing. While peers have been shown to be fairly consistent raters of instruction when using validated instruments, they do demonstrate more positive bias than do students. As a side note, this makes me wonder why we haven’t heard the most common complaints about student ratings turned to peer evaluation (for example, stating that “It’s just a popularity contest” or that “You can buy good peer ratings by giving good peer ratings”. Would this be ‘peer inflation’?).

More seriously, there has really been little empirical research on peer evaluation. Recent investigations (reported by John Centra and by Jack Robinson at 1993 or 1994 AERA ... I can get the citations if necessary) have shown that more complex peer evaluations processes like portfolio assessment can be problematic because faculty are uncomfortable with evaluating others’ work and because of elements of subjectivity in portfolio assessment. Even with respect to interpreting more quantitative results like student ratings data, there are questions about the reliability of faculty decisions (e.g., Franklin & Theall, 1989 AERA paper, “Who reads ratings? Knowledge, attitudes and practices of users of student ratings of instruction”). Keeping formative and summative purposes completely separate remains the wisest idea.

Erin Porter, Ph.D.
Center for Teaching Effectiveness
Phone: 512/471-1488
FAX: 512/471-0596
University of Texas at Austin
Main 2200
Austin, Texas 78712-1111
notes these references:

Dilts, D.A., Lawrence J. Haber & Donna Bialik
Assessing What Professors Do: An Introduction to Academic Performance
Appraisal in Higher Education
Greenwood Press, Westport, Connecticut

Barbara B. Helling
Looking for Good Teaching: A Guide to Peer Observation
Journal of Staff, Program & Organiztion Development
Vol. 6, no.4, Winter 1988, pp 147 - 158

John A Centra
Reflective Faculty Evaluation
Critical roles of Colleagues and Department Chairs
Chapter 6
Jossey-Bass Publishers

James Eison
Designing EffectivePeer Observation Programs
Journal of Staff, Program, & Organization Development
Vol 6, No 2, Summer 1988, pp. 51 - 59

Barabara J. Millis
Colleagues Helping Colleagues: A Peer Observation Program Model
Journal of Staff, Program, & Organization Development
Vol 7, no 1, Spring, 1989, pp 15- 21

Newer items:

Braxton, Bayer, Finkelstein
Teaching Performance Norms in Academia
Research in Higher Education, 1992, 33(5), 533-569

Arreola, R.A.
Developing a Comprehensive Faculty Evaluation System
Boston, Anker, 1995.

M. Weimer’s,
How Am I Teaching?
(Magna Pub.)

W.Alan Wright
Executive Director,
Office of Instructional Development and Technology,
Dalhousie University,
Halifax, Nova Scotia,
Canada, B3H 3J5
Telephone:902-494-1622 Fax:902-494-2063
OIDT Home Page http://WWW.dal.ca/~oidt/oidt.html
Lifelong learning Network http//www.dal.ca/lifelong/

Wrote:
Wright’s Rules for Peer Observation

1. In general, the purpose of an observation must be made clear from the start.
2. An observation undertaken for summative purposes likely provides information leading to ‘career’ or ‘personnel’ decisions.
3. Summative observation MAY incidentally lead to feedback which has, potentially, some formative impact. (If it does, so much the better.)
4. Formative observation is intended as a help mechanism. It should not lead to a ‘report’ or a mention for a professor’s file. Feedback to the professor observed is confidential.
5. Notwithstanding point 4 (above) the observed professor may voluntarily report on the observation process in his/her teaching portfolio or other report that may eventually be reviewed by a committee making a summative judgement.

In short, the professor must know in advance ‘the rules of the game’.

Art Crawley Wrote:
For those who like a little POD history the earliest article on using Small Group Instructional Diagnosis (SGID) was written by Joe Clark and Jean Bekey for the POD Quarterly, Vol. 1, No. 2 (Summer 1979). The POD Quarterly was published for a few years before POD decided that it would be easier and probably cheaper to do an annual publication. The article’s title is “Use of Small Groups in Instructional Evaluation.” Joe Clark, who (I think) has his own company headquatered in Seattle called Videodiscovery was the Project Director of the SGID Project out of the University of Washington, Seattle Biology Learning Resource Center and was funded by FIPSE. I have an original copy of their handbook they developed for presenting SGID workshops for faculty and administrators. The handbook has served me well since then, and it is, as Cynthia says, a more qualitative approach to instructional consultation/assessment than the standard paper and pencil assessment instruments.

Mary Floyd Wrote:
Ed shared the guidelines in place at UNC and mentioned the need for two observers for a class and the need for the observers to be trained. From what Ed said and the comments of others, the peer review process is time consuming. I have two questions: one, who does the training, and two, what’s the payoff for the observers? I imagine that if we are talking about peer review for self-improvement, the faculty being reviewed will gain something, but here also I have my doubts. It depends on the department. Do observers ever come from other departments and who chooses them? I have an endless string of questions about this process. BTW, I’m one of those faculty who asked a colleague to come observe a class or two and give me some feedback many years ago. I also chose someone whose teaching style paralleled my own, in other words, not a lecturer, reading from a notebook.

Ed Neal (ed_neal@unc.edu)
Director of Faculty Development
Center for Teaching and Learning
Campus Box #3470
University of North Carolina at Chapel Hill
Chapel Hill, NC 27599-3470
Ph: (919) 966-1289
Fx: (919) 962-5236

Wrote:
Mary Floyd has posed some very good questions about the peer observation process, and I’d like to respond to her concerns.

(1) Who trains the peer observers? Observer training is one of the services of the Center for Teaching and Learning--last year I conducted 16 sessions to train peer observers. Normally, we work with a school or department to set up their evaluation system and perform the training as part of that service. I think this is one of the most valuable contributions that teaching centers can provide.

(2) What is the payoff for the observers? In most cases, I’m afraid, the payoff is simply what the observers learn about teaching in the process of trying to evaluate it. Some departments actually provide release time for faculty who serve as observers (typically, there is a rotating committee of observers), but most departments provide no extrinsic rewards. It has been our experience, though, that a well-run system provides ample intrinsic rewards--including the opportunity (or excuse) to talk about teaching with one’s colleagues.

(3) Can observers come from other departments? In the professional schools on our campus (in both Academic Affairs and Health Affairs), it is not uncommon for one member of an observation team to come from outside the home department, but in Arts and Sciences it rarely occurs (the exception being “Curricula” such as African and Afro-American Studies). In many ways, it is advantageous to have someone from outside the department perform the observation, since they are presumably less influenced by departmental politics.

(4) Are there real benefits in peer observation for self-improvement? Formative assessments are not only easier to perform, but they have been shown to provide strong benefits for both the observer and the observee. As a matter of fact, most faculty members seem to accept the idea of this kind of peer observation, no doubt because it is “collegial” in the old-fashioned sense and because some teachers (such as Mary Floyd) have invited colleagues into their classes for informal feedback. The really difficult task is convincing the faculty that the process can be used summatively as well as formatively.

I think it is important to understand that ALL forms of teaching evaluation are imperfect. Bias can be introduced in hundreds of ways. However, I think this fact argues that we need to use as many different methods of evaluation as we can afford when it comes to making tenure and promotion decisions.

I also think that if we don’t provide formative assessments (and resources and opportunities for faculty to develop their teaching skills), summative evaluation of teaching is unfair and in many respects simply punitive. I believe that one shouldn’t even begin to talk about summative evaluation of teaching (including student evaluations and peer observations) unless one is prepared to talk about faculty development initiatives at the same time. Faculty members are right to resist evaluation of skills for which they have no training, and pedagogical training is not normally part of a professor’s education.

Cheryl Knight
Appalachian State University
Knightcs@appstate.edu
704-262-2590

Wrote:
A peer Review group at ASU has begun the process. We have actually found it most valuable and, yes, even fun. The lessons have been informative to both the observer and the observee. The preconference is most important if we are to understand how the current lesson fits into the big picture. We scripted the observations, each person wrote a reflection then participated in a post conference. Wow! It was really a great experience. On Friday afternoon last week, our team met with other teams to share the experience. Both teams that had completed the process felt the same way. We feel the key is building a trust base prior to beginning the process, participants selecting who they will work with, and insisting the proceses is formative. My teaching will never be the same, nor will I deprive myself of the growth possible in a positive peer review experience.

No kidding, it was well worth the time, a tremendous amount of time, put in on the process.

She also wrote:
I would like to respond to some of Mary’s concerns about peer review. I know my prospectives are different due to the extraoraadinary experience I have had with our peer review process in the College of Education at ASU. Our people were trained through our Professional Development Center for the most part, however the most valuable training came from the discussion of readings we shared in our group. Our observers were in our college but not necessarily from our department. The process is very time consuming but I feel it is worth every minute required if we are to improve our teaching. (which I understand is the ultimate purpose.) The payoff for me as an observer is the insight I gained into other teaching styles, class management techniques, means of interacting with students, creative ways of grouping and a variety of assessment techniques. I was a learner. Another benefit is the professional friendship of other faculty members. We have formed a learning/teaching community like none I have ever experienced before. We truly dialog about the scholarship of teaching. We actually enjoy dialoging about our “mental models” (Senge). The professional development is a great benefit to me that actually extends into my class.

The key to peer review as our group experienced it is the building of a trust base. It can be one of the most valuable professional experiences a professor can have.

Barbara B. Levin
345 Curry Bldg.
University of North Carolina
Greensboro, NC 27412-5001
(910) 334-3443

Wrote:
We do peer review in the CUI department at UNCG. It is very helpful in my opinion (as an untenured assistant professor) but it is time consuming for the tenured faculty member who conducts the peer review. They spend time with us scheduling a time, talking with us before class, attending the class, and then after class writing up our review and discussing it with us. I would guess that it takes a minimum of 6-8 hours to do a thorough job. As the one being reviewed, I learn a lot from the opportunity to receive constructive feedback. The people doing the reviews should get some reward for their efforts.

Ed Neal (ed_neal@unc.edu) Wrote:
Mary Floyd asked if anyone has been engaged in peer review and discovered the kinds of problems it might cause. Since our Center has been directly involved in creating peer review systems on our campus, I thought I would share some of our experiences in this area.

First, we define “peer review” as the assessment of teaching evidence by colleagues, only *part* of which is peer observation of classroom teaching. When we work with a school or department (typically with a faculty committee appointed by the dean or department chair), we make this distinction clear and try to set up a comprehensive review of teaching by peers rather than focusing exclusively on peer observation. We usually suggest teaching portfolios as a way to present teaching evidence for peer review (our portfolio model includes peer observation reports and student evaluations as part of the documentary evidence). We present peer review of teaching as an analog to peer review of research, which helps faculty understand it better.

Although all departments at UNC currently practice peer observation of classroom teaching, only a minority do so in ways that provide valid and reliable assessments. The following guidelines promote validity and reliability and are the ones we use when helping a department create a peer observation system:

1. There should be at least two observers to provide more than one perspective on the observed performance.
2. Both observers should attend the same classes so they can compare their impressions of the same teaching behaviors.
3. The observers should be trained. Untrained observers are subject to many kinds of biases, and although subjectivity can never be removed from the system, training can help reduce bias. Training is also necessary to insure that the observers fully understand the purpose and procedures of the observation system.
4. The observers should be of ranks equal to or above that of the teacher being observed.
5. The observers should use a standardized observation/report form so that they focus their observations on the same dimensions of teaching.
6. At least two classes should be observed in order to provide an adequate sample for review. The observation schedule must be negotiated in advance with the person being observed.
7. A pre-observation conference between the observers and the instructor is required so the instructor can provide the appropriate context for the observation.
8. A post-observation conference is required, as soon as possible after the obervation occurs, to provide feedback to the instructor and to insure that the outcomes of the evaluation are open and above-board.
9. The observation report must be jointly written by the two observers to insure that the product is one of consensus.
10. All procedures for the evaluation process must be carefully followed by the participants. Any deviation from the specified procedures will jeopardize the validity, reliability, and fairness of the assessment. Departments should have some way of monitoring the system to insure that the procedures are being followed.

Most of the problems that arise in peer observation are the result of failing to follow these guidelines (e.g., having junior faculty observe senior faculty or visitors not observing the same classes). The only other major problem is amount of time it takes to perform this kind of observation. The departments that we have worked with believe that the time investment is worth it for cases of promotion and tenure, but that modifications are necessary for other purposes (such as review of senior faculty).

On our campus, the best model of a peer observation system is that of the Kenan-Flagler Business School. I would be happy to provide their guidelines to anyone on the ICUT network.

Return to Instructional Development Services

Page updated: 02-Sep-2008

Accessibility Policy