Using the IR Item Bank to Create Your Own Form

Download as PDF (Introduction and Item Bank)

Student ratings of teaching effectiveness have been researched for over 50 years and are the subject of over 2,500 books and articles. They are the most widely studied of all forms of teaching evaluation and the most supported by empirical research. This extensive work clearly demonstrates that properly designed student rating instruments can provide feedback that is valid and reliable, relatively free from bias, correlates well with external measures of student learning, and serves multiple purposes.

Student ratings of teaching effectiveness are a valuable source of information and can guide efforts for improving teaching. They are, however, only one source of data for assessing teaching effectiveness and most useful in conjunction with a more comprehensive evaluation process. Evaluation specialists recommend considering multiple sources of data (e.g., student ratings, self-assessment, classroom observations, peer review of materials, teaching portfolios, alumni ratings) for both improvement purposes and personnel decisions.

What Is Effective Teaching?

The development of the OIR item bank began with a review of the literature from several areas related to student ratings of teaching effectiveness. This encompassed research on effective teaching, learning, principles of good teaching practice, evaluation, and student ratings. The culmination of this analysis resulted in the creation of the OIR item bank that is broad based and reflects the currents ideas and work of leading researchers in these fields.

Teaching and Learning Literature

The extensive literature in the areas of teaching and learning provided the framework for The Seven Principles of Good Practice in Undergraduate Education (Chickering and Gamson). Originally published in 1987, the principles were developed by a distinguished group of researchers and commentators on higher education. The principles assert that several common features characterize good practice in undergraduate teaching.

  1. Encourages student-faculty contact is evident through interaction and rapport with students, acceptance of students’ view and discussions, concern with student progress, and willingness to help students with problems.
  2. Encourages cooperation among students occurs through the active involvement of students in small groups activities.
  3. Encourages active learning is addressed through talking and writing about learning, relating learning to past experiences, applying learning to present events, and projecting learning into the future.
  4. Gives prompt feedback is accomplished by providing timely feedback that is supportive and appropriate for enhancing improvement. The quality of the assessment tool (e.g., exam, paper, oral presentation) and the frequency of feedback are also important components.
  5. Emphasizes time on task is evident through effective use of class time, appropriate time allocation for course components, enhancement of students’ time management skills, and course pacing.
  6. Communicates high expectations is reflected in setting high but attainable goals for students’ academic performance, difficulty of coursework, and course workload.
  7. Respects diverse talents and ways of learning encompasses recognizing different talents and learning styles of students, being sensitive to differences among students, and recognizing unique interests and talents of students.

Evaluation and Student Ratings Literature

Several meta-analysis studies from the evaluation and student ratings literature reviewed research in which students, faculty, administrators, and alumni identified components of effective teaching (Feldman). Results showed considerable overlap between the qualities selected by each group. The studies defined common characteristics of effective teaching.

  1. Appropriate student learning outcomes place the focus on the outcomes of effective teaching. They can be classified by a variety of schemes including cognitive vs. affective, or low-level (i.e., acquisition of knowledge) vs. high-level (i.e., analysis). It is important to ensure that student learning is consistent with course objectives.
  2. Flexibility in teaching approaches permits faculty to enhance student learning by responding to diverse student backgrounds and learning styles. Research indicates that engaging students in the learning process contributes to their cognitive development. It has also been documented that faculty who implement a variety of appropriate instructional approaches are more enthusiastic in their teaching.
  3. Good organization and preparation of course is reflected in the level of preparation, use of class time, and in the course objectives, syllabus, assignments, activities, and evaluation methods of student performance. Organizational strategies are important as research indicates they are related to how much students learn.
  4. Knowledge of and enthusiasm for the subject matter and teaching should be evident. Faculty must be knowledgeable in their subject matter in order to organize it in a meaningful way for students. They should be able to communicate their knowledge at a level students can comprehend. An infectious enthusiasm comes with confidence and excitement for the subject and teaching.
  5. Effective communication is important in all types of educational settings from large lectures to one-on-one conversations. It is manifested through good speaking, writing, and listening skills. Communication also occurs in a variety of other ways including a course syllabus, presentation, explanation, or course activities.
  6. Positive attitude toward students is evident in a teaching environment that supports student learning and provides a positive self-image for students. It is reflected in mutual respect and rapport, concern for students’ learning, availability to students outside of class, and encouragement of students to participate in discussion and express their opinions.
  7. Fairness in evaluation and grading is reflected in the consistency between course objectives, course content, assignments, and evaluation strategies. The standards of grading should be clear and consistent, feedback timely and useful, and the equity of the workload appropriate for the credits received.

Organization of the IR Item Bank

Research indicates that effective teaching is a complex, multidimensional process that should be reflected in the design of student rating instruments. Consequently, most published student rating forms have a well-defined factor structure and provide measures of distinctive components of teaching effectiveness. The organizational scheme of the IR item bank was modeled on this work.

The item bank was developed to guide faculty in designing student rating forms that reflect the questions they would like to have answered with respect to their own teaching. With this collection of items, faculty, departments, and schools/colleges can customize their forms in the areas of student learning, teaching practice, and course elements.

The item bank contains questions organized around various dimensions of effective teaching and special instructional settings. The items were categorized on the basis of their apparent content. The dimensions include:

Student Learning

  • student outcomes (cognitive, critical thinking, interests and values, social awareness, self-concept)
  • collaborative/cooperative learning
  • student effort and involvement

Teaching Practice

  • organization and preparation
  • communication
  • faculty/student interaction

Course Elements

  • grading
  • examinations
  • textbook
  • assignments (reading, writing, oral presentation, computer-based, other)
  • audiovisual aids
  • technology usage
  • course difficulty, pace, and work load

Overall (Global) Questions

Student Demographic Information

Open-ended Questions

Specific Activities

  • guest speakers
  • field trips
  • research and field projects

Special Instructional Settings

  • team teaching
  • laboratory sessions
  • discussion/recitation sections
  • studio work
  • community-based and service learning
  • clinical/field placements
  • graduate seminar
  • research supervision

The selection of a student response scale for items within the OIR item bank is based on a review of the research and examination of nationally published instruments. Items are written so student responses are recorded on a 5-point agreement scale with:

1 = strongly disagree
2 = disagree
3 = neutral (neither disagree nor agree)
4 = agree
5 = strongly agree

Open-ended questions asking for written responses from students are also included in the item bank.

Creating Your Student Rating Form

Step 1: Determine the purpose of the instrument

Historically, student ratings of college teaching were designed for purposes of improving teaching practice or performance (i.e., formative evaluation). Over time, however, student ratings have increasingly been used to assist in personnel decisions (i.e., summative evaluation). In too many cases, this has become their exclusive purpose. Evaluation that serves both purposes is possible, however, research suggests that it is preferable to separate formative and summative evaluation – both conceptually and in practice. Furthermore, guidance for those who use student ratings of instruction is important in order to avoid misuse and misinterpretation of the results.

Step 2: Select the appropriate teaching dimensions

National studies suggest that the areas of student learning, teaching practice, and course elements be included on student rating forms. A comprehensive instrument would reflect the following:

  • student outcomes
  • student effort and involvement
  • organization and preparation
  • communication
  • faculty/student interaction
  • assignments, exams, and grading
  • course difficulty, pace, and work load
  • overall (global) questions
  • student demographic information

Step 3: Choose the specific questions

The number of questions you select from each of the areas you identified in step two will determine the length of your form. Three of the leading national surveys (SIR II, SEEQ, IDEA) contain 40 to 45 questions, but you may choose the number of questions that you find appropriate for your purposes. You may also want to make selections from the more specialized sets of questions within the item bank. These may be appropriate if you have specific activities or instructional settings that you would like to assess. Open-ended response items are a final option for your student rating form. Generally, two or three open-ended questions are appropriate. Possible selections are available in the item bank.

Step 4: Organize the selected items

Consideration should be given to: How will the items be grouped (e.g., clustered by teaching dimension, chronologically)? Will labels be used to identify the subtitles (e.g., student outcomes, communication)? How will the items be arranged for ease of reading and answering?

More Information

If you would like more information on this topic, please contact the Office by email or by phone at 443-8700.