Gathering and acting on feedback
Peer observation of teaching and other collaborative activities
Responding to student feedback
Collecting feedback and reading it is only the first step in using student feedback. Academic staff may wish to consult the QA staff on their interpretation of the feedback or the action planned on the basis of such feedback. Such consultation is part of the QA group's usual role with staff and is of course entirely confidential.
Gathering your own Student Feedback
1. Quick & Easy Feedback Strategies
While many staff already collect student feedback at the end of semester using the Feedback for Teachers questionnaires and student group interviews, it is usually collected too late to allow staff to respond by implementing changes to improve the teaching of the students offering the feedback. Mid-semester FFT surveys will be available as a resource for teachers to use from semester two 2016 onwards and will allow consistency in the form of feedback between the middle of semester and the end of semester and across different classes.
However, staff who would like to use other tools to monitor their teaching during the semester in order to act immediately to improve their teaching might find the following suggestions on the collection of student feedback useful. As with all aspects of feedback and evaluation of teaching, it is useful to draw upon a range of different yet complementary sources of data.
There are a range of strategies suitable for gathering informal feedback from students. Perhaps the most obvious way is to ask the students questions directly. If classes are small and interactive, and there is mutual trust and respect between students and teachers, then posing questions directly to a class may be feasible. However, there is no scope for anonymity of student responses and some students may feel uncomfortable voicing less popular opinions in such a public manner. Following are some of the various strategies that have been developed to address these issues yet still permit quick and easy collection of feedback from students.
Three open questions
This strategy can be adapted to different modalities and purposes. Three questions usually provide a substantial amount of feedback without taxing student or staff resources unnecessarily. This uses students' individual written responses to open questions asked at or before the end of the class. The questions might be written on the board, projected on-screen, provided as an actual questionnaire, or supplied on Blackboard (LMS). The questions you choose to ask will be determined by what you want to gather feedback on, however they are typically along the lines of the following:
"What was the most useful thing you learned today?"
"What was the best thing about today's class?"
"How could I change my teaching to help students learn more from this class?"
In some cases it might be appropriate to ask very specific questions such as:
"Which of the set pre-readings was most helpful in preparing for today's class?"
"How did today's computer learning task help you understand the concept of...?"
In larger classes the time required to read written responses from every student can be a barrier to using this technique. In such cases, sampling procedures can be helpful. There are many ways to select a random sample of students. One simple technique is to have all students complete a questionnaire and then randomly select one out of every five responses to read.
Other techniques used include selection based on sub-groups in the class, for example asking all those in a Tuesday tutorial group to complete a survey. If sampling is used then there is only a probability that the results of your sample are representative of the whole population.
Group responses: An alternative to sampling in larger classes is to divide the class into groups of 5-10 students and collect the collated responses to the questions from each group of students after the individual students have contributed to a group discussion. For example:
"I want you each to write down your answers to these three questions. In a few minutes I want you to discuss your answers in your groups. Once everybody has had their say I'd like one person in each group to write down their group's response to each question."
This technique is useful as it ensures discussion and some degree of consensus amongst students before they respond. At the same time it streamlines the amount of feedback that the teacher has to read. In some settings this technique can be adapted to verbal presentation of group responses.
Critical learning statement – ‘Three clear and three muddy points’
This is a useful technique which gathers feedback on student learning as a way of investigating the effectiveness of teaching. Students are asked to write down the three critical points they have most clearly learned from the class and the three points they are still most unclear about. This strategy is also known as three clear and three muddy points.
A consideration of the responses and identification of any recurring themes is an effective way of identifying where teaching has worked well in terms of student learning and where it could be improved. It also provides feedback that can be acted upon quickly, as the teacher can review any particularly difficult points in the following session.
A variant of this strategy involves asking students to write down three questions they still have on the topic of the class. This can provide pointers to any remaining areas of confusion for students and also provide staff with an indication of what students are interested in learning more about. This technique can also be adapted for group responses.
2. Student Group Interviews
In units of study with low enrolment it is possible to obtain student feedback using structured group interviews. In this process the staff member appoints an agreed interviewer (e.g. fellow staff member or RA) who visits the class on an appointed day while the staff member leaves the room. The class is then interviewed and the responses to the preset questions, as well as an indication of the extent of support for different opinions and views within the group, are documented by the interviewer. Comments and suggestions from the students can be followed up during the interview for further inclusion in the report. The interview usually takes about 45 minutes. A report is written and sent to the staff member.
This process is particularly suitable for small classes of less than 20 students, however the process can be easily adapted for use with larger groups. The structured group interview is particularly useful when information about the reasons for the students' opinions or perceptions is required. The additional information typically volunteered by students can prove a valuable source of data for new staff, new units of study, or when seeking feedback on innovations in existing teaching practices.
3. Focus Group Interviews
Focus Groups are a variant of the Structured Group Interview method usually adopted for surveying larger classes. Students are asked to nominate or self-select a group of approximately 15 students to participate in a structured group interview. The process is then the same as that of the structured group interview but conducted with this subgroup of the complete class. On the day of the interview the interviewer visits the class, the interview group of students remains, and the rest of the class leaves. After the interview, a report is written and forwarded to the staff member. It is recommended that the feedback from the report be validated with the student group as a whole in a subsequent class to ensure that the comments are representative.
This process is an effective way of obtaining qualitative data from large student groups when particularly detailed information is required, such as in a formal unit of study review or evaluation.
Lizzio et al. (2002) have developed focus group protocols for investigating students’ experiences of workload and assessment at the unit of study level. These appear in the appendices of the following paper:
Lizzio, A., Wilson, K., & Simons, R. (2002) University students’ perceptions of the learning environment and academic outcomes: Implications for theory and practice. Studies in Higher Education, 27, 27-51.
4. Getting Feedback Online
You might consider gathering student feedback on their learning experience via online means. You could invite comments on particular topics and tasks via:
Using one-to-one email guarantees a certain privacy for each student, in that the exchange will be between you and them only. You could invite your students to email you privately at particular points in the Unit with their questions, and some brief notes or comments about what they find useful or aren't sure about. Your availability and the feedback you provide via this private channel will be appreciated by many of your students.
In a discussion list, all messages and posts to the list are able to be read by everyone enrolled in your Unit (unless you set it up otherwise). In LMS, the 'Discussion' tool is structured around discussion threads housed within forums. You could create an 'Evaluation' thread where students are encouraged to post feedback on specific aspects of the teaching & learning process. They might discuss, for example, what they understood were the key points in a particular talk, topic, or reading. This kind of discussion will enable you to determine whether the teaching & learning processes are effective in supporting and developing the students' understanding.
The composition of the group that accesses the 'Evaluation' thread will influence the nature of what is posted. Therefore you will need to indicate to your students whether the 'Evaluation' thread is public and accessible by all, including yourself and other teaching staff. You might consider setting up 'private' group-based discussions. After a period of online discussion, one student representative from each group could email you a summary of the group's discussion; or group representatives could post their summaries to a public thread for you and others to discuss.
Peer observation of teaching and other collaborative activities
5. Peer Observation of Teaching
Student feedback on teaching and units of study provides only a limited perspective for the evaluation of teaching. There are aspects on which students are sometimes not in a position to provide feedback. In addition to data drawn from students it is often beneficial to gather feedback from academic colleagues. This may be done by inviting a colleague to visit your class to observe a session prior to your engaging in a reflective discussion with them. Colleagues may be peers in the same department, colleagues in a different field, or staff from the QA Group. Please read Teaching Insight 6 for further information.
6. Collaborative Peer Teaching Activities
An alternative to engaging in peer observation of teaching is to ask a colleague to participate in the:
- collaborative planning of a class
- collaborative teaching of a class
collaborative review of unit of study documentation
(i.e. unit of study outline, assessment tasks and criteria, etc.)
Such collaboration and reflection on teaching activities can provide you with additional insights or a different perspective on your teaching practice.
Collaborative planning of a class or teaching activity
Inviting a colleague to participate in the joint planning of a class or similar teaching activity can provide useful feedback on this aspect of your teaching practice. It is also an opportunity to share ideas and teaching strategies, and can be a rewarding learning experience for both people. Doing so can encourage you to reflect on and consider:
- Your reasons for teaching the way you do.
- The effectiveness of the teaching strategies you select.
- Possible alternative teaching strategies and approaches.
In the joint preparation of a class it may be useful to consider the following issues:
- What are the intended learning outcomes for the class and how do these relate to the aims and objectives of the unit of study as a whole?
- What teaching and learning activities have been selected to achieve these outcomes?
- How will the teaching and learning tasks or activities selected achieve these learning outcomes?
- How will the student learning be monitored?
The role of the critical partner is to prompt a consideration of these and similar issues, to support their colleague's reflection, and contribute to the development of alternative teaching strategies. Such feedback and reflection may be usefully documented in the form of a learning outline with accompanying rationale and a list of potential alternative teaching strategies or ideas to follow up at a later date. Evidence of such reflection and subsequent action to improve teaching can support claims of quality teaching.
Collaborative teaching of a class or teaching activity
Co-teaching can be a useful experience for staff seeking to investigate and reflect on their own teaching. Negotiating with colleagues to co-teach a class can usually be arranged. As a strategy for supporting reflection on teaching, many staff report this to be a useful variant of peer observation of teaching.
In inviting a colleague to collaborate in the teaching of a class it is useful to consider the following:
- Select a co-teacher from the same unit of study team or department, or somebody with an interest or expertise in the topic of the class.
- Select a class that provides interesting scope for co-teaching activities.
- Agree that the observation and subsequent discussion will remain confidential and clarify the purposes of the co-teaching exercise.
- Meet with your colleague beforehand to discuss the topic of the class and to prepare. Such collaborative planning is a worthwhile source of feedback on teaching in its own right. Clarify your individual roles and responsibilities within the session and share your ideas about how you might teach the session.
- During the co-teaching session try to note how you and your colleague approach and handle different teaching tasks. What different strategies are employed and how do students respond? Consider the possible implications of different teaching strategies and techniques for student learning.
- After the session, meet to review the class and share your observations and insights on each other's teaching. Such a discussion is often best initiated by a reflection on your own teaching before seeking additional observation and input from your co-teacher. Note aspects of your teaching that you could modify to further facilitate student learning, and jointly generate strategies to address these issues.
- Both staff members may find it useful to document the issues raised during the discussion and any actions or initiatives that are proposed to address relevant teaching and learning issues. Evidence of such reflection and subsequent action to improve teaching can support claims of quality teaching.
Collaborative review of unit of study documentation
Collaborative critical review of unit of study documentation can also provide useful feedback to staff on aspects of their teaching practice. There are many examples of documentation that can provide a useful basis for collaborative review. Examples include:
- Unit of study Outlines
- Unit of study aims, objectives, or learning outcomes
- Assessment tasks and assessment criteria (e.g. exam papers, essay questions)
- Lecture handouts and slides
- Tutorial questions
- Tutors' & demonstrators' teaching guides
- Student laboratory and practical manuals.
Sets of prepared readings, etc.
Review questions might encompass the following issues:
- What is the intended purpose of the documented aspect of teaching that is to be reviewed? How does it contribute to student learning?
- Against what criteria would you decide how well it achieves the intended purpose?
- Based on these criteria, what are its shortcomings and strengths with respect to promoting student learning?
- How could this aspect of teaching be improved?
Feedback from this process of collaborative review and discussion can be documented under the questions noted above. Action statements and possible improvements arising from this process can also be documented. Evidence of such reflection and subsequent action to improve teaching can support claims of quality teaching.
Responding to student feedback
7. Interpretation of the feedback
General Student Feedback
The following guidelines may assist you in interpreting the results. Please also see Teaching Insight 9 for further information.
(i) You should consider the representativeness of the responses. If the number of students completing the questionnaire as a percentage of the number of students enrolled in the unit is less than about 60% the responses may not be representative of those students who have experienced your teaching.
(ii) The statistical report gives the response analyses for all individual questions from your questionnaire. For each question the analysis shows:
- the number of students who gave each of the possible responses
- the percentage of students who gave each possible response (calculated over the number answering the question, which may be less than the number completing the questionnaire)
- a graphical representation of those percentages
- the mean and standard deviation
A summary of the teaching and learning practices that characterise each scale is attached to the report.
(iii) The students' responses to the open ended questions will be returned with the statistical report. These comments usually provide additional information worth considering when interpreting the statistical report. As the responses to the open ended questions typically include the most critical positive and negative aspects of the students' experience of the teaching and learning process, they are often the form of student feedback most useful for indicating potential improvements in teaching.
With your knowledge of the unit and its material you will be in a position to interpret the results of your particular survey; however you should bear in mind that student perceptions are shaped by various aspects of the teaching and curriculum context. For instance, a unit with a small number of students taught predominantly in small groups and using continuous assessment will invariably be rated more highly than a unit with large student numbers in which lectures are the predominant mode of delivery and where students are assessed solely on the results of a final unseen examination. Optional units also tend to be rated more highly than compulsory sections of a program.
It should be noted that students assign "ratings" individually in the absence of any agreed upon criteria for what constitutes "satisfactory." For example, a rating of "4" may be assigned by different students for different reasons and reflect different standards.
Standard Open Response Questionnaires
Student responses to open-ended questions typically include the most critical positive and negative aspects of the students' experiences of the teaching and learning process and are well worth considering. Such responses have been noted by academic staff to be the form of student feedback most useful for indicating potential improvements in teaching.
Student evaluation by questionnaire is only one source and method of collecting information. Other sources, such as peers, and other methods, such as group discussion, also provide valuable information and should be considered in course or teaching evaluation.
If you would like to discuss the interpretation of feedback, a member of the QA team academic staff will be happy to meet with you once you have considered the comments and report. Please do not hesitate to contact the QA team on extension 13725 or email firstname.lastname@example.org to arrange an appointment. Staff at the Institute for Teaching and Learning are available to discuss any aspect of your teaching or of the unit curriculum which you would care to develop further as a result of student feedback.
8. Feedback Cycle
Staff are encouraged to reflect on the results of their student feedback:
You may wish to use the format similar to the Unit of Study Evaluation template. Feedback should be obtained from a range of sources. This could include your own reflections of the teaching and learning in the unit of study, as well as any collaboration with colleagues.
- Context or Unit of Study Description and Background
- Key Issues Identified by Students
- Key Peer Review Issues
- Self Review Issues
9. How to tell students what you are going to do as a result of their feedback
Students commonly report that they rarely see any action arising from their feedback. Consequently, they come to perceive their opinions and views as being of little real interest to staff. In order to recognise the value to staff of constructive student feedback it is recommended that students be advised of actions that will or will not be taken on the basis of their feedback. This can be done at the beginning of a subsequent class or via student notice boards. One framework for feeding such information back to students is to tell students:
- Which suggestions will be acted upon and how
- Which suggestions you would like to act on but are unable to and the reasons why
- Which suggestions you will not be acting on and why
For further information please see Responding to Student Feedback.