[The following article appeared in two parts, in the September and October editions of _TQM_in_Higher_Education_.] TQM FOR PROFESSORS AND STUDENTS [by] George Bateman and Harry Roberts For us, the following definition catches the essence of TQM: Continually serve customers better and more economically, use scientific methods and teamwork, and concentrate on removing all forms of waste. We believe that faculty can successfully adopt TQM efforts because: ù Professors are relatively free to change the way they teach. ù Professors want to be good teachers, and there are ways--even for college presidents, deans, and department heads--to encourage good teaching. ù The key TQM idea is customer satisfaction and students play the role of customers. ù The TQM movement has already led some faculty to begin thinking of students as customers. The view of students as customers isn't universal; many faculty often resist or resent this view. The idea of students as customers can be construed much too narrowly. Students aren't customers in the sense that the custom- er is always right, nor are students the professors' only cus- tomers. But the idea of students as customers is more healthy than the extreme paternalistic assumptions that faculty have tradi- tionally made, namely, that professors know what's best for stu- dents, and that students can't judge their own long-term self- interest and have to be given a lot of medicine they don't like to take. This traditional professorial paternalism can lead to complacency, stagnation, failure to check how much is really being learned and retained, and the working hypotheses that students' needs coincide with professors' interests. Worse, it can lead to accepting poor student performance. The idea of students as customers encourages professors to take responsibility for the success of teaching, and therefore become interested in methods of teaching improvement. We can testify from personal experience that teaching looks very different when you think of students as customers. Profes- sors begin to try to figure out why students perform poorly or challenge the relevance of the material. They begin to think about getting relevant data. We believe professors need more data than they usually get, and they need it in a more timely fashion. Role of Course Evaluations in Improving Teaching If an institution is to achieve a customer focus, some information about customer satisfaction is essential. Since the late 1960s, the Graduate School of Business at the U. of Chicago, where we teach, has used student course evaluations based on questionnaires in all courses, with systematic public reporting of results. Other business schools--for example, Northwestern's Kellogg School--also have used public course evaluations. Just as grading often makes students uncomfortable, course evaluations make professors uncomfortable. But, in spite of minor technical reservations, the Chicago faculty generally believe that course evaluations provide the best available information we have about teaching effectiveness. Faculty members don't believe that the evaluations are mere popularity ratings. Although we can't prove it, we believe that teaching at Chicago is much better than it would be in the absence of public course evaluations because evaluations encourage the faculty to treat students as customers, whether or not the word "customer" is used. In promotion decisions, a summary of course evaluations is always included in reports and discussions of the Appointments Committee. This, and the fact that, both at Chicago and North- western, deans take these evaluations seriously, encourages good teaching. Inspirations Unfortunately, course evaluations have limitations: ùThey're available only after the course has ended. ùThey use general-purpose questions that apply to all cours- es. ùThey can't include course-specific questions. ùThe numerically scaled questions tell almost nothing about what worked and what didn't. Some information can be gleaned from tabulating free re- sponse questions to see which themes occurred most frequently, but these highlight pervasive problems rather than specific difficulties. There have been attempts at many colleges to employ simple feedback questionnaires, typically informal but focused on the specific class, at the end of class sessions. At Chicago, two developments led us to experiment systematically with feedback questionnaires and other TQM methodology: ùIn the fall of 1990, U. of Wisconsin-Madison professor Ian Hau was teaching a large undergraduate statistics course. From his students, Hau formed a small quality improvement team to help him improve the course while he was teaching it. ùIn March, 1991, Andrew Appel, a Chicago M.B.A. student suggested that we use the Chicago "laboratory course" format to help Chicago faculty members apply TQM ideas and tools to improve their teaching, curriculum development, and re- search. Originally, the "laboratory course" was a "new product laboratory" where teams of students worked with client companies to develop and implement new product ideas. Faculty and execu- tives from client companies coached the students. The laboratory format has been extended to other kinds of applications such as implementation of TQM. The Teaching Laboratory Thus was born "Business 712, The Laboratory to Achieve Organizational Excellence: Improvement of Teaching, Curriculum, and Research"--"teaching lab." In the lab, the clients are usu- ally faculty members, and most student activity during the first year (1991-1992) focused on helping these clients. For example: ù Eleven faculty members have worked with lab course stu- dents or student teams to improve ongoing courses. ù A team of five students worked with the behavioral science group as a unit to design a new required course in behavior- al science. ù Two students worked with marketing faculty on curriculum issues in introductory marketing courses. ù A student worked with a faculty member to develop a course on high-tech marketing. ù One student in the lab has benchmarked the performance of two of the school's most outstanding case teachers. These efforts were generally very successful. For ongoing courses, students developed feedback mechanisms that tell the instructor, continually and quickly, what is and what isn't working--both in class and in the readings--so that the instruc- tor can make appropriate adjustments quickly. Students used various tools, including focus groups, video- taping, and broader surveys, but the survey tool turned out to be a simple fast-feedback questionnaire, used at all or almost all the class sessions. The questionnaire evolved from lengthy to streamlined, and the process was simplified so that faculty could do it themselves. Many of them are doing now. They also designed simple questionnaires (often confined to one side of one page), which faculty could administer and inter- pret. The use of fast-feedback has become widespread, though far from universal. In the lab's second year, students have turned to broader issues of curriculum development--benchmarking business efforts in general management training--and administrative facilitation of education--the use of information technology in M.B.A. educa- tion. [The Fast-Feedback Questionnaire and its Results on Teaching and Learning] Here we focus on what we learned developing the fast- feedback questionnaire. Because of our experiences, we've reached several conclusions and promising hypotheses about ways of improving teaching. 1. It's essential for the students to be sold on the feed- back questionnaire. Emphasizing at the start that responses will benefit the current class, not just future ones, is key. 2. Using feedback questionnaires can hurt the instructor's ego because sometimes there are very negative, even hostile student reactions, even when a course is going well. However, it's more helpful to learn about problems while one can address them rather than encounter them on the end-of-the course ques- tionnaire. 3. Ordinarily, instructors must rely on subjective impres- sions as to what does and doesn't work. Our lab experiences suggest that these impressions are often untrustworthy, and that they tell almost nothing about variations in individual students' or groups of students' reactions. 4. Often student feedback has suggested "obvious" problems that weren't obvious to the instructor. For example: ùIn almost every class there were problems hearing or under- standing the instructor, reading the writing on the board, or seeing the visuals. ùAlmost always, students have wanted more examples and applications to illustrate concepts. ùStudents were impatient with fellow students who try to dominate class discussions. ùIt was very hard for the instructor to judge whether the pace of the class is too fast or too slow, and casual stu- dent comments weren't a reliable guide. 5. The feedback questionnaires can probe into deeper prob- lems, such as students' understanding of basic ideas, motivation for course preparation, or reaction to outside readings. We've been surprised to find a common student tendency to skip readings that have no potential impact on the course grade. 6. Probing into these deeper problems, however, requires the instructor's intense involvement in the feedback process: s/he must provide reverse feedback. This can be oral, written, or both. It can take the form of course modifications, answers to specific questions, elaboration of obscure points, clarification of the grading system, fuller comments on student papers or cases, additional references, or outside speakers. 7. The processes of feedback and reverse feedback tend to draw students and instructors together to improve the learning experience. An instructor's written reverse feedback can explain points singled out by the fast-feedback questionnaires, and even answer specific questions asked on the questionnaires. Reverse feedback can require substantial time and effort of the instruc- tor, but the payback in avoidance of rework is great. 8. Regarding reverse feedback: students want instructors to provide feedback, preferably fast, not only on the question- naires, but on all work they hand in. Students aren't happy with a grade on a written assignment that doesn't include comments. 9. Course ground rules should be made explicit: students should understand what's expected of them and what the instructor expects to provide. It's appropriate to discuss the ground rules, and possibly modify them with the aim of a mutual understanding-a course "contract." 10. Instructors should devote some time to "marketing" their courses, including the outside readings, both in advance and during the course. 11. The fast-feedback questionnaire can discover how stu- dents are actually using their study time. Instructors can then use this data to help improve students' study efforts. 12. Another useful TQM aid for students may be the personal quality checklist, developed at AT&T. This simple tool applies TQM to personal work processes, and is adaptable to student work processes. 13. There should be some structured instruction, even in courses where faculty are primarily coaches and facilitators, such as laboratory courses. Experiences and Suggestions In addition to facilitating and coaching students in the lab, we've also applied some of the lessons to our own teaching in statistics and quality management. Observing what we've learned from the lab, we've made some immediate changes in our own courses. We now: ù Put a copy of the course syllabus and a short background questionnaire into student mail folders before the first class meeting. ù Reduce and focus the outside readings. ù Provide a clear idea of what each reading should accom- plish, rather than just the general feeling that it will be interesting or good for the students. ù Try to "sell" the readings. ù Use short fast-feedback questionnaires. All these steps have proved to be helpful, but the last one was the most important. We make no claim for novelty, because we know of many uses of feedback questionnaires that pre-date ours. The lab however, helped us to develop a systematic approach that others may find useful. The questionnaire's goal is to get systematic feedback after every class meeting, analyze the results, and make appropriate adjustments. Even without formal questionnaires, instructors can actively seek out feedback on their own. The time-honored approach to assessing one's work is to evaluate quizzes, problems, and other assignments. Quizzes and problems, however, often tap only limited aspects of learning; in particular, it's often hard to tell how well students can make the connection from abstract ideas and highly simplified examples to real-world applications. Since we teach mainly in the statistics area, we assign projects that require real-world application of statistical tools. Progress reports on these projects provide excellent feed- back, especially on pervasive misunderstandings of statistical ideas that students have somehow acquired before reaching our courses. In the spirit of the lab approach, however, instructors can go further. They can ask questions in class, ask for a show of hands on student experiences or problems, institute a suggestion system, and administer--often to only a few students--very short questionnaires, such as Mosteller's famous, "What was the muddi- est point in this lecture?" Course Strategy The usefulness of feedback tools stems from the fact that students know when they can't see or hear or are confused or unclear about content, and can tell the instructor when a par- ticular topic seems irrelevant to their interests. Ideas for course strategy improvement, by contrast, must come from the instructor--from an improved understanding of the subject matter and its connection with other topics. For example: ù Which topics are essential, which can be left out or de-emphasized? ù How can we better exploit what students already know? ù What new topics are needed to keep the course up-to-date? ù Are there simpler and better frameworks for understanding the subject matter? Can one general idea unify several specific ideas, which can then be seen as special cases of the general idea? ù Can process mapping and flowcharting be used to improve course strategy? TQM can contribute to course strategy. For example, TQM's insistence on continual and substantial improvement is essential to combat the tendency toward simply accepting the slow evolution of textbooks and courses. TQM encourages instructors to widen their horizons beyond minor issues, such as, "Should we teach the median before we teach the mean?" TQM tools such as benchmarking, brainstorming, and focus groups can bring out new opportunities in course strategy and in curriculum design. For more information, contact: George Bateman or Harry Roberts, Graduate School of Business, U. of Chicago, 1101 East 58th St., Chicago, IL 60637; Ph: 312/702-7301; Fax: 312/702-0458. [For more information or to subscribe, contact: TQM in Higher Education Magna Publications, Inc. 2718 Dryden Drive Madison, WI 53704-3086 Phone: 608-246-3580 or 800-433-0499]