[The following series of articles comes from the _Public_Sector _Quality_Report_, April and May 1995 editions. The editor, Vice Giorgi, planned the series as an inter-related set of complementary items on bechmarking in government. This file contains the complete set.] BENCHMARKING SEARCH FOR BEST PRACTICES IS KEY IMPROVEMENT STRATEGY Remember during elementary school exams, how your teacher marched sentinel-like between the rows of desks, ever on guard against you "borrowing " from your neighbor's paper? It turns out, in the "real" world, "borrowing" from a more-knowledgeable neighbor can be good Borrowing is acceptable. Borrowing works. In fact, "borrowing" is a key strategy in the continuous improvement efforts of many organizations, both corporate and government Only working adults don't call it borrowing. We call it "benchmarking. " The process of comparing our organizational practices and processes with those of others, to discover ways we can learn and improve. Benchmarking. It's one of those improvement strategies which causes some people to claim that quality management is nothing more than systematic common sense. After all, what could be more obvious than to try and better your personal and organizational performance by borrowing good ideas and proven techniques from other, successful individuals and organizations? Of course, truly effective benchmarking isn't quite so straightforward, as anyone who's attempted it can probably relate. It turns out your teacher was right in at least one respect: Simply "borrowing" an answer doesn't mean you've solved the problem. Most benchmarking experts and practitioners agree that what's far more critical is understanding how the answer was arrived at, and then carefully working through the problem- solving methodology on your own. Only then will you have the correct answer for your organization. That's probably what W. Edwards Deming was getting at in Out of the Crisis, when he cited the "search for examples" as a common obstacle to effective adoption of quality. In hopes of furthering your organization's benchmarking efforts, the next few issues of PSQR will look more closely at several government benchmarking initiatives, relate some "dos and don'ts" from experienced practitioners, and point you to resources that can be useful as you build and refine your own benchmarking methods. ****** PORTLAND BASES BENCHMARKING ON ANNUAL SEA REPORT In the late 1980s, when the Governmental Accounting Standards Board first began floating the idea of requiring state and local governments to regularly report "service performance" information as well as financial data, the city of Portland, OR, decided to get ahead of the curve. Since 1991 the city auditor's staff has been compiling an annual Service Efforts and Accomplishments (SEA) Report, a document which captures spending, workload and performance/results measures on the city's six major public services: fire and emergency services, police, parks and recreation, transportation, environmental services and water. The report includes the findings of an annual citizen satisfaction survey mailed to a random sampling of 10,000 residents, segmented among Portland's seven major neighborhoods. In addition, the report presents comparison data on the six public service areas from six cities of comparable size -benchmarking "partners" in a sense, though Portland's comparison efforts to date largely have been unilateral. Those six cities are: Charlotte, NC; Cincinnati, OH; Denver, CO; Kansas City, MO; Sacramento, CA; and Seattle, WA. Dick Tracy, Portland's director of audits, says the report has several uses within city hall. For starters, it's popular with elected officials as a tool for reviewing budget requests in light of performance by various bureaus within the city. In addition, Mayor Vera Katz quotes often from the report in her annual "state of the city" speech. Perhaps most valuable, however, is the information the report provides on how neighborhoods within the city compare based on service levels received and citizen satisfaction expressed. In other words, the report helps Portland officials do benchmarking, on a neighborhood by neighborhood basis, to spot service and customer satisfaction problems and take corrective action. For example, in the city's eastern sections, residents consistently were less satisfied than in other areas with parks and recreation services. Parks and rec staff looked more closely and found that the trouble stemmed, in part, from the relatively few number of parks in the area, the age of certain parks, and problems with maintenance. The data helped city officials make capital spending and maintenance decisions to gradually bring those neighborhoods more in line with others. Similarly, city officials are monitoring whether residents know the name of their neighborhood police officer. Tracy says name recognition is one measure of the effectiveness of community policing programs. In neighborhoods where familiarity lags, he says, "that's an indication we need different kinds of programs, or perhaps better training." If you're still struggling with defining performance measures in your local jurisdiction, Portland's report is rich with metrics you might consider tracking. In fire services, for example, the city measures fires per 1,000 residents, lives lost per 1,000 residents, total fire loss (in dollars) per capita, structural loss as a percentage of value of property exposed to fire, and percentage of travel times within four minutes. "We've actually found that benchmarking works best when we do it internally," Tracy notes. "We compare ourselves to ourselves over time, and compare ourselves within our jurisdiction, as to how that crew is doing against this crew or this neighborhood against that one. Intra-neighborhood comparisons have been really useful for all our bureaus." Tracy says inter-city comparisons also are helpful in pinpointing service areas where Portland either performs quite well, or quite poorly, in relation to the six comparable cities. Over the years, for instance, Portland's wastewater treatment costs have grown out of proportion to other cities. City officials examined that trend and elected to make changes in capital improvement plans and tighten operating cost controls. Tracy notes, however, that truly comparable data from other cities has been difficult to gather, and requires a fair amount of interpretation, based on operational and philosophical differences from city to city. For example, one city might rely heavily on community policing, while another favors a more traditional, patrol car strategy. He says inputs and outputs can vary widely in those two instances, making meaningful benchmarking somewhat problematic. "We encourage our city officials to pay attention when we are really high, or really low, compared to other cities," Tracy says. "That should key us and make us want to understand why." Assembling an SEA report is not without cost and effort. Tracy estimates the audit staff spends about 1,800 hours each year (more when the report was first developed) gathering and compiling data, at an average cost of $50 or more per hour. In addition, the customer satisfaction survey costs about $20,000 each year. But Portland's SEA reporting process apparently makes sense to others. Chuck Schwabe, a senior assistant city manager in Coral Springs (FL), reports that his city recently completed its first SEA report, borrowing directly from the model established by Portland. CONTACT- Richard Tracy, director of audits, city of Portland, OR, (503) 823-4005. Chuck Schwabe, senior assistant city manager, Coral Springs, FL, (305) 344-1004. IG DEVELOPS LARGE-SCALE PERFORMANCE REPORTING SYSTEM Identify a few key performance measures for 43 local government services, from animal control to recycling. Now dump those measures into a spreadsheet program, and plug in actual input, output, efficiency and effectiveness data gathered from approximately 150 local governments within the United States. Next, use the spreadsheet to calculate high performers, low performers, average performance within each service area, etc. Then, once each year, give each of the 150 participants a computer disk which lets them study, sort and manipulate the data. Let them compare their performances to jurisdictions of comparable population, budget and staff size, whether within the same geographic region or from across the nation. Do all that, and what you'll have is a potentially powerful benchmarking tool. And, in fact, that's exactly what Florida- based The Innovation Groups (IG) has spent the past year developing. IG calls its system Standard Reporting of Performance Measures (SRPM), and after a year in operation participants are just now getting a feel for the kinds of things they can learn and do with the information. Mark Glover, IG vice president, says few if any SRPM participants are ready to engage in a full-fledged search for best practices based on the data. He says some simply are using the data to add a touch of performance-based measurement to otherwise standard budget documents. Others use the data in accountability reports distributed to the public. Still others incorporate the data into their budget review processes, weighing funding requests and departmental performance against similar data from comparable jurisdictions. But ultimately, Glover says, "my interest is in trying to identify some better and best practices. SRPM is the front-end of a good benchmarking tool. I think it can be used for that, and we're certainly encouraging participants to try and do that." In a few cases, Glover has been called upon to help an SRPM participant sort through the data, focusing on likely benchmarking partners in the areas of water treatment and building maintenance, laying the groundwork for some one-to-one benchmarking visits. Glover also is noticing interest among participants and other government quality practitioners in creating function -specific, more-detailed versions of SRPM on, for example, fire services or police. Glover expects the number and specificity of performance measures included in SRPM will increase as participants grow accustomed to assembling, reporting and using the data. IG will begin gathering data for its fiscal year 1995 SRPM in June. Local government organizations need not be IG members to participate, although members enjoy discounted fees ($500 per year, versus $995 for non-members). CONTACT: Mark Glover, vice president, The Innovation Groups, Tampa, FL, (813) 622-8484. SELECTED BENCHMARKING RESOURCES: Organizations * American Productivity & Quality Center operates the International Benchmarking Clearinghouse (IBC), with databases of outstanding practices, a list of benchmarking contacts, documents (including surveys and process maps), and other resources. For a fee, IBC staffers conduct customized information searches of their databases. To spread the costs of a major benchmarking study, IBC does "consortium studies" on behalf of multiple organizations with common interests. In addition, APQC conducts seminars and on-site training in benchmarking and other quality-related subjects. IBC charges a one-time initiation (sliding scale based on number of employees) plus annual dues. CONTACT: APQC, International Benchmarking Clearinghouse, (800) 776-9676 or (713) 6814020. * The Benchmarking Exchange (TBE), an electronic information system for individuals and organizations doing benchmarking. Users can search a menu-driven database filled with thousands of best practice citations, or "post" benchmarking requests via e-mail. TBE users pay a flat-rate subscription- 3-month ($255) or 12-month ($595)-with no additional fees or connect charges. Users need a computer, a modem, a telephone line or Telnet access to the Internet. TBE has a free, 30-minute on-line "tour" for prospective subscribers. CONTACT: The Benchmarking Exchange, Inc., Aptos, CA, (800) 662-9801 or (408) 662-9800. Books * Benchmarking: The Search for Industry Best Practices That Lead to Superior Performance, Robert C. Camp, ASQC Quality Press, 1989. * Benchmarking: A Practitioner's Guide for Becoming and Staying Best of the Best, Gerald J. Balm, QPMA Press, 1992, (708) 619-2909 * The Benchmarking Book, Michael J. Spendolini, AMACOM Press, 1992. ______________________________________________________________ Editor's Note: Benchmarking--comparing your organization's practices and processes with others, to discover ways you can learn and improve--has become a key strategy in the continuous improvement efforts of many organizations. In hopes of furthering your benchmarking efforts, last month's and this issue of PSQR take a closer look at government benchmarking initiatives, along with some "dos and don'ts from experienced benchmarking practitioners. ****** NPR BENCHMARKS BEST IN TELEPHONE SERVICE The National Performance Review, coordinating the joint efforts of ten U.S. federal government agencies, recently conducted a benchmarking study best practices in telephone customer service. If you sometimes wonder if your customers are more likely to get a busy signal or the run-around rather than prompt, responsive service when they call your organization, you'll want to read this report. Why phone service, you might ask? Three reasons, according to the report. First, there's the phone's critical role in connecting U.S. citizens with their government. For example, on a single day the Social Security Administration (SSA) can receive as many as 1.7 million phone calls. In 1994, the Internal Revenue Service (IRS) fielded 68.7 million calls, an average of more than a quarter million each business day. Second, the telephone is a cost-effective mechanism for delivering services and information, so it's likely to remain an increasingly important element of the government service-delivery equation. Third, there's plenty of room for improvement. The report notes that world-class organizations answer their phones in less than 15-20 seconds, and work hard to eliminate busy signals and abandoned calls. By contrast, the report relates the experience of newspaper columnist Ann Landers, who once spent 45 minutes on the phone before connecting with a "live, warm body" at the Immigration and Naturalization Service (INS). Landers then wrote about her unhappy experience for millions of people to read. With "teleservice" identified as the service area they wished to examine, team members conducted research studies to identify "world class" phone service operations, which would in turn became potential benchmarking partners. Team members developed questions for those partners, then conducted telephone interviews and site visits with senior managers and front-line customer service agents alike. In broad terms, the team found several common characteristics among the benchmarking partners, including: * Strong organizational cultures focused on not just satisfying customers, but delighting them. * A commitment of resources, including hiring and training, necessary to meet ambitious customer service standards. * Open communication between senior managers and employees. * Employees involved in all aspects of planning the phone center operation, and empowered to handle calls to completion at the first contact. On a more detailed level, the benchmarking team identified dozens of telephone service "best practices" in each of the following organizational areas: leadership, information management/analysis, planning, human resource management, business results, and customer focus/satisfaction. For example, in the realm of human resource management, the team found that most high-performing organizations take extra pains to solicit service improvement suggestions (including implementing electronic, on-line suggestion systems) from front-line operators, the people interacting with customers on a virtual minute-by-minute basis each day. In addition, "coaches," not "cops," monitor 5-10 calls per front-line worker each month, providing instant, improvement-oriented feedback and identifying training needs. Internal best practices and service improvement reports are compiled and distributed to all front-line workers. Front-line workers are given ownership of each call, and as a result first-call resolution rates of 85 percent or greater are not uncommon. The report notes that participating federal agencies now are using the study's findings to assess their own telephone service levels, identify performance gaps, then take steps to close those gaps based on the best practices established by benchmarking partners. ______________________________________________________________ You Rang? Phone Service Measures Predictive of Customer Satisfaction * Average speed of answer: < 15 seconds * Abandoned call rate: < 2 percent * Busy signal rate: < I percent * Service level (total calls - busy/abandoned): 98 percent * First-call resolution: 85 percent (Excerpted from Serving the American Public: Best Practices in Telephone Service) ________________________________________________________________ Ten federal agencies/organizations had employees participating on the benchmarking study team, including the U.S. Department of Defense, SSA, Office of Personnel Management, INS, State, Environmental Protection Agency, Bureau of the Census, IRS, General Services Administration, and the American Federation of Government Employees. The eight private-sector benchmarking "partners" included American Express Travel Related Services, AT&T Universal Card Services, Bell Canada, Citibank, Duke Power Company, GE Answer Center, Saturn Corporation and USAA Insurance. For a copy of the report, Serving the American Public: Best Practices in Telephone Service, call the NPR offices at (202) 632-0150. To order the report via e-mail, the address is almanac@esusda.gov, with the one-line message: send npr benchmark report. CONTACT: Pamela Johnson, benchmarking study organizer, National Performance Review, 750 17th Street NW, Washington, DC 20006, (202) 632-0150, ext. 151. BENCHMARKING BUNGLED: 10 MISTAKES TO AVOID The January 1995 issue of Quality Progress, the American Society for Quality Control's member magazine, included a concise, thought-provoking article entitled "The 10 Pitfalls of Benchmarking" by Irving DeToro, a Rochester, NY, benchmarking consultant and trainer. Following is quick review of DeToro's 10 "most unwanted" benchmarking blunders. If you'd like to add DeToro's article to your file of benchmarking ideas and information, photocopies of Quality Progress articles are available from the ASQC Quality Information Center, (800)248-1946 or (414)272-8575. 1. Lack of sponsorship. 2. Selecting the wrong people for the team. 3. Teams not fully understanding their own work. 4. Teams taking on too much. 5. Managers failing to understand the necessary commitment. 6. Focusing on metrics rather than processes. 7. Not positioning benchmarking within a larger strategy. 8. Misunderstanding the organization's mission, goals and objectives. 9. Assuming every project requires a site visit. 10. Failure to inspect benchmarking. CONSULTANT'S CORNER BENCHMARKING POINTERS: DISCOVERING THE "HOW" PSQR recently invited management consultant Peter Griffin to share some thoughts on how, and how not, to benchmark effectively. Griffin, of United Kingdom-based P Griffin & Associates, specializes in helping clients achieve quality improvement. Benchmarking is one of the most effective means to identify improvements which can make a significant difference in your organization. Benchmarking is not new. We all perform it to some extent every day, and never give it a second thought, let alone spend time describing it as benchmarking. For example, we may realize that our colleague has found a quicker route to work. The majority of us will be keen to learn how this improvement has been achieved, so we ask. We get the maps out and clarify exactly the route taken, and implement the action to achieve the improvement...until an even better route is identified. Improvement is a never ending journey. What can be so difficult about examining how other organizations have achieved improved performance? The answer is nothing, but "examining others" is a world away from really learning "how" they achieved the improvement. How do you identify a suitable activity to benchmark? As with all improvement activities, it is better to start with a known problem area that is able to be defined, or an activity where improvement will provide maximum benefit (the 80/20 rule works everywhere). Once the subject activity is identified, spend as much time as possible defining it. The more clearly you know your problem, the easier it will be to identify differences that will lead to major improvement. How do we gather the information? 1. Develop a questionnaire with all the information you want to obtain. Remember to phrase questions to gain maximum comparative information (as you will want to compare the outcome from a number of partners). Absolute values are meaningless without all the supporting details. Ask open-ended questions-How? Where? When? Who? What?-and allow for scaled answers (very important, important, not important). Review the questionnaire with a team of others to identify ways to improve it. 2. Complete the questionnaire for your own organization. This is a good test of the questions and also ensures that you can respond to similar requests for information from your benchmarking partners. 3. Write down your reasons for asking each question. Again, this will test if all the questions are really necessary, and will provide you with a ready-made answer when the partner asks, "Why do you want to know that?" 4. Talk to the benchmarking partner. Describe and clarify your areas of interest, objectives and primary questions. Ask if these areas have been covered before (the information you seek might already be available). 5. Arrange a visit (eyes are as useful as ears-and they double your chances of learning). How should a visit be performed? Prepare thoroughly. Find out as much as possible about the benchmarking partner from other sources (publications, consultants, trade associations, etc.) so that you can maximize knowledge gained from direct contact with the partner. Use a small team (two people is ideal) and nominate one as the leader. Thank the partner (again and again). Debrief as soon as possible after the visit. What to do with the information gained? Use it to compare the similarities and differences in your two organizations and activities, in order to clearly identify improvement opportunities. Share the knowledge with all interested parties in your own organization, and then take the improvement action! Some final points: * Benchmarking alone will not tell you what customers actually want. If your product or service is obsolete, no amount of improvement in production processes will make it competitive. * Benchmarking is only of benefit if the improvement actions are implemented. * Always seek to find out how an organization has improved its performance. This information often comes from the front-line people, not the management, who often are more eager to tell you how much performance has improved, but not necessarily how. * The Golden Rule of Benchmarking: Be prepared to give a partner more information than you receive. * Remember, improvements are continuous and benchmarks go out of date quickly. Your partners' performance will probably continue to improve in advance of your own. Always remain honest and thoroughly professional, and you will be welcome to return for more information in the future. CONTACT: Peter Griffin, P Griffin & Associates, 0 1525 716767, or e-mail griffinp@dircon.co.uk JOURNAL FOCUSES ON SEA REPORTING In the April PSQR you learned how the city of Portland, OR, uses its annual Service Efforts and Accomplishment (SEA) Report to benchmark the performance of major city service departments, both internally (neighborhood by neighborhood) and externally (against comparable cities). If your organization is interested in exploring SEA tracking and reporting as an improvement tool, you'll want to get your hands on a recent double-issue (February/ March, Vol. 18, Numbers 2&3) of the International Journal of Public Administration (IJPA). It's a special issue devoted to research relevant to SEA reporting. The issue's editor is Jean Harris, a Penn State University professor and a frequent contributor of article ideas and organizational improvement-related information to PSQR. The issue includes research articles on: * Key SEA indicators for higher-education, in areas ranging from academics to athletics, drawn from a field study of 28 college presidents and 56 chief administrators. * Some thoughts on using SEA measures as a foundation for compensation/incentive plans. * A description of Texas' new planning and budgeting process, which combines strategic planning, SEA indicators/reporting, and line-item appropriations. * A study of the usefulness of SEA measurements in the decision-making surrounding road maintenance. * Eight unintended consequences of SEA reporting, consequences to be avoided in developing your own SEA methodology. * Performance reporting at the federal government level, based on the initial impact of the Chief Financial Officer Act of 1990. * One city--Boca Raton, FL--and its early experiences with SEA reporting. If your organization's internal reference library, or your local public library, doesn't stock the IJPA, individual copies of 18, Numbers 2&3 (Cost: $170, prepaid) can be ordered by writing or calling the contact below. CONTACT: Marcel Dekker Journals, P.O. Box 7 5017, Monticello, NY 127501-5176, (800) 228-1160. [For further information about PSQR or to subscribe, contact: Public Sector Quality Report 17733 Kingsway Path Lakeville, MN 55044-5209 Phone: (612) 898-5058 Fax: (612) 892-7710 e-mail: 74363.3644@compuserve.com]