[The following article appears in the January 1995 edition of
_Public_Sector_Quality_Report_, pages 2 and 3.]

           COAST GUARD CHARTS COURSE TOWARD EFFECTIVE TRAINING

     Does your organization hope its employee training efforts
are effective, or do you take proactive steps to make certain
it's so?
     Last year the U.S. Coast Guard Institute (the organization's
primary, non-residential training school) completed in-house
design and development of an automated Training Effectiveness
Measurement Program (TEMP). TEMP lets the Coast Guard measure the
effectiveness of training programs using surveys of both students
and instructors.
     Using standard computer hardware and a software platform
called Progress, TEMP automatically generates two computer-
scannable bar-coded survey forms anywhere from 180 days to a year
after a Coast Guard employee graduates from any of the hundreds
of resident training courses (occupational skills, computer,
quality tools, etc.) offered at various training sites.
     Survey questions are determined by the individual training
sites. Significantly, survey forms and return envelopes are
mailed both to the student and the student's supervisor, which
provides two views on whether training is carrying over to the
workaday world.
     Bar coding lets TEMP track each survey form by location,
course and instructor. As completed surveys are returned they are
scanned into the TEMP system, results are compiled, and that data
is returned to the school in question. TEMP data thus becomes the
customer feedback -a "report card," if you will-by which training
facilities and individual instructors can gauge effectiveness and
identify opportunities for improvement.
     For example, a question on one survey asked graduates
whether they understood how to transfer computerized data from a
hard disk to a floppy disk. Although a segment of the training
class dealt with this technique, student responses consistently
showed that classroom training was not carrying over to real-life
practice. Armed with that feedback, the training officer and the
instructor were able to take steps to improve the course.
     In addition to course-specific data, the Institute also will
use TEMP to conduct a six-month, system-wide training
effectiveness survey based on the SERVQUAL "service gap" model.
     Commander Bill Baker, head of the Institute, summarizes the
concept behind a SERVQUAL survey this way: Ask customers 22 pairs
of questions designed to reveal what they expect from a
world-class product or service, and how the service/product they
currently get compares. Responses to the questions serve to
illustrate where critical "service gaps" exist.
     Baker says automation is essential to making the survey
process practical. He says the Institute distributed, scored and
reported the results of 10,000 surveys last year, its first year
with the TEMP system, with no additional staffing.     Because
TEMP programming and system design was done in-house, using
existing computer equipment, Baker says the only real incremental
cost of the survey process is for distribution: postage, survey
forms and envelopes.
     Technology aside, Baker says he likes the TEMP survey
methodology for other reasons:
     * It's private. and therefore non-punitive. Class- and
instructor-specific effectiveness data gathered from a particular
training site is provided only to that school's leadership. Only
system-wide SERVQUAL data (which does not identify individual
schools, classes or trainers) is handed up through the
organization to the Coast Guard's training chief.
     "If you take this report card up the chain, as you might in
a smokestack, hierarchical organization, what's the 
reason for a school to participate? The feedback is given only to
school chiefs. They can use the feedback how they) want."
     * It's (relatively) unbiased. Baker says a more typical
survey scenario would have the instructor distributing feedback
forms to students upon completion of the course, or just prior to
receiving a final grade. He says having a third party (the
Institute) conduct the surveys encourages more candid,
unvarnished responses. Also, the interim between completion of a
course and the survey allows time to see if training has carried
over into the workplace.
     * It's customized. "A school can come to us with any set of
questions and we can modify the program for each customer," Baker
says. "If you buy something off the shelf, you sometimes have to
make them comply with the program."
     Is the TEMP program available for sale or "borrow" to other
government organizations? Baker replies: "Any good computer
programmers can sit down and write you a program. I think the
idea is what needs to be sold. The idea of assessing students
requirements and expectations for training, finding out through
surveys what we're giving them, and then closing that gap."
______________________________________________________________

     When the U.S. Coast Guard Institute set about developing a
system for measuring training effectiveness, they did lots of
reading about customer service. The "Bible," according to
Commander Bill Baker?
     Delivering Quality Service: Balancing Customer Perceptions
and Expectations, Zeithamal, Parasuraman and Berry, 1990, The
Free Press.
     The authors are credited with developing the SERVQUAL model
for spotting "service gaps."
________________________________________________________________

CONTACT: Cmdr. Bill Baker, U.S. Coast Guard Institute, Oklahoma
City, OK, (405) 954-7232, uscg-mmac @mmacmail jccbi.gov.

[For further information about PSQR or to subscribe, contact:
Public Sector Quality Report
17733 Kingsway Path
Lakeville, MN 55044-5209
Phone: (612) 898-5058
Fax:   (612) 892-7710
e-mail: 74363.3644@compuserve.com]