Re: Deming Was Wrong/Constantine

[John Constantine <rainbird@trail.com>]

Once more, into the breach... While Ed was responding a la "Deming Was
Wrong" response, it seems to be dealing with Performance Evaluations.
So...off we go.

Ed suggested:

> The recent postings on performace appraisal systems and their relative evils
> and/or necessity in different types of organizations suggests at least some
> straight forward hypotheses that may be worth testing:  within comparable
> organization types, certain kinds of perfromance appraisal systems are better
> than others.  Clearly organizational types i.e. structured around
> subordinate/superior , team based organizational structures, and other
> hybrids - each of which could be reasonably operationalized - could be
> conceptualized and examples of each identified.  Similarly, the effects,
> desired and otherwise, of performance appraisal systems, could be cataloged
> and quantified.

This is what a perfectly rational, sensible person would ask who is
looking for some reasonable answers to his hypothesis. And later, he
eloquently points out one of the defects in our culture:

> I know that I, and most other Americans have been exposed to
> such organizational practices - performance appraisal - beginning in school -
> (hello Myron, and Mr. Kohn) - and I still know much less about the empirical
> bases for numerous generalizations that could be spun from my own experience.

He asks for the evidence in this matter, as to "this" vs "that" type of
organization, and whether the measurement/evaluation/appraisal process
is "good" or "bad".

For the record, in response to his and others' inquiries on the same
subject, such a process as performance measurement/evaluation/appraisal
requires at least three things, as put forth and reinforced in
organizations which claim to utilize Management be Results, or
Management by Objectives:

	1. Operational Definitions
	2. "Data"
	3. Control
Operational Definition - one which gives communicable meaning to a
concept by specifying how it is measured and applied within a particular
set of circumstances. It a. gives precise meaning to the spoken or
written word, thereby forming a common language between two or more
persons; and, b. defines how it is to be used when applied in a specific
context. ("Fast" in situation x may be entirely different than in
situation y.)

Data - in the context of performance appraisals, in order that it be
meaningful deals with measurement of some kind. This is often "pulled
out" of the subject person as part of the required ritual, of stating
what the person "commits to" prior to signing the form at the beginning
of the evaluation period. (I've listed an excellent URL which offers
many different "measuring sticks" by which a person can be evaluated. It
is: http://www.llnl.gov/PBM/handbook/Appendix-C.html. *I'll withhold
comment about these until later in the post.) As can be seen from these,
and others from your own experience and perhaps even your own design,
these are numerically based, able to be counted and used as evidence to
support the finding of "good" or "bad", "acceptable" or "unacceptable"
performance. If the measuring stick is set to 12 and the person only
achieves ten, the finding is in the negative.

If the "data" cannot be counted, there must be some other format which
allows for a judgement to be made. A written report, memo from an
associate, or a host of alternatives allow for the person to be
evaluated, or appraised, despite the lack of data which can be counted
and amassed by period. Most, if not all, periods seem (by coincidence
perhaps?) to fall within a budgetary cycle, either quarterly,
semi-annually, or annually.

Control - the person being measured/evaluated/appraised is responsible
for actions which generate the data upon which that person will be
judged; OR, the evaluator BELIEVES that the person is responsible for
those actions.

Thus, Pete has met with his supervisor last year, at which time the
supervisor asked Pete to identify his "performance goals" for the
period. Pete obliges, using those measures provided to him by his
supervisor as what are "good", based on their discussions. Pete has to
deal with lowering the "Percent of errors in cost estimates" as his
goal. Pete goes through the year and shows up for his performance
evaluation, but unfortunately, the Percent of errors did not go down,
but in fact went up! Pete gets a negative evaluation, and perhaps misses
out on his lifelong career goal, to be able to attend post-grad training
in his field of research, which required consistently superior ratings.

Back to the beginning, the above are "real-time", not hypothetical, not
theoretical, and not anecdotal. 

They are "bad", in the sense that they are dishonest. Such systems as
MBR and MBO are promoted as "good" in that they provide a focus for
looking forward, and a means of identifying what was stated to have been
performance which was lacking, as defined by the performance goals of
the individual workers themselves, particularly when they were not
achieved. In this sense, they are fraudulent, dishonest and misleading.

The person needs to know what he/she is being held accountable for, and
whether or not that has a snowball's chance in hell of being "counted",
or whether it simply will come down to a supervisor's opinion, hopefully
not on a bad hair day. He/she has a right to know what might impact on
the evaluation.

All this is under the old, or current, model for performace evaluations.
It is perhaps based on the supposition that someone must watch over what
is being done by the workforce. As Ed pointed out, we've been
indoctrinated by this line of thinking since we were young.

Operational definitions become meaningless as they become more defined,
due to the chances that there will be a likely error by the subject
person; e.g., "Will not produce typing errors in advertising
presentations other than once per year." As they are made broader in
scope, they also become meaningless; e.g., "Shall cut costs."

While Ed asks for his own "data" to confirm his suppositions, one flawed
effect of such a system is that that kind of information (what is the
effect of the use and abuse of performance evaluations in America?) is,
like much quality related information, unknown, and unknowable. We don't
know what we don't know about how much or how many or how often this has
produced negative ripple effects which produced effects years later.

*When looking at the Performance Measures listed on the linked site,
remember that they are identified as Performance-Based by this agency
(TRADE) and supported by YOUR tax money. Do the measures make sense, in
what circumstances, and does it say anything at all about what systems
or processes are involved? Think about it. As I said in another post
earlier, performance appraisals as they currently exist are a sin. (I'm
reflecting back to my youth and whey I say it's a sin, I simply mean it
is very, very bad. It has nothing whatsoever to do with such things as
Cardinal sins, mortal sins, venial sins, and the like. I apoligize if I
gave anyone the impression that I was making a moral judgement by so
doing.)

If you think you can create a Performance Evaluation system which does
not have serious flaws, why doesn't everyone give it a try, instead of
quoting Deming or anyone else. Be creative. It just takes three steps.

Thank you for all your intensity on this subject.

Regards, 
John Constantine
rainbird@trail.com
Rainbird Management Consulting
PO Box 23554
Santa Fe, NM 87502
http://www.trail.com/~rainbird
"Dealing in Essentials"

======================================================================