What is the Question? David and Sarah Kerridge Where education deals with problems at all, it shows how to solve them. Each examination question has a "right" answer. In the same way, many "TQM" programmes teach a systematic approach to problem solving - finding the "best" solution. It comes as a shock to discover that in the real world, the difficulty is not to solve the problem, but to find out what the problem is. The solution to a problem, or the answer to a question, are treated here as equivalent. As Dr Deming put it "The questions are more important that the answers". When we asked him why, he said that there is seldom one right answer, but there usually is one key question to ask. In one sense, the key question is always the same: "What is your aim?" Without this, no other question is meaningful. In another sense, the key question is the one which no one else has thought to ask. There may be no "right" question in an absolute sense. But improving the question is far more important than improving the answer. Years of answering questions made me see this for myself. Someone came with a question, such as "Have I used the right statistical method?". Experience showed that the more clearly the question was initially stated, the less likely it was to be the right question. At first, I usually made the mistake of answering the question that was asked. Curiosity led me to ask what was to be done with the answer. Often, I could see that the answer, though right for the question asked, was not what was really wanted. This made me cautious, and so I began to check routinely. Eventually I learned to clarify the question before I tried to answer it. This certainly changed things. A friend described my typical consultation as "You have done the wrong analysis on the wrong data, and you should be solving a different problem". This shows how often I thought I had discovered a wrong form- ulation: and in most cases, the client agreed. Some of those who consulted me found this questioning disturb- ing, and some were impatient. But most were interested, often enthusiastic. If nothing else, we had an interesting discussion on the problem itself, and the nature of research. Once or twice, as the real problem emerged, someone said "That's brilliant! How did you think of that?" Naturally, I smiled modestly and took the credit. But all I was doing was running through a check-list of the questions found fruitful in the past. Besides, it is far easier for an outsider to stand back from the problem, and see it in perspective. If you are applying what follows to your own problems, get the help, if you can, of someone interested in, but detached from, the problem. The management field is no different. The greatest waste comes not from inaccurate or inefficient solutions, but from slick and precise solutions to the wrong problem. So, instead of more and better problem-solving techniques, we need a systematic way to find better and more relevant questions. Profound Knowledge provides a very good framework for doing this. The difficulty is that managers are not used to a questioning approach. They are always under pressure to be seen to do something. This means that they will often press ahead with solutions to the wrong problem. In what follows, I try to explain what has worked for me. It may not work for anyone else. It is a first, woolly, attempt to formalise a system that has grown gradually and unconsciously. So it is bound to have mistakes, raw edges, and vagueness. What is more, according to the system explained below, it should, at this stage, have them. The idea that exactness can ever do harm comes as a shock. It goes against training, inclination, and perhaps against human nature. When we see a problem we are impatient to solve it. It is satisfying to follow a precise and logical problem-solving process, and exciting when "the right answer" is found. The logical process requires strict definitions, precise statements, and rigorous attention to detail. We then end up with a unique right answer. Obviously something must be wrong, because, as Dr Deming pointed out, there seldom is one right answer. Any method which produces one must be misleading. To see why this is so, consider an imaginary, but realistic, consultation. Suppose that we are asked to solve a quadratic equation. We solve it, and give the "right" answer, because a mathematical equation really does have a right answer. Then we ask "Where did that equation come from?" "The trajectory of a shell from a naval gun" we are told. Immediately we know that the answer can not be right, because we only get a quadratic equation by ignoring complications like air resistance, the earth's rotation, and so on. Perhaps, to the accuracy required, these things can be ignored. But strictly speaking, the "right" solution that we found is a solution to the wrong problem: wrong, that is, in the sense that it is a simplified, unrealistic version of the real problem. The important problem is now clear. It is not solving a quadratic equation, which is trivial. It is assessing how safe it is to use this oversimplified model. This relies more on judgment and experience than on theory. But then, we should ask, why are the figures wanted? What action depends on them? This determines the accuracy required. Yet again, is there another way to get the information that is needed? And so on... We have turned a trivial problem of calculation into a far more interesting investigation of aims and methods. And this problem in ballistics is a hundred times simpler than anything we will meet in management. So here is our first principle: 1 In the real world, only wrong questions have right answers. Any statement about the real world is imprecise. This does not matter so long as we know how imprecise it is. For example, in a Census, the population of a city is counted, giving an exact number of people. This exactness is artificial. UK census rules count everyone who was alive within the city boundaries at midnight on the census day, or who arrived there later, not having been counted anywhere else. Hardly anyone who asks for the population of the city wants the number those rules produce. It includes visitors, and leaves out anyone away from home on the census day. It uses a definition of the city boundary that is exact for electoral purposes, but is different from the postal district boundaries, which is how most people define where they live. Again, we have a right answer to what is, for most purposes, the wrong question. To give the population as 65,713, is in fact less accurate than to say "about 66,000". This expresses the degree of accuracy that will apply to nearly all uses of the figure. Here is our second principle: 2 A precise question, unless the precision is clearly related to a well defined use for the answer, is the wrong question. This sounds obvious, but we usually ignore it. Misleading precision is easy to see in the case of measurements, because we are suspicious of figures anyway. It is clearly wrong to measure a child's height with a tape-measure, and announce "The height is 4 feet, 5.3879 inches". But the same mistake is more often made with words and ideas, than with figures. Dr Deming frequently quoted the example of manufacturers who thought their business was to make carburetors, and so missed the idea of making fuel-injection systems. This is an example of misplaced precision. He said that they would have done better to have defined their business as getting a stoichiometric mixture of fuel and air into the engine. "Stoichiometric" means the mixture that has exactly the right balance for complete combustion, and so greatest chemical effic- iency. But could this too be misplaced precision? This definition excludes lean-burn engines, and other improvements, where the aim has changed from efficiency to environmental protection. This is covered by Dr Deming's warning that aims must continually be re-examined. The vague question "How should we get air and fuel into the engine?" would avoid misleading definiteness. In complete contrast, as we move from planning towards action, the need is for greater and greater precision, not vagueness. Operational Definitions enable us, and even compel us to achieve a level of precision previously undreamed of. But that comes later in the process. What we have to try to do is to introduce precision as a conscious act, at exactly the right moment: neither too soon, nor too late. This is like using a microscope. We should always start with the low-power magnification, so as to get the broad picture, before changing to the high-power, which concentrates on fine detail. We are unable to do both at once. What is precision for? We need precision for any statement that we are to test: a statement of logic, or fact. We can not test the statement "Some people are fairly tall". We also need precision to make sure that different people mean the same thing when they use the same words. Finally, absolute precision is essential before we turn thought into action. But in early stages, with different people contributing to a study, it may be helpful if the people involved do think differently. The gradual process of defining terms will reveal these differences, which may lead to creative new ideas. The statement of a problem can rarely be tested as strictly as a statement of fact. But we should test that each step in refining the statement decreases the differences of interpret- ation. Each step should represent a deliberate choice to focus attention in a particular way. It must not be imposed by an arbitrary choice of words. Often the way to formulate the question depends on values, and guesses about the future, where there may be important differences of opinion. The purpose of precision is to exclude alternatives. So our third principle is: 3 Unnecessary precision excludes alternatives which are better retained. When we think about a problem we bring to it all sorts of assumptions based on experience. It is easier if we have no experience at all, but that is rarely possible. An outside adviser may help through sheer ignorance. It is useful to collect examples of the problem, to see if our ideas cover every case, and if any new way of looking at the examples arises. As we progress through the problem-defining steps which follow, we can check back to see how the examples fit into what we are saying. Our aim should be to stop assumptions creeping in unawares. The most common way for them to do this is to wear the disguise of familiar language. All language makes assumptions, and the more precise the language, the more assumptions it makes. Perhaps too, for psychological reasons, we are less likely to question a statement that sounds precise. This may be why it has been so difficult to persuade statist- icians to look again at SPC. The "orthodox" theory is, in one way, perfect. It is based on a model of random variation, and rigorously derived distribution theory. All of it is mathemat- ically proved, and so "must be true". It is true, of course, so long as the axioms hold. "Every theorem is true in its own world", as Dr Deming put it. Unfortunately, in the real world, such axioms never hold, just as the simplified model in ball- istics is never true. But the apparent precision of the theory makes it hard to go back to first principles and start again. Then it would be easier to see that it answers the wrong question. If we can do without any particular assumption, we should do so, however harmless, or obvious, it seems. It may prevent us seeing things in a new way. If not, we must bring it clearly into the open, and question it, however time-wasting, disturbing and subversive this seems. It seems to be natural to start with too narrow a view, as well as to make too many assumptions. Before we define our aim, in solving any problem, we should look at the wider context. This means studying the various systems within which the problem arises. How does the aim of our subsystem contribute to the overall aim? Think deeply about this, and we are less likely to make the same mistake as the makers of carburetors. The wider context also includes the whole of Profound Know- ledge. Have we thoroughly considered the four aspects of Systems, Variation, Theory of Knowledge, and Psychology, separately and in combination? Most people enjoy this discussion of the wider problem, as it enables them to show how much more they know than the adviser. They are less happy to have familiar assumptions questioned, and this requires tact. But even the questioning can be made to seem like part of the natural curiosity of an outsider. 4 See every question in context. The real problem often lies "upstream". In formulating the problem, we have to take account of information about the world in which the problem arises. How much of this is based on inescapable facts of life, how much on theories we currently hold, and is any based, not on facts, but habits? Even habits are not necessarily bad. At least they make observations taken at different times comparable. But the main problem comes from theory, particularly if it is not recognised as such. We can not state a problem, or observe anything, without theory. This does not mean that theory totally limits what we can observe, or every question we can ask. If it did we would never learn anything new. We should not build into our thinking anything that might change, or that others will dispute. If there are several competing theories, can we use just the things that all theories agree on? Scientific theories, however widely accepted at one time, vary enormously in their factual support. Build up the picture slowly. Form the problem structure by adding in the most clearly established facts first, and then use the least amount of theory that is needed. There is another trap with using theory. Theories deal with prediction, but they are always approximate. A theory that is complicated enough to be true is too complicated to be useful. So what we think of as "the theory" is usually a simplified version, adequate for most purposes. But do our particular purposes need the same level of accuracy as usual? Often, in system problems, only some aspects of the components affect the behaviour of the whole system. We can predict the behaviour of a gas by visualising molecules as billiard balls. So we may be able to use an even simpler version of the theory, which almost certainly assumes less. Often this is the same as an older, "outdated" theory. Conversely, for our immediate purposes, is a more accurate version of the theory needed? It is rare to err in this direction, but it could happen. 5 Assume the minimum possible, and make sure that each necessary assumption is explicitly recognised as such. Look again at Dr Deming's statement of the System of Profound Knowledge, and see how very few theoretical assumptions he made. Everyone who first sees it is tempted to "fill in the gaps". Of course there is a great deal more to Psychology than the fact that people are different, and learn in different ways, and that motivation may be either intrinsic or extrinsic. In a particular case, even to state the problem, we may need to assume far more than this. But so long as we can assume only this bare minimum, we should do so. Even while we define the terms more precisely, we should broaden our vision. An important way is to look at analogous problems, however strained the analogy may seem. Another is to see if we are narrowing the problem unnecessarily. It is some- times easier to state and solve a more general problem than a special one, because we are thrown back to basic principles rather than the particular circumstances of the case. We can use analogies from anywhere: from medicine, or biology, or any subject whatever in which a problem that is even vaguely similar arises. These analogies prove nothing. People are not rats, but studies of laboratory rats may still make us notice things about people that were too familiar to be seen. The purpose of an analogy is to make us ask questions. 6 Relate the question to similar questions in a different context. If all these rules are strictly followed, so many questions will be asked that nothing ever gets done. We have to use judgement, to select the most productive questions. Some, about the wider system, and about the aim, are always necessary. Look for suspiciously precise statements, or assumptions unsupported by evidence. As a general rule, we can assume that some questions will already have been asked. It is the unfashionable and uncomfort- able questions that are least likely to have been asked already. Surprisingly, the question "How strong is the evidence for that?" has often not been asked. Again, look for the things that everyone wants to believe, and question them. This is another reason why help from an outsider is so important. When the dangers of precision first occurred to me, I immedi- ately cornered the nearest philosopher. Then, like the Ancient Mariner, I fixed him with my glittering eye, and compelled him to listen. All he said was "Ah. Wittgenstein wrote a whole book about that". Well, you can't always be the first. It is not even the first time that an abstract philosophical idea has proved of practical importance. Operational meaning is a good illustration. The danger of emphasising that imprecision has a place, is that this could sound like a licence to be imprecise all the time. In the same way, some people grasp the importance of the "unmeasured and unmeasurable" and then think that nothing should be measured at all. This is quite wrong. We want the maximum precision achievable, but approached gradually, systematically, and persistently. At the right time we must be obsessively precise: this is a more rigorous approach, not less, whatever it looks like. A final principle sounds too good to be true, but experience supports it: 7 The wrong question is usually far harder to answer than the right question. If a problem is difficult, look for an easier one. Perhaps this is because most problems are simple: but only difficult ones are analysed in such detail. So any other problem is usually easier to solve. Please remember, once again, that this is a first attempt to formalise a system which I have used informally and unconsciously for many years. I expect I have left out important steps, because I take them without thinking. Think about the system, try it, and help me fill in the gaps.