by David S. Olton, Ph.D.
Dr. Olton was Professor of Psychology at The Johns Hopkins University and an associate of Milton Cole for many years. He died of pancreatic cancer in 1994 but left this advice to junior faculty and students as only a portion of a very full academic and scholarly legacy.
A proposal is like a long chain which must be entirely intact in order to obtain the desired goal. Consequently, my own strategy in reviewing a proposal is to give a gentle tug on the whole chain and find out what is the weakest link. I review the proposal briefly throughout its extent, and hunt for what I perceive might be a “fatal flaw.” Then I pull very strongly on that particular link in the chain. If it breaks, the proposal is doomed, irrespective of the value of the rest of the components. Thus, no aspect of the proposal can be ignored. Like a chain, a proposal is as strong as its weakest link.
If the chain survives a gentle tug, then I go back and review the proposal in detail. That most important point in the entire proposal has to do with the issue that is being addressed, and the way the results will be relevant to that issue. You should assume that your reviewer is intelligent, but absolutely naïve. In some cases, you may get a reviewer that can quickly scan through any of the points that are already familiar. If, however, you are in the more likely position of having a reviewer who knows very little about you area, you should start at the beginning and explain exactly why you are doing what you are doing in terms that are clear to an intelligent person.
Be brief, and be organized. Even though you may have spent hundreds of hours in preparation of this proposal, the reviewer is going to attempt to make a decision in as a short a time as possible. In the meeting itself, the fate of your proposal will probably be decided within fifteen minutes at the most. Everything you can do to help the review process become efficient will aid your cause. Some think that if a weak spot is present in an experiment, it can be protected by enough random words so that it won’t be detected. That strategy never works. You have the responsibility as the candidate to make your application clear. If it is not, the reviewer will assume the problem is yours and lower your priority scores appropriately.
In terms of organization, many headings help a great deal. Proposals that use very many headings and layout all the details clearly have received rave reviews when they address important issues well. In a manuscript, you can use at least four orders of heading without any trouble. Examples are listed below:
Emphasize the general issues, the rationale, and the conclusions that can be drawn from the experiment. Describing techniques, procedures, and general experimental designs can often be done easily. But the ultimate bottom line has to do with the new information that is obtained. Discuss the hypotheses that drive the experiments, the possible results that may be obtained, how these are interpreted, and how they relate to overall general issues.
Remember, everything that the reviewer knows about you is contained in the few pages that you submit as the proposal. Everything that is in those few pages will be used to make a judgment about you. If the physical aspect of the proposal is sloppy, the reviewer will interpret that sloppiness. Appropriate conclusions include the candidate being uninterested in the project so that it was just slapped together, that the candidate doesn’t think very clearly or take time to carry out experiments carefully, that the candidate is attempting to insult the reviewer, etc. None of these are very flattering. Be incredibly careful about the technical details of the proposal: grammar, style, the appearance of the type, the quality of the copies, the accurate filling out of forms, etc. Sloppiness should not be confused with creativity. The reviewer can tell the difference, and so should you.
Science in general is very cooperative enterprise. If you have been in an appropriate environment, you have developed very close relationships with your colleagues and depend on them for many different kinds of assistance. In the proposal process, however, you are in a competition. The funds available are always substantially less than the number of applicants. If you don’t care enough about the outcome to organize an effective campaign, and your competitor does, the results will be clear in the priority scores and in the funding outcome.
Performance and behavior are influenced by many different variables. One of them is the general intelligence of the individual doing the performing. You are probably well qualified on the basis of your competence to get funded. However, the reviewer judges you on the basis of performance. You should seek to take the competence that you have and produce the very best performance from it.
One of the best predictors of future behavior is past behavior. Consequently, the past record of the candidate is a very important piece of information that is used to indicate the likelihood that the work in the proposal will actually get carried out. Even the most brilliant proposal will suffer in terms of priority scores if the candidate has been unproductive or is unable to demonstrate the necessary skills and techniques have been learned. The issue is not so much absolute output, but productivity relative to the opportunities that have been available.
With respect to productivity, you should make clear what you have actually accomplished. Listing lots of articles in preparation will not help your case at all. The question is “what have you actually done?” Published articles and abstracts presented at talks are the best way to document output. Copies of written material should be submitted as an appendix whenever possible to document what you claim you have done.
You may have significant professional accomplishments that don’t necessarily appear as publications. If you do, be sure to list them and describe them. For example, if you spent six months in a laboratory learning a particular technique and that technique is relevant to your proposal, be sure the reviewer sees the information clearly. Get a letter from someone relevant certifying your skill.
Your training is also of importance, because it indicates the skills that you should have available. Indeed, the way in which you document your skills is by a statement of relevant publications and training experiences. Listing of information may be useful if the reviewer is personally familiar with the training in that list. However, rarely is such the case. Consequently, if you have had extensive training in an area, be sure to provide a paragraph describing the quality of that training.
For predoctoral or postdoctoral proposals, letters of recommendation are solicited by the committee. You should obviously provide the person who writes your letter with as much information as possible. At the very least, that person should have a copy of your vita and relevant publications. Preferably, you should also include a copy of your proposal itself. Additional information about the points with which you are concerned will help a great deal. If you let the recommender know particular points of strength or weakness that you wish to have addressed, the writer may be able to do a great deal of good.
Research plans are composed of another set of links in this long chain. As with the entire proposal, all the links must be intact. However, my experience says that the weakest link is almost inevitably the rationale for the experiment and the interpretation of the results that can be obtained. You are unlikely to be the only one carrying out research in your area. A reviewer will ask what you’re going to contribute that is new and that is not being done by anyone else. Data for the sake of data aren’t interesting. Data for the advancement of the scientific enterprise are interesting. You should be able to state clearly the general issue that you are addressing in the most general way, without reference to scientific jargon or previous experiments. The issue should be inherently interesting and solvable with an experimental approach.
Two steps can help to produce this integration. One begins with a set of hypothesis, and then considers the possible outcomes of the experiments. The other begins with the possible outcomes of experiments, and considers what these have to say about various issues. You should use both. Outcomes may be obtained that don’t support your hypothesis. Every reviewer wants to know what you will do if you don’t get your expected outcome. A set of contingencies should describe the direction you will pursue given any set of data. A flow chart is a lovely aid. You should at least use the flow chart in the construction of your research, and you may want to put it in the proposal itself.
The end of the proposal is also critical. The question at the end has to do with what you have learned as a result of the particular experiments. If you lay out the issue clearly in the introduction, you should be able to indicate in the final discussion how the results you expect to obtain are relevant to this issue. Again, a flow chart can be very helpful. You do an experiment; you get one of several alternative outcomes. Those particular outcomes should say something significant about the validity of one or more hypothesis.
Especially when dealing with complex, interrelated systems, this organization of hypotheses and results is absolutely critical. Parametric manipulations can be beautiful sometimes but again the issue has to do with the interpretation of the outcomes from various experiments.
The emphasis should be on positive solutions to interesting issues. Tearing down previous research is very easy, as is outlining the apparent contradictions or problems in a field. The inadequacy of previous work is not sufficient justification for your own. The justification for your own lies in the positive contributions that it will make, irrespective of how good the rest of the work has been done previously.
Details of the experiments are also very important. You should be able to indicate the number of subjects that are going to be used in experiments, the various control groups, and most important the data collection analysis. Your ability to sue each of the techniques required in experiment should also be document.
The timetable for the experiments is evaluated to determine if you have judged your goals realistically. The length of time that each experiment should take in both the best and worst of circumstances should be indicated. The integration of all of the experiments into the proposal should also be described so the reviewer knows how you are going to begin and what you will do during each period of support.
For predoctoral and postdoctoral applications, an individual sponsors the candidate. The quality of the sponsor and of the training resources are both evaluated. You should choose your sponsor and the training institution very carefully to insure high quality. This quality must also be apparent to the reviewer. Thus the sponsor should write a very clear statement saying why the sponsor is appropriate for you, and in that statement should indicate very clearly what you will be doing in the training. The facilities available, the courses, the timetable for your training, and many other aspects of the period should be documented. The sponsor’s statement should indicate what you will be doing during all the time that you’re requesting support. An important statement has to do with how this training is going to build on the skills that you already have, and develop you to become an independent researcher.
In addition to evaluating the sponsor and you as a candidate independently, the reviewers determine the extent to which you provide a good match. If your sponsor says that you’re going to spend the first six months in classes, and you say that you’re going to spend the first six months learning research techniques, the reviewer is going to question whether or not the two of you can work together effectively. You should see your sponsor’s statement and your sponsor should see your own statement, and they should be closely integrated. The integration can not be done at the last minute. Consequently, the two of you should work carefully before the proposal is due to make certain that you have covered all the appropriate details. You as the candidate should make clear to the reviewer that you know the environment in which you are going for training. You should know your sponsor’s work intimately, and also the facilities that you‘re going to use.
The best place to begin writing a proposal is put yourself in the reviewer’s place. As a reviewer, you have two goals: First, make a correct decision about funding. Second, review as quickly as possible with the minimal investment. Given these goals, your optimal strategy is to search for a fatal flaw. If you find one, you can kill the proposal on the basis of that piece of information alone. You will have made a correct decision and minimize the time and effort to make it.
A single fatal flow is sufficient to kill an entirely proposal. All of the components of a research project are lined up like links in a chain. In order for a chain to be effective in pulling an object, every single link in it must be good. A chain is as strong as its weakest link; a proposal is as strong as its weakest element. As a reviewer, think of all the links that you wish to see. Each one must be present and it must be forged correctly. A missing link, or a present but weak link, is all that is necessary to complete your job.
Once you have identified all the links that you wish to examine as a reviewer, and the criteria that you are going to use to determine their strength, you are ready to change roles and forge a chain that meets the requirements that you initially set up.
The three key elements of a proposal are: An idea, a plan, and proof of your ability to carry out the plan.
The idea must stand by itself. The fact that others have investigated a problem is not sufficient to justify that problem. They may have made a fundamental mistake in their work. Consequently, the idea must be justified on the basis of a logical analysis of the relevant issues, irrespective of the existing literature. (As indicated below, the existing literature should not be ignored. However, the fact that it exists is not sufficient in and of itself to support the merit of your idea.)
The idea may be justified in many different ways. It may be a new approach to a problem that has been significant but unsolved. It may be the finding of a new problem one that people had ignored in previous work. It may be a new perspective on a field, or an interrelationship of one field with another. Many different approaches may be taken.
The plan is the second component of the holy trinity. Even the best idea is of little use in a research project unless a systematic approach can be taken to answer the questions raised. The plan must demonstrate that through logical analysis and correct experimental design, the idea can be tested. The ideal plan is an elegant one. It is designed to obtain the necessary information with the minimal amount of effort, and is arranged in such a way that the probability of attaining this goal is very high.
Even a good idea and an appropriate plan do not guarantee success. You must demonstrate proficiency in the ability to put the plan to work. All your previous work will be used to make a judgment about your competence. Preliminary data are the most convincing. If you can demonstrate that you have already been able to carry out key procedures effectively, this information will provide the strongest testimony of your ability to implement the proposed research.
Each of these elements by itself is not sufficient either. The idea, plan, and your competence must all be interrelated. Every element of the idea should be closely linked to a component in the plan and in your competence. All three of these are necessary to support the proposal. In terms of our lovely chain, each link must not only be strong, it must be interdigitated with its neighboring links. A lovely idea and a lovely plan that are not related together are of no more use than two links that are not joined together.
The format of the proposal should help the reviewer in every way possible. The reviewer expects a certain format which is usually laid out by the agency providing the funds. Use that format down to the last detail unless you have an incredibly compelling reason not to. The reviewer is expecting the format and knows how to process the information effectively in it. If you break the format, you are creating a difficulty for your reviewer, who will not aid your cause. Any proposal can be put into almost any format. Remember, you are writing for your reviewer – not for you. A failure to follow the correct format raises difficulty for your reviewer and raises a number of questions about the proposal, none of which are very flattering. For example, if you cannot follow the instructions on the proposal, how can you ever follow the instructions to get your research accomplished? If you decided not to bother to follow the instructions, even though you understood them, how interested are you in getting the money? The list goes on for some time. The cardinal sin is to exceed the page limit. By doing so, you interfere with the reviewer’s goal to minimize time and effort expended in the review. Any proposal should be able to fit in any length as well as any format. The only sin worse than violating the rules about length blatantly is meeting them with subterfuge (i.e., using very small margins, no space between paragraphs, small type). All of these tricks will be noticed immediately by the reviewer and raise a whole series of additional and unflattering questions. For example, if you cheat on a proposal, do you also cheat in your research?
Given the constraints imposed by the requested format, you should do everything you can to make the rest of the proposal clear. You should know the types of information being sought and arrange these in such a way that they are easy to find. A well organize proposal, lots of headings, integrative transitional statements, and a detailed table of contents can all help a great deal.
The details of production should be attended to carefully. The ribbon should be new, the keys should be clean, and the printing should be excellent. Many dot matrix printers are unacceptable, unless they can be modified to produce quality printing. All of the components of the proposal should be bound together in such a way that even a two-year-old child could not pull them apart. Finally, the copying should be first-rate. Every copy of every page of the proposal should be checked to be sure that is without flaw.
The reviewer has some expectations about the general size of the budget that is appropriate. The general policy is most likely to provide the support that is necessary to conduct the proposed research. However, implicit or explicit limits are almost always present. You should know that the range of support normally provided, and make sure that your proposal falls within that range. This adjustment should be carried out by varying the amount of research that you propose rather than manipulating the price on it. The error on your part will raise questions about your competence as an investigator. Estimates in either direction are equally damaging. For each item in the budget, you should indicate why that item is necessary to conduct the research project, the way in which the cost of that item was determined, and why it is preferable to alternatives that might have been used.
For major pieces of equipment, and for personnel, particular attention must be made to the percentage of time that will be used for the proposed research. Reviewers are aware that people and equipment may be shared in different projects, and are very sensitive to the possibility that support might be used to help subsidize other projects. If the item is for sole use in the project, its dedication to this project should be justified. If the item is to be shared, the percentage of use in the project should be justified along with a percentage of the total cost, and an indication of where the remaining cost and time will go. The basis of the prediction for the time-cost estimate, and a way of monitoring the actual use to determine if that estimate is accurate, should be included.
- The problem is trivial or is unlikely to produce new or useful information.
- The proposed research is based on a hypothesis that rests on doubtful, unsound or insufficient evidence.
- The problem is more complex than the author realizes.
- The problem is local in significance, production, or control, or otherwise fails to fall clearly in the mainstream of the discipline.
- The problem is intellectually premature – only a pilot study.
- The problem as proposed is overly involved with too many elements required to be investigated simultaneously.
- The description of the research leaves the proposal nebulous, diffuse, and without a clear aim.
- The proposed methodology, including tests and procedures, are unsuited to the objective. May be beyond the competence of the investigator.
- The over-all design is not carefully thought out.
- Statistical aspects are not given sufficient consideration.
- Approach lacks imagination or originality.
- Controls are either inadequately conceived or described.
- Proposed material for research is unsuited or difficult to obtain.
- The number of observations proposed is unsuitable.
- Available equipment is unsuited to the research.
- Investigator does not have experience or training for the proposed research.
- Investigator appears to be unfamiliar with pertinent literature or methods, or both.
- Investigator’s previously published work in the field does not inspire confidence.
- Investigator relies too heavily, or insufficiently, on experienced associates.
- Investigator is spreading himself too thin.
- Investigator needs more contact with colleagues in this or related fields.
- Requirements for equipment, personnel or time are unrealistic.
- Other responsibilities prevent the investigator from devoting sufficient time to this project.
- Institutional setting is unfavorable.
- Current research grants held are adequate in scope and funding to cover the proposed research.