Deciding which projects to fund

By Jack M. Kloeber, Jr. Ph. D.

Many important problems faced by decision makers involve multiple conflicting objectives. When faced with truly difficult decisions involving high stakes, risky and conflicting objectives, this transparent approach will help to guide decision makers toward better, more defensible choices.

What do the following have in common?

1)    bioterrorism threats
2)    military base closure candidates
3)    new technology efforts for radioactive waste remediation,
4)    IT projects within a medical devices company, and
5)    new drug candidates in a pharmaceutical company

First, the organizations facing the task of prioritizing in each situation need to achieve more than one key objective.  Because there are multiple, sometimes conflicting objectives, there is no one magic metric such as number of lives lost, long term cost reduction, speed of remediation, number of projects completed, or Net Present Value that adequately communicates the value (or harm) of any project.

Second, these have actually been successfully prioritized for decision makers using the same Multiple Objective approach- Value Focused Thinking(VFT).

What is Value Focused Thinking?

There are other methods for prioritizing projects with multiple conflicting objectives: SAW, AHP, TOPSIS, Electre, Outranking, ANP, and others. However, we have found the VFT methodology to be superior and it's superiority is supported by many articles in peer-reviewed journals which compare multiple objective methods.

Many quite experienced decision makers, faced with the task of prioritizing projects or assets for the purpose of funding, cutting, or otherwise allocating resources, move directly to the alternatives at hand and start working on which project is the least or the most value-adding to the organization.  Ralph Keeney, however, says to first look at the values of the organization.  Keeney4 affirms that “Values are what we care about…[they] should be the driving force for our decision making…[and] the basis for the time and effort we spend thinking about decisions.”  The VFT method, shown in Figure 1, does just that.  The emphasis is on uncovering and discussing the driving values of the company, and then structuring the objectives and supporting objectives which support those values.

Once the company develops a structured view of their own objectives, projects or assets can be compared against these objectives to reveal value gaps.  New projects or hybrid projects are often developed that help eliminate the value gaps.  The final analysis is one of transparency and relevancy, bringing needed clarity to the decision maker.

Case Study

The management team was trying to prioritize their lead compounds, all in stages prior to acceptance into development. They knew they had to focus on a few compounds but were unable to agree upon the prioritization.  Kromite proposed using a multiple objective approach to the prioritization.  Since nothing seemed to be working, they agreed to try the approach.  
Figure 2 shows the steps taken by the team, facilitated by the analyst to build a suitable model for the prioritization.  The small decision team immediately focused on defining the values, then objectives, of the therapeutic area (Oncology).  The analysts then helped the team arrange the objectives into a suitable hierarchy, with the top level being the key objectives that were relevant for this decision.  (See Figure 3, below).

With the objectives identified, the work of finding metrics which would align with these objectives began.  The functional experts (oncologists, biologists, and marketing experts) assisted in developing measures which aligned best with the objectives.  Keeney and Raiffa5 developed a mathematically sound and defensible method for combining the performance scores into one overall number.  The Value number communicates the value of the program to the organization.  For example, if your objective was to fill an apparent project launch gap in 2018, a measure should be developed which takes this objective into account.  Such a measure was “Launch Timing” (see Figure 3, above).

Metrics are meaningless unless they are tied not only to a key objective, but also reflect the value added to the company.  There may be a certain level of achievement in a metric which adds a disproportionate amount of value.   Using the same example and Figure 3, the earlier launches are of course of greater value.  However, launching at or before 2018 would add disproportionate value and was reflected in the return function.  The curve shows significant value increase (on the vertical axis) for any launch date before 2018 (horizontal axis).  This is called developing a Value Function.

Now, with objectives in place, aligned metrics developed, and value functions agreed upon, the programs can be consistently assessed against a set of metrics which are aligned with the objectives and values of the organization.  A transparent analysis, giving clarity to the decision makers regarding the strengths, weaknesses, and relative value added by each project was now possible.  Tough discussions took place.

The analysis revealed the real strength of two competing programs, 1 and 2.  Although Program 7 was thought to be an excellent program due to its focus on unmet medical need, it was dominated by nearly every program in every other objective.  Program 4 originally was assessed to have much lower value. The team modified the program to allow for an earlier launch in an alternate disease with just as much commercial potential, increasing its value to the organization significantly. (Upper-Left in Figure 4, below.)

Total Multiple Objective value can be traded off against the Probability of Technical Success (POS).  Those with low value and low POS should be closely examined (Upper Right in Figure 4).

One of the objectives was Novelty.  The measure categorized each Program into First in Class, Best in Class, or neither(minimal differentiation).  From the bottom-left graph in Figure 4 one can assess how balanced the portfolio is in its short term spend, long term spend, and overall  projected financial value.

All of this modeling and interaction with company experts and management has enabled us to look at the Drug Discovery programs in a new light.  Clarity for the decision makers has been increased.  The decision making body now is confident the valuation results have captured the key areas of concern, have captured them consistently, and have given more influence to the areas of most concern.

We will end the same way we began this editorial, with a question:    What decision problems or prioritization issues exist in your company which should involve multiple objectives? 

References

1      Department of Homeland Security Bioterrorism Risk
Assessment:  A Call for Change, Committee on Methodological Improvements to the
Department of Homeland Security's Biological Agent Risk Analysis, National Research Council National Academies Press, 2008.
2     2005 Congressional Base Realignment and Closure Act.
3     Grelk, Kloeber, Jackson, Parnell, and Deckro, ‘Making the CERCLA Criteria Analysis of Remedial Alternatives More Objective’, Remediation, Spring, 1998, John Wiley.
4     Keeney, Ralph L.,   Value Focused Thinking: A Path to Creative Decision-making, Harvard University Press, 1992.
5     Keeney, Ralph, and Raiffa, Howard,  Decisions with Multiple Objectives, Cambridge Books, Cambridge University Press, 1993.