Earned Value Management Misconceptions and Limitations


Posted on 7th May, by JimYoung in Blog. Comments Off on Earned Value Management Misconceptions and Limitations

Although EVM offers excellent benefits, and is undoubtedly a very clever thing, it’s not perfect. In this final chapter I’ve discussed in detail a number of gaps or anomalies with EVM that somewhat limit its effectiveness and discourage its universal acceptance. Despite these shortcomings, and despite the cries from Agile practitioners that EVM is a step back in the evolution of project management, I suggest that EVM is sufficiently useful to ensure it’s here for the long run. Among EVM limitations and criticisms, which if resolved will further enhance this tool, are these:

Lack of Commitment

The old adage, “what gets measured, gets done,” has a corollary in “what gets promoted, get used.” EVM may take time to set up and can involve a rethink of how our projects are planned and how their progress is measured. If our organisation’s senior managers and project managers are ambivalent about it or aren’t fully committed to this change, then EVM is unlikely to gain traction.

Introducing EVM is a change management challenge and most of us to some degree are wary about change. EVM’s a venture into the unknown, which if not managed effectively may see passive and active resistance escalate, project team members’ motivation deteriorate and project management productivity nosedive rather than improve.

To help ensure the successful embedment and ongoing success of EVM, we need a persuasive business case to justify the change, senior managers who are committed to its introduction, who speak with one voice, and who arrange for EVM coaching and mentoring for project team members as required.

Confusing Terminology

A frustrating feature of EVM is the need to familiarise ourselves with the associated terminology. In particular, we need to be careful when communicating with our external stakeholders (people or groups outside our organisation who have a stake or interest in our endeavour), some of whom could find our use of specialised EVM terminology patronising or even alienating.

EVM is rife with a not-very-intuitive vocabulary and obtuse mathematical formulae and acronyms, and thus communicating EVM information to stakeholders can be challenging. It’s seen as linguistic tribalism. Such language can be annoying and even intimidating if you’re not in the tribe.

The use of EVM language might also cement unfortunate stereotypes. Project managers are sometimes cited as serial offenders. While our project management colleagues may appreciate and even be impressed with our command of EVM jargon, gobbledygook and acronyms, others are not. One source of confusion is that EVM misuses the term “value”, since in reality, no true value is delivered until the customer receives the working solution.

However, despite the slew of intimidating language, formulae and acronyms that define EVM, it isn’t too complicated. The glossary at Appendix One may help.

Lack of Project Management Maturity 

Some of the literature written by advocates of EVM lauds the benefits of EVM without regard to the project management maturity of the organisation. Attempting to implement EVM within an organisation that is lacking project management fundamentals will invariably result in failure and serve to demoralise and frustrate project teams.

Project management maturity refers to the progressive development of organisations’ project management processes, which greatly influences the success rate of EVM implementation. Such maturity or capability levels could be described as naïve, novice, normalised and ultimately natural. The introduction of EVM is best targeted at the normalised level of maturity where project management is already usual practice, is part of the organisation’s culture, and training in project management is endorsed.

Organisations with low levels of project management (ie, naïve or novice) may, for example, fail to include the cost of internal staff time when estimating project labour costs. Rarely do such organisations use timesheets to track actual staff time devoted to project work. Although some organisations establish project schedules, their project managers are not required to set a baseline that is essential for EVM. It’s also difficult to clearly identify costs when organisations use poor scope definitions and lack proper change control procedures.

Quality Ignored

At present EVM is solely focused on cost and time, whereas quality management, risk management and customer satisfaction are also important considerations for project management success. In particular, quality management and stakeholder engagement must occur to ensure that our project products are fit for purpose and meet stakeholders’ requirements.

EVM metrics may indicate that our project is on track, but product quality may be lacking unless separately controlled. We may have done all the work on time and within budget, but this is pointless if the resultant product or service doesn’t meet specifications or isn’t fit for use.

However, researchers propose new metrics in the EVM methodology to take into account risk and quality (eg, reducing the EV for risk or when poor quality occurs) for a more efficient project control. With regard quality, perhaps one answer for progress reporting purposes would be to add a “Quality Performance Index” (QPI) as a measure of how well the product appears to be conforming to customer requirements/specifications, where say a rating of 0.90 would mean that, in the opinion of a reviewer, there is 90% conformance with requirements. By multiplying the three performance indices together (CPI x SPI x QPI), we would have a single number that provides an overall rating for our project’s performance.

Although extending conventional EVM methodology by embracing quality and risk management, the added complexity may be too much for some project managers who already find EVM very demanding.

Benefits Ignored

The purpose of projects is to implement changes that improve the performance of our business.  While EVM cost and time are very important, often the ultimate determinant of project success is that business case benefits, which justified the investment, are realised, and these may still be achieved even when the project is finished late or over budget. Conversely, there have been plenty of projects that met their cost and schedule objectives but failed to deliver an appropriate level of business value.

EVM doesn’t directly address benefits or their post-project realisation. EVM has no relationship to the investment value or benefit for which our project has been funded and undertaken. Yet due to the use of the word “value” in the expression EVM, this fact is sometimes misunderstood. True value is not delivered until the customer is able to use the solution. Therefore, “earned value” might be better described as “earned scope”?

The last thing I want to say under this heading is to restate that projects are done to deliver benefits. If the benefits are not actually delivered, it’s irrelevant that the project is on time and under budget.

Business Need

EVM performance might seem great, but this doesn’t ensure that our project business case, which is a “living document”, remains valid and that our project will now deliver a suitable solution to a business need that has changed or even evaporated. Periodic business case reviews are necessary to confirm, terminate or amend our project – particularly those longer duration, expensive or risky investments. The business case is not static. It should not be used only to gain initial funding for our project and then ignored, but should be actively maintained throughout the life of our project and be continually updated with current information on costs, risks and benefits.

One trap is often the “sunk-cost fallacy” – continuing to invest additional resources such as people, time or money in our project in our desire to not waste the already used, unrecoverable resources. Once a resource is spent and is non-refundable, it’s considered “sunk”. Non-value adding projects need to be stopped promptly in order to free their resources, funds and energy for other, more useful business needs. This stop decision would normally be made by a business analyst, project sponsor or project steering committee.

Inaccurate Estimates

Sometimes the duration or cost for a proposed project are inadvertently under or over estimated, or may be deliberately so in order to gain project approval. Then, during detailed planning, we find that a more accurate bottom-up estimate based on the project WBS often exceeds the original top-down guesstimate. Unless the project budget is based on a comprehensive and realistic estimate, the PV will be an unsuitable baseline against which to measure project performance. See the estimating checklist at Appendix Two.

Measurement Challenges

Another EVM challenge is accessing the actual cost of partially completed tasks (ie, WBS terminal elements). EVM is only as good as the numbers crunched. If a task has not yet started, we have no trouble saying that zero percent of the task has been completed, and if the task is finished, it’s obviously 100 percent completed. However, it can be difficult to estimate how much of a task has been completed for anything between these extremes.

As discussed earlier, one approach is to assume that a task is 50% complete (ie, half actual cost for a task has been incurred) as soon as it starts, and is 100% complete once the task has been completed, which is referred to as the “50-50 rule”. Another approach is the “0/100 rule”, whereby no credit is earned for a task until it’s finished. In order to give more weight to the completion of work, other percentages may be used, such as 25/75 and 20/80. The 20/80 rule is often favoured because the emphasis remains on completing a task rather than just starting it.

Thus, for more accurate measurement of completed work, it’s useful to keep WBS tasks small, both in their duration and cost.

Distorts Work Effort

It’s argued that EVM might encourage unnecessarily high progress on non-critical tasks to report a higher SPI, and also encourages us to preferentially complete lower cost/higher value tasks to guarantee a higher CPI. However, achieving our project duration target depends on completing all tasks in a timely manner according to task dependencies, and accelerating non-critical tasks will not get us there any sooner, but may simply add cost to the detriment of our CPI measure.

Any slippage in tasks on the critical path will result in slippage of the project as a whole. Delays in tasks outside the critical path won’t result in slippage of the whole project unless those delays become so acute that these tasks also create a new or parallel critical path. Rather than paying attention to making sure tasks off the critical path don’t slip or applying additional resources to accelerate non-critical tasks, we project managers should keep our eyes on the tasks that matter more.

Doesn’t Reveal Cause of Problems

While EVM helps us identify cost and schedule variances (ie, time and cost problem symptoms), it doesn’t tell us the true cause of such problems. To help determine the cause we may need to employ root cause analysis (RCA) using tools such as Pareto charts, five-whys, fishbone diagrams, scatter diagrams, and failure mode and effects analysis (FMEA). My point is that EVM is but one tool among many available for our use, and needn’t be used in isolation.

Forecasts Based on Past Performance

Some criticism has been made of EVM because it merely looks at past data to predict future performance, while such history is not necessarily a good indicator of future performance. EVM assumes that the prevailing SPI and CPI will continue right up to project completion, whereas in practice it could vary significantly along the way.

Predicting the future has always been a dicey proposition. “It is like trying to drive at 100 kph while looking out our rear view mirror,” is one such criticism. Furthermore, while EVM can identify the EAC and the project end date, it cannot generate the data needed to create AC or EV forecast trends at any date prior to project completion.

Index Issues

Two indexes that indicate the performance of our project are the cost performance index (CPI = EV/AC) and the schedule performance index (SPI = EV/PV). However, CPI is dependent on AC for accuracy. If AC does not include all appropriate costs, CPI will be an unreliable measure. Also, EVA cannot distinguish a critical task from a non-critical task. SPI may therefore be misleading when an ahead-of-schedule non-critical task overshadows a behind-schedule critical task. The SPI may then indicate a healthier project than is reality.

EVM and Fixed Price Contracts

There’s a perception that EVM is not needed for fixed-price contracts where risk is transferred to the contractor. A fixed-price or lump sum contract means someone has agreed to deliver a project or chunk of work for a fixed amount of money. With this type of contract there needs to be a well-defined scope of work. In effect, from the customer’s perspective, with a fixed price, AC will always equal PV and the CPI will always be 1.0. Yet, why would a contractor who’s taking on a fixed-price contract that has them bearing all the cost risks not apply EVM at least for their internal use? Without EVM they may not properly know their potential cost and schedule exposure risks.

For this type of contract the benefit of EVM is with the contractor who has assumed the cost risk, although the contractor may not wish to share EVM reports with the customer, often because the contractor prefers to keep any commercially sensitive risk contingency built into the price confidential. The customer and contractor will have differing perspectives on actual and budgeted costs.

With such contracts, the customer’s interest is mainly in schedule performance, but using standard EVM reports would expose cost. While the two parties will agree what information is to be shared, the preferred solution for the contractor is often to simply express schedule performance in terms of time only without mention of cost.

Unsuitable for Agile

In recent years, there have been various theories regarding the best way to reconcile the conflicting theories of EVM, which needs a reliable baseline, with an as-we-go incremental Agile development in which schedule and cost grow as scope emerges – essentially predictability versus adaptability – two ends of the project management practice spectrum.

Without doubt, EVM is best suited for base-lined task-driven projects with well-defined requirements and detailed scope that are based heavily on planning and control to achieve predictability over project costs and schedules.

w1

Because EVM requires quantification of a project plan, it’s not appropriate for discovery-driven or Agile projects that can’t be planned far in advance, where requirements evolve and change occurs even late into project implementation. With Agile, scope is under constant review and is the big variable, whereas time and cost simply adjust to accommodate it throughout project planning and implementation, and where there is little documentation and no fixed schedule or budget against which to measure progress. As we know, EVM calculations are based on detailed and accurate upfront planning that simply doesn’t exist in an Agile project.

While there are challenges with adopting the Agile framework, with its feature backlogs, burn charts, and iterations to the traditional EVM view of project baselines, some Agile proponents (“Agilists”) suggest that a simplified version of EVM could be used with hybrid-Agile projects – those with a blend of these two seemingly disparate approaches.

Nevertheless, Agile is of course a legitimate philosophy and is particularly suited for those projects with a high degree of uncertainty where the solution isn’t well defined, requirements and their priorities frequently change, and creativity and innovation dominate. Imagine attempting to develop a traditional and detailed project plan to find a cure for coronavirus?

w2

There’s no one best project method. We need to fit the project management approach to the characteristics of our project challenge. However, my view is that trying to force-fit EVM to adaptive Agile initiatives, such as SCRUM, completely misses the boat – quite simply EVM and Agile are incompatible. One Agile expert puts it thus: “Trying to apply EVM discipline to an Agile work effort is like an oil-slimed albatross with one wing missing and the other badly broken. It will never fly.”

Final Comments

EVM is a project planning and management tool that is used to evaluate project progress and predict potential outcomes. While there are plenty of project management software options that allow us to monitor some of these aspects, EVM gauges the cost, scope, and schedule of our project to better evaluate its performance and also help us make a clear, objective determination of a completed project’s final cost and overall effectiveness.

Exceeding the set budget or going over the planned time are two of the most common concerns faced by us project managers when tackling a project. Fortunately, implementing EVM can help solve both of these problems and enable us to keep our finger on the pulse every step of the way. And while it may seem difficult to justify the adoption of EVM, there are some very clear benefits to using it. Have we ever asked ourselves questions such as these?

  • My actual costs are less than my budget. Is my project doing well, or is it behind schedule?
  • My project’s actual costs are now higher than budgeted, and the project is only halfway complete. What’s it likely to cost at completion?
  • The rest of the work will cost less than budgeted. Is this probable?

If we’ve asked ourselves these questions, then EVM is a tool we should seriously consider using to determine our project’s health.

When faced with a new project, we no doubt want it to be successful. We’ve heard all the horror stories about cost overruns, and we want an early warning system to let us know when problems are arising before it is too late to resolve them. We’re in search of a solution. That solution may be Earned Value Management.

EVM is designed to make sure projects are performing the way they’re supposed to or identify early when something challenges their success. EVM enables us to close the loop in the plan-do-check-act management cycle.

The most conspicuous attribute of EVM is its unique metrics for evaluating and forecasting project performance. As such, EVA is a significant part of our project management professional arsenal. Remember, it all begins with a robust project baseline and the objective measurement of progress if we are to fully benefit from this compelling tool.

Finally, a comment about EVM and project risk management. They are complementary processes. Both are key aspects of the overall project management discipline. Risk management is largely related to what may happen in the future; EVM is concerned with using what has already happened to predict and control the future. There may be great synergies to be realised by integrating these two processes.

Another challenge would be to develop forecasting models that are not solely dependent on a project’s past performance. Also, future developments may include new metrics in the EVM methodology to take into account stuff like risk, quality and technical performance for more efficient project control.

It’s worth reiterating that organisations will not reap the benefits of EVM until they successfully implement sound project management processes and possess the willingness to change. Given these circumstances and a willingness for some trial and error, we can implement the EVM regime easier than we might expect. An experimental start might be to apply EVM to just one or two tasks on our next project. Give it a go. Good luck!

 





Comments are closed.



Blog

My latest thoughts on Project Management and life.

LATEST BOOK: EARNED VALUE MANAGEMENT

My big thanks to the coronavirus lockdown that provided me with an undisturbed opportunity to put together this 104 page booklet on Earned Value...

Project Estimating Checklist

Hofstadter’s Law: “It always takes longer than you expect, even when you take into account this law.”

Hofstadter was right....

Earned Value Management Misconceptions and Limitations

Although EVM offers excellent benefits, and is undoubtedly a very clever thing, it’s not perfect. In this final chapter I’ve discussed in detail a...