Faster, Easier, Clearer Scoring - Anchored Scales

A Breakthrough in Tender Evaluation: Faster, easier, more defensible scoring using Anchored Scales:

Your challenge as a tender evaluator is to assess bidders’ capability efficiently, without compromising the quality of your decision, or the value delivered for the money spent. Learn how to use the cleverest evaluation tool yet: designed to make your assessments accurate, targeted, transparent, and cost-effective >>

Procurement Professionals everywhere complain about the process of assessing tenders. It’s hard to stay impartial, to find the information you need to score against, and to work out what matters and what doesn’t, before assigning a score. Agreement between Evaluation Team members is hard to achieve, and it’s common to worry that the decision could be challenged by a disgruntled bidder, even resulting in Court action. And if that’s not enough, report writing at the end of the process is painful and long-winded.

In this series of articles, we introduce the first in a range of tools that have been proven to reduce evaluation time and costs significantly as well as providing evaluators with greater confidence in their decisions, and reducing reporting requirements.

How anchored scales can slash your evaluation and reporting time

Investing some time ‘up front’ can significantly improve the effectiveness and efficiency of your procurement process. Using an anchored scale can not only make it easier to compare responses, but it also reduce variability in tender assessments across members of the Evaluation Team.

When an evaluation team meets after they’ve completed their individual assessments of the bids, there is generally a bit of horse trading to be done. Everyone has a different perception of what constitutes a ‘good’ response, and what’s a ‘bad’ one. This can result in significant variation and spread between the marks which take time, require intensive discussion, and occasionally introduce conflict before they are resolved. This challenging process also introduces uncertainty and lack of faith in the robustness of the decision that has been made.

Arguments during moderation of the scoring about why one response is better than another can be much reduced. Some evaluators like to select the ‘best’ response, and then benchmark the others against it. This still has the disadvantage of subjectivity in the assessment.

A fairer and more objective approach is to put together an ‘anchored’ scale that describes in detail the quality and level of information that will be provided within a band of marks, for each attribute. It’s essential to discuss and agree what components of each attribute will be important to differentiate the bidders on, first. Then, the process simply requires defining the characteristics within each component, that correspond to each of the score bands.

Unlike existing scales that rely on subjective descriptors (such as ‘exceptional compliance’, ’good’ or ‘inadequate’), an anchored scale is based on objective facts that are tightly linked to the components that will differentiate the bidders based on the value for money they bring to the contract.

For example, an anchored scale for Relevant Skills might use the following descriptors:

90, 95, 100 All positions included. Project Manager has BEng or equivalent and >20 years’ experience in complex concrete reinforced structures. Committed to project 90% or more.

Stakeholder Manager has consulted with Maori stakeholders on at least five projects in past seven years.

Earthworks Engineer > 5 years’ experience with local geotechnical conditions.

Traffic Manager has Level 3 CoPTTM.

75, 80, 85 All positions included. Project Manager has BEng or equivalent and >10 years’ experience including concrete reinforced structures. Committed to project 70 - 90%.

Stakeholder Manager has consulted with Maori stakeholders on at least three projects in past five years.

Earthworks Manager and crew have completed at least two projects in soft ground conditions in past five years.

Traffic Manager Level 2 plus at least three years’ project experience on State Highways.

60, 65, 70 All positions included. Project Manager has tertiary engineering quals and concreting experience on reinforced structures is mentioned within CVs of at least two staff at supervisor level or above. PM committed to project 50% or more.

Stakeholder Manager has taken leadership role on community engagement for at least two infrastructure projects.

Earthworks Manager/ crew have soft ground experience on at least one project in the past five years.

Traffic management at CoPTTM Level 2, 1-3 years’ experience on State Highway projects.

50, 55 All positions included.

Project Manager has either tertiary engineering quals or concrete structures experience, but not both. Less than 50% commitment for PM.

Stakeholder experience does not include leadership on any form of community consultation.

Earthworks Manager/ crew have soft ground experience more than five years ago. Traffic management at Level 1 CoPTTM or with no experience on State Highways.

40, 45 PM is either committed less than 50%, or has no tertiary qualifications, or has no reinforced structures experience.

Stakeholder experience does not include any form of community consultation.

Earthworks manager/ crew have soft ground experience more than five years ago. Traffic management at Level 1 CoPTTM and with no experience on State Highways.

< 35 Some positions excluded: Did not answer the questions.

With this approach, there will be less argument about what marks should be awarded for each response. You can adjust your descriptors (before the evaluation starts) to either spread out or condense the scores. You may choose to make the scale transparent by issuing it within the RFT document, or keep it up your sleeve for evaluators only.

Putting together effective anchored scales does take some practice – but the more you do it, the better you get. A well structured scale like this for all the attributes in a tender will make your evaluations simple, consistent, quick and fair.

Anchored scales are being used increasingly by government organisations on procurement projects in New Zealand. A Council who used this approach for a Passenger Transport project reported that it reduced the time needed for evaluation from a couple of weeks to a couple of days. Moderation and writing the evaluation report were done in a few hours (rather than the previous painful process taking 2-3 days).

All for a few hours invested up front in developing the right RFT tools. That’s got to be a sensible investment, hasn’t it?

We’ll have more examples of time-saving evaluation tools in our next newsletter.