A vendor scorecard makes NGO ERP evaluation more transparent only when the scoring rules reflect real decisions. A spreadsheet that averages every answer can rank an unsuitable supplier first if attractive features outweigh a failed essential control.
Put mandatory gates before weighted scoring. Then evaluate suitable candidates using clear criteria, evidence states and an agreed scale. Preserve the reasons behind each score so reviewers can distinguish a demonstrated workflow from a claim, a workaround or a future contractual commitment.
Which requirements cannot be traded away?
Start with the essential ERP requirements. A gate should state the condition, acceptance evidence and consequence of failure. For example, an organisation may require country-level access segregation to be demonstrated before sensitive records can be introduced.
Give unresolved evidence its own status. “Not demonstrated” is not the same as “failed,” but it is also not a pass. Decide whether a candidate can remain under consideration while the evidence is obtained and which decision cannot proceed until the gap is closed.
Do not rescue a failed mandatory gate by adding points elsewhere. If the organisation chooses a compensating control, record the revised requirement, cost, risk acceptance and approval explicitly.
Weights reflect the operating priorities
Weights express relative importance among criteria that can reasonably trade off. Agree them before reviewing supplier-specific scores. Use workflow groups that avoid counting the same capability repeatedly under finance, grants and reporting.
An example model could assign 35% to essential workflow fit beyond the gates, 20% to reporting and data, 20% to delivery and support, 15% to security evidence beyond mandatory requirements and 10% to commercial fit. These are example weights, not an industry benchmark.
| Score | Meaning To Define Before Evaluation |
|---|---|
| 0 | Does not satisfy the criterion |
| 1 | Major gap or impractical workaround |
| 2 | Partial fit with material work remaining |
| 3 | Meets the agreed requirement |
| 4 | Useful additional fit relevant to the stated need |
Avoid awarding extra points for unrelated features. A larger catalogue is not necessarily more valuable for the organisation's requirements.
Include the work outside the finance office
Weight categories against actual programme risk. A partner-funded network needs evidence for sub-award monitoring; a public fundraising organisation needs gift and settlement reconciliation. Field-heavy delivery may make offline recovery a gate rather than a low-weight convenience.
| Evaluation Area | Representative Evidence |
|---|---|
| Fundraising and gifts | Preference history, duplicate resolution and refunded gift |
| Payments and treasury | Settlement matching, rejected transfer and verified payee change |
| Partner awards | Release hold, reviewed expenditure and closeout balance |
| Field operations | Disconnected capture, conflict resolution and lost-device response |
| Vendor trust | Relevant assurance scope, recovery evidence and named responsibilities |
| Total cost | Common-period quote with growth assumptions and exclusions |
Keep the total weight at 100% after adding categories. Avoid awarding the same evidence points under several headings. Prioritising features by operating model helps determine which categories deserve weight and which can remain outside the current phase.
A claim and a demonstration are different evidence
Record an evidence state alongside the score: claimed, documented, demonstrated or contracted for future delivery. These states are not interchangeable and should not be treated as a simple maturity ladder. A contractual promise can be important while the functionality remains undelivered.
Ask specific vendor questions to resolve unclear claims, then test the answers through live demo scenarios. Retain the scenario, environment, result and limitations. A video from a different configuration should not automatically satisfy a requirement about the proposed deployment.
For each score, write a short reason tied to evidence. If two evaluators disagree, compare their interpretation of the requirement before averaging the numbers. An average can conceal that one evaluator scored current capability while another scored the roadmap.
Gaps have delivery and operating consequences
Keep configuration, integration, custom development and unsupported needs visible. A criterion can have acceptable eventual fit while carrying significant delivery risk. Record dependencies, owner, effort assumptions and the acceptance condition separately from the capability score.
Consider three example suppliers. Supplier A scores 82 out of 100 but fails a mandatory data boundary. Supplier B scores 76 and passes all gates. Supplier C scores 79 but has an unresolved gate awaiting a test. A is ineligible under the agreed rule; C is conditional; B is the currently evidenced eligible option. The total alone does not decide the outcome.
Use the selection decision process to handle these conditions. The scorecard should make the decision clearer, not turn uncertainty into a misleadingly precise number.
Test whether weight changes alter the decision
Run a sensitivity check with plausible alternative weights. If two suppliers exchange rank when a category changes slightly, examine the underlying tradeoff. The decision may depend on whether the organisation values reporting depth or implementation simplicity more strongly.
Check arithmetic as well as interpretation. Weights should sum to 100%, scales should be consistent and excluded criteria should not silently distort the denominator. Keep missing evidence separate from a genuine zero score.
Finish with a decision note explaining gates, weighted results, material gaps and accepted conditions. Preserve the original scoring version and later revisions. This makes it possible to understand why the chosen supplier was considered suitable when implementation questions arise.
Weight the evidence as well as the fit
A worked scoring rule is weighted points = criterion weight × fit score / 4 × evidence factor. With a 35-point workflow weight, fit score 3 and demonstrated-evidence factor 1.0, the contribution is 26.25 points. The same fit assertion supported only by a claim at factor 0.5 contributes 13.125 points.
One usable example rubric assigns claimed evidence 0.5, documented evidence 0.75 and demonstrated evidence 1.0. Agree the rubric before evaluating vendors. A contracted future deliverable remains a delivery condition and must not receive credit as demonstrated current capability. Mandatory controls still need their explicit gate result, regardless of the multiplier.
Keep the evidence reference, environment, date and evaluator beside the score. Recalculate after a retest rather than changing a score without an explanation. The weights must total 100 and every scored criterion needs a defined scale.
Score mandatory gates before weighted criteria in the Vendor Evaluation Scorecard. A strong total cannot compensate for a failed mandatory requirement.
Record expected and observed results side by side in the NGO ERP Demo / POC Workbook.
Conclusion
A sound scorecard exposes tradeoffs without hiding essential failures. Apply gates first, explain weights, preserve evidence states and test sensitivity. The final decision should remain understandable from the underlying observations and accepted risks, even if the spreadsheet's numerical ranking is removed.
Score Causeway against your requirements
Evaluate Causeway against your own scorecard and evidence requirements. Share the mandatory gates, scoring definitions and representative scenarios before the discussion.
Ask for a response that identifies demonstrated capability, configuration needs, dependencies and proposed future work against the same requirement identifiers. Keep the weighting and final evaluation under your organisation's control. This produces a more useful comparison than accepting a supplier-authored score for its own product.
Frequently asked questions
How should missing evidence be scored?
Mark it as not demonstrated or unresolved under a defined rule. Do not assume either a pass or a technical failure without evidence. Assign a follow-up owner and deadline, and identify which decisions remain conditional until the evidence arrives.
Can a vendor pass with a failed mandatory requirement?
Not under the original gate definition. The organisation can formally revise the requirement or accept a documented compensating approach through the appropriate authority, but it should not hide that change inside a weighted score.
How should future roadmap features be treated?
Keep them separate from current demonstrated capability. Record the dependency, proposed delivery date, acceptance conditions and consequences of delay. A contractual commitment may support a future plan, but it does not prove that the feature works today.
