Meeting the requirements
The main factors to consider are obviously the degree to which the different systems meet the specific needs and requirements of the institution. If they do not offer some desired features ‘out of the box’, the question is how flexible is the system, how much can it be customised and how much time and money will that entail (both vendor time and your time). Vendors will often say that their systems are fully customisable, but it is well worth probing beneath the surface to discover whether this is borne out in previous cases. On the other hand, it is also worth considering whether what has been specified by your institution in advance is actually worth the cost and time of the customisation.
What indications are there about the trajectory of future development?
Can the vendor provide a detailed implementation plan and how well do they seem to understand the constraints under which they will be working in a particular institution? How much time are they prepared to provide in conducting workshops and training sessions and at what cost? Can the system be implemented in a modular way, offering some short-term demonstrable benefits before the whole system is in place?
Who are the existing customers, how close are there requirements to yours? What has been their experience with implementation, training, support etc.? Is there an active User Group?
What are the lifetime costs of the system? How much is upfront cost and how much will support, maintenance and upgrades cost? How likely are upgrades to be adding features that are important to your institution (relates to customer base and trajectory of development).
Health of the company
How long has the company been in existence? Does it have a sustainable business model? What are its recent financial results? How many employees does it have, how qualified and knowledgeable are they, how good is staff retention, and how many employees are available to implement your system and support you? A rapidly developing market niche such as CRIS systems in the UK is good for these companies in principle, but could over-stretch their staff resources in the short-term and even lead to financial problems.