Establishment of performance criteria

Themes: Targeted transport functions; other transport functions affected by ITS implementation

Why are performance criteria needed?

ITS systems are implemented to improve performance, to allow new services to be provided, to strengthen organisational capacity, etc. (see Objectives and Requirements in Steps 1 the Planning phase).

It is essential to know whether the expected improvements have actually been achieved. If the actual performance is below expectation, it may be possible to adjust or adapt the system. However, many organisations fail to measure this properly, and as a result they fail to get the properly functioning system that they have paid for.

Suitable performance metrics need to be defined well in advance of deployment – ideally at the design stage - and included in the supply Contract. The pre-ITS system should be measured to provide a benchmark, and then measured at various stages following deployment.

Setting performance criteria

In monitoring and evaluating an ITS implementation, it is crucial to have developed a relevant set of performance metrics for the targeted transport functions and for any others that might be affected by the implementation. Data should be gathered for these metrics over a sufficient timescale that seasonal or special factors can be identified and taken into account in any subsequent evaluation.

Metrics for the targeted transport functions should have been identified in the planning and design processes, and taken into account in the feasibility and investment justification. If this has not been the case, then the metrics will need to be created in relation to managing the supply contract and its performance conditions.

Of potentially greater difficulty is identifying any transport functions that have not been targeted by the ITS implementation, but may well be affected by it in either a positive or negative sense. Potential positive impacts should have been identified in the Design Phase,  “What else can the technology, resources and data be used for?”, but these may only become apparent in the normal dialogue with the system integrator where the potential for system enhancements might well be promoted.

However, almost by definition, any negative impacts will most likely relate to unforeseen consequences of the implementation – and hence the relevant data would not be available unless it formed part of a comprehensive management information system. A typical issue might be the use of ITS systems in operations management to reduce manning levels at stations and terminals, and the informal services that the staff had provided in passenger reassurance would then be lost.