Project level

The process of evaluation particularly that of ex post evaluation is greatly enhanced by the availability of robust and comprehensive information and data relating to project activities and outcomes. The availability of such data greatly facilitates the task of the evaluators and obviates much of the need to collect retrospective information, some of which may have been forgotten or lost as a consequence of participant mobility.

In addition, the collection of such data ex post necessitates significant time and resources on the part of both the evaluators and the project participants. This underlines the advantages of implementing a process of monitoring which can be aligned with the normal reporting requirements. Through a 'light touch' monitoring process, a significant amount of information may be collated which will greatly assist the final evaluation process.

A suggested optimal period for monitoring reporting for projects is on an annual basis, 12 months from the project start, and at the end of the project. Yet, certain outcomes and impacts are unlikely to be manifested until after the lifetime of the project itself, or after some time of the partnership's existence. Thus, a further ex post reporting period is desirable, i.e. one year following the end of a project is, even though for some impacts it may be relevant to allow for a 3-5 year period running the risk, however, of limited 'project memory' and low ability of impact attribution.

Below the needs for data and information are mentioned at the start, during and at the end and beyond the lifetime of a project.

At the start of the project

It is essential that proposals should provide certain types of information and contain clearly defined research (or related) objectives, targets and milestones. Contingent on the overall objectives of the project, the following types of information appear to be the most relevant to collect in this phase:

  • number, country and type of project participants
  • information about type of activities and research area addressed
  • information about intended dissemination and follow-up research
  • information about availability of resources
  • total budget, sources/amounts of co-funding for the project
  • centrality of research project to core activities of organisations
  • expected outputs, outcomes and impacts
  • etc.

This information can be requested (and may be found) in a standardised proposal application form that is usually filled in by the proposal coordinator in collaboration with the proposal participants.

During the lifetime of the project

At this phase of the project, data collection may be concentrated on:

  • the project activities (those completed against those scheduled) and their features (e.g. number, types of activities, outputs of activities, etc.)
  • the project budget (absorbed against overall)
  • initial project outputs, including, for example
    • conference/workshop attendances
    • project meetings
    • degree theses
    • products, process etc. (licensed/patented or otherwise)
    • student/staff exchanges
    • contributions to standards, public awareness, policy
    • further development of research networks
    • publications (articles, conference proceedings, books, book chapters, reports, grey literature, datasets)
    • etc.

This information can be requested in standardised progress reports. Such reports usually take the form of Periodic and Final Reports of projects.

At the end of the project and beyond

It is anticipated that the monitoring process should, by this stage, already have resulted in the collection of a substantial amount of information relating to output, outcome and possibly impact indicators.

This information will possibly be available in the Final Reports of completed projects. This information is a valuable input for examining the effectiveness of the partnership but also aspects of efficiency, utility and added value. However, it is not enough. Monitoring data show 'what is there' at specific points in time, without the ability to provide explanations or report on the value judgements of participants and stakeholders engaged. The additional inputs that are needed to examine the evaluation questions can be collected through targeted surveys, interviews, case studies etc.

The combination of the monitoring and the additionally accumulated data and information at the project level can then be aggregated to an overall evaluation of the partnership. It can also provide an additional source of consistent, comparable data which could be used by any future "meta-evaluations" of the partnership.