Partnership level
Baseline information may be collected at the moment the partnerships become operational (i.e. as soon as their administrative mechanisms are in place).
Monitoring / interim information and data should, ideally, also be collected at regular intervals through the course of the partnerships, i.e. every 12 months, and again when the partnerships cease their operation. The end of the partnerships' operation however may not be the point in time when the EU support finishes.
Some partnerships continue their operation even after the end of the EU-supported life cycle based on their own contributions. In these cases it would be good to gather monitoring information at the end point of the EU support period to have a reference point for the whole EU supported operation of the partnerships as well as a baseline reference for their "independent life".
Below the needs for data and information are mentioned at the start, during and at the end and beyond the lifetime of a partnership.
While designing the partnership
- Aims and Objectives: The partnership aims and objectives are usually defined at the commencement stage as one of the key parts of the partnership's design. Clearly articulating the objectives of the specific partnership will help build up the objective hierarchy, i.e. general, specific and operational objectives.
- Information about the partnership: types of partnerships, countries represented, thematic priorities and S&T fields addressed, budgets, national contributions, EU contributions, funding sources, funding modes.
- Information about the partners: number, types, contact data, role of organisation, funding source of organisation for the participation in the partnership, etc.
- Planned joint activities: number, types of activities, sources of funds, national budget, EU budget, other budget, funding mode.
- Planned joint calls: number, types of research and research fields addressed, sources of funds, national budget committed, EU contribution requested, other budget, funding mode.
This information can be requested (and may be found) in a standardised application form for the establishment of the partnership that is usually filled in by the proposal coordinator in collaboration with the partners.
During the lifetime of the partnership
The information that needs to be collected at the interim stage i.e. while the partnership is 'running' has to reflect the progress made against the initial plans, i.e.:
- the partnership activities (those completed against those scheduled) and their features (e.g. number, types of activities, outputs of activities, etc.)
- the partnership budget (absorbed against overall)
- joint calls completed against planned and their specific features (such as number, number of proposals, accepted proposals, number of types of participants, countries represented, national budgets committed and spent, etc.)
- etc.
This information can be requested in standardised progress forms about the partnership overall. Such reports usually take the form of Periodic and Final Reports or Annual Reports of the partnerships.
At the end of the partnership and beyond
At this stage, the evaluation will be largely concerned with issues relating to the overall aims of the partnership e.g. coordination or alignment or integration of national / regional programmes, excellent science, industrial leadership, tackling societal challenges, etc. and the degree to which these were achieved or not.
Thus, it is the time to address specific evaluation questions. Most often it is effectiveness, efficiency, relevance, and added value that are addressed in ex-post evaluations. However, it is also worthy to consider examining questions in relation to network health, connectivity or utility of effects to the wider target population.
The monitoring data gathered during the lifetime of the partnership will be valuable to answer questions such as:
- Have all national contributions and the EU contributions been absorbed? How many proposals/projects were funded? What other activities were completed? Who was involved?
- How has the partnership membership evolved over the years? Who were the more 'successful' country memberships?
- What are the outputs, outcomes and impacts that have been achieved?
- etc.
This information will possibly be available in the Final Report of the partnership. Answers to these questions are important inputs to the process of examining the evaluation questions but they are not enough. Monitoring data show 'what is there' at specific points in time, without the ability to provide explanations or report on the value judgements of the participants and stakeholders engaged. The additional inputs can be collected through targeted surveys, interviews, case studies etc.
The project level monitoring will prove quite useful at this stage of evaluation because of the wealth of information they will have produced. When aggregated, they could provide valuable insights about impacts at the partnership level overall.