The Welsh transport appraisal guidance (WelTAG) 2022 consultation: Transport Planning Society response

We particularly like your statement: “… transport planning is not an exact science. It is about designing good programmes and projects that meet the needs of people in Wales, not just adding up costs and benefits”. As Transport Planning Society we don’t however think that cost-benefit ratios and wellbeing appraisal have to be exclusive. Cost-benefit ratios in many infrastructure appraisals have been extended to include, in addition to metrics related to transport efficiencies (travel times, operating costs, reliability, accidents), wider impacts be they social, environmental or economic. For example, some or many of the Welsh Government’s well-being objectives can be quantified and potentially monetised. This includes reflecting absolute values that cannot be exceeded.

What matters is how the Government’s objectives are reflected in the weighting or valuation of each of the contributing factors to the cost-benefit analysis. In that respect, the government’s intention to produce benefit cost ratios (BCRs) with and without travel time savings, is a good example of how the BCR is still a useful catch-all, but also that it is possible and desirable, to separate each of the contributing components.

We recommend the Government looks into the discussion and recommendations in the recent report by the International Transport Forum on broadening transport appraisal: https://www.itf-oecd.org/broadening-transport-appraisal . In particular, the section on ‘Modifying cost-benefit analysis’ on pages 14-17 provides good insights and useful directions.

Assessment tools have three main purposes, first in project/programme assembly to select and rank candidates, second by funders to verify eligibility for grant/revenue support, third by professionals to justify projects/programmes to senior colleagues and politicians who are the local decision makers. Transport both responds to and leads development, but also policy and transport related impacts (such as air quality and congestion). Not everything is quantifiable. That which cannot be quantified still requires professional and evidenced inputs on the influence on return on investment, ranking etc. Non-quantifiable factors can in principle be reported in terms of increasing/decreasing the calculated quantified and/or monetised return on investment.

If it is the intention to rely less on formal modelling and appraisal techniques, the evidence used to support decision-making must remain robust. There is a danger that decisions will be influenced more by what transport planners might want to see happen, rather than what analytical methods tell us is likely to happen as a result of policy, planning and project interventions. Even a ‘decide and provide’ approach needs a ‘predict’ element to ensure that the measures taken are effective. In that respect we are pleased to see your statement on page 45: “It is particularly important to support it with good data, especially if you are not providing additional benefit-cost modelling”. We hope that further guidance will be provided, including a WelTAG databook, to ensure quality and consistency.

Even with a reduced focus on benefit-cost ratios, we expect WelTAG to provide advice on tools and data sets to use, particularly where the Integrated Well-Being Appraisal requires the analysts to assess impacts that until now have received less attention. As at the moment the document contains absolutely no advice on modelling, so we expect that practice will default to TAG guidance. The Government must be clear if that is their intention. We notice that in Figure 9, reference is made to using regional transport models as sources of data for quantitative measures. It is our advice that supplementary guidance is provided on which models exist, and how to use these in a proportional manner (we understand that National Highways’ Regional Traffic Models for England have been very successful in simplifying and speeding up appraisals).

The Transport Planning Society finally recommends that a reduced emphasis on cost-benefit ratios requires not just changes in the appraisal methods, but also on how decision-makers use the results from modelling, appraisal and other analytics. This involves confidence in decision-making under uncertainty.

As a professional society TPS recommends clear ownership and accountability by the right players and at the right time in any of the processes involved in transport planning. Therefore, carrying out stage 0 in-house make excellent sense.

The proposed approach reminds us of the ‘Aqua Book: guidance on producing quality analysis for Government’, released in 2015. The document contains practical advice on responsibilities both within and outside of government, with the aim of ensuring more robust quantitative analysis. We recommend the government takes note: https://www.gov.uk/government/publications/the-aqua-book-guidance-on-producing-quality-analysis-for-government

If the in-house team assumes responsibility, sufficient skills should be ensured. There is a general skills shortage in transport appraisal throughout the UK, and as the TPS we recommend that the Welsh Government invests in its in-house capabilities. Transport Scotland is a strong supporter of the Chartered Transport Planning professional qualification, and the Professional Development Scheme that underpins this accreditation. Our Skills Director would be pleased to explore with you how the Transport Planning Society, and the CTPP qualification, could ensure that in-house teams are sufficiently trained and have the appropriate competencies.

The proposed stage 0 has such similarities with the DfT’s ‘strategic dimension’ (‘the section of the business case that describes how the transport proposal contributes to achieving strategic priorities and how it aligns with existing portfolios, programmes and projects in the DfT, across government and in the geographical area(s) of scope. This dimension sets out the strategic context for the proposal and therefore provides an overarching framework for the business case’) that we believe stage 0 would benefit from clarifying how it is intended to be used differently in the Welsh context.

Other similarities exist with how Oxfordshire County Council intends to implement a decide and provide approach to transport assessment. Their September 2022 document identifies many similar considerations that stage 0 aims to address and we recommend you read it to identify practical advice that can be transferred to WelTAG: https://mycouncil.oxfordshire.gov.uk/documents/s62102/CA_SEP2022R12%20Annex%201_Implementing%20Decide%20and%20Provide%20-%20TA%20Requirements.pdf

In our experience, the main problem in the appraisal of strategies or programmes is that it is important, but difficult, to separate out the contributions of individual components – some may be complementary, some my work against each other. Guidance is required on how you expect this to be handled when programmes or strategies are assessed.

If WelTAG is used at strategic or programme level, it remains critical that any inputs into the quantification of benefits is supported by an analytical or modelling approach that is transparent, defensible and robust. This will require a clear statement, and possibly peer review, of the mechanisms used to reflect strategies and the assumptions made.

The document refers regularly to evidence-based assessments of benefits but is non-committal on how these will be derived. We expect a whole range of evidence will be claimed and supported by different levels of robustness in data or in the techniques used to analyse these. We welcome innovation and challenges to the status quo prescribed in TAG, but moving away from tried-and-tested has risks, too. Any alternative approaches must be tested and peer-assessed, to ensure that decisions will continue to be based on evidence rather than wishful thinking or worse, manipulation.

In all cases, uncertainty needs to be allowed for, through the development and application of scenario modelling. A decision needs to be made by the Welsh Government whether the DfT’s Common Analytical Scenarios will be applied (all of them or a subset), or if specific scenarios should be developed in support of Welsh projects, reflecting uncertainties pertinent to Wales.

The TPS is a strong advocate for the integration of land use and transport planning, and many of our previous reports (for example, State of the Nations (2020 - https://tps.org.uk/public/downloads/VFHEc/State%20of%20the%20Nation%20FINAL%20v2.pdf and our response to the Department for Transport’s Transport Decarbonisation Plan in 2021 - https://tps.org.uk/public/downloads/egbH-/TPS-Response-to-TDP-and-COP.pdf ) have made the point that by their integration many of the Government’s objectives can be met more easily (including decarbonisation but also objectives related to health and equality).

We believe the greatest challenge in modelling and appraisal of aligned land use and transport planning is the representation of the land use response to alternative transport interventions. A successful integrated approach will require each project or programme alternative to have different, project-specific land use development patterns. Here again we repeat our warning that, also for integrated land use and transport planning, the assumptions of such land use responses must reflect what is likely to happen, rather than what would be desirable (the latter may not be achievable).

The benefits of integrating land use and transport planning extend beyond just the shortening of trips, and its impact on e.g., decarbonisation targets. There are wider transport efficiencies. By reducing the length of trips that people need to make to satisfy their daily needs, new and mainly active modes come into play: walking, cycling and emerging micro-modes, reducing the carbon impacts of transport even further.

From a policy perspective, we suggest that the government considers an alternative approach to integration than usually pursued: this is to let land use plans follow transport policy and investment rather than the usual situation in which transport projects are developed to support land use plans. This may be a better way to reduce the transport intensity of new development patterns.

That said, the greatest opportunity lies in transitioning existing communities, which are not fulfilling the Welsh Government’s core values, into sustainable communities. How do we transform existing semi-functional community environments into harmonized sustainable communities focusing on public, active, and shared transport, promoting healthy living, and economic growth? WelTAG as presented focuses on future planning, not retrofitting. The transition plan for each individual element required to transition a current community to a healthy sustainable community may not necessarily be able to demonstrate value for money or demonstrable improvements in line with the core strategic WelTAG values at each of the project delivery stages. For existing transformational strategic projects, encompassing both land use and transport planning, WelTAG would need to be focused firmly at the strategic level.

The proportionate approach underpins much of TAG – although there has been a tendency to play it safe, and to make appraisal more complex than has often been required. We therefore welcome the Welsh Government’s explicit statement that for most projects in Wales it will be sufficient to sue WelTAG Lite, giving confidence to promotors and their consultants that a simpler approach is best.

In the case of WelTAG Lite, we note that stages 1-3 are proposed to be combined into one report. As TPS we believe that this leads to the risk of up-front exclusion of options; whereas options generation is a critical part of appraisal. Criticism in the Green Book review was directed at business cases arriving too early at a limited set of options. There is also a risk of ignoring negative and unintended impacts – and as a remedy we suggest that the template or guidance sets out the expectation that identifying these secondary impacts is an explicit requirement. The Strategic Outline Case is relevant for WelTAG Lite also; and should not be ignored.

We are pleased to see the use of gateway reviews. We recommend that one such review takes place early enough to provide evidence that the Lite, Standard or Plus approach that was originally assumed to be appropriate, is indeed suitable now further details of the project including qualitative and quantitative analyses of costs and benefits have emerged.

Guidance will be required when determining whether to follow the WelTAG Lite, WelTAG Standard or WelTAG Plus route. We recommend that, rather than approaching the need for more or less complexity on the basis of project costs, this is done by an explicit assessment and reporting of the anticipated complexity of the responses to and benefits of a transport intervention (project, policy or program) in stage 0, 1 or 2.

It is disconcerting to note that in Figure 7, modelling is limited to just WelTAG Plus. We accept that in both WelTAG Lite and WelTAG Standard, modelling should be less intense and data requirements less comprehensive, enabling a faster and lower cost turnaround (and the DfT’s TAG proportionate approach also recognises this). But we don’t support that appraisal is defensible where no modelling, no structured, repeatable quantification of project, plan or programme impacts has taken place. This is where additional guidance is most necessary.

Finally, guidance is required when dealing with different types of projects, and particularly projects aimed at different modes For example road vs bus vs rail, or the introduction of new modes that may support or extract from traditional modes. Existing TAG guidance is not existent or inconsistent, and the WelTAG guidance as presented is silent on the different modal requirements for modelling and value for money assessment. This needs addressing when assessing projects or programmes that are modally integrated (such as e-scooters at stations), as we expect to be increasingly the case. It would be easy to call for more detailed modelling – but it should be possible as an alternative to describe the potential unintended consequences and whether they strengthen or weaken the case.

As opposed to the DfT’s highly technical TAG strategic modelling and appraisal guidance, the new WelTAG majors on the principles of how appraisal must support decision-making, but is light on directions on how to do this well. It is possible to continue to rely on TAG to provide such details on methodology, underpinning data and reasonable assumptions. But TPS believes that Wales-specific model guidance would strengthen the delivery of robust WelTAG appraisals. This includes templates for minimum data and model requirements for WelTAG Lite, WelTAG Standard and WelTAG Plus, and a databook with Wales-specific parameters for use. Given the thoroughness of TAG, we suggest extensive referencing to those documents.

Many practical modelling and appraisal debates are driven by disagreements around assumptions about the future that are inputs to strategic modelling and appraisal As such, we suggest a strong and mandatory Assumptions Log to be produced and signed off by the project sponsor. This should not be left to the consultants building the supporting models – key assumptions with substantial bearings on appraisal results are often buried in the tools themselves, or in report appendices. They are too important to be overlooked.

In terms of topics for further guidance, we base our answer on work done for National Highways on improving their ability a) to support NH staff with modelling, analytics and appraisal; b) to respond to emerging issues that require modelling and analysis and c) make best use of new technologies and innovations to respond to issues a) and b). We recommend providing guidance on the following topics:

• Dealing with freight
• Integration of land use and transport planning
• Reflecting uncertainty in appraisal, at all levels

See (The future of transport modelling: a structured approach for identifying future innovation areas - C Rohr, T Pollard, Mott MacDonald; T Metcalfe, A Stoneman, WSP; R Himlin, M Boother, National Highways; T van Vuren, Veitch Lister Consulting; L Oakes-Ash, City Science, UK - https://aetransport.org/past-etc-papers/search-all-etc-conference-papers?abstractId=7640&state=b )

We support the more easily accessible wording of the new WelTAG guidance, compared with the more technocratic language in the DfT’s TAG documents. However, this may risk a of lack of precision and the possibility of confusion and (accidental or purposed) misuse. These are our main observations where this may occur, and we suggest a more careful wording to reflect the government’s exact objectives in the document:

• Page 2: ‘groups of sustainable travel interventions’. This suggests that all interventions will be sustainable. We anticipate that WelTAG will itself identify whether a proposal is sustainable, rather than that being an up-front expectation. Sustainability needs to be defined.
• Page 2: ‘an affordable, sustainable, efficient transport system’. It can be argued that an efficient transport system minimises travel times, and that therefore travel time reductions should feature strongly in the decision-maker’s support. We understand this is not your intention, and we suggest that the term efficient is defined in advance
• Page 2: The percentage of journeys by sustainable transport has been set to 48% by 2040. We believe it is more important to state and develop policies and programs that achieve a percentage of overall miles travelled to be reduced, preferably to this same level. If these 48% are mainly short journeys this will lead to much smaller sustainability benefits than if these are representative of the mean or median distance travelled per trip.
• Page 2: ‘all projects and programs funded … must address behaviour change’. The term behaviour change can mean different things to different people. Think for example of drivers changing their mode of travel to sustainable alternatives. This can be achieved by making car travel more difficult or alternatives cheaper without actually changing behaviour. Behaviour change would involve a re-evaluation of how people value the need to travel and the alternatives available to do it. The recent House of Lords report ‘In our hands: behaviour change for climate and environmental goals’ suggests that without changes to people’s behaviours now, the target of net zero by 2050 is not achievable. We strongly suggest making it clearer what the government’s behaviour change objectives are exactly.
• Page 3 and in general: Well-being is at the heart of the proposed improvements to WelTAG but is not defined. To enable SMART objectives to be set and evaluated, TPS suggests that this term is defined more clearly, so that quantification is not just possible but always considered. SMART needs to be properly implemented, not just be a good intention poorly implemented.
• Page 4: Under the Value-for-Money header, five criteria are set out that the government will use to consider supporting a programme or project. The TPS supports this approach, and the criteria, but neither individually nor combined do these cover value for money. For example, a particular project can deliver in all five dimensions, but another project could satisfy these dimensions better, faster, cheaper, or with less risk. The term Value for Money also creates an expectation that a project or programme’s performance against the five dimensions can be monetised – and we understand that this being expressed through a BCR is exactly what the new WelTAG aims to avoid. Our advice is that quantification of impacts remains a key element of appraisal in WelTAG, to enable such comparisons to be carried out.
• Page 5: This same concern arises under the ‘Integrated well-being appraisal’ heading. Although you state clearly that the appraisal should answer four simple questions, this only shows that a project or programme meets the government’s objectives, but not how well. To order or sift proposals for priority and progression, this level of fit should be quantified.
• Figure 1. The increase in efficiency of transport freight by 4% needs to be defined – in numbers of trips, in miles or in emissions?
• Page 9: TPS is pleased to see that in the enhanced WelTAG document, specific attention is paid to the need for delivery and evaluation. This is often lacking in current appraisal practice.
• Page 13: We repeat our concern that meeting the five value-for-money criteria cannot be equated with demonstrating value for money, let alone good value for money. There will have to be a mechanism for prioritising and sifting alternative or even competing schemes and programmes.
• Page 15: We are very pleased to see, in Figure 3, attention paid to existing infrastructure, and a commitment to upgrade in support of modal shift. We expect that increasingly appraisal will need to be applied to existing infrastructure in need of maintenance, improvement or replacement, which will inevitably include highway infrastructure. In that respect we repeat what we stated in The TPS’s response to the consultation to update the Strategic road network and the delivery of sustainable development (circular 02/2013) (https://tps.org.uk/public/downloads/k8b8G/TPS%20formatted%20response%20to%20strategic%20road%20network%20for%20website.pdf), that efficient roads need to, and can, cater for more than just motorised traffic.
• Page 27: We disagree with the explanation of what a do-minimum representation is. A do-minimum option is not just about maintaining existing infrastructure, it is meant to reflect committed or likely to have been constructed changes (transport, land use and policy) that affect the project under consideration. Particularly how its need, operation and hence value for money is influenced by these. If the assumed do-minimum is just maintaining existing infrastructure, each project and policy will be assessed in isolation from the wider government policy and how that is being implemented in advance of a scheme coming on-line.
• Page 60: We welcome the explicit requirement to report on stakeholder engagement as part of the business case for each project, including whether the project is acceptable to different groups, and how their views were taken into account in designing the project.
• Page 77: In the Quality checklists for WelTAG reports, advice is lacking on how WelTAG Plus differs from WelTAG Standard. This needs addressing.
• Page 84: Under Modelling and technical appraisal, reference is made to induced demand. There is much confusion in practice what induced demand is, and a careful definition would be beneficial. The example that increased road capacity might result in additional traffic is dealt with well and robustly in TAG guidance, and in strategic transport models that have been developed along TAG guidelines. Generally, induced demand does not need special attention if the model approach is appropriate for the project or policy under investigation. For example, including departure time, mode destination and strategic re-routing responses for the most complex schemes.
• Page 87: We welcome this explicit naming of relevant policy documents, as this is sometimes difficult to achieve in isolation. TPS suggests that this list is kept up to date and accessible from the internet as a dynamic resource, similar to the current TAG databook.

The TPS has three recommendations for ensuring that studies are high quality, meet the needs of users and represent good value for money.

1. All studies should be supported by peer review or a critical friend, who can be the honest broker between the sponsor, their consultants and the government. This role is well-established in the USA and has been employed successfully in certain projects in the UK as well – such as for the Sheffield supertram and in Liverpool when developing a walking and cycling modelling tool. A register of trusted advisers could be set up. We believe that this is particularly relevant when considering the make-up of the suggested Review Group on page 61, which should include data, analytics and modelling experts, as well as the identified policy and local experts. The five ways of working, promoted on page 63, make the point that it must be ensured that the WelTAG process takes a long-term view by providing the data and analysis to understand the longer-term impacts of the programme or project. We cannot see how this can be achieved without some form of modelling; not modelling will implicitly assume that the long-term future will be the same as now.
2. Taking this one step further Government should externalise the responsibility for modelling and appraisal, away from the project team. This would not just enhance the independence of the analytical team, it would also increase credibility of this work to outside stakeholders. A mechanism would need to be established to ensure collaboration, which could be the peer reviewer or critical friend described above.
3. Increasing internal, in-house skills. As the TPS we recommend that the Welsh Government invests in its in-house capabilities. Transport Scotland is a strong supporter of the Chartered Transport Planning professional qualification, and the Professional Development Scheme that underpins this accreditation. Our Skills Director would be pleased to explore with you how the Transport Planning Society and the CTPP qualification, could ensure that in-house teams are sufficiently trained and have the appropriate competencies.