CHAPTER 7

Machine Learning Guide for State DOTs

Building on the content from the previous chapters of this report, a guide is prepared to help state DOTs and other transportation agencies in identifying promising ML applications, assessing costs, benefits, risks, and limitations of different approaches, and building a data-driven organization conducive to capitalizing on and expanding ML capabilities in a broad spectrum of transportation applications. The ML guide is published as a separate document and only a brief description is provided in this chapter. Please see the complete “Guide on Implementing and Leveraging Machine Learning at State Departments of Transportation” for more information (Samach et al. 2024).

While state DOTs are the target audience and most of the examples included in the guide are from state DOTs, this guide could support other transportation agencies seeking to advance their understanding and use of ML tools and techniques. The ML guide seeks to help readers answer questions at the project, program, and portfolio level, including but not limited to:

• What are some promising ML applications in transportation today?
• Is ML a suitable approach for the identified use case(s)? If yes, in what capacity?
• How should the user plan an ML pilot project and how might it differ from other projects?
• What are some of the challenges and risks the user should be aware of in ML pilot planning and execution?
• How can a user build the agency’s capabilities in ML?

Agencies need to be aware of ML’s potential benefits as well as its challenges, limitations, and risks to make informed decisions about how it is deployed within the agency and across the transportation network. To help state DOTs and other transportation agencies kickstart (or expand) their journey into the rapidly evolving world of ML, the research team developed a 10-step roadmap to building agency ML capabilities, see Figure 15. The roadmap attempts to be sufficiently specific to ML at state DOTs while remaining method-agnostic and adaptable to the changing ML landscape.

The roadmap shown in see Figure 15 consists of 10 steps and includes a loop from step 5 to step 2 to emphasize the iterative nature of the ML development and implementation process. This roadmap is broken down into the 10 steps summarized below:

Step 0. Develop an understanding of ML key concepts and applications in transportation. This step summarizes what is needed to develop an ML model, highlights differences between ML and traditional methods that agencies are familiar with, and summarizes core and emerging ML concepts. It also includes examples of how ML has been used successfully in transportation so far as well as a discussion of ML techniques expected to impact transportation in the near future, such as Large Language Models (LLMs). It is labeled as “Step 0” for two main reasons: (1) it assumes some agency staff may already understand the basics of ML, and (2) in true data scientist fashion the roadmap is indexed at zero.

Step 1. Identify candidate transportation use cases where ML could support agency needs. This step discusses primary ML capabilities useful for transportation use cases (i.e., detection, classification, prediction) and includes examples of the types of transportation problems that can potentially be solved by ML methods, rooted in agency success stories.

Decision Gate #1. After getting a sense of the landscape and possibilities for ML, the agency is faced with its first major decision gate: is ML a suitable approach for one or more of our agency’s needs? If

so, in what capacity? This decision gate includes a checklist to help agencies decide whether ML is a feasible and desirable approach for one or more of their needs, based on the main criteria of suitability and maturity and their assessment of candidate transportation use cases.

Step 2. Assess the availability of current resources to support ML and any gaps, including data, data storage, computing, workforce and organizational considerations, funding, and other considerations, including privacy and policies.

Step 3. Build a business case to secure leadership buy-in. This step discusses best practices in building a case for leadership to pursue an ML pilot project to support broader agency capabilities. It also includes information on identifying the value proposition of leveraging ML, including potential benefits and ROI, as well as spreading awareness of uncertainties, potential risks, and costs to different stakeholder audiences.

Decision Gate #2. The agency team will need to decide if and how to plan an ML pilot project given the resource availability and gaps identified in Step 2 and leadership appetite from Step 3. If a “go ahead” decision is made, options may include executing the entire ML pipeline and developing all code in-house, leveraging available open-source code and applying it to the identified use case, or acquiring proprietary services, data, software, and/or consulting to support one or more aspects of the pilot. This step includes important questions to help teams decide which option to pursue for the pilot and summarizes the different benefits and risks of each approach.

Step 4. Plan the ML pilot project, including the pilot scope, schedule, and budget. This step includes a detailed hypothetical example of pilot scope planning for a transportation asset management use case. It also includes important best practices, lessons learned, and examples concerning pilot schedule and cost considerations.

Step 5. Execute the ML pilot project, including one or more aspects of the ML pipeline if the agency plans to build a custom model. During pilot execution, the agency may discover they need more data or other supporting elements for the project to improve model performance or system integration. Therefore, there is a loop back to Step 2: Assess Gaps to illustrate this iterative process.

Step 6. Communicate results of the ML pilot. At this stage, the team should share the results of the pilot project, including relevant evaluation metrics, with others across the agency to spread awareness and build buy-in.

Step 7. Scale from a pilot to a larger deployment. If the pilot project demonstrates promising results, it can be scaled by location, time, user base, and/or scope into a larger deployment.

Step 8. Conduct operations and maintenance (O&M) of the newly deployed system. It may be helpful to assign one or more team members to be responsible for overseeing regular O&M for the ML application, which includes monitoring the input data and the ML model’s performance for signs of drift. Additionally, it is crucial to consider the deployed system’s life cycle costs and secure ongoing funding to ensure it keeps operating smoothly.

Step 9. Continue to expand ML capabilities across the agency. This includes building out a broader enterprise AI/ML strategy and enterprise data management strategy, building workforce capabilities and training staff, fostering a data-driven culture at the agency, and collaborating within and across agencies.

Insights in the guide are based heavily on best practices and lessons learned synthesized over the past two years from a variety of tasks, including a literature review, a survey of agencies that received 43 responses, and five case studies based on interviews with state DOT teams developing and/or deploying ML in different capacities. The top guide takeaways are summarized below.