Hoosier Highways Flow More Safely, Smoothly With Video

Indiana DOT Removes Network Roadblocks with Extreme Fabric Connect

When an incident occurs on Indiana’s urban Interstate highways, motorists can rest assured that emergency personnel are often on the way before anyone even dials 911. Thanks to a statewide network of around 350 cameras, the Indiana Department of Transportation (INDOT) closely monitors roads around the clock for any sign of trouble, and dispatches the appropriate responders, whether that’s Indiana State Police, other emergency vehicles or INDOT Hoosier Helpers sponsored by State Farm, INDOT personnel who patrol freeways to help motorists and keep traffic moving. Cameras assist INDOT in providing an online map of real-time traffic images, and enable the state to deliver video feeds to news station partners. With those cameras dispatching video across the network, the flow of network traffic at INDOT directly impacts the flow of roadway traffic.

Navigating Network Roadblocks

While INDOT response may look polished to motorists, back end management of the network supporting the
camera system has – at times – hit roadblocks. “We had a sprawling, undocumented network that was kind of unruly and hard to manage,” explains Ben Fischer, Network Engineer at INDOT. “We had multicast problems, especially related to PIM (Protocol-Independent Multicast), with many different versions of PIM running on different codes and different boxes.”

Those challenges often resulted in intermittent video or outages. Then troubleshooting could take hours, or even a couple of months as was the case of an outage that affected two-thirds of the network. Fischer himself spent, on average, approximately 20-30 percent of his work week on PIM and multicast issues.

Extreme: Reducing Complexity

Fischer and the rest of the small network team began exploring the market for a solution to reduce multicast complexity and add redundant links. Integration Partners, a partner in Indianapolis, introduced INDOT to Extreme Fabric Connect as a way to simplify configuration and management, and increase network flexibility and availability.

“We ultimately chose Extreme after comparing several vendors,” Fischer says. “Shortest Path Bridging, along with Fabric-related devices, and the simplicity that SPB provided over PIM would reduce our ongoing costs of operations and provide the best value to the State of Indiana.”

The Flash Cutover

As INDOT prepared to make the move to Extreme, the team spent several months planning and preparing with Integration Partners, which included setting up configurations, getting devices ready and testing in the lab. Fischer ramped up with a one-day, hands-on class that walked him through building the entire network.

“We spent a lot of time talking through everything – potential issues and what was most critical to have up and running within about a half hour of the transition,” Fischer says. “We walked through it multiple times from different angles.”

Then, the team set an ambitious goal: transition all 28 locations at once, during the day between the morning and evening rush hours. “The overall simplicity of deploying the SPB network made it possible for us to consider a flash cut and to do it at a reasonable price,” Fischer says.

INDOT kept manpower demands and costs down by using Cradlepoint wireless routers to connect some of the sites remotely, without having to physically send someone out. “First we thought we’d need somebody with a high level of expertise at all 28 sites, but we had 2-3 true experts with console access, and then 15 others supporting us, significantly reducing our labor costs,” Fischer says.

“Indiana Department of Transportation wanted a network that could face the challenges not just of today, but also tomorrow,” comments Ed Walton, VP of North American Channels and Alliances at Cradlepoint, Inc. “We were happy to provide a solution that not only met their system architecture needs, but helped them to avoid unnecessary operational cost.”

When the day arrived, extensive planning resulted in a smooth cutover, despite the fact that a legacy link failed 10 hours before. “Leading up to it, there was a lot of stress, and a little bit of terror that we’re going to essentially tear down the whole network one Tuesday morning and then stand it right back up,” Fischer says. “But within a few minutes of the cutover, stuff started coming back up, and we saw better capacity than 10 minutes before the change. Everything just worked. The whole thing was pretty magical.”

Easier, Safer Management

With Extreme Fabric Connect, INDOT gained a more stable, easier-to-manage network that will support its current infrastructure and growth. If something happens to go out, Extreme Fabric Connect uses redundant routes to direct traffic automatically via the shortest path. For the team, that means not having to drop everything during the day – or worse, at night – to troubleshoot.

“We don’t get near the volume of call outs as we did before, helping to improve retention and quality of life for our support staff,” Fischer says.

The same goes for routine maintenance, where the team has swapped trepidation for confidence. Where before the team chose not to stand up redundant links, especially in the middle of the day, now they do it without worry.

“It could have easily been an 8-plus hour day to just troubleshoot why one link failed,” Fischer says. “And it would have involved numerous technicians who would have to stop their normal jobs trying to get it back up. Whereas now, if we have a link like that fail, we won’t have to drop everything.”

Standing up devices also takes just minutes, compared to hours or days before. In the case of a fiber cut, before Extreme, it took 30 man-hours to get VLANs fixed to enable video at a busy point on a highway.

With sites on the side of highways, remote capabilities now mean safer operations and less time behind the windshield.

When INDOT needs to reach out to Extreme, techs get rapid resolution. “Getting tickets resolved with Extreme is always prompt and they are willing to partner with us to find solutions,” Fischer says. “Problems are escalated to the appropriate groups and people will work around the clock to get something resolved. Extreme, Integration Partners and Indiana DOT, it’s just been a wonderful partnership.”

Up Next

Extreme Fabric Connect gives INDOT a stable network on which to scale easily as needed. Next up: more cameras for an expanded view of road activity. Again, technicians will be able to troubleshoot remotely. On the back end, they can view the camera in real time and make adjustments without the risks that come with sitting in the middle of a busy road.

All this results in safer technicians and roadways. “At INDOT, we strive to try to be on the forefront of technology to reduce the need for always adding expensive new lanes,” Fischer says. “With Fabric Connect, we are more efficient; our engineers spend less time troubleshooting so they are able to focus on other business needs.”

About Indiana Department of Transportation

The Indiana Department of Transportation (INDOT) plans, builds, maintains and operates a superior transportation system enhancing safety, mobility and economic growth. Hoosier Helpers is the INDOT freeway service patrol designed to provide an extra measure of safety on Indiana Interstates and around major cities with the goal of keeping traffic moving.

Solutions