Some agency chief information officers are trying to remove some security tools from their networks.

Why? Because there are just too many.

Larry Grossman, acting chief information security officer (CISO) at the Federal Aviation Administration, said he has a team looking at how to reduce the number of tools. Grossman was speaking Oct. 1 at the Northern Virginia Technology Council’s Cybersecurity Summit.

Across the federal government, agencies have so many security tools deployed on their networks that they may not know the exact number. Several agencies don’t know what tools even sit on their networks, according to several press reports. In addition, industry officials have told Fifth Domain that some agencies have so many vendor products on their networks that it obstructs visibility of what’s actually happening.

Grossman said he’s adopted a policy of removing two tools for every one tool the FAA purchases for network security, similar to a rule for adding regulations that’s been set by the Trump administration. The Department of Homeland Security also has “too many” cybersecurity products on its network, said Paul Beckman, the agency’s chief information security officer.

Beckman said he’s trying to determine a way to measure the best tools deployed on his network.

“If you’re not doing what you purport to do ... it’s something I’m certain not to be reinvesting in next budget cycle,” Beckman said.

To help evaluate the effectiveness of the tools, Beckman is using the Lockheed Martin Kill Chain and attack simulation technology. The attack simulation technology allows Beckman to run a cyberattack through his entire network infrastructure and find vulnerabilities within it. By doing so, Beckman can identify which security tools aren’t working.

“When I’m looking at the phases, I’m asking the question every single time ‘where’d I catch [the attack]?'and ‘what failed?,’" Beckman said. “Because if I caught it at the very end ... everything leading up to that has failed me."

If Beckman finds that a security tool is failing continuously, “you’re going to go out the door.”

“All these things should’ve caught it before it got to Point B — if it didn’t, why?,'” Beckman said. “And a lot of times it will be a misconfiguration ... but sometimes it’s just not doing what it purports to do and it needs to get out.”