Department of Homeland Security Secretary Janet Napolitano recently made news when she suggested that the Transportation Security Administration’s shift to a more “risk-based” approach would eliminate some of the more frustrating airport security rituals enacted since the 9/11 attacks. Americans, she said, might now be able to keep their shoes going through airport security. But according to the Government Accountability Office, it’s still unclear how effective a key part of that approach actually is.
The TSA’s Screening of Passengers by Observation Techniques (SPOT) program works by deploying “Behavior Detection Officers” who keep an eye out for passengers displaying nervous or erratic behavior. Instead of pulling grandma out of the line or screening for people based on ethnicity, BDOs are taught to look for certain behavior markers that might indicate a threat. A “validation study” completed in April concluded that SPOT “was more effective than random screening to varying degrees.” No terrorists were caught, but the program helped identify thousands of people who had outstanding warrants, were in the country illegally, or had false documents. TSA Director John Pistole said in a speech on September 6 that the TSA was looking into expanding SPOT.
However, the GAO report notes several technical problems with data-gathering in the study, which made “meaningful analyses” of the information gathered on suspicious behaviors in the program prior to 2010 impossible. TSA fixed those problems, but DHS’ validation study included data from before the problem was resolved. The results may also have been biased by the fact that BDOs knew whether individuals were being recommended to them on the basis of SPOT or through random screening. In other words, it wasn’t a blind test.
“It just raises the potential for bias,” says Steve Lord, director of homeland security and justice issues at the GAO. “We’re not sure how significant it was, but this is something that would need to be studied and evaluated in a subsequent analysis.”
Does that mean the results suggesting SPOT is more effective than random screening are useless? Lord says no. “It certainly answered one question, is it better than random?” Lord says. “But there’s some additional work they have to do.”
That said, we also have no idea how much more effective than random screening the SPOT program is. According to the GAO report, TSA considers that “sensitive information.”
