While I agree with this the security clearance process is broken, Clapper is probably the worst person to cite to make a case here. Strike that. Second worst. After Comey.
A book can be written on the main reasons for the process being broken, but I would say the biggest problem, the central and overarching problem, is the complete bureaucratization of the process. Atul Gawande, M.D., a noted Harvard Medical School professor and bestselling author, pointed out in his book “Checklist Manifesto” that in situations involving real-world complexity (e.g., security clearance investigations) workers need room to move and adapt. If not then the system will breakdown. All the planning and drills and operating procedures will become worthless and dysfunction will reign. In discussing the Hurricane Katrina disaster-- textbook example of the complete breakdown in coordination and functioning of federal, state, and local emergency management agencies-- Gawande had as interesting take on it:
"“The lesson of this tale has been misunderstood. Some have argued that the episode proves that the private sector is better than the public sector in handling complex situations. But it isn’t… No, the real lesson is that under conditions of true complexity—where the knowledge required exceeds that of any individual and unpredictability reigns—efforts to dictate every step from the center will fail. People need room to act and adapt. Yet they cannot succeed as isolated individuals, either—that is anarchy. Instead, they require a seemingly contradictory mix of freedom and expectation—expectation to coordinate, for example, and also to measure progress toward common goals.
This was the understanding people in the skyscraper-building industry had grasped. More remarkably, they had learned to codify that understanding into simple checklists. They had made the reliable management of complexity a routine.”
Security clearance background investigations have become quintessentially ‘dictating every step from the center’. Exactly what Gawande says will inevitably lead to failure. But there are no live camera shots of people stranded on rooftops so there is no sense of urgency or willingness to change things. Except the movement toward greater and greater bureaucratic requirements.
This job-- at least on the contractor side-- attracts men and women with extensive investigative backgrounds and experience. Or at least it used to. Former LEOs, S/As, et al. And during background investigations most of their time will be spent getting to the bottom of how a Subject could actually work full time employment concurrently with a part-time Army Reserve employment (and usually 6 years ago on a 5-year PR case). Or how Subject can go to school part time and work part time. Concurrently! And most of the attention of the investigator during the ESI is NOT spent bluffing and acting asking the Subject to scour his brain for any compromising aspects of their past life-- NB: bluffing and acting elicits the best derogatory information and stuff not found in records or known by friends-- but forcing the Subject to scour his brain for people that can cover certain time periods and employments and residences and who can tell the investigator what a “straightshooter and the most sterling character of anyone they know” the Subject is.
Having former detectives, special agents, investigators, et al., doing the investigative work of security clearance investigations within the confines of the extremely bureaucratic process is like having experienced and talented artists creating art with only an Etch A Sketch. (RZ…IHB memo 4.2.7-43 “OPM requires the use of at least three straight lines that cross over when drawing a…”).