Measuring Secure Software Maturity


Robert Vamosi, Contributor

October 22, 2015

All software is flawed. That’s a given. But what can be done about it? In the late 1990s researchers began studying ways that a programmer might contribute to (or unintentionally undermine) the security of a software system. It became clear that getting involved at the earliest software development level made sense. However, at some point, it also became apparent that software security includes business, social and organizational teams, that a more holistic approach was needed. And it was out of studying the existing software security initiatives in place at several major organizations that the Building Security in Maturity Model (BSIMM) was born.

“BSIMM began as a science experiment,” said Gary McGraw, CTO at Cigital and one of the BSIMM co-founders. “What started with a nine firm ‘test tube’ has escaped the lab and expanded into a de facto measurement standard describing the work of over 104 firms.”

On Monday, McGraw, along with co-authors Jacob West, Chief Architect at NetSuite, and Sammy Migues, Principal at Cigital, released a new report, their sixth in the seven years of the BSIMM series.

“We built BSIMM because there was simply too much ungrounded opinion flying around in the computer security space,” McGraw said. “We figured a set of objective, descriptive facts would help. They do, and the measurement tool is very powerful.”

The current data set includes 208 measurements from 78 firms in multiple vertical markets. The BSIMM report is not necessarily a How To. It is a snapshot of the state of secure software development lifecycle, a reflection of the current state of secure software development within these representative organizations. Although tools on the site will help any new organization track and adopt these best practices for themselves.

BSIMM is meant for executives responsible for initiating and maintaining Security Software Initiatives (SSI). Often, McGraw said, they are part of some internal group which BSIMM calls the Software Security Group (SSG). In terms of sheer numbers, BSIMM draws its current strength from 1,084 SSG members, with 2,111 additional members all trying to secure software developed by 287,000 developers.

“The most remarkable change in the BSIMM has been the growth of the data set, which is currently 29 times as big as it was when we started,” McGraw said. “The BSIMM gets its power from real data carefully gathered through field observation. Data don’t lie.”

For example, it’s not enough to have a SSG, mature organizations need a satellite system of others such as developers, architects, and people within the organization who are actively engaged in and promoting software security. One hundred percent of the top scoring companies have satellites. Not surprising, zero percent of the lowest scoring companies have them. This suggests that as a secure software development initiative matures, according to the report the work gets more distributed within the organization.

“We were surprised that the measurement tool we built has always worked to measure every software security initiative we have ever come across,” McGraw said. “We say ‘there are no special snowflakes’ to signify the power of the tool. With measurement comes science. And with science comes rapid improvement.”

The twelve categories are architecture, code review, security testing, penetration testing, software environment, configuration management and vulnerability management, strategy and metrics, compliance and policy, training, attack models, security features and design, and standards and requirements. The report tracks 112 activities within 12 categories within each organization.

As a longitudinal study 26 of the 78 firms were measured twice, about two years apart. In the second survey, the raw score went up in 21 of the 26 firms with perhaps the most growth seen in training activities. This, the authors conclude, shows a clear trend for increased maturity.

“We’re very proud of what we’ve accomplished in the BSIMM,” McGraw said. “The measurement tool is a testament to the dedicated BSIMM Community who together are solving the hardest problems in software security.”

This article was written by Robert Vamosi from Forbes and was legally licensed through the NewsCred publisher network.

Comment this article

Great ! Thanks for your subscription !

You will soon receive the first Content Loop Newsletter