Interesting Forrester / RSA report

[bigmike82]

Found here.

It's titled "The Value of Corporate Secrets."

I'm still analyzing and composing a response, but if nothing else, the article is a great way to see how CISOs of larger companies think about their security.

One of the biggest flaws that I've found so far is that the authors basically assume that compliance has no impact in protecting a companies 'secrets'. It also understates the impact that spending on compliance has on overall security.

[socalblue]

That's exactly right. Far too much is spent on simple compliance (Corporate policies or government regulations) without regard to how it will help secure the enterprise.

A company can be 100% compliant with government regulation & still be easily hacked. Security must be handled in a pragmatic, multi-layered manner to be effective.

[bigmike82]

I agree but disagee.

The base assumption that they made in the report is that spending on compliance has 0 benefit to security as a whole. That is a bad assumption to make.

[bigmike82]

Here's a more thorough response I wrote up. If anyone's got any feedback, I'd much appreciate it.

IT Security spending is subject to the same debates as any other budgetary area in todays enterprise. Just like in Macroeconomics, budgets are designed to allocate finite resources among infinite demand. Finding the right balance between different areas is often the object of long and intense discussions among the key decision makers.

Within Information Technology, different areas are all competing for a certain amount of dollars. The slice that is allocated toward Information Security is must be prioritized to meet the requirements of the enterprise and include satisfying regulatory, policy, personnel and security as a whole. Due to the lack of indepth research, many key findings of this report are cast into doubt. For example, it is easy to suggest that compliance spending is greater than it should be based on the value the enterprise receives. This key finding, however, must be analyzed more thoroughly before it can be accepted. Each enterprise must ensure that its compliance spending does not increase security as a whole before it can follow the logical step and reallocate this spending to other areas.

Forrester Consulting was engaged by Microsoft and RSA to conduct an assessment of IT Security practices among large enterprises throughout North America, Europe and Australia. After an analysis of the results, the key findings included the value of corporate secrets, the spending pattern in relation to those secrets, and how the value thereof influences the number of incidents.

While many of the key findings are accurate based strictly on the surveys conducted, this report fails to correctly analyze the influence of spending on compliance. A key point in the paper is that “investments are overweighed toward compliance.” This conclusion can not be supported by the evidence provided, similarly to the way that the report also criticizes the role that accident prevention plays in Enterprise Security policy

Value of Secrets
Forrester allows the CISO decision makers to place a value on the secrets under their care. While this may be a good way of determining overall metrics, I do not believe that these values are inherently accurate. CISO’s areas of responsibility lie in determining security policies and enforcing them, not in determining accurate business data values. Only someone who is knowledgeable about the material itself as well as the market as a whole can place a true value on these secrets. CISOs, by the nature of their position, do not have the time and budget to accurately perform market research or perform competitor analysis on all pieces of information in their inventory. Therefore, great care must be taken when determining action based on the 'value' of these corporate secrets.

Compliance spending
The Forrester report fails at correctly analyzing its data in this key finding. The Report justifies its finding by the following statements:
Enterprises devote 80% of their security budgets to two priorities: compliance and securing sensitive corporate information, with the same percentage (about 40%) devoted to each. But secrets comprise 62% of the overall information portfolio’s total value while compliance-related custodial data comprises just 38%, a much smaller proportion. This strongly suggests that investments are overweighed toward compliance.
More details are provided later in the paper. Forrester found that, on average, 39% of the budget is related to compliance of various types. More significantly, however, the report glosses over the fact that a fairly large portion of this is dedicated to complying with INTERNAL security policies. The report assumes that compliance with internal security policies has no impact on security as a whole. Considering that internal security policies are designed to secure the corporate network (and, by extension, corporate secrets), compliance with these policies DOES have a positive impact on security as a whole. Their findings indicate that 41% of the budget is dedicated to protecting corporate secrets while 39% is dedicated to compliance-driven projects and technology. Assuming that half of this compliance spending is spent on internal compliance, and further assuming that one half of this is dedicated to protecting secrets in some way, an additional ten percent is added to protecting corporate secrets. You now have 50% of your budget dedicated to protecting secrets, while only 20% dedicated to complying with external requirements. These assumptions are conservative in nature, though I have not, as yet, found hard evidence to support them. In any case, failing to properly quantify these issues in their analysis destroys one of their most important key findings.

Accident prevention
Forrester found that Enterprises focus on preventing accidents, but indicates that this focus is misplaced because of the ratio of cost between accidents and intentional damages. The report does show that accidents were the cause of a majority of security incidents over the past two years. The report then goes on to quantify the damage caused by these security incidents, and correctly indicates that intentional incidents (that is, incidents caused intentionally by an employee or an outsider) are much costlier to fix than the accidental ones. What the report fails to analyze is how a focus on preventing accidents has lowered the cost of these damages, and how the cost of preventing accidents factors into the security budget as a whole. Implementing and verifying data encryption can be done with a relatively low cost, and will serve to drastically minimize the damage wrought by accidents. Furthermore, the report bridges the perception of risk with budgetary investments, an assumption that does not appear to be backed up with hard data. The report found that the likelihood of incidents occurring were greatly focused on accidents, and this is the case. They then drew a conclusion that, because of this risk perception, enterprise security investments are overly biased toward preventing employee mistakes. The data found in the report does not substantiate this finding.

Number of incidents
The report appears to correctly quantify the number of incidents, and the cost thereof, as compared with the size of the enterprise. The report indicates that “Enterprises with more valuable information must spend more time and effort securing them.” This conclusion is accurate, as is their statement that “Enterprises are not spending enough effort protecting data from theft and abuse by outside parties.”

That said, this finding is not without its issues. It does not clarify sufficiently how the enterprises reach the figure of cost per incident. While it is entirely reasonable that an exploited server could cost $300,000 to repair and recover from, it would seem that the survey overstated the impact of a lost smart-phone. $11,000 for a lost smart-phone appears to be excessive. It would have been preferable to include a survey on the items that affect the costs of the response to the security incidents.


CISOs don’t know how good their controls are
This key finding is another area where Forrester fails to correctly make their case. The data does not support their conclusion that “CISOs do not know how effective their security controls are.” Security controls are generally of thee types: Preventative, Detective, and Corrective. In PCI terms, there is also a compensating control, which is a control that is designed to compensate for a failure to comply with a certain aspect of the Standard. The loss of a laptop is a security incident. However, using a preventative control such as FDE minimizes the damage. Strong controls can reduce the occurrence of security incidents, but not always. Because of this, the conclusion that CISOs overstate the effectiveness of their controls can not be made as simply as Forrester attempts to.

Recommendations
Despite its troubling method for reaching conclusions, the Reports’ recommendations are solid, and should be given serious consideration by large and small enterprises. Identifying the most valuable information is not a task simply accomplished by the CISO or even a team of security analysts. Other decision-makers and data owners must be consulted as well. For example, financial forecasts can be very valuable, but the CFO should be consulted to determine an accurate value for this data. Only after consulting the various data-owners, performing market research and competetitve analysis can the enterprise accurately determine the value of data under his care.

I disagree with their two risk categories, as it oversimpliefs the risk management process. Regulatory punishment should be added as a consequence of the risk not being compensated for. This ensures that risks are controlled based on their total consequence, not simply by a compliance policy. One additional aspect which must be considered is the PR and marketing fall-out from an exposure. The report fails to adequately quantify the cost of a high-exposure data leak, and this must be looked at in any risk assessment.

The most important recommendation made by the Report is to “increase vigilance of external and third-party business relationships.” This is of vital importance based on the results of the surveys, and Enterprises must ensure that any access to data is only given to those parties who are willing and able to secure this data to the same standard of the data owners themselves.

[bigmike82]

Methodology issues
The reports methodology is adequate for a shallow survey, but falls well short of a thorough piece of research. It furthermore fails to provide sufficient backing for many of the conclusions made. There are many questions left unasked, and without those answers, the Report should be unable to come to the conclusions it did. For example, the lack of thoroughly determining how internal compliance spending is allocated completely destroys the legitimacy of the compliance spending conclusion.

The conclusions and key findings reached by this Report are cast into doubt upon a closer examination of the methods used to attain them. The survey, as conducted, simply is not capable of providing sufficient data to reach them. Nevertheless, many of the recommendations reached by this report should be closely looked at by Enterprises wishing to strengthen their security. This report is useful as a stepping stone for more in-depth research, but its conclusions should not be used without an in-depth analysis of enterprise-specific spending.