DevSecOps and Audit Compliance

Integrating Compliance Auditing with DevSecOps

By now you have probably at least read about the benefits of merging development, security, and operations into a cohesive unit (if not also implemented to some degree). Now it’s time to take it a little further: integrating security compliance audits. Whether you face mandated audits, like PCI, FISMA and agency specific implementations like Security Controls Testing (SCA) or NISTIR 8011 flavored Adaptive Capabilities Testing (ACT), or self-imposed assessments for your own [good] reasons (or both), integrating an external audit with your system development and maintenance process can help your organization more efficiently remediate vulnerabilities and weaknesses, and the overall audit process will be less costly.

As a side note: I personally like the term “Secure Agile Development”, or SAD, more than DevSecOps…but that’s not important right now. What’s important is coming up with a new catchy, clickbaity term. DevSecOpsAud? AudDevSecOps? No, those are terrible. Let’s keep it simple: Security + Operations + Development + Audit, or SODA.

Summary

Like Agile development and DevSecOps, SODA is easier said than done. I will address the easy part here; implementation is left as an exercise for the reader (or contact me). In theory, SODA involves the following main points:

• Technical/Penetration testing with every deployment, agile sprint or other dev cycle, and major change. Testing will be focused on the changes, much like other testing conducted as part of the development process, and will be constrained to appropriate short periods of time. All of the logistics around testing have been handled before we get to this point, so there’s nothing left to do except the actual testing.

• Gather evidence for other controls as they happen, by being part of the process, or by simply being there to observe. For example: auditors require test accounts; accounts must be reviewed on a regular basis; therefore, the auditee can involve the auditor during the account reviews. Evidence obtained. Contingency plan test scheduled for next Thursday? Invite the auditor. Evidence obtained.

• Audit findings (or the issues that would otherwise be written as annual audit findings) are addressed just like issues reported by non-audit parties. Integrate with ongoing development efforts (e.g. add to sprint backlog) or related tasks, like updating the beloved system security plan.

Adaptive Capabilities Testing

Agency-specific next generation auditing based on NISTIR 8011, relying heavily and automation and focusing on capabilities rather than individual controls, is an ideal alignment with SODA. Both SODA and ACT also lend themselves well to something we have always done instinctively: risk-based assessments and testing. Focus the most time and effort on the areas that present the most risk to the organization. And don’t worry if your organization is still near the starting line of implementing ACT–or possibly not yet at the starting line–SODA can be enjoyed just as much with the tried-and-true NIST 800-53 security controls assessment as it can with ACT.

Benefits of SODA with DevSecOps and Agile Environments

So why drink the SODA? The answer is pretty much the same reasons for security to be integrated with development and operations: improved risk management and better efficiency. Ultimately, this will give your organization a better return on audit (ROA). Audits require a decent amount of coordination, logistics, and time/effort by all involved parties. This may involve planning meetings, audit/test plans, test accounts, on-site visits and network access, VPN/remote access, test accounts (more on this later), evidence gathering–the dreaded Document Request List (DRL), assessment meetings, status meetings, meetings to discuss findings, probably more meetings, eventually a report of some kind, maybe more meetings to discuss the report… It just makes sense to streamline this as much as possible.

Did you notice that I didn’t mention two major aspects around an audit? Neither one actually involves the auditor:

• Preparation. I mean…who doesn’t love an eleventh-hour mass deployment of patches? So many hours spent by so many people trying to ensure that the security vulnerabilities and weaknesses are either fixed just before the auditors arrive or are hidden by enough smoke and mirrors to prevent the auditors from finding them. And yes, the auditor knows you do this. But this approach leaves the organization open to attack–at least outside of audit time–and it defeats the real value of an audit, which is to discover problems and develop effective solutions. Our goal: no more last-minute patch and bug fix scramble.

• Recovery. The work may not stop with the final report, especially not if there are findings. Simple fixes could be resolved relatively quickly, possibly incorporated into the next agile sprint and may even be remediated before the audit ends. The more complex issues, remediation activities requiring more time, risks that may be accepted by the CISO/Security Officer, and other finding paths forward could easily cover months or years until final resolution. Depending on the organization’s compliance requirements, this may involve the also dreaded POA&Ms and CAPs (and regular status reporting that goes with them).

Another benefit of integrating audits with DevSecOps is increasing the chance of finding issues that very dedicated attackers with relatively unlimited resources could find, but may remain hidden from even talented security professionals restricted to the annual 1-2 weeks of testing. This is the same basic concept and one of the reasons for integrating security with DevOps. The last point I want to make here may be the most controversial of the entire discussion: it should be relatively easy to implement SODA at the same cost as doing an annual review all at once (if not less expensively). I should probably repeat that: at the same cost!

We have almost completely automated the planning and reporting requirements–as well as other aspects–of our assessments (of the audit, consulting/advisory, and other varieties), which means we can put more focus and effort into the actual testing. SODA could give you a similar benefit, even without the automation efforts.

Recommendations for Success

I like to say no two agile environments are alike. However, there are some common ideas and attributes of integrating otherwise disparate processes. For SODA–and, quite frankly, for DevSecOps and for integrating any kind of security testing with development or operations or IT in general)–we present the following recommendations:

• Break down testing into short segments that align to the changes in the system/functionality since the previous testing (i.e. align to your agile sprints, if I wasn’t clear enough).

• Develop your people into subject matter experts (SMEs), both on the application security side and on the development side. With DevSecOps, you can develop your SMEs internally. With SODA, you can get an auditor that also knows your system well.

• Integrate the teams (audit team with DevSecOps team) and the testing. The SMEs can bridge the gaps and help the audit team better understand the system and integrate better with DevSecOps at your organization. Yes, there are always independence concerns when an audit is involved. However, everyone should work together to make the annual audit run as smoothly as possible; the only difference here is that the same coordination and audit is being cut into slivers and spread out across the audit period (1 year / 365 days).

• Define and refine the testing logistics. Make it seamless:

  • Points of contact for every involved organization for every component; ensure everyone is kept up-to-date.
  • Scheduling - too many variables for a summarized recommendation, but coordinate the audit testing along with all other testing and development cycles.
  • Test environment dedicated to audit testing if possible; bonus points for automated environment builds and pushes. Otherwise: coordination, coordination, coordination!
  • Convert findings into remediation tasks (bug tracking).

• Face the Agile dirty word head on: documentation. This is a good recommendation even if you never integrate audits into the process. Auditors always need documentation and other evidence, so you may as well make it as easy as possible. Give the auditors access to the system documentation, even in draft form, for faster and better ROA.

• Auditors should be able to collect evidence for various controls simply by participating in the process. For example, if you are required to review accounts and access on a regular basis, then the auditor can be included in the communications between relevant parties performing these reviews.

Don’t Fear the Auditor

They (or We, when wearing the auditor hat, of course) are here to help. Seriously, we are not “out to get you”. While the hat, I have been questioned or outright accused many times about getting paid by the finding; I have never heard of this actually happening. On a similar note, we the “sometimes auditors” don’t want people to lose their jobs or otherwise be reprimanded due to audit findings. Much like peer reviews or taking a break while writing a report, it is often very helpful to have an outsider or rested eyes look at your stuff.

It can be helpful to know the things auditors need, like documented controls and system documentation (to see how things should be implemented), test environment with working test accounts (to verify how things are actually implemented), and other evidence (more implementation validation). Almost equally helpful is knowing that an auditor cannot accept words as proof of implementation, which is part of Auditing 101 classes.

Questions and Decisions

Auditors are good at finding things (well, some are). With audit testing occurring more frequently than once per year and closer to development, it is quite possible there will be more findings. The main question for management, the auditors, and the compliance committee to answer is: which issues become official audit findings? and when? and how? How much time does the auditee get to fix identified issues before they become audit findings?

Another basic question, but this time with many possible answers, is: how do we integrate audit testing with other security testing? Should we allow audit testing to occur simultaneously with DevSecOps testing? Run it as part of each 2-week sprint, or maybe just every other sprint? Can the organization easily support another environment dedicated to audit testing, or do we need to coordinate testing schedules with other teams/personnel?

The answers to these questions and the resulting decisions should be customized to fit your environment, people, processes, technology, security goals, and (of course) compliance requirements.

Conclusion

So there it is: the next logical step beyond DevSecOps. Go drink your SODA. Maybe just try the diet or zero-calorie version first.

If you have any questions about reducing costs and efforts in your compliance audits with seasoned penetration testers, please reach out to us at 877-752-7170 or contact@fyrmassociates.com