07 Mar, 2019

Optimizing a Security Assessment Engagement

by Mark Moses

Having examined real-world threats, the regulatory environment, shareholder demands, customer concerns, and pressure from the C-suite, you have wisely decided to engage a trusted partner to perform a security assessment. This might be an assessment of a web or mobile application, a cloud environment, or a device destined to join the "Internet of Things." Regardless of what the scope will be, you surely want to maximize the return from your investment in the assessment.

The hope I have in writing this blog post is that you will start to see your security partner as an actual partner in the process of identifying potential vulnerabilities in your application, website, network, or device. To start, we'll go through the steps and describe the actions you can take throughout the process to ensure that you and your entire team, from development to operations to the security team, ends up feeling the exercise was worthwhile and walk away with actionable information.

First Steps

Let's start with the elephant in the room when any security assessment or audit is discussed. People generally don't like them. They take up time and resources from the day-to-day work required to complete projects, implement upgrades, develop an exciting new application, or release a new product. Having spent most of my career on the other side of these assessments and audits, I can say I never recall anyone celebrating that a security assessment was scheduled. From the start, the very people who could gain the most from the assessment can be set against it. Turning adversaries into allies can be a daunting challenge, but a rewarding one.

Communication is absolutely the key here. Letting the team – software developers, cloud or network engineers, DevOps teams – know that the purpose of the assessment is not to play a game of "gotcha" to point out where somebody missed something or to pick apart coding practices, but that it is crucial. Use language that is positive and shows the value-add of having a third party perform this work. Various types of assessments will have different attributes to stress. Briefly, third-party evaluations of web and mobile applications often reveal weaknesses that arise from outside libraries, configuration items on servers or databases, or legacy applications that the application is leveraging. Evaluations of physical devices with their operating system and application software can yield simple changes that drastically reduce the attack surface.

Again, it's vital that leadership stresses that findings resulting from a security assessment will not be viewed as mistakes. It is this very mindset that can create a negative adversarial relationship. We need to stress to our teams from the beginning that findings will happen when professional, white-hat security teams utilize specialized knowledge and tools to attack a target. Real-world threats and white-hat hackers share a desire to seek out weaknesses and attempt to exploit both known and novel vulnerabilities. Some of the most talented development teams, bulwarked by top-notch security teams, have still been breached, often with disastrous consequences. The takeaway here is that it will take an outsider with a very different perspective to locate and exploit a weakness in your system. It's best if that outsider is working with you, going through your system methodically and carefully looking for any foothold, and able to report to you what was tested, what works, and what doesn't.

In cases where the team you employ utilizes security specialists who originated in the development space, like nVisium's consultants, you are not just getting a long list of findings to sort through. Instead, you are getting expert guidance on not only how to fix the underlying code or configuration, but the methods and best coding practices to avoid that weakness in future development. A proper security assessment is more than checking a box to assuage your stakeholders' concerns – it is also a staff development exercise where everyone wins.

TL;DR – Positive communications: get your team on-board early, and the third-party security provider isn't the enemy.


Carefully consider the scope of the assessment. Is this just an API, an entire application, a suite of applications, or the suite of applications and the infrastructure it is running on? Do you prefer a "Black Box" style, where your security partner is going in blind, or would you prefer to supply source code and architecture diagrams that will allow a team to provide solid feedback on secure coding techniques and architecture design elements?

When scoping, gather as much information as you can. What languages and frameworks are involved? How many lines of code are in each application? How many static and dynamic pages, APIs, methods, and user roles? How many hosts? What does the architecture look like?

Representatives from operations, security, and the development teams should be involved in creating the scope to ensure that related or dependent systems are either deliberately included or excluded. A solid foundation for an assessment is a scoping questionnaire that clearly delineates what is being tested and what type of assessment is being performed. While determining what information, code, or credentials may be delivered to a partner, referring to the scope will inform what code base, what documentation, and which credentials will be required for the assessment.

Planning Around Deployment Schedules

During the scoping and scheduling phase, integrate the planned test window into your development and deployment schedule. Avoid rolling out new software versions, database upgrades, library upgrades, operating system patches, and so on during the test period. It is detrimental to the efficacy of an application assessment when the environment is not stable. Security assessments are often targeted against a quality assurance or staging environment rather than production. This elevates the importance that the test is included in your deployment schedule. Ideally, the target environment will be configured exactly as the production environment is currently or as the production environment will be at go-live. The closer the match of the test environment to the planned or existing production environment, the more relevant your results will be to your stakeholders. While a one or two week freeze to major code updates to staging environment may seem like a serious pain point, it will also minimize the time spent discussing false positives that occur because the staging environment was in flux during testing or does not match the production environment.

Preparing for the Kickoff Call

The type of assessment you are embarking on will determine what information will have to be provided for testing. Let's run through some of the most common scenarios to give you some ideas to make the kickoff call go smoothly and set you up for a successful and cost-effective assessment.

No matter what kind of security assessment you are performing, some common decisions need to be made.

  1. Who will be the primary administrative and technical points of contact? They should have the knowledge or connections to get information or make decisions quickly from their respective standpoint. Though nVisium assessments require little overhead from the client when testing, the primary point of contact should be available by email, corporate chat client (Slack, Teams), or phone should any issues during testing.
  2. What is the escalation protocol should testing uncover a serious issue? Which point of contact is appropriate in those cases? Ensure that your testing partner has a clear escalation procedure and points of contact to quickly resolve any issues should any arise.

In a pure "Black-Box" assessment, what the application does, how your clients/users access the application, and the URLs in scope are still relevant and should be provided.

How is your application accessed? Is it only accessible after employees have been validated via Single Sign-On? Is it behind a firewall, load balancer, or both? Is it on an internal network that is only accessible via a VPN? Consider your firewall or VPN configuration. Will you have to whitelist the team's IP addresses? Publicly exposed web applications and APIs do not exist in a vacuum but may be part of a larger system of authentication that will have to be either tested with the target or opened to the testing team. Be prepared to provide the required network, VPN, and application credentials to your security partner.

Does a third-party host your application infrastructure? It is likely that if the target of an application assessment is hosted in the cloud or at a hosting provider, they require notification ahead of testing; effectively granting permission for the assessment to proceed. It is prudent to check with your providers when you are scoping an assessment and notify your security assessment partner if an outside party hosts the servers. Your security partner will be prepared to walk you through the notification process with major vendors.

Are users to the application created via self-service or do you need to provision them? How many roles does the application utilize? Best practices for your assessment will have the team trying to exploit both vertically and horizontally. To best accomplish this testing thoroughly, this will require multiple accounts. For example, in a web-based system that is licensed out to multiple clients and that has roles for users, client-specific administrators, and cross-client administrators (your internal team), it would be prudent to provision:

  • 1 user role for each of 2 clients
  • 1 client-specific administrator for each of 2 clients
  • 1 cross-client administrator

In this example, those five users will allow the testing team to attempt vertical access escalations and horizontal access escalations with the ability to validate success or failure. Your environment may be set up for a single client but have many more roles. Think through your particular situation and anticipate this discussion on the call.

In a hybrid assessment, your partner will be reviewing your source code and validating findings with a dynamic assessment in the runtime environment. How will you be providing access to source code? Temporary GitHub access? A secure file-sharing site? Sometimes the political environment inside a company makes this decision more difficult than it might sound; thinking through it before the kickoff call will facilitate a smooth discussion.

During the Assessment

While in progress, you may hear very little from your testing partner depending on the reporting requirements in your Statement of Work. You will likely get a status report at the end of each week. nVisium, as typical with most security partners, will notify you immediately if they find a critical security vulnerability or evidence of a pre-existing breach. It is important that the identified points of contact be available should anything critical come up that needs to be addressed to keep the assessment moving and on schedule or to communicate to the leadership team if required.

Maximizing Your Read-Out Call

The wait is over, and your draft report has been delivered and distributed internally to the key players. Have a schedule in place early with reasonable expectations for your team to review the report and formulate their questions and concerns. Consider entering the findings into your bug-tracking platform, such as Jira, to manage all assessment results and get the remediation efforts into your development cycle as soon as possible.

Next, we recommend scheduling a call with your security partner to go over the report. This is a great time for you to discuss the suggested remediation strategies and pin down any changes to the report language that everyone can agree on. Most importantly, make the most of your time with the authors to get their input into the strategies you can use to improve on your adoption of secure coding techniques, configuration, and system architecture.

Wrapping Up

Are you ready to maximize the return on this investment? Approach a security assessment not as a required chore, but as an opportunity to minimize risks, reduce the threat surface, mitigate security vulnerabilities, and identify areas of possible improvement. With a well-considered scope and some advance planning, a robust security assessment can occur without disruption to the daily routine or the deployment schedule. Most importantly, a security assessment will add significant value to your development and operations teams, while satisfying the requirements of those auditors, shareholders, and other stakeholders.

Plan ahead. Communicate positively to the team. Ensure you have the necessary resources available for preparation, the kickoff meeting, provisioning of information and access, and finally for a thorough review of the findings. You just might find that the entire exercise was not only worthwhile and rewarding but also – dare I say it – fun.