Going Beyond Fuzzing for Airtight Security

In Part 1 of this series , we introduced the software verification technique called fuzzing.  We described what it is, who uses it and why, the tools involved, and the benefits of fuzzing.

In Part 2 , we examined the limitations of fuzzing and how most of them can be overcome by pairing a good fuzzer with formal-methods-based analysis tools.

In this final installment, we’ll look at how to go beyond fuzzing to formally prove you have eliminated every last vulnerability from your most cybersecurity-critical applications.

This post is Part 3 of a 3-part series derived from TrustInSoft’s new guide to fuzzing for cybersecurity entitled “Fuzzing and Beyond.” To obtain a FREE copy, CLICK HERE.

Beyond fuzzing: How to guarantee air-tight security

Fuzzing is a great first step toward eliminating undefined behavior in your code.

However, for applications where security is key and you want to be absolutely sure your code is totally free of all vulnerabilities that can be exploited by hackers, you’ll need to go beyond fuzzing.

While fuzzing with TrustInSoft Analyzer is a great way to detect and eliminate more vulnerabilities than conventional testing or classic fuzzing can reveal, it cannot guarantee perfect coverage of all possible input vectors.

In interpreter mode, the Analyzer makes iterative test runs on the discrete input sets it has been given. But, as we’ve already seen in Part 1 and Part 2 of this series, the input space for most software applications—consisting of billions upon billions of possible combinations—is too immense to be covered completely through iterative testing. If you try, you’ll never finish.

So, while a coverage-guided grey-box fuzzer like AFL can explore your program’s input space efficiently, it can’t explore it completely. Fuzzing alone cannot guarantee you’ve found every bug that may be lurking in your code.

For applications where assurance of a high level of cybersecurity is required, TrustInSoft Analyzer offers a more advanced solution. As we’ll see shortly, this solution is a complement to fuzz testing that can guarantee perfect cybersecurity. We call this solution exhaustive static analysis.

Exhaustive static analysis

Exhaustive static analysis goes beyond fuzzing. Instead of performing individual executions on individual inputs in an iterative fashion, it relies on a formal method called abstract interpretation to fully explore a program’s input and execution space.

Abstract interpretation allows TrustInSoft Analyzer to perform abstract executions for the entire range of values defined by your input variables. It turns your existing tests with discrete inputs into a generalized test covering your code’s entire input space.

For example, let’s say you were testing a function of an integer

I using values of I = –10  and I = +10.  Thanks to the power of formal methods, you would be able to test this function over the full interval of values for the integer  I, from -231 to 231-1, in a single test.

This input generalization works for all variable types in C/C++: integer, float, pointer, function pointer, etc. You can easily generate a generalized test from one of your existing tests with discrete inputs or from your API interface.

Thanks to the power of mathematics, exhaustive static analysis allows you to run the equivalent of billions upon billions of test cases simultaneously in just a few seconds, in a single test run. It is guaranteed to detect all undefined behaviors in your code, regardless of compilation optimization level or memory layout. Plus, once all the bugs it detects have been eliminated, it provides formal proof that your code is totally free of exploitable vulnerabilities.

So, why use fuzzing with TrustInSoft Analyzer in the first step before exhaustive static analysis?

At this point, you may be asking yourself, “So, if you can eliminate every last vulnerability with exhaustive static analysis, why use fuzzing at all in the first step?”

The first thing to mention before we address this question is that there are some types of code that cannot be analyzed with discrete test inputs at all. Hence, analyzing this type of software with exhaustive analysis and input generalization is the way to go.  For example, when a device starts up, the value of the registers can be any value, and the tester of a bootloader must take this into account.  There is no point in testing this state of the device with discrete values as the test will not be representative enough. An exhaustive analysis is required to test the vulnerability of a bootloader.

Having said that, when it comes to the type of software where both Fuzzing in interpreter mode or exhaustive analysis techniques are suitable, sometimes it can still be more efficient over the entire test campaign to start initially with fuzzing in interpreter mode and then move in the second stage to exhaustive static analysis.

Even though exhaustive static analysis generates far fewer false positives than any other classic static analysis tool, it can generate some false positives due to the approximations made. In exhaustive static analysis, there are only two ways to determine which warnings are true vulnerabilities and which are false positives. One way is to manually investigate each warning, one by one. This method can be time-consuming . This is the same method that is being used with classic static analysis tools. 

The second way, which we prefer, is to re-tune and repeat your analysis and then compare results.

By tuning, we mean adjusting the approximations of the acceptable and forbidden zones with slightly different parameters—to change their “shape,” if you will in order to eliminate false positives.

This is a far more efficient way to eliminate false positives.

Unfortunately, the more vulnerabilities you have in your target program, the more laborious this re-tune-and-compare process becomes. It becomes much more efficient after you’ve eliminated the more obvious vulnerabilities in your code.

Now, you’ll remember that TrustInSoft Analyzer’s interpreter mode will automatically run any set of inputs automatically and generate no false positives. That’s why we called fuzzing with interpreter mode a great first step. By first fuzzing your program with AFL and running the resulting fuzz inputs through TrustInSoft Analyzer in interpreter mode, you can quickly detect and eliminate many true vulnerabilities before running the Analyzer in exhaustive static analysis mode.

Interpreter mode thins your vulnerability herd considerably. This greatly simplifies the task of re-tuning your analysis. It makes the elimination of the hard-to-find undefined behavior that exhaustive static analysis reveals much easier and quicker. Ultimately, it saves you a lot of time over the course of your debugging campaign.

You’ll achieve formal proof that your code is 100% vulnerability-free much, much sooner.

The advantages of proceeding to exhaustive static analysis

In today’s world, software providers need assurance of a high level of cybersecurity in their source code. To do this, there are several significant advantages to proceeding to exhaustive static analysis after fuzzing in interpreter mode.

First, it’s exhaustive. You’ll have peace of mind knowing you have found and removed every undefined behavior—every single vulnerability from your code.

Second, once you’ve removed all undefined behavior, TrustInSoft Analyzer provides a mathematical guarantee that you’ve removed every vulnerability from your code. This is formal proof you can use as evidence in reviews with security specialists, customers, and regulators.

Finally, having accomplished exhaustive static analysis once for a given program, you’ll find it is much less work than fuzzing when you modify your code. You’re now working from a much cleaner baseline.  You simply re-run the analyses you’ve already set up.

This post is Part 3 of a 3-part series derived from TrustInSoft’s new guide to fuzzing for cybersecurity entitled “Fuzzing and Beyond.” To obtain a FREE copy, CLICK HERE.

For additional information, see our white paper

This post concludes our series on fuzzing. If you found it useful, our new white paper, Fuzzing and Beyond, contains still more information on the use of fuzzing and exhaustive static analysis for making your code more secure. You’ll discover why fuzzing and exhaustive static analysis with TrustInSoft Analyzer form an efficient two-step process for securing your most critical code. You’ll also learn of an additional benefit fuzzing provides. You’ll even find two case studies on how TrustInSoft Analyzer formally verified the total eradication of security vulnerabilities from two highly sensitive applications.