index

Exercises marked with (C) will be covered in class, and exercises marked with (H) are left as homework.

0. Setup (C)

Access to the shared VM

In this class we will again use of the clang VM. You may download the cssh.sh script, and execute it as follows to connect to the clang VM using your GCP account:

Get the examples

Inside the VM, fetch the examples for today's class and extract them. The following script will create a lab6code directory containing the examples.

$ curl https://www.dcc.fc.up.pt/~edrdo/aulas/qses/lectures/lab6/lab6code.tgz | tar xfz -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   165  100   165    0     0   1006      0 --:--:-- --:--:-- --:--:--  1006

$ cd lab6code

1. Basic unit testing (C)

1.1. Run the test example

Access the passwd directory. We will test a small function isPasswordOK, with code given in passwd/passwd.c and using a GoogleTest suite in passwd/test_passwd.cpp .

The expected behavior of isPasswordOK is that a password is considered valid ("OK") if:

To execute the tests and also generate a gcov coverage report, execute make test.

eduardorbmarques_gmail_com@clang:~/lab6code$ cd passwd

eduardorbmarques_gmail_com@clang:~/lab6code/passwd$ ls
Makefile  passwd.c  test_passwd.cpp

eduardorbmarques_gmail_com@clang:~/lab6code/passwd$ make test
g++ -g -fprofile-arcs -ftest-coverage   -c -o test_passwd.o test_passwd.cpp
gcc -g -fprofile-arcs -ftest-coverage   -c -o passwd.o passwd.c
g++ test_passwd.o passwd.o -g -lgtest -lgtest_main -lpthread -ftest-coverage -fprofile-arcs -o test_passwd
Running tests ...
[==========] Running 6 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 6 tests from PasswdTest
[ RUN      ] PasswdTest.invalid_length5
[       OK ] PasswdTest.invalid_length5 (0 ms)
[ RUN      ] PasswdTest.invalid_length13
[       OK ] PasswdTest.invalid_length13 (0 ms)
[ RUN      ] PasswdTest.invalid_length10_more_than_one_punct
[       OK ] PasswdTest.invalid_length10_more_than_one_punct (0 ms)
[ RUN      ] PasswdTest.valid_length6_no_punct
test_passwd.cpp:19: Failure
Value of: isPasswordOK("1234Ab")
  Actual: false
Expected: true
[  FAILED  ] PasswdTest.valid_length6_no_punct (0 ms)
[ RUN      ] PasswdTest.valid_length12_no_punct
test_passwd.cpp:23: Failure
Value of: isPasswordOK("123456789Abc")
  Actual: false
Expected: true
[  FAILED  ] PasswdTest.valid_length12_no_punct (0 ms)
[ RUN      ] PasswdTest.valid_length12_with_punct
test_passwd.cpp:27: Failure
Value of: isPasswordOK("123456789Ab!")
  Actual: false
Expected: true
[  FAILED  ] PasswdTest.valid_length12_with_punct (0 ms)
[----------] 6 tests from PasswdTest (1 ms total)

[----------] Global test environment tear-down
[==========] 6 tests from 1 test case ran. (1 ms total)
[  PASSED  ] 3 tests.
[  FAILED  ] 3 tests, listed below:
[  FAILED  ] PasswdTest.valid_length6_no_punct
[  FAILED  ] PasswdTest.valid_length12_no_punct
[  FAILED  ] PasswdTest.valid_length12_with_punct

 3 FAILED TESTS
Deriving coverage report ...
File 'passwd.c'
Lines executed:100.00% of 30
Branches executed:100.00% of 32
Taken at least once:81.25% of 32
No calls
Creating 'passwd.c.gcov'

1.2. Find and fix a few bugs

Notice that a few tests have failed. There are 4 lines in the code with faults that you should fix. As you detect those faults and change the code, re-execute the tests.

1.3. Increase branch coverage

For every test execution (make test), a summary of coverage information is displayed. Add more tests to achieve 100 % branch coverage, that is, add tests until you obtain in the coverage summary the following output:

You may open passwd.c.gcov for a detailed coverage report to identify what branches have not yet been covered by the tests. Branches not covered are identified as "not executed" (not reached during execution) or "taken 0 %" (every conditional statement leads to 2 branches, and both branches for a conditional statement shoud be exercised by at least one test).

2. Fuzz-testing (C)

2.1. Getting started

Compile the files, then execute escapeProg. Note that Address Sanitizer will be enabled, as it will helpful to expose bugs during fuzzing.

eduardorbmarques_gmail_com@clang:~/lab6code$ cd escapeHtml/

eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ ls
Makefile  corpus  escapeHtml.c  escapeProg.c  fuzzTest.c  seeds

eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ make
clang -g -Wall -fsanitize=address escapeProg.c escapeHtml.c -o escapeProg
clang -g -fsanitize=address,fuzzer escapeHtml.c fuzzTest.c -o fuzzTest

eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ cat seeds/example.html 
<html>
 <body> 
  <b> Some HTML ...</b> 
          & 
  <i>some more HTML </i>
 </body>
</html>

eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ ./escapeProg seeds/example.html 
&lt;html&gt;
 &lt;body&gt; 
  &lt;b&gt; Some HTML ...&lt;/b&gt; 
          &amp; 
  &lt;i&gt;some more HTML &lt;/i&gt;
 &lt;/body&gt;
&lt;/html&gt;

2.2. Black-box fuzzing with radamsa

Let us first generate a "fuzzed-up" version of seeds/example.html and feed it to escapeProg, as shown below. The -s 666 arguments below indicate that a fixed seed with value 666 will be used. If the seed is not specified, radamsa will use a random one read from /dev/urandom.

eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ radamsa -s 666 seeds/example.html | tee 666.html
<html>
 <body> 
  <b><b><b><b><b><b> Some HTML ...</b></b></b></b></b><b><b><b><html>
 <body> 
  <b><b><b><b><b><b> Some HTML ...</b></b></b></b></b><b><b><b><html>
 <body> 
  <b><b><b><b><b><b> Some HTML ...</b></b></b></b></b><b><b><b><html>
 <body> 
  <b><b><b><b><b><b> Some HTML ...</b></b></b></b></b><b><b><b><html>
 <body> 
  <b><b><b><b><b><b> Some HTML ...</b></b></b></b></b><b><b><b><b><b> Some HTML ...</b></b></b></b></b></b> 
          & 
  <i>some more HTML </i>
 </body>
</html></b></b></b></b> 
          & 
  <i>some more HTML </i>
 </body>
</html></b></b></b></b> 
          & 
  <i>some more HTML </i>
 </body>
</html></b></b></b></b> 
          & 
  <i>some more HTML </i>
 </body>
</html></b></b></b></b> 
          & 
  <i>some more HTML </i>
 </body>
</html>

eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ ./escapeProg 666.html
&lt;html&gt;
 &lt;body&gt; 
  &lt;b&gt;&lt;b&gt;&lt;b&gt;&lt;b&gt;&lt;b&gt;&lt;b&gt; Some HTML ...&lt;/b&gt;&lt;/b&gt;&lt;/b&gt;&lt;/b&gt;&lt;/b&gt;&lt;b&gt;&lt;b&gt;&lt;b&gt;&lt;html&gt;
 &lt;body&gt; 

[... remaining output ommited - no crash ...]

The program did not crash. Let us generate 10 inputs at once (using -n 10) and feed them all at once to escapeProg. We will observe again that the program does not crash.

radamsa -s 666 -o 666-%n.html -n 10 seeds/example.html
eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ ls 666-*.html
666-1.html   666-2.html  666-4.html  666-6.html  666-8.html
666-10.html  666-3.html  666-5.html  666-7.html  666-9.html

eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ cat 666-*.html | ./escapeProg
&lt;html&gt;
 &lt;body&gt; 
  &lt;b&gt;&lt;b&gt;&lt;b&gt;&lt;b&gt;&lt;b&gt;&lt;b&gt; Some HTML ...&lt;/b&gt;&lt;/b&gt;&lt;/b&gt;&lt;/b&gt;&lt;/b&gt;&lt;b&gt;&lt;b&gt;&lt;b&gt;&lt;html&gt;
 &lt;body&gt; 
 
[... remaining output ommited - no crash again ...]

Perhaps the 666 seed is not "malicious" at all. We can try different seeds, and see what happens. You may execute the radamsa_batch.sh script to try seeds 1 to 1000, each used to generate one input only. You will get 3 program crashes.

Fuzzing, please wait ...
=================================================================
==30399==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000016 at pc 0x0000004c2f64 bp 0x7ffe269cccc0 sp 0x7ffe269cccb8
WRITE of size 1 at 0x602000000016 thread T0
    #0 0x4c2f63 in escapeHtml /home/eduardorbmarques_gmail_com/lab6code/escapeHtml/escapeHtml.c:23:14
[...]
Got a crash with seed 423 - check crash-radamsa-423.html
=================================================================
==30665==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000013 at pc 0x0000004c2f64 bp 0x7fffaa9ac140 sp 0x7fffaa9ac138
WRITE of size 1 at 0x602000000013 thread T0
    #0 0x4c2f63 in escapeHtml /home/eduardorbmarques_gmail_com/lab6code/escapeHtml/escapeHtml.c:23:14
[...]
Got a crash with seed 511 - check crash-radamsa-511.html
=================================================================
==31867==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000013 at pc 0x0000004c2f64 bp 0x7ffd2f7797c0 sp 0x7ffd2f7797b8
WRITE of size 1 at 0x602000000013 thread T0
    #0 0x4c2f63 in escapeHtml /home/eduardorbmarques_gmail_com/lab6code/escapeHtml/escapeHtml.c:23:14
[...]
Got a crash with seed 911 - check crash-radamsa-911.html
Done ... 3 program crashes in 1000 executions

Line 23 seems to be problematic ... is it the root cause of the program crash? Try to identify what may be wrong and fix it; using gdb with a breakpoint set for the escapeHtml function may be useful. To test your code fixes, use the "crash inputs" generated previously, and then run radamsa_batch.sh again.

2.3. White box fuzz-testing with libfuzzer

White-box fuzz can potentially do better (i.e. faster, generating less inputs). Undo your changes in escapeProg.c and execute the fuzzTest program as follows.

eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ rm -f corpus/* 

eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ ./fuzzTest -seed=666 corpus seeds
INFO: Seed: 666
INFO: Loaded 1 modules   (13 inline 8-bit counters): 13 [0x5a5f70, 0x5a5f7d), 
INFO: Loaded 1 PC tables (13 PCs): 13 [0x56dce0,0x56ddb0), 
INFO:        0 files found in corpus
INFO:        1 files found in seeds
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: seed corpus: files: 1 min: 98b max: 98b total: 98b rss: 27Mb
[...]
=================================================================
==6548==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6020000006f3 at pc 0x00000054f3f9 bp 0x7ffe282d9500 sp 0x7ffe282d94f8
WRITE of size 1 at 0x6020000006f3 thread T0
    #0 0x54f3f8 in escapeHtml /home/eduardorbmarques_gmail_com/lab6code/escapeHtml/escapeHtml.c:23:14
[...]
artifact_prefix='./'; Test unit written to ./crash-78dd4412249fcb254c9c1ee22f5cfb05a95fd451
Base64: PAAAAABodG1sPgogPGJvZHk+IAogIDxiPiBaaG1lIEinTUwgLi4uPAH8AAAgCiAgICAgJiAKICA8PGk+aHQ=

The execution indicates that the program crashes. The crash input is identified at the end of the execution output (./crash-./crash-78dd4412249fcb254c9c1ee22f5cfb05a95fd451 above). In the corpus directory you may see other outputs generated by libfuzzer that were deemed significant in the sense of inducing broader coverage, but that caused no program crash.

eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ ls corpus/
200b3d3e2634c55b93efae29f7b97dd60b95abfb  7062ed59b23a8ea2505ecdafe6c6d95c704d9f62
2bd7ed1fe632eab360036f5d1896180816250acb  96b40b533a8d9c56001bfedde8ccdb08872e18c2
3bcefad04e18c40aa9ea6df87fed0d65cf41eb7e  ddb843de2955cde757092487bffe1d4de25d6bd9
6217a61d792385e9a7ef2ac91b7370eba576fca0

eduardorbmarques_gmail_com@clang:~/lab6code/escapeHtml$ cat corpus/200b3d3e2634c55b93efae29f7b97dd60b95abfb
<html>
 <body> 
  <b> Some HTML ...</b> 
    & 
  <i></html>

QSES - Laboratory Exercises 6