Project 1: Build It; Break It; Fix It

version 1.01

Overview

In this project, you will implement a secure log to describe the state of an art gallery: the guests and employees who have entered and left, and persons that are in rooms. The log will be used by two programs. One program, logappend, will append new information to this file, and the other, logread, will read from the file and display the state of the art gallery according to a given query over the log. Both programs will use an authentication token, supplied as a command-line argument, to authenticate each other. Specifications for these two programs and the security model are described in more detail below.

You will build the most secure implementation you can; then you will have the opportunity to attack other teams’ implementations.

You will work in teams of 2-3 people in groups assigned by us. Your first task is to choose a team name and email it to our TA, Aymeric Fromherz. Make sure you include both your team name and the names all team members in your writeups.

All implementations must be written in C, and we will provide you with some basic starting code and a makefile. Feel free to make whatever changes you feel are necessary, as long as you stay within the rules outlined below. You don’t need to feel obliged to keep the starter code the way we gave it to you; it’s just there as one suggested way of organizing your code.

This project will operate using infrastructure from the Build it, Break it, Fix it contest developed at UMD. Please note that your grade in for the project and your score in the contest are not the same (although they are likely to be correlated). Details of grading and scoring are below. Scoring well in the contest is good for bragging rights and for extra credit.

Each student must register at our contest website (which requires an on-campus IP address) and then affiliate with your team. Choose a password for the contest website that is not the same as the one you use for your regular CMU login. The team leader can then create the team, invite the other team members, and register the team for the contest.

Details

Programs

Your team will design a log format and implement both logappend and logread to use it. Each program’s description is linked below.

The logappend program appends data about activity in the museum to a log. The logread program reads and queries data from the log logread contains a number of features that are optional. If you do not implement an optional feature, be sure to print unimplemented to standard output.

Look at the page of examples for examples of using the logappend and logread tools together.

Security Model

The system as a whole must guarantee the privacy and integrity of the log in the presence of an adversary that does not know the authentication token. This token is used by both the logappend and logread tools, specified on the command line. Without knowledge of the token an attacker should not be able to:

• Query the logs via logread or otherwise learn facts about the names of guests, employees, room numbers, or times by inspecting the log itself.
• Modify the log via logappend.
• Fool logread or logappend into accepting a bogus file. In particular, modifications made to the log by means other than correct use of logappend should be detected by (subsequent calls to) logread or logappend when the correct token is supplied.

Oracle

An oracle reference implementation is provided to demonstrate the expected output of a series of commands run on logread and logappend. Contestants may run the reference implementation by going to the team participation page on the website and clicking on “Oracle Submissions”. Here is an example of the expected input format for the oracle, structured as a series of command-line calls within brackets:

[
	"logappend -T 1 -K secret -A -E Fred log1",
	"logappend -T 2 -K secret -A -G Jill log1",
	"logappend -T 3 -K secret -A -E Fred -R 1 log1",
	"logappend -T 4 -K secret -A -G Jill -R 1 log1",
	"logread -K secret -S log1"
]

Deliverables

Part A: Build It

Each team should initialize a git repository on the ECE GitLab and share it with the 732s18-grading user. You must grant the 732s18-grading user at least reporter permissions. You MUST NOT make your repository public; doing so will be treated as an academic integrity violation.

Create a directory named build in the top-level directory of your repository and commit your code into that folder. Your submission will be scored after every push to the repository. You may keep your code other directories to keep it from being scored.

To score a submission, an automated system will invoke make in the build directory of your submission. The only requirements on make are that it must function without internet connectivity, it must return within ten minutes, and it must build from source (committing binaries only is not acceptable). We will provide a sample makefile.

Once make finishes, logread and logappend should be executable files within the build directory. An automated system will invoke them with a variety of options and measure their responses. The executables must be able to be run from any working directory.

Checkpoint

To ensure you’re making progress on the assignment, we will have a checkpoint deadline.
By this deadline, your team must achieve at least 10 points on the contest scoreboard. You can then continue to refine your system until it’s time for the final submission described below.

Final Submission

Via Canvas, you should submit:

• Your implementation, including all your code files and your makefile. Even though we will have access to your git repo (details below), you will submit a “final” version to the submit server for grading.
• A design document (PDF) in which you:
  • Describe your overall system design in sufficient detail for a reader to understand your approach without reading the source code directly. This must include a description of the format of your log file.
  • List three specific classes of attacks you have considered, and describe how your implementation counters these threats. (Where possible, please note the relevant lines of code for your defense.) If you were not able to completely prevent a threat you identified, you may still mention it. In that case, describe any partial mitigation you implemented and explain anything you wanted to implement but were for whatever reason unable to. Please be clear about distinguishing what you have implemented vs. what you would have implemented.

Part B: Break It

We will assign you implementations developed by four other teams. Your goal will be to find and validate bugs and vulnerabilities in that code according to the Break-It specifications.

Via Canvas, you should submit:

• A vulnerability analysis document (PDF).
  • Choose one of the implementations you were given and run the Coverity Static Analysis tool on it (more details will be forthcoming).
    • Identify two bugs reported by the tool that you consider to be false positives
    • Explain why the false positives are not actually bugs, as well as why the tool flagged them anyway.
    • Identify two distinct bugs reported by the tool that you consider to be true positives.
    • Explain why the true positives could be real risks.
  • From the implementations you were given, identify a total of four additional, distinct bugs. You can use any technique you want, and you can continue to use the Coverity Static Analysis tool.
    • At least two of these bugs must be security vulnerabilities.
    • At least two of the bugs must be from different implementations.
    • If you did not find any attacks or vulnerabilities, describe how you looked for attacks and why you believe none are present.
  • All bugs must be validated via the contest server.
• Submit any code you wrote to implement your attacks. Make sure the vulnerability analysis explains how to use your code to launch the attack. In particular, be sure to include your test case JSON file as well.

For points in the contest, you are welcome to also look at other implementations beyond the three you have been assigned, but you can only submit at most 5 breaks per implementation, and your breaks can only target at most 8 implementations.

If one of your assigned implementations doesn’t work well enough to analyze, please request an alternate implementation from an instructor.

Part C: Fix It

You will be given access to all breaks against your initial implementation, and you are now responsible for fixing them. For each bug, submit a small patch that fixes it. Each patch should only fix one bug (though fixing that one bug may fix many break reports). Please see the Fix Specifications for more details.

You should also submit a fix-it report that should organize your fixes into categories (e.g., these three fixes address buffer overflows). Explain how you developed the fixes, and your level of confidence that these fixes address all such bugs in your program.

As discussed on Piazza, you will be graded on:

• The proportion of breaks fixed
• The explanation of each fix
• The organization of fixes into general categories (i.e. don’t explain a buffer overflow, then an integrity fix, then another buffer overflow)
• The general clarity and presentation of the writeup
• The name and group name/id included in the writeup

For each fix, please:

• Explain what the issue was
• Explain how the attack impacted your system/what the security risks were
• Explain in text how it was corrected. You can include pseudocode, a small snippet of actual code, and/or a pointer to the fix commit to help illustrate your explanation.
• Explain why it defends against all bugs of the same class
• Explain your development process to fix the break
• If you claimed that you defended against it in the build-it phase, explain why your defense wasn’t enough

Grading

Part A: Build It will be worth 100 points

• 75 points for passing the automated correctness tests. To receive full credit, your implementation must pass all of the core tests and at least three of the optional tests.
  • The checkpoint is worth 10 of these 75 points. To get these 10 points, your program must pass at least one correctness test by the checkpoint deadline.
• 25 points for your design document
  • 10 points for your description of your approach
  • 5 points each for your explanation of how you defend against a class of attacks

Part B: Break It will be worth 100 points

• 40 points for successfully completing the Coverity portion
• 60 points for finding four additional bugs beyond those covered by your Coverity description

Part C: Fix It will be worth 50 points

• Your score will be proportional to the number of breaks you successfully fix. If you have no breaks against your implementation, you will automatically receive full credit.

Extra Credit

The assignment as a whole is worth 250 points. The top 50% of teams, based on contest score, will be awarded an additional 10 points (4%). The top 10% of teams will be awarded an additional 15 points for a total of 25 (10%). The very top team will receive adulation and eternal bragging rights.

Rules

General

• Do not hack the contest’s infrastructure, or you will not get credit for the assignment.
• Teams must do their own work and not share solutions with others.

Build-It

• Do not obfuscate the source code in your submissions or you will not get credit for the assignment.
• Make sure that your build-it submissions do not time out. You will be given 10 minutes before a single test times out and 30 minutes for all tests to complete before timing out.
• Beware of changing your build submission between the build-it and fix-it rounds because if you make changes, they will be considered as part of your first fix, which may end up disqualifying it for contest purposes.

Change log

1.01: Clarified points received at checkpoint.