Welcome to the evaluation of CFGConf. Thank you for taking the time to partipate in the evaluation. Your participation will help us gather valuable feedback to improve the language and the system in the future versions.
This study has been approved by the University of Arizona Institutional Review Board (IRB). For more details, view the consent information for the study and the accompanying survey. Only Sabin Devkota will have access to data regarding whether or not you choose to participate. The other researchers analyzing the results (i.e., Kate Isaacs) will only have access to anonymized data and will not know who participated.
Instead of conducting the study through a remote video session, we are conducting the study offline. You will be asked:
- To download CFGConf and perform the tasks described below to familiarize yourself with the system.
- Send us the CFGConf files JSON files you create in the process.
- Fill out a survey regarding your experience.
You do not need to perform the tasks in one continuous session. You may leave and return. We estimate all tasks and feedback should take about an hour.
We are evaluating the usefulness of our language/system and not your knowledge on CFGs or graph drawing. You can refer to the CFGConf wiki throughout. If you get stuck or have any questions during or after, please contact devkotasabin@email.arizona.edu.
CFGConf is a JSON-based language for visualizing Control Flow Graphs (CFGs) as node-link diagrams. CFGConf is designed to make it convenient to generate CFG-specific drawings with just a few lines of JSON code. Features include filtering the graph to nodes and loops of interest and automatically collapsing and de-emphasizing uninteresting functions.
First, follow the setup guide to create the Hello World example to verify CFGConf is working on your system.
init.json: Apart from syntax errors, the most common error is caused by the init.json file not pointing to the correct JSON file. For more on troubleshooting, refer to the troubleshooting section.
syntax errors: If your CFG does not load after 15 seconds, you may want to validate your JSON file. See validating your JSON file to find out whether the syntax is correct and the JSON file conforms to the schema.
Please perform the tasks below. Each will ask you to create a CFGConf JSON file. Please save these files and send them to us after the study.
Support files for each task are located in cfgConf > static > CFGConf > files > tasks.
Throughout these tasks, you can access any resources, including:
- the quickstart guide with common usage examples of CFGConf
- the reference guide which describes all available keys
Replicate the drawing below. In other words, create a CFGConf JSON file that creates the provided drawing.
Step 2.1: Produce a drawing of the graph from the dot file t2.dot. Loops inside the graph are provided in the file t2_loops.json.
Step 2.2: Compare the resulting drawing from CFGConf to a drawing produced using dot graphviz on the file t2.dot. Note: These drawings are expected to appear different. Do not be concerned if they do not look the same.
To produce a pdf file using graphviz, you can run the following command in the terminal. The drawing t2.pdf will be created in the same directory.
dot -Tpdf t2.dot -o t2.pdf
Step 3.1: Produce a filtered drawing using the graph specified in ltimes.dot. A dyninst analysis file named ltimes.json with the functions and loops is also provided.
In this drawing, show only the node ids in the boxes, rather than the disassembly like the previous example.
Use the following set of nodes as the starting nodes for filtering:
"B1973", "B1974", "B1978", "B1986", "B1993", "B4052", "B4183", "B4205", "B4206", "B4430"
The drawn graph should be limited to nodes within 3 hops of the above set and have no more than 25 nodes total.
Step 3.2: Turn off the filtering and view the changed output.
Produce a filtered drawing using the same graph files ltimes.dot and ltimes.json with collapsed functions.
Specifically, the functions in the filtered graph should be collapsed unless they contain loops, with the exception of the function __kmpc_fork_call. Ensure __kmpc_fork_call is still drawn.
Use the following set of nodes as the starting nodes for filtering:
"B3805", "B4451"
Thats it! We are done with the tasks. Please fill out the survey regarding CFGConf below:
You may upload your JSON files through this survey or email them to devkotasabin@email.arizona.edu
Thank you for giving your valuable time for the evaluation of CFGConf. We look forward to your feedback.