Microbat


Acknowledge*: This work is supported and inspired by the NRF2015NCR-NCR003-003, Secrity and Privacy in Smart Grid Systems for the trustful implementation of smart grid system.

Mirobat is a feedback-based debugger for interactively recommending suspicious step in buggy program execution. It records the trace of the execution of a buggy program and allow developers to provide light-weight feedbacks (correct-step, wrong-variable-value, wrong-path, and unclear) on trace steps. Microbat is able to learn, analyze, and reason based on developers' feedback and program information and recommend suspicious step in buggy program.

The metaphor of Microbat lies in that microbat is a species of bat which hunts bug by iteratively sending ultra wave as feedback to locate the prey.

A demo video can be checked as follows. In addition, the source code of Microbat is available at https://github.com/llmhyy/microbat. Our paper is available here.

Evaluation

We evaluate our approach and tool with a simulation experiment and a user study.

Simulation Experiment

In the simulation experiment, we simulate human feedbacks with mutation technique. More specifically, we mutate the code for test cases in Apache projects (namely, Math, Lang, and CLI) so we can have a correct trace before mutation and a buggy trace after mutation. Therefore, we can know the correct feedback on each step on the buggy trace by referencing the correct trace.

We do this by matching the original trace and mutated trace. We first transfer each trace into a contextual tree in which the method invocation/loop head trace step is the parent of its follow-up steps in the invocation/loop scope. Then we match the steps in a top-down manner. That is, we match the root step first, then their direct children next. For two matched parent steps, their direct children will be matched by a dynamic algorithm (Longest Common Subsequence algorithm). The code for trace matching can be checked at https://github.com/llmhyy/microbat. A more detailed technique report can be checked at our technique report.

The results shows that Microbat can find 92.8% mutated bugs and 65% bugs can be found within 20 feedbacks. The detailed statistics can be checked here. In addition, all the mutation files can be checked here.

User Study

We recruited 16 participants to conduct user study on Microbat. We survey the programming experince of the participants (see our survey form) and devide them into two equivalent groups. Our results show that the participants using Microbat can complete the debugging tasks with 55.8% less time on average.

The detailed statistics can be checked here