The Daikon Invariant Detector Developer Manual

images/daikon-logo

Next: , Up: (dir)   [Contents][Index]

Daikon Invariant Detector Developer Manual

This is the developer manual for the Daikon invariant detector. It describes Daikon version 5.0.6, released March 28, 2014.

Table of Contents


Next: , Previous: Top, Up: Top   [Contents][Index]

1 Introduction

This is the developer manual for the Daikon invariant detector.

For information about using Daikon, see the Daikon User Manual. This manual is intended for those who are already familiar with the use of Daikon, but wish to customize or extend it.

Additional information can be found in technical papers available from http://plse.cs.washington.edu/daikon/pubs/.


Next: , Previous: Introduction, Up: Top   [Contents][Index]

2 Extending Daikon

This chapter describes how to customize or modify Daikon.


Next: , Up: Extending Daikon   [Contents][Index]

2.1 Compiling Daikon

To compile Daikon, type make in daikon/java/ or any of its subdirectories. The distribution includes compiled .class files, so you do not need to compile them yourself unless you make changes.

When you compile Daikon, environment variables DAIKONDIR and JAVA_HOME should be set. This is already done if you source the daikon.bashrc, as recommended in the installation instructions (see Complete installation in Daikon User Manual). Thus, a complete .bashrc or .bash_profile shell setup file that would enable you to compile Daikon might look like the following:

export DAIKONDIR=$HOME/daikon
export JAVA_HOME=/usr/java/current
source $DAIKONDIR/scripts/daikon.bashrc
export CLASSPATH=.:$CLASSPATH

Daikon is written in Java. In order to compile Daikon, you need a Java compiler such as ‘javac’ on your path. To override the default Java compiler (‘javac’), create a Makefile.user file in the daikon/ directory and add a line like the following.

JAVAC ?= jikes -g +E +F

In order to compile Daikon, you need the C preprocessor cpp, which is used to convert each .jpp file in the distribution into multiple .java files, which are then compiled. If you have a C compiler, you almost certainly have cpp. If you do not have cpp (or gcc, which can emulate cpp via gcc -E), you may run make avoid-jpp, in which case changes to .jpp files will not be reflected in the .java files or the compiled .class files. (The purpose of the .jpp files is to avoid code duplication by placing common code in a single file, then generating other files that need to include that common code.)

To make the documentation (via make -C $DAIKONDIR/doc), you will need a recent version of makeinfo; version 4.7 or newer.

For more information about compiling Daikon, see the comments in the Makefiles.

To compile Daikon on Windows, the best approach is to install the Cygwin toolset (available at http://cygwin.com/), which contains everything you need to compile and run Linux programs under Windows. You can install Cygwin by running one of the setup programs (based on your machine type) found at http://cygwin.com/install.html.

When setting up environment pathname variables under Windows/Cygwin (such as JAVA_HOME) make sure that the pathname is specified in Linux format (e.g., /cygdrive/c/daikon rather than C:\daikon). However, the CLASSPATH environment variable is a special case; as java is a Windows program, it expects CLASSPATH to be in Windows format. This is the only exception to Cygwin looking like Linux. Thus CLASSPATH must always be in Windows format; i.e., it should use semicolons (;) rather than colons (:) as a path separator. Using the wrong filename format or the wrong pathname separator is a common source of errors that will result in odd error messages and build failures.

Please note this applies to a -CLASSPATH argument to java as well: it must have Windows pathnames. One easy way to deal with this is to use the cygpath utility:

java -cp "`cygpath -wp linux/path/a:linux/path/b`" <remaider of arguments>

Note both the back quotes (‘`’) to force cygpath to be evaluated before calling java and the double quotes (‘"’) to protect against special characters and spaces in Windows paths.


Next: , Previous: Compiling Daikon, Up: Extending Daikon   [Contents][Index]

2.2 Version control repository

The Daikon Mercurial repository is located at Google Code; see http://code.google.com/p/daikon/.


Next: , Previous: Version control repository, Up: Extending Daikon   [Contents][Index]

2.3 Using Eclipse

[To be improved.]

Here is one way to use Eclipse to edit Daikon.

First, make sure that Daikon builds cleanly from the command line.

File > Import > General > Existing Projects into Workspace

Choose the java directory of your Daikon checkout

Project > properties > Java build path: libraries : add external jars everything in the lib/ directory, plus also the tools.jar file in the lib/ directory of your JDK. (I’m not sure why, but add jars doesn’t show all .jar files in the directory.)

Source: add Daikon, remove Daikon/src. Default output folder: change from Daikon/bin to Daikon.


Next: , Previous: Using Eclipse, Up: Extending Daikon   [Contents][Index]

2.4 New invariants

You can easily write your own invariants and have Daikon check them, in addition to all the other invariants that are already part of Daikon. Adding a new invariant to Daikon requires writing one Java class, adding a line to another file to inform Daikon of the new class, and recompiling Daikon.

The file java/daikon/inv/unary/scalar/Positive.java in the Daikon distribution contains a sample invariant. This invariant is true if the variable is always positive (greater than zero). This invariant is subsumed by other invariants in the system; it is provided only as a pedagogical example. To enable the invariant, comment out the appropriate line in Daikon.setup_proto_invs(), then recompile Daikon.

A Java class defining an invariant is a concrete subclass of one of the following abstract classes:

SingleScalar

Invariants over a single numeric (scalar) variable, such as ‘x != 0’.

TwoScalar

Invariants over two numeric variables, such as ‘y = abs(x)’.

ThreeScalar

Invariants over three numeric variables, such as ‘z = ax + by + c’.

SingleSequence

Invariants over one sequence (array) variable, such as ‘a[] contains no duplicates’.

TwoSequence

Invariants over two sequences, such as ‘a[] is a subsequence of b[]’.

SequenceScalar

Invariants over a scalar and a sequence, such as ‘x is a member of a[]’.

A complete list of invariants appears in Daikon.setup_proto_invs().

Daikon’s invariants are first instantiated, then are presented samples (tuples of values for all the variables of interest to the invariant; this might be a 1-tuple, a 2-tuple, or a 3-tuple) in turn. If any sample falsifies the invariant, the invariant destroys itself. All remaining invariants at the end of the program run can be reported as likely to be true.

The key methods of the new invariant class InvName are

protected InvName(PptSlice ppt)

Constructor for class InvName. Should only be called from instantiate_dyn. Its typical implementation is

super(ppt);
protected Invariant instantiate_dyn (PptSlice slice)

Returns an invariant of this class on the specified slice. Its implementation is almost always

return new InvName(slice);
public static InvName get_proto()

Returns the prototype invariant used to create other invariants. Its typical implementation is

if (proto == null)
  proto = new InvName (null);
return (proto);
public boolean enabled()

Returns whether or not this invariant is enabled. Its implementation is almost always

return dkconfig_enabled;
public boolean instantiate_ok (VarInfo[] vis)

Returns whether or not it makes sense to instantiate this invariant over the specified variables. If not present, the invariant is created over all comparable variables of the correct types.

public InvariantStatus check_modified (..., int count)

Presents a tuple of values to the invariant; these are the first N arguments, which have appropriate types for the particular invariant. The count argument indicates how many samples have this value. For example, three calls to check_modified with a count parameter of 1 is equivalent to one call to check_modified with a count parameter of 3. Returns whether or not the sample is consistent with the invariant. Does not change the state of the invariant.

public InvariantStatus add_modified (..., int count)

Similar to check_modified except that it can change the state of the invariant if necessary. If the invariant doesn’t have any state, then simply calls check_modified.

protected double computeConfidence ()

Returns the probability that the observed data could not have happened by chance alone. The result usually falls between 0 and 1, where 0 means the values seen so far certainly happened by chance and 1 means they could never have happened by chance. The method may also return one of the following constants in the Invariant class (see the class for documentation): CONFIDENCE_JUSTIFIED, CONFIDENCE_UNJUSTIFIED, CONFIDENCE_NEVER.

For example, suppose your new invariant has a 50% chance of being true by chance for each sample. (‘x is even’ is an example of such an invariant.) Then a reasonable body for computeConfidence would be

return 1 - Math.pow(.5, ppt.num_samples());

If 5 values had been seen, then this implementation would return 31/32, which is the likelihood that all 5 values seen so far were even not purely by chance. An invariant is printed only if its probability of not occurring by chance alone is large enough (by default, greater than .99; see Daikon’s --conf_limit command-line option.

public String format ()
public String repr ()
public String format_using (OutputFormat format)

Returns a high-level printed representation of the invariant, for user output. format produces normal output, while the repr formatting routine produces low-level, detailed output for debugging. When first writing an invariant, you can make repr and format_using simply call format, then fix up the implementations for the different output formats later as needed. See also New formatting for invariants.

After the invariant is written, add a call to its get_proto method in the Daikon.setup_proto_invs method.


Next: , Previous: New invariants, Up: Extending Daikon   [Contents][Index]

2.5 New derived variables

A derived variable is an expression that does not appear in the source code as a variable, but that Daikon treats as a variable for purposes of invariant detection. For instance, if there exists an array ‘a’ and an integer ‘i’, then Daikon introduces the derived variable ‘a[i]’. This permits detection of invariants over this quantity.

(Describing how to create new variety of derived variable is still to be written. For now, see the derived variables that appear in the Java files in directory $DAIKONDIR/java/daikon/derive/.)


Next: , Previous: New derived variables, Up: Extending Daikon   [Contents][Index]

2.6 New formatting for invariants

Daikon can print invariants in multiple formats (see Invariant syntax in Daikon User Manual).

To support a new output format, you need to do two things:


Next: , Previous: New formatting for invariants, Up: Extending Daikon   [Contents][Index]

2.7 New front ends

A front end for Daikon converts data into a form Daikon can process, producing files in Daikon’s input format — data trace declarations and records. For more information about these files, see File formats.

The data traces can be obtained from any source. For instance, front ends have been built for stock data, weather forecasts, truck weight data, and spreadsheet data (see convertcsv.pl in Daikon User Manual), among others. More often, users apply a programming language front end (also called an instrumenter) to a program, causing executions of the program to write files in Daikon’s format. (For information about existing front ends, see Front ends (instrumentation) in Daikon User Manual.) When a general front end is not available, it is possible to manually instrument a specific program so that it writes files in Daikon’s format. The resulting instrumented program is very similar to what an instrumenter would have created, so this section is relevant to both approaches.

Conceptually, instrumentation is very simple. For each program point (say, a line of code or the entry or exit from a procedure) at which you wish to detect invariants, the front end must arrange to create a declaration (see Declarations) that lists the variables in scope at that program point, and the front end must arrange that execution creates a data trace record (see Data trace records) for each execution of that program point. Conventionally, the way to create a data trace record is to insert a printf (or similar) statement that outputs the current values of the variables of interest.


Next: , Up: New front ends   [Contents][Index]

2.7.1 Example instrumented Java program

This section gives an example of how an instrumenter for Java might work; other languages are analogous. Suppose we wish to instrument file Example.java.

class Example {
  // Return either the square of x or the square of (x+1).
  int squar(int x, boolean b) {
    if (b)
      x++;
    return x*x;
  }
}

The .decls file might look like the following.

DECLARE
Example.squar:::ENTER
x
int
int
1
b
boolean
int
2

DECLARE
Example.squar:::EXIT
x
int
int
1
b
boolean
int
2
return
int
int
1

The instrumented .java file might look like the following. This example does not compute the “modified bits”, but simply sets them all to 1, which is a safe default.

class Example {
  static {
    daikon.Runtime.setDtraceMaybe("daikon-output/StackAr.dtrace");
  }

  // Return either the square of x or the square of (x+1).
  int squar(int x, boolean b) {
    synchronized (daikon.Runtime.dtrace) {
      daikon.Runtime.dtrace.println();
      daikon.Runtime.dtrace.println("Example.squar:::ENTER");
      daikon.Runtime.dtrace.println("x");
      daikon.Runtime.dtrace.println(x);
      daikon.Runtime.dtrace.println(1);  // modified bit
      daikon.Runtime.dtrace.println("b");
      daikon.Runtime.dtrace.println(b ? 1 : 0);
      daikon.Runtime.dtrace.println(1);  // modified bit
    }

    if (b)
      x++;

    int daikon_return_value = x*x;
    synchronized (daikon.Runtime.dtrace) {
      daikon.Runtime.dtrace.println();
      daikon.Runtime.dtrace.println("Example.squar:::EXIT");
      daikon.Runtime.dtrace.println("x");
      daikon.Runtime.dtrace.println(x);
      daikon.Runtime.dtrace.println(1);  // modified bit
      daikon.Runtime.dtrace.println("b");
      daikon.Runtime.dtrace.println(b ? 1 : 0);
      daikon.Runtime.dtrace.println(1);  // modified bit
      daikon.Runtime.dtrace.println("return");
      daikon.Runtime.dtrace.println(daikon_return_value);
      daikon.Runtime.dtrace.println(1);  // modified bit
    }

    return daikon_return_value;
  }
}

(Daikon’s Java front end, Chicory, does not actually insert instrumentation into the Java source code of your program. Rather, it instruments the bytecode as it is loaded into the JVM. This is more efficient, and it avoids making any changes to your .java or .class files. We have shown an example of Java source code instrumentation because that is simpler to explain and understand than the bytecode instrumentation.)


Previous: Example instrumented Java program, Up: New front ends   [Contents][Index]

2.7.2 Instrumenting C programs

Daikon comes with a front end for the C language: Kvasir (see Kvasir in Daikon User Manual). Kvasir only works under the Linux operating system, and it works only on “x86” (Intel 386, 486, 586, 686 compatible) and “x86-64” (AMD64, EM64T compatible) processors.

You may wish to infer invariants over C programs running on other platforms; for instance, you want a robust C front end that works under Microsoft Windows. This section will help you to either write such a front end or to hand-instrument your program to produce output that Daikon can process.

We welcome additions and corrections to this part of the manual. And, if you write a C instrumenter that might be of use to others, please contribute it back to the Daikon project.

A front end for C (or any other language) performs two tasks. It determines the names of all variables that are in scope at a particular program point, and it prints the values of those variables each time the program point executes.

Determining the names of the variables is straightforward. It requires either parsing source code or parsing a compiled executable. In the latter case, the variables can be determined from debugging information that the compiler places in the executable.

The challenge for C programs is determining the values of variables at execution time: for each variable, the front end must determine whether the variable’s value is valid, and how big the value is.

A front end should print only variables that have valid values. Examples of invalid values are variables that have not yet been initialized and pointers whose content has been deallocated. (A pointer dereference, such as ‘*p’ or ‘p->field’, can itself be to uninitialized and/or deallocated memory.) Invalid values should be printed as ‘nonsensical’ (see Data trace records).

It is desirable to print ‘nonsensical’ rather than an invalid value, for two reasons. First, outputting nonsense values can degrade invariant detection; patterns in the valid data may be masked by noise from invalid values. Second, an attempt to access an invalid value can cause the instrumented program to crash! For instance, suppose that pointer ‘p’ is not yet initialized — the pointer value refers to some arbitrary location in memory, possibly even an address that the operating system has not allocated to the program. An attempt to print the value of ‘*p’ or ‘p->field’ will result in a segmentation fault when ‘*p’ is accessed. (If you choose never to dereference a pointer while performing instrumentation, then you do not need to worry about invalid references. However, you will be unable to output any fields of a pointer to a struct or class, making your front end less useful. You will still be able to output fields of a regular variable to a struct or class, but most interesting uses of structs and classes in C and C++ are through pointers.)

C relies on the programmer to remember which variables are valid, and the programmer must take care never to access invalid variables. Unfortunately, there is no simple automatic way to determine variable validity for an arbitrary C program. (Languages with automatic memory management, such as Java, do not pose these problems. All variables always have an initial value, so there is no danger of printing uninitialized memory, though the initial value may not be particularly meaningful. Because pointed-to memory is never deallocated, all non-null pointers are always valid, so there is no danger of a segmentation fault.)

An instrumenter needs information about validity of variable values. This could be obtained from the programmer (which requires work on the part of the user of Daikon), or obtained automatically by creating a new run-time system that tracks the information (which requires a more sophisticated front end).

In addition to determining which variables are uninitialized and which pointers are to allocated memory, there are additional problems for a C front end. For example, given a char pointer ‘*c’, does it point to a single character, or to an array of characters? If it points to an array of characters, how big is that array? And for each element of the array, is that element initialized or not?

The problem of tracking C memory may seem daunting, but it is not insurmountable. There exist many tools for detecting or debugging memory errors in C, and they need to perform exactly the same memory tracking as a Daikon front end must perform. Therefore, a Daikon front end can use the same well-known techniques, and possibly can even be built on top of such a tool. For instance, one C front end, named Kvasir, is built on top of the Valgrind tool (http://valgrind.org/), greatly reducing the implementation effort. Valgrind only works under Linux, but a C front end for another platform could build on a similar tool; many other such tools exist.

There are two basic approaches to instrumenting a C program (or a program in any other language): instrument the source code, or instrument a compiled binary representation of the program. In each case, additional code that tracks all memory allocations, deallocations, writes, and reads must be executed at run time. Which approach is most appropriate for you depends on what tools you use when building your C instrumentation system.

In some cases, it may not be necessary to build a fully general C instrumentation system. You may be able to craft a smaller, simpler extension to an existing program — enabling that program (only) to produce files for Daikon to analyze.

For instance, many programs use specialized memory allocation routines (customized versions of malloc and free), in order to prevent or detect memory errors. The information that such libraries collect is often sufficient to determine which variable values should be printed, and which should be suppressed in favor of printing ‘nonsensical’ instead.

The presence of memory errors — even in a program that appears to run correctly — makes it much harder to create Daikon’s output. Therefore, as a prerequisite to instrumenting a C program, it is usually a good idea to run a memory checker on that program and to eliminate any memory errors.


Next: , Previous: New front ends, Up: Extending Daikon   [Contents][Index]

2.8 New suppressors

As mentioned in Daikon internals, one way to make Daikon more efficient, and to reduce clutter in the output to the user, is to reduce the number of redundant invariants of various kinds. This section describes how to add a new suppressor relation, such that if invariant A implies B, B is not instantiated or checked as long as A holds, saving time and space. Suppression implications use some terminology. A suppressor (defined in the class NISuppressor) is one of a set of invariants (NISuppression) that imply and suppress a suppressee invariant (NISuppressee). The set of all of the suppressions that suppress a particular suppressee is stored in the class NISuppressionSet.

Adding a new suppression is straightforward when the invariants involved do not have any state. Define the suppressee and each of the suppressions that suppress it using the corresponding constructors. Add the method get_ni_suppressions to the class of the invariant being suppressed and return the appropriate suppression set. Make sure that get_ni_suppressions always returns the same suppression set (i.e., that storage to store the suppressions is only allocated once). Normally this is done by defining a static variable to hold the suppression sets and initializing this variable the first time that get_ni_suppressions is called.

The following example defines suppressions for ‘x == y’ implies ‘x >= y’ and ‘x > y’ implies ‘x >= y’.

private static NISuppressionSet suppressions = null;

public NISuppressionSet get_ni_suppressions() {
  if (suppressions == null) {
    NISuppressee = new NISuppressee (IntGreaterEqual);

    NISuppressor v1_eq_v2 = new NISuppressor (0, 1, IntEqual.class);
    NISuppressor v1_lt_v2 = new NISuppressor (0, 1, IntLessThan.class);

    suppressions = new NISuppressionSet (new NISuppression[] {
      new NISuppression (v1_eq_v2, suppressee),
      new NISuppression (v1_lt_v2, suppressee),
    });
  }
  return (suppressions);
}

For suppressions depending on the state of a particular invariant, each Invariant has an isObviousDynamically(VarInfo[] vis) method that is called once the state of other invariants has already been determined. This method returns a non-null value if this invariant is implied by a fact that can be derived from the given VarInfos.

For example, suppose division was not defined for divisors smaller than 1. The following example defines an obvious check for ‘x <= c’ (where c < 1 is a constant) implies ‘y % x == 0’, written in the Divides class.

public DiscardInfo isObviousDynamically(VarInfo[] vis) {
  DiscardInfo di = super.isObviousDynamically(vis);
  if(di != null) {
    return di;
  }

  VarInfo var1 = vis[0];

  PptSlice1 ppt_over1 = ppt.parent.findSlice(var1);

  if(ppt_over1 == null) {
    return null;
  }

  for(Invariant inv : ppt_over1.invs) {            
    if(inv instanceof UpperBound) {
      if(((UpperBound) inv).max() < 1) {
        return new DiscardInfo(this, DiscardCode.obvious,
                               ‘Divides is obvious when divisor less than one’);
      }
    }
  }

  return null;
}

Previous: New suppressors, Up: Extending Daikon   [Contents][Index]

2.9 System.exit

The Daikon codebase does not call System.exit(), except in a dummy main method that catches TerminationMessage, which is the standard way that a component of Daikon requests the JVM to shut down.

The reason for this is that calling System.exit() is usually a bad idea. It makes the class unusable as a subroutine, because it might kill the calling program. It can cause deadlock. And it can leave data in an inconsistent state (for example, if the program was in the middle of writing a file, still held non-Java locks, etc.), because the program has no good way of completing any actions that it was in the middle of. Therefore, it is better to throw an exception and let the program handle it appropriately. (This is true of instrumentation code as well.)


Next: , Previous: Extending Daikon, Up: Top   [Contents][Index]

3 Debugging Daikon

This chapter describes some techniques that can be used for debugging Daikon. Because Daikon processes large amounts of data, using a debugger can be difficult. The following logging techniques provide alternatives to using a debugger.

Daikon’s logging routines are based on the java.util.logging utilities (built into Java 1.4 and later).


Up: Debugging Daikon   [Contents][Index]

3.1 Track logging

Often it is desirable to print information only about one or more specific invariants. This is distinct from general logging because it concentrates on specific invariant objects rather than a particular class or portion of Daikon. This is referred to as Track logging because it tracks particular values across Daikon.

The --track class|class|...<var,var,var>@ppt option to Daikon (see Daikon debugging options in Daikon User Manual) enables track logging. The argument to the --track option supplies three pieces of information:

  1. The class name of the invariant (e.g., IntEqual). Multiple class arguments can be specified separated by pipe symbols (‘|’).
  2. The variables that are used in the invariant (e.g., return, size(this.s[])). The variables are specified in angle brackets (‘<>’).
  3. The program point of interest (e.g., DataStructures.StackAr.makeEmpty()V:::ENTER). The program point is preceded by an at sign (‘@’).

Each item is optional. For example:

IntEqual<x,y>@makeEmpty()
LessThan|GreaterThan<return,orig(y)>@EXIT99

Multiple --track switches can be specified. The class, program point, and each of the variables must match one of the specifications in order for information concerning the invariant to be printed.

Matching is a simple substring comparison. The specified item must be a substring of the actual item. For instance, LessThan matches both IntLessThan and FloatLessThan.

Program points and variables are specified exactly as they are seen in normal Daikon invariant output. Specifically, Ppt.name and VarInfo.name.name() are used to generate the names for comparisons.

Invariants are not the only classes that can be tracked. Any class name is a valid entry. Thus, for example, to print information about derived sequence variables from sequence this.theArray[] and scalar x at program point DisjSets.find(int):::EXIT, the tracking argument would be:

SequenceScalarSubscriptFactory<x,this.theArray[]>@DisjSets.find(int):::EXIT

There are two configuration options that can customize the output. The option daikon.Debug.showTraceback will output a stack trace on each log statement. The option daikon.Debug.logDetail will cause more detailed (and often voluminous) output to be printed. For more information, see Configuration options in Daikon User Manual.

Note that all interesting information is not necessarily currently logged. It will often be necessary to add new logging statements for the specific information of interest (see Adding track logging). This is covered in the next section.

More detailed information can be found in the Javadoc for daikon.Debug and daikon.inv.Invariant.


Next: , Up: Track logging   [Contents][Index]

3.1.1 Adding track logging

When you add a new invariant, derived variable, or other component to Daikon, you should ensure that it supports track logging in the same way that existing components do. This section describes how to do so.

Track logging is based around the class name, program point name, and variables of interest. Track logging methods accept these parameters and a string to be printed. Debug.java implements the following basic log methods:

log (String)
log (Class, Ppt, String)
log (Class, Ppt, Varinfo[], String)

The first uses the cached version of the Class, Ppt, and VarInfo that was provided in the constructor. The second uses the specified variables and the VarInfo information from Ppt. The third specifies each variable explicitly.

When logging is not enabled, calling the logging functions can take a significant amount of time (because the parameters need to be evaluated and passed). To minimize this, a function logOn() is provided to see if logging is enabled. It is recommended that code of the following form be used for efficiency:

if (Debug.logOn()) {
  Debug.log (getClass(), ppt, "Entering routine foo");
}

Track logging also can work with other loggers. Each of the logging methods has an alternative version that also accepts a logger as the first argument. In this case, normal track logging is performed if the class, ppt, and vars match. If they don’t match, the same information is logged via the specified logger. For example:

if (Debug.logOn || logger.isLoggable (Level.FINE)) {
  Debug.log (logger, getClass(), ppt, "Entering routine foo");
}

The above will print if either the tracking information matches or if the specified logger is enabled.

Convenience methods are available for track logging invariants. In this case the class name, ppt, and variable information are all taken from the invariant. The available methods are:

logOn()
logDetail()
log (String)
log (Logger, String)

These correspond to the Debug methods described above. They are the recommended way to log information concerning invariants.

Track logging also provides one additional level of detail. The function logDetail() returns whether or not more detailed information should be printed. This should be used for information which is not normally interesting or especially voluminous output. Often statements using logDetail() should be commented out when not in active use.


Previous: Adding track logging, Up: Track logging   [Contents][Index]

3.1.2 Track log output

Each call to a track log method will produce output in the same basic format. Space for three variables is always maintained for consistency:

 daikon.Debug: <class>: <ppt>: <var1>: <var2>: <var3>: <msg>

If showTrackback is enabled, the ‘traceback’ will follow each line of debug output.


Next: , Previous: Debugging Daikon, Up: Top   [Contents][Index]

4 Daikon internals

This chapter describes some of the techniques used in Daikon to make it efficient in terms of time and space needed. These techniques can be enabled or disabled at the Daikon command line, as described in Running Daikon in Daikon User Manual.


Next: , Up: Daikon internals   [Contents][Index]

4.1 Avoiding work for redundant invariants

Daikon reduces runtime and memory by avoiding performing work for redundant invariants that provide no useful information to the user. There are three basic types of optimization that can be performed for uninteresting invariants: non-instantiation, suppression, and non-printing.

Non-instantiation prevents the creation of an invariant because the invariant’s truth value is statically obvious (from the semantics of the programming language), no matter what values may be seen at run time. Two examples are ‘A[i] is an element of A[]’ and ‘size(A[]) >= 0’. Non-instantiation is implemented by the by the isObviousStatically method. With the equality sets optimization (see Equality optimization), non-instantiation can only happen if all equality permutations are statically obvious. Note that isObviousStatically should be used only for invariants that are known to be true. Other code presumes that any statically obvious invariants are true and can be safely presumed when determining if other invariants are redundant.

An invariant can be suppressed if it is logically implied by some set of other invariants (referred to as suppressors). A suppressed invariant is not instantiated or checked as long as its suppressors hold. For example ‘x > y’ implies ‘x >= y’. Suppression has some limitations. It cannot use as suppressors or suppress sample dependent invariants (invariants that adapt themselves to the samples they see and whose equation thus involves a constant such as ‘x > 42’). Suppression also cannot use relationships between variables. For example, it cannot suppress ‘x[i] = y[j]’ by ‘(x[] = y[]) ^ (i = j)’. Suppressor invariants can only use variables that are also in the invariant that is being suppressed. In this example, only invariants using the variables ‘x[i]’ and ‘y[i]’ can be used as a suppressors. See New suppressors for more information.

Non-printing is a post-pass that throws out any invariants that are implied by other true invariants. It is similar to suppression, but has none of the limitations of suppression. But since it is only run as a post pass, it cannot optimize runtime and memory use as suppression can. Non-printing should be used only in cases where suppression cannot. Non-printing is implemented by ObviousFilter, which calls the isObviousDynamically method on invariants. The isObviousStatically method is also used by the non-printing checks; it can be called at the end without reference to equality sets.

More detail can be found in the paper “Efficient incremental algorithms for dynamic detection of likely invariants” by Jeff H. Perkins and Michael D. Ernst, published in Foundations of Software Engineering in 2004; the paper is available from http://plse.cs.washington.edu/daikon/pubs/invariants-incremental-fse2004-abstract.html.


Next: , Previous: Avoiding work for redundant invariants, Up: Daikon internals   [Contents][Index]

4.2 Dataflow hierarchy

Dataflow hierarchy is a means to relate variables in different program points in a partial ordering. Variables in program point X are related to variables in another program point Y by a flow relation if every sample seen of X’s variables is also meant to be seen at Y. Y is called a parent program point of X. For example, all the field variables in the :::ENTER program point of a method in class C relate to the field variables in the :::CLASS program point of C. This is because the state of C, when in context at the entry :::ENTER program point, is also in context at the :::CLASS program point. Any invariant that holds true on a parent program point must hold on the child program point. The purpose of dataflow hierarchy is to reduce the presence of redundant invariants by only keeping invariants at the highest parent at which they apply. This saves both time and space.

Daikon currently provides four ways that program points can be connected. First, :::CLASS program points are parents of all their method program points. Second, between two classes that are related by inheritance, corresponding program points relate — for example, java.util.Vector:::CLASS is a child of java.util.List:::CLASS. Third, when a program point contains variables of a type whose :::CLASS program point is also available to Daikon, the former program point’s variables relate to the latter program point’s :::CLASS method. For example, if X.y is of type Y, and Y contains fields a and b, X.y, X.y.a and X.y.b relate to Y.this, Y.b and Y.a. Fourth, variables at :::ENTER program points are related to the ‘orig’ versions at :::EXIT program points.

When using Daikon, the above four ways of relations in the dataflow hierarchy will result in some true invariants that are not reported at some program points. However, the invariant will be present in some parent program point. Dataflow hierarchy is enabled by default, but can be disabled by the --nohierarchy flag. When dataflow is enabled, the only samples that are examined by Daikon are the :::EXIT program points (plus ‘orig’ variables) since these contain a complete view of the data.


Previous: Dataflow hierarchy, Up: Daikon internals   [Contents][Index]

4.3 Equality optimization

When N variables are equal within a program point there will be N(N-1)/2 pairwise invariants to represent the equality within the equal variables, and N copies of every other invariant. For example, if a, b, and c are equal, then ‘a == b’, ‘a == c’, ‘b == c’ will be reported as pairwise invariants, and ‘odd(a)’, ‘odd(b)’ and ‘odd(c)’ will be reported. If the variables will always be equal, then reporting N times the invariants is wasteful. Daikon thus treats equality specially.

Each group of variables that are equal from the start of inferencing are placed in equality sets. An equality set can hold an arbitrary number of variables, and replaces the O(N^2) pairwise equality invariants. Every equality set has a leader or canonical representation by a variable in the set. Non-equality invariants are only instantiated and checked on the leader. When printing invariants, Daikon reports only invariants on the leader. The user can easily determine that ‘odd(a)’ and ‘a == b’ imply ‘odd(b)’. Equality optimization can be turned off at the command line with the --noequality flag.


Next: , Previous: Daikon internals, Up: Top   [Contents][Index]

5 Testing

Daikon has two sets of tests: unit tests (see Unit testing) and regression tests (see Regression tests). If there are any differences between the expected results and the ones you get, don’t check in your changes until you understand which is the desired behavior and possibly update the goals.

The Daikon distribution contains unit tests, but not regression tests (which would make the distribution much larger). The regression tests appear in Daikon’s version control repository (see Version control repository).


Next: , Up: Testing   [Contents][Index]

5.1 Unit testing

The unit tests are found in daikon/java/daikon/test/; they use the ‘JUnit’ unit testing framework. They take a few seconds to run. They are automatically run each time you compile Daikon (by running make in $DAIKONDIR/java or any of its subdirectories). You can also run them explicitly via make unit. When you write new code or modify old code, please try to add unit tests.


Next: , Up: Unit testing   [Contents][Index]

5.1.1 Invariant format testing

This tests the formatting of invariants with specified input. The tests are configured in the file InvariantFormatTest.commands under daikon/test/. Make sure the InvariantFormatTest.commands file is in the classpath when this tester is run or the tester will not work. (It will just tell you that the file is not in the classpath.)

The file is formatted as follows:

<fully qualified class name> [<instantiate args>]
<type string>
<goal string>+ <- 1 or more goal strings
<sample>* <- 0 or more samples

The file format should be the same regardless of blank or commented lines except in the samples area. No blank lines or comments should appear after the goal string before the first sample or between parts of samples (these lines are used currently to determine where samples lists end). This will be remedied in a future version of the tester.

Instantiate args

These are optional additional arguments to the static instantiate method of the class. Each argument consists of the type (boolean or int) followed by the value. For example:

boolean true
int 37 boolean false
Type string:

A type string must consist of one or more of the following literals: ‘int’, ‘double’, ‘string’, ‘int_array’, ‘double_array’, or ‘string_array’, separated by spaces. This string represents the types that an invariant is supposed to compare For instance, a binary integer comparison would have type string ‘int int’. A pairwise sequence comparison would have type string ‘int_array int_array’.

Goal string:

The goal string must start with the prefix ‘Goal ’, and then continue with ‘(<format type>): ’, where format type is the format in which the invariant will print. After this the representation of the invariant must occur. It must represent the invariant result exactly as printed, even white space is significant (as proper formatting should be correct down to the whitespace). The first variable (the one corresponding to the first type in the type string) corresponds with ‘a’, the second with ‘b’ and so on. Format the type string accordingly. (In samples, the value of ‘a’ is read first, possibly followed by ‘b’, and then possibly ‘c’, depending on the arity of the invariant.)

Example:
Type string, Goals
 |             |
\|/            |
int           \|/
Goal (daikon): a >= -6
Goal (java): a >= -6
Goal (esc): a >= -6
Goal (ioa): a >= -6
Goal (jml): a >= -6
Goal (simplify): (>= |a| -6)

Note that the spacing on the goal lines is exact, that is, no extra spaces are allowed and no spaces are allowed to be missing. So the exact format is again:

Goal<1 space>(<format name>):<1 space><goal text>
Samples:

Values formatted according to the type string, one value per line Make sure that the samples provided are actually instances of that particular invariant (That is, if the desired invariant is ‘a < b’, then the first number of each sample better be less than the second)

Arrays and strings must be formatted according to the Daikon .dtrace file convention (for a full description, see File formats). This states that arrays must be surrounded in brackets (start with ‘[’, end with ‘]’), and entries must be separated by a space. Strings must be enclosed in quotes (‘"’). Quotes within a string can be represented by the sequence ‘\"’.

For example:

[1 2 3 4 5] - an array with the elements 1, 2, 3, 4, 5
"aString" - a string
"a string" - also legal as a string
"\"" - the string with the value "
["a" "b" "c"] - an array of strings

int int        <- type string
Goal: a < b    <- goal string, no comment/blank lines after this
1              <- or before this
2
2              <-|__ Pair of values (a = 2 , b = 3)
3              <-|

Other examples are in the existing test file (InvariantFormatTest.commands).

The output of a test run can be converted into goals by using the --generate_goals switch to the tester as follows:

java daikon.test.InvariantFormatTester --generate_goals

Note that this test is included in the set of tests performed by the master tester, and so it is not necessary to separately run this test except to generate goal files.

Furthermore, this framework cannot parse complex types from files unless they contain a public (Object) valueOf(String s) function. Otherwise the program has no was of knowing how to create such an object from a string. All primitives and the String type are already recognized.


Previous: Invariant format testing, Up: Unit testing   [Contents][Index]

5.1.2 Sample Testing

Sample testing tests various components of Daikon as samples are being processed. A file (normally daikon/test/SampleTester.commands) specifies a .decls file to use, the samples for each ‘ppt/var’, and assertions about Daikon’s state (such as whether or not a particular invariant exists).

Each line of the file specifies exactly one command. Blank lines and leading blanks are ignored. Comments begin with the number sign (‘#’) and extend to the end of the line. The type of command is specified as the first token on the line followed by a colon. The supported commands are:

SampleTester Command: decl: decl-file

This command specifies the declaration file to use. This is a normal .decls file that should follow the format defined in the user manual.

SampleTester Command: ppt: ppt

This command specifies the program point that will be used with following vars, data, and assert commands. The program point should be specified exactly as it appears in the .decls file.

SampleTester Command: vars: var1 var2...

Specifies the variables that will be used on following data lines. Each variable must match exactly a variable in the ppt. Other variables will be treated as missing.

SampleTester Command: data: val1 val2...

Specifies the values for each of the previously specified variables. The values must match the type of the variables. A single dash (-) indicates that a variable is missing.

SampleTester Command: assert: assertion

Specifies an assertion that should be true at this point (see Assertions). The negation of an assertion can be specified by adding an exclamation point before the assertion (for example: !inv("x > y", x, y)).


Next: , Up: Sample Testing   [Contents][Index]

5.1.2.1 Assertions

Assertions are formatted like function calls: <name>(arg1, arg2, ...). The valid assertions for the assert: command are:

Assertion: inv format var1 ...

The inv assertion asserts that the specified invariant exists in the current ppt. The format argument is the result of calling format() on the invariant. This is how the invariant is recognized. The remaining arguments are the variables that make up the invariants slice. These must match exactly variables in the ppt. The inv assertion returns true if and only if the slice exists and an invariant is found within that slice that matches format.

Optionally, format can be replaced by the fully qualified class name of the invariant. In this case, it is only necessary for the class to match.

More assertions can easily be added to SampleTester.java as required.


Previous: Assertions, Up: Sample Testing   [Contents][Index]

5.1.2.2 Example file

The following is an simple example of sample testing.

decl: daikon/test/SampleTesters.decls

ppt: foo.f():::EXIT35
  vars: x y z
  data: 1 1 0
  data: 2 1 0
  assert: inv("x >= y", x, y)
  assert: inv(daikon.inv.binary.twoScalar.IntGreaterEqual,x,y)
  assert: !inv("x <= y", x, y)

Previous: Unit testing, Up: Testing   [Contents][Index]

5.2 Regression tests

The regression tests run Daikon on many different inputs and compare Daikon’s output to expected output. They take about an hour to run.

The regression tests appear in the $DAIKONDIR/tests/ directory. Type make in that directory to see a list of Makefile targets. The most common target is make diffs; if any resulting file has non-zero size, the tests fail. You do not generally need to do make clean, which forces re-instrumentation (a possibly slow process) the next time you run the tests.

As when you install or compile Daikon, when you run the tests environment variable DAIKONDIR (or INV, whose effect is the same) should be set. Additionally, environment variable JAVA_HOME should be the directory containing the Java JDK.

You should generally run the regression tests before checking it a change (especially any non-trivial change). If any of the regression test diffs has a non-zero size, then your edits have changed Daikon’s output and you should not check in without carefully determining that the changes are intentional and desirable (and you should update the goal output files, so that the diffs are again zero).

There are several subdirectories under $DAIKONDIR/tests/, testing different components of the Daikon distribution (such as Kvasir, see Kvasir in Daikon User Manual). Tests of the invariant detection engine itself appear in $DAIKONDIR/tests/daikon-tests/.

Each Makefile under $DAIKONDIR/tests/ includes $DAIKONDIR/tests/Makefile.common, which contains the logic for all of the tests. Makefile.common is somewhat complicated, if only because it controls so many types of tests.


Next: , Up: Regression tests   [Contents][Index]

5.2.1 Kvasir regression tests

Note on Kvasir tests: The Kvasir (Daikon C front-end) tests appear in the $DAIKONDIR/tests/kvasir-tests directory. These tests run Daikon to ensure that the Kvasir output is valid Daikon input. To run them, go to $DAIKONDIR/tests/kvasir-tests or any test sub-directory within here and run make summary-w-daikon. If any tests return ‘FAILED’, then you should look at the appropriate .diff file. If you feel that the failure was actually a result of your Daikon changes and should be in fact correct output, then run make update-inv-goals to update the Daikon invs.goal file.


Previous: Kvasir regression tests, Up: Regression tests   [Contents][Index]

5.2.2 Adding regression tests

Most Daikon regression tests appear in subdirectories of $DAIKONDIR/tests/daikon-tests/. (There is also a $DAIKONDIR/tests/chicory-tests/ directory, but it is usually better to put tests in $DAIKONDIR/tests/daikon-tests/, even if they are exercising Chicory-specific behavior.)

To create a new test, perform the following steps.

  1. Create a new subdirectory of $DAIKONDIR/tests/daikon-tests/.
  2. Put the source files for the test under $DAIKONDIR/tests/sources/, not in the test directory itself. You may be able to re-use existing Java source code that appears in that directory.
  3. It is expedient to copy a Makefile from another subdirectory, such as $DAIKONDIR/tests/daikon-tests/StackAr/, then modify it.

    The Makefile must contain at least the following entries.

    MAIN_CLASS

    Dot separated fully qualified name of the class that contains the main entry point for the test. For example,

    MAIN_CLASS := DataStructures.StackArTester.
    
    include ../../Makefile.common

    This includes the common portion of the test Makefiles that does most of the work. See it for more information on the details of regression testing.

    instrument-files-revise:

    A target that writes the list of files to instrument. For example,

    instrument-files-revise:
        echo "DataStructures/StackAr.java" >| ${INST_LIST_FILE}
    

    If you run make (without a target), you will see a description of all of the Makefile’s functionality. Most of that is inherited from $DAIKONDIR/tests/Makefile.common.

  4. The .goal files are the expected results of running Daikon and its associated tools.

    For example, the StackAr directory contains the following .goal files, among others:

    Makefile
    Stackar.spinfo-static.goal
    StackAr.txt-daikon.goal
    StackAr.txt-esc.goal
    StackAr.txt-jml.goal
    StackAr.txt-merge-esc.goal
    StackAr.txt-merge-jml.goal
    

    If you omit some .goal files, then the related tests will not be run. (If you can’t figure out how to ensure the tests are not run, it may be easier to just add the additional .goal files.)

    You can start out with the .goal files empty. Execute make diffs, to produce output; the tests will fail because the output is not identical to the empty .goal files. When the test output is satisfactory, execute make update-goals to copy the actual results to the .goal files. Then, commit the goal files, Makefile, and source files to the repository.

  5. The test can be added into the standard tests (either ‘everything’ or ‘quick’) by adding the test the appropriate list in $DAIKONDIR/tests/daikon-tests/Makefile.

Next: , Previous: Testing, Up: Top   [Contents][Index]

6 Editing Daikon source code

The Daikon source code follows these coding conventions: http://homes.cs.washington.edu/~mernst/advice/coding-style.html.


Next: , Up: Editing   [Contents][Index]

6.1 Emacs commands

The command M-x daikon-tags-table sets up Emacs to use the Daikon TAGS table, which lets you easily find definitions and search through the entire codebase in Emacs. (You need to run this command once per instance of Emacs.)

The commands M-x daikon-info and M-x daikon-developer-info view the Daikon manual and the Daikon Developer manual using the Info viewer within Emacs. These commands also update all other versions of the manuals (HTML, PDF, etc.). You may also choose to use the stand-alone Info viewer, to use a Web browser to read the manual in HTML form, or to read the PDF version either electronically or on paper.


Next: , Previous: Emacs commands, Up: Editing   [Contents][Index]

6.2 Eclipse setup


Next: , Up: Eclipse setup   [Contents][Index]

6.2.1 Start up Eclipse

Just run the command eclipse at the command shell.

There is one big catch: take note of where you launch Eclipse and always launch Eclipse from the same location. For example, you might want to alias run-eclipse with cd $HOME; eclipse. For convenience, this is already done for you if you source the daikon-dev.bashrc startup script. There are ways around the launch location inconvenience, such as the -data switch, but the alias trick should be the simplest and least confusing workaround.

It is strongly recommended that you make the following customizations to Eclipse:

Window >> Preferences >> Workbench >> Refresh Workspace on Startup
Window >> Preferences >> Java >> Compiler >> Problems >> Unused imports >> Ignore
Window >> Preferences >> Java >> Compiler >> Style >> Non-static access >> >> Ignore

Next: , Previous: Start up Eclipse, Up: Eclipse setup   [Contents][Index]

6.2.2 Work on existing coding projects


Next: , Up: Existing projects   [Contents][Index]

6.2.2.1 Import an existing project

First, do cp $DAIKONDIR/java/.classpath-pag $DAIKONDIR/java/.classpath

Second, import Daikon into Eclipse. To import: start Eclipse, then choose menu item ‘File->Import->ExistingProjectIntoWorkspace’, then enter (the full name of) directory $DAIKONDIR/java for Project Contents.

Those two steps are all there is to it!

(This simple procedure works because there are already .project and .classpath files in $DAIKONDIR/java. To import a project that doesn’t have these files, you would need to create them (for instance, copy them from Daikon’s), or else follow a different procedure to work on your project in Eclipse.)


Previous: Import existing project, Up: Existing projects   [Contents][Index]

6.2.2.2 Importing a different existing project

Before working on code in Eclipse, you must import the code into the Eclipse workspace. You also need to set up various compilation settings and specify jars and classes that should be in the compilation classpath. (By default, Eclipse does not use your environment Classpath.)

  1. Change to the Java perspective by selecting "Java Perspective" in the perspective menu on the far-left tool panel. You should see some of the windows move around to a structure similar to the screenshot below.
  2. Select the menu item File->New->Project
  3. On the popup window, select Java Project and click Next
  4. Call the project ‘Daikon’ or some other appropriate name and click Next.
  5. On the next window, create a new folder in the project called src, and answer Yes to the next dialog box asking you to confirm a change in the build directory. Select ‘Ok’ and you should see the new Project as a small folder icon in the package explorer.
  6. Select the menu item File->Import
  7. On the next popup window, select ‘File System’ and click Next. Browse to the folder containing your source code. Using the standard ‘PAG’ setup, the folder should be $DAIKONDIR/java. The import destination should be Daikon/src. Check the boxes for including Daikon, ‘jtb’, and plume.
  8. Finally, add the required jars for Daikon by right-clicking on the project and selecting the Properties menu. Select "Java Build Path", then "Libraries", then "Add External Jars" and add tools.jar, junit.jar, jakarta-oro.jar, and java-getopt.jar. (See your CLASSPATH environment variable for full pathnames to these files.)

Note that if you follow these directions, then the compiled classes do not appear in the directory from which you imported in the previous step. Instead, the compiled classes appear in the Eclipse workspace directory. If you followed the setup instructions above, then the compiled code will be in "$HOME/.eclipse/Daikon/bin", which you should add to your CLASSPATH in order to use the updates you make from Eclipse.


Next: , Previous: Existing projects, Up: Eclipse setup   [Contents][Index]

6.2.3 Interaction with external tools

Eclipse compiles your project whenever you save. However, it parses frequently during editing and issues most or all of the errors and warnings that the compiler would have. In order to clear a warning/error (in the Task List), you must save the file, however.

If you change files outside Eclipse, you should refresh Eclipse, via right-clicking on the project or via menu item File >> Refresh.


Next: , Previous: External tools, Up: Eclipse setup   [Contents][Index]

6.2.4 Using the Eclipse debugger

To begin debugging, you can click on the bug icon in the toolbar or select Run->Debug from the menu. You will see the same window as if you selected Run->Run, but when the Java application launches, Eclipse will switch to the Debugging Perspective, which reveals many windows such as the stack trace window, the variable values window, and the step control panel.

Debugging features (window locations given for default setup):


Previous: Eclipse debugger, Up: Eclipse setup   [Contents][Index]

6.2.5 Using Mercurial with Eclipse

This section has yet to be completed, although you can just do all the editing, compiling, and running from the Eclipse IDE while doing version control commands from the shell or whatever current system you use.


Previous: Eclipse setup, Up: Editing   [Contents][Index]

6.3 Editing daikon.texinfo

The Daikon manual appears in doc/daikon.texinfo. The manual is written using the Texinfo formatting system, which makes it easy to produce HTML, info, or printed (PDF) output.

You can edit the manual using an ordinary text editor. Emacs has a useful Texinfo mode. You can mimic the formatting commands that already appear in the manual, but it’s also a good idea to read the Texinfo documentation.

If you wish to create a new section of the manual, insert two lines like the following:

@node    @Short title
@chapter @Long title

where you select the short and the long titles (which may be the same), and you may replace @chapter by @section, @subsection, or @subsubsection.

Follow the Texinfo conventions for marking up text. In particular:

Once you have edited the manual, run make twice (yes, you must run it twice) in order to generate formatted versions of the manual (daikon.{html,pdf}).


Next: , Previous: Editing, Up: Top   [Contents][Index]

7 Making a distribution

This section provides the instructions for publishing a Daikon distribution; aka making a release. If you only want to create the daikon.tar or daikon.tar.gz file in your own directory, then simply run make with the desired file as the target.

We have a general policy of using even version numbers (e.g., 4.6.4) for official releases and odd version numbers (e.g., 4.7.3) for intermediate work. This means as you prepare for a release the current version number is probably odd. It will be reset as one of the first steps in the release process. After making the distribution, one of the final steps is to increment the version number again to prepare for subsequent development. Note, this system has the useful side effect of allowing the build and test process to be repeated to fix a problem (steps 5-14 below) without having to worry about updating or resetting the version number.

The Daikon distribution site is located at //plse.cs.washington.edu/daikon/; internally, this can be accessed as part of our file system at /cse/web/research/plse/daikon. In order to be able to write to the distribution site, your ‘CSE’ user id must be a member of the ‘plse_www’ group. Finally, for each of the major steps below, an approximate elapsed time is listed. Currently, this is for a quad x86-64 based machine at 3.4GHz with 16GB of memory. Barring any difficulties, the entire process will take at least two hours - and could be much more depending on the number of different platforms on which you test the release.

NOTE:
Each of the steps below assumes that the current working directory is $DAIKONDIR and that you are using the Bash shell.

  1. Update the Daikon source files to their most recent version.
    hg fetch && hg outgoing && hg status
    

    The last line printed should be ‘no changes found’.

    [Time: moments]

  2. Make sure we have the latest version of bcel.jar. Generally, you can ensure this by checking that the latest version of plume.jar is checked into daikon/java/lib/plume.jar.
  3. Adjust the version number in file $DAIKONDIR/doc/VERSION. Set it to an even number (as noted above) in preparation for the release. You may do this manually, or if the value just needs to be incremented by 1 (to go from odd to even) you may use:
    make update-dist-version-file
    

    Optionally, override the distribution date (default: today) by redefining the environment variable TODAY:

    export TODAY='November 1, 2013'
    

    Now, update the appropriate files with the date and version number of the release.

    make update-doc-dist-date-and-version
    

    [Time: moments]

  4. Update the publications pages in a local copy of plume-bib. If you have not done so previously, you may create this repository now:
    (cd; hg clone https://code.google.com/p/plume-bib/ bib)
    

    Now update the contents:

    (cd ~/bib; hg fetch)
    

    Alternatively, you may set the environment variable BIBDIR to the location of the bibliography you wish to use.

    [Time: moments]

  5. Do a clean rebuild of everything.
    make rebuild-everything
    

    [Time: 13 min]

  6. Run the Kvasir regression tests. There should be no diff files marked FAILED.
    make -C $DAIKONDIR/tests/kvasir-tests clean-all nightly-summary-w-daikon
    

    [Time: 15 min]

  7. Run the Daikon regression tests. The last line printed should be "All tests succeeded."
    make -C $DAIKONDIR/tests clean diffs
    

    [Time: 20 min]

  8. Edit the doc/CHANGES file to indicate what has changed in this version. A good way to determine what has changed is to diff the Texinfo source files in the doc directory against the previous versions:
    cd $DAIKONDIR/doc
    diff -b -u -s --from-file /cse/web/research/plse/daikon/download/doc *.texinfo
    

    Another good source of information are the repository logs since the last release:

    cd $DAIKONDIR/java
    hg log -v -r "tag(5.0.2)::"  (substitute last release number for 5.0.2)
    

    As we normally only tag the repository when we make a release, this can usually be simplified to:

    cd $DAIKONDIR/java
    hg log -v -r "last(tagged())::"
    
  9. Build everything to be distributed and copy it to the staging area (http://plse.cs.washington.edu/daikon/staging-daikon/). The final output will be a list of files that were added/removed since the last release.
    make staging
    

    [Time: 4 min]

  10. Check the website to see if it has any broken links.
    make check-for-broken-doc-links
    

    Review check.log and make corrections as appropriate.

    Errors 301 and 501 are normally fixed by adding an additional line to $DAIKONDIR/plume-lib/bin/checklink-args.txt.

    In some cases a "403 Forbidden" error is transient, due to the website being down. You can check such links by hand.

    Error 404 usually indicates an error in our links that needs to be fixed.

    [Time: 2 min]

  11. Test the distribution, currently in the staging area, on a 64-bit machine. Since the machine you are currently running to build the release is probably such a machine, you may simply run the command below. The last step in this test runs Chicory/Daikon on the StackAr example. Success is indicated by invariant output being written to the screen.
    make test-staged-dist
    

    [Time: 7 min]

  12. Test the distribution, currently in the staging area, on a 32-bit machine. This is probably best done on a virtual machine running a current version of Fedora. The instructions below assume you are running on such a machine. They will download the staged daikon.tar.gz file, extract the files, set the appropriate environment variables and run the top level Makefile. Finally, a short verification test is run; success is indicated by invariant output being written to the screen.
    mkdir -p ~/tmp
    cd ~/tmp
    rm -rf daikon.tar.gz daikon
    wget http://plse.cs.washington.edu/daikon/staging-daikon/download/daikon.tar.gz
    tar xzf daikon.tar.gz
    export JAVA_HOME=/usr/lib/jvm/java
    export DAIKONDIR=`pwd`/daikon
    source daikon/scripts/daikon.bashrc
    cd daikon
    make
    make installation-test
    

    [Time: 10 min]

  13. Test the distribution on a Windows machine. On a Windows machine with Cygwin installed, as a guest user (or at least one without environment variables such as $DAIKONDIR defined!), download the staged daikon.tar.gz file, extract the files, set the appropriate environment variables and run the top level Makefile. The last line of output from the first make should indicate that Kvasir was not built because this is not a Linux platform. After running the second make command, success is indicated by invariant output being written to the screen.
    [Time: 10 min]
    mkdir -p ~/tmp
    cd ~/tmp
    rm -rf daikon.tar.gz daikon
    wget http://plse.cs.washington.edu/daikon/staging-daikon/download/daikon.tar.gz
    tar xzf daikon.tar.gz
    # Convert OSTYPE from a bash shell variable to a system environment variable.
    export OSTYPE
    export JAVA_HOME="/cygdrive/c/Program Files/Java/jdk1.7.0_45"
    export DAIKONDIR=`pwd`/daikon
    source daikon/scripts/daikon.bashrc
    cd daikon
    make
    make installation-test
    
  14. Test the distribution on a MacOSX machine. On a MacOSX machine with the development tools installed, download the staged daikon.tar.gz file, extract the files, set the appropriate environment variables and run the top level Makefile. The last line of output from the first make should indicate that Kvasir was not built because this is not a Linux platform. After running the second make command, success is indicated by invariant output being written to the screen.
    [Time: 4 min]
    mkdir -p ~/tmp
    cd ~/tmp
    rm -rf daikon.tar.gz daikon
    # or: wget http://plse.cs.washington.edu/daikon/staging-daikon/download/daikon.tar.gz
    curl -O http://plse.cs.washington.edu/daikon/staging-daikon/download/daikon.tar.gz
    tar xzf daikon.tar.gz
    # Convert OSTYPE from a bash shell variable to a system environment variable.
    export OSTYPE
    export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Home
    export DAIKONDIR=`pwd`/daikon
    source daikon/scripts/daikon.bashrc
    cd daikon
    make
    make installation-test
    
  15. Save the previous release in the history archives.
    make save-current-release
    

    [Time: moments]

  16. Update the website.
    make staging-to-www
    

    [Time: 1 min]

  17. Commit changes back to the repository:

    Could this be done with cat $DAIKONDIR/doc/VERSION?

    hg -q commit -m "Change version to X.Y.Z"
    hg push
    cd kvasir
    hg -q commit -m "Change version to X.Y.Z"
    hg push
    

    [Time: moments]

  18. Add a version label to the repository:
    hg tag X.Y.Z
    hg push
    cd kvasir
    hg tag X.Y.Z
    hg push
    

    [Time: moments]

  19. At this point the distribution has been completed; however, we now need to adjust the version number in file $DAIKONDIR/doc/VERSION - it should be bumped up to an odd value for continuing development.
    make update-dist-version-file
    

    If you previously set the environment variable TODAY, you need to reset it:

    export -n TODAY
    

    Now, update the appropriate files with the date and version number for ongoing development.

    make update-doc-dist-date-and-version
    

    [Time: moments]

  20. Verify these changes by doing partial product builds:
    make rebuild-everything-no-clean
    

    [Time: 3 min]

  21. Commit changes back to the repository:
    hg -q commit -m "Bump version number for ongoing development."
    hg push
    cd kvasir
    hg -q commit -m "Bump version number for ongoing development."
    hg push
    

    [Time: moments]

  22. Send mail to the ‘daikon-announce’ mailing list. This list is located at ‘daikon-announce@googlegroups.com’. (We have just moved the location of our mailing lists. For now, you can find old sample messages in the archive of our previous mail list.)

Next: , Previous: Distribution, Up: Top   [Contents][Index]

8 Analyzing historical versions of Daikon

Daikon’s Mercurial repository, available at http://code.google.com/p/daikon/, contains complete development history starting from October 1, 2010, and partial development history before that time. The reason for this is that converting the full CVS repository to Mercurial would create a repository that would be too large for convenient use, and that would also be too large to upload to Google Code. The original CVS repository is available for download from http://plse.cs.washington.edu/daikon/download/inv-cvs/, as files inv-cvsaa through inv-cvsai. Download all 9 files, and then run the following commands to re-create the CVS repository, which will be named invariants/:

cat inv-cvsa? > inv-cvs-full.zip
unzip inv-cvs-full.zip 

Developers can usually make do with the Mercurial repository, which has complete history for the Java source code.

The CVS repository is most often used by researchers — for example, in the testing community — because it contains both a CVS repository and a set of tests that is more extensive than those in the Daikon distribution. If you are a researcher who uses the Daikon version control history, please let us know of any publications so that we can publicize them at http://plse.cs.washington.edu/daikon/pubs/#daikon-testsubject.

The layout of the Daikon CVS repository differs slightly from that of the distribution. For example, the top-level directory is named invariants/ instead of daikon/.

The remainder of this section points out some pitfalls for such researchers. Although these problems are easy to avoid, some previous published work has made these mistakes; don’t let that happen to you!

Recall that Daikon contains two sets of tests (see Testing); you should include both in any analysis of Daikon’s tests. (Or, if you can analyze only one of the two sets of tests, then clearly explain that the regression tests are the main tests.) The regression tests use Makefiles to avoid re-doing unnecessary work, so any description of the time taken to run Daikon’s tests should be a measurement of re-running the tests after they have been run once, not running them from a clean checkout or after a make clean command.

Daikon intentionally does not contain tests for third-party libraries that are included (sometimes in source form) in the Daikon distribution. As one example, the java/jtb/ directory contains an external library. Therefore, any measurement of Daikon’s code coverage should not include such libraries, whether distributed in source or as .jar files.

Historically, the file doc/www/mit/index.html in the CVS repository contains information about how Daikon’s developers use Daikon. This file changes from time to time — for instance, it changed when a CVS branch was created and later when development on it ceased (see Branches).


Up: Historical   [Contents][Index]

8.1 Branches

The Daikon CVS repository contains two branches: a main trunk and a branch (named ‘ENGINE_V2_PATCHES’) for version 2 of Daikon.

The CVS manual (see section “Branching and merging” of the manual CVS — Concurrent Versions System) describes CVS branches:

CVS allows you to isolate changes onto a separate line of development, known as a branch. When you change files on a branch, those changes do not appear on the main trunk or other branches.

Later you can move changes from one branch to another branch (or the main trunk) by merging. Merging involves first running cvs update -j, to merge the changes into the working directory. You can then commit that revision, and thus effectively copy the changes onto another branch.

In early January 2002 (or perhaps in late 2001), we created the ‘ENGINE_V2_PATCHES’ branch at the invariants/java/daikon level of the Daikon CVS repository. Primary development continued along the CVS branch ‘ENGINE_V2_PATCHES’, which we called “Daikon version 2” We called the CVS trunk “Daikon version 3” it was experimental, and very few people ran its code or performed development on it. Periodically, all changes made to the branch would be merged into the trunk, as one large checkin on the trunk. Later, development on version 3 became more common, some changes were merged from the trunk to the branch, and version 2 was finally retired (and no more changes were made to the branch) in December 2003.

A regular cvs checkout gets the trunk. The -r flag specifies a branch. For example, to get the branch as of June 9, 2002, one could do

cvs -d $pag/projects/invariants/.CVS co -r ENGINE_V2_PATCHES \
    -D 2003/06/09 invariants/java/daikon

Some warnings about analyzing historical versions of Daikon:

  1. When analyzing 2002 (and at least parts of 2003) you should be careful to use the branch, not the trunk. Or, you could analyze both (but as a single development effort, not as separate efforts).
  2. When a programmer periodically merged changes from the branch to the trunk (or vice versa), that operation resulted in very large checkins. The times at which these merges occurred is indicated in file invariants/java/daikon/merging.txt in the repository; for example, this happened 34 times during calendar year 2002. CVS checkins for the branch properly attribute and time-stamp the work that appears as a single large checkin on the trunk.
  3. There may be long periods of time in the branch (respectively, the trunk) with no checkins, but that does not necessarily indicate a lacuna in development, as checkins might have occurred in the meanwhile in the trunk (respectively, the branch).

Next: , Previous: Historical, Up: Top   [Contents][Index]

Appendix A File formats

This chapter contains information about the file format of Daikon’s input files. It is of most information to those who wish to write a front end, also known as an instrumenter (see Front ends (instrumentation) in Daikon User Manual). A new front end enables Daikon to detect invariants in another programming language.

Daikon’s input is conventionally one or more .dtrace data trace files. (Another, optional type of input file for Daikon is a splitter info file; see Splitter info file format in Daikon User Manual.) A trace file is a text file that consists of newline-separated records. There are two basic types of records that can appear in Daikon’s input: program point declarations, and trace records. The declarations describe the structure of the trace records. The trace records contain the data on which Daikon operates — the run-time values of variables in your program.

Each declaration names an instrumented program point and lists the variables at that program point. A program point is a location in the program, such as a specific line number, or a specific procedure’s entry or exit. An instrumented program point is a place where the instrumenter may emit a trace record. A program point declaration may be repeated, so long as the declarations match exactly (any declarations after the first one have no effect).

A data trace record (also known as a sample) represents one execution of a program point. The record specifies the program point and gives the runtime values of each variable. The list of variables in the data trace record must be identical to that in the corresponding declaration. For a given program point, the declaration must precede the first data trace record for the program point. It is not required that all the program point declarations appear before any of the data trace records.

There exist some other declaration-related records; see Declaration-related records.


Next: , Up: File formats   [Contents][Index]

A.1 Declarations in a separate file

Instead of placing both declarations and data trace records in a single file, it is permitted to place the declarations in one or more .decls declaration files while leaving the data trace records in the .dtrace file. This can be convenient for tools that perform a separate instrumentation step, such as dfepl (see dfepl in Daikon User Manual). Such a tool takes as input a target program to be analyzed, and produces two outputs: a .decls file and an instrumented program. Executing the instrumented program produces a .dtrace file containing data trace records for all the program points that appear in the .dtrace file. This approach works fine and is easier to implement in certain situations, but has a few disadvantages. It requires the user to perform at least two steps — instrumentation and execution — and the existence of two versions of the program (instrumented and uninstrumented) can lead to confusion or extra work. It is also more convenient to have a single file that contains all information about a program, rather than multiple .decls files that must be associated with the .dtrace file.


Next: , Previous: Declarations in a separate file, Up: File formats   [Contents][Index]

A.2 File format conventions

Daikon files are textual, to permit easier viewing and editing by humans. Each record is separated by one or more blank lines. To permit easier parsing by programs, each piece of information in a record appears on a separate line.

Outside a record, any line starting with a pound sign (#) or double slashes (//) is ignored as a comment. Comments are not permitted inside a record.


Next: , Previous: Conventions, Up: File formats   [Contents][Index]

A.3 Declarations

The trace file (or declaration file) first states the declaration file format version number (see Declaration version). It may also specify some other information about the file (see Declaration-related records). Then, it defines each program point and its variables.

Indentation is ignored, so it may be used to aid readability. Fields with defaults can be omitted.

As a rule, each line of the declaration file is of the form <field-name> <field-value>.

Some additional details about the declaration file format appear in file daikon/doc/decl_format.txt in the Daikon source code.


Next: , Up: Declarations   [Contents][Index]

A.3.1 Declaration-related records


Next: , Up: Declaration-related records   [Contents][Index]

A.3.1.1 Declaration version

The declaration version record must be the first record in the file.

The declaration version record is as follows:

decl-version <version>

The current version is 2.0.

Previous versions (see Version 1 Declarations) did not include a version field and are identified by the lack of this field.


Next: , Previous: Declaration version, Up: Declaration-related records   [Contents][Index]

A.3.1.2 Input-language declaration

You can specify the language in which the program was written with a record of the form

input-language <language>

The language string is arbitrary and is not currently used.


Next: , Previous: Input-language declaration, Up: Declaration-related records   [Contents][Index]

A.3.1.3 Variable comparability

The Variable comparability record indicates how the comparability field of a variable declaration should be interpreted.

Its format is:

var-comparability <comparability-type>

The possible values for comparability-type are implicit and none.

implicit means ordinary comparability as described in Program point declarations. (The name implicit is retained for historical reasons.)

This record is optional. The implicit type is the default.


Previous: Variable comparability, Up: Declaration-related records   [Contents][Index]

A.3.1.4 ListImplementors declaration

This declaration indicates classes that implement the java.util.List interface, and should be treated as sequences for the purposes of invariant detection. The syntax is as follows:

ListImplementors
<classname1>
<classname2>
...

Each classname is in Java format (for example, java.util.LinkedList).

The --list_type command-line option to Daikon can also be used to specify classes that implement lists; see Options to control invariant detection in Daikon User Manual.


Next: , Previous: Declaration-related records, Up: Declarations   [Contents][Index]

A.3.2 Program point declarations

The format of a program point declaration is:

ppt <ppt-name>
<ppt-info>
<ppt-info>
...
<variable-declaration>
<variable-declaration>
...

The program point name can include any character. In the declaration file, blanks must be replaced by \_, and backslashes must be escaped as \\. Program point names must be distinct.

The following information about the program point (ppt-info) can be specified:


Previous: Program point declarations, Up: Declarations   [Contents][Index]

A.3.3 Variable declarations

The format of a variable declaration is:

variable <name>
  <variable-info>
  <variable-info>
  ...

The variable name is arbitrary, but for clarity, it should match what is used in the programming language. All characters are legal in a name, but blanks must be represented as \_ and backslashes as \\.

If the variable is an array, .. marks the location of array indices within the variable name. Some examples of names are:

this.theArray
this.theArray[..]
this.stack.getClass()

The following information about the variable (variable-info) can be specified:


Next: , Previous: Declarations, Up: File formats   [Contents][Index]

A.4 Data trace records

A data trace record (also known as a sample) contains run-time value information. Its format is:

<program-point-name>
this_invocation_nonce
<nonce-string>
<varname-1>
<var-value-1>
<var-modified-1>
<varname2>
<var-value-2>
<var-modified-2>
...

In other words, the sample record contains:


Next: , Up: Data trace records   [Contents][Index]

A.4.1 Nonsensical values for variables

Some trace variables and derived variables may not have a value because the expression that computes it cannot be evaluated. In such a circumstance, the value is said to be nonsensical, it is written in the trace file as nonsensical, and its modified field must be 2. Examples include

For trace variables, it is the responsibility of the front end to perform a check at run time whenever a variable’s value is about to be output to the trace, and to output the value ‘nonsensical’ (see Nonsensical values) rather than crashing the program or outputting an uninitialized or meaningless value. (Determining when an expression’s value is meaningless is the most challenging part of writing an instrumenter for a language like C, since it requires tracking memory allocation and initialization.) For derived variables created by Daikon, Daikon does the same thing, setting values to ‘nonsensical’ when appropriate. For controlling Daikon’s output in the presence of nonsensical values, see the daikon.Daikon.guardNulls configuration option (see General configuration options in Daikon User Manual).


Previous: Nonsensical values, Up: Data trace records   [Contents][Index]

A.4.2 Variables that do not appear in trace records

A trace record should contain exactly the same variables as in the corresponding declaration. There is one exception: for efficiency, compile-time constants (e.g., static final variables in Java) are omitted from the trace record, since they would have the same value every time.

Furthermore, neither the declarations nor the trace records contain derived variables (see Variable names in Daikon User Manual).


Next: , Previous: Data trace records, Up: File formats   [Contents][Index]

A.5 Example files

Here are portions of two files StackArTester.decls and StackArTester.dtrace, for a Java class that implements a stack of integers using an array as the underlying data structure. You can see many more examples by simply running an existing front end on some Java, C, or Perl programs and viewing the resulting files.


Next: , Up: Example files   [Contents][Index]

A.5.1 Example declaration file

This is part of the file StackArTester.decls, a declaration file for the StackAr.java program (see StackAr example in Daikon User Manual).

ppt DataStructures.StackAr.push(java.lang.Object):::ENTER
ppt-type enter
parent parent DataStructures.StackAr:::OBJECT 1
  variable this
    var-kind variable 
    dec-type DataStructures.StackAr
    rep-type hashcode
    flags is_param
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable this.theArray
    var-kind field theArray
    enclosing-var this
    dec-type java.lang.Object[]
    rep-type hashcode
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable this.theArray.getClass()
    var-kind function getClass()
    enclosing-var this.theArray
    dec-type java.lang.Class
    rep-type java.lang.String
    flags synthetic classname
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable this.theArray[..]
    var-kind array 
    enclosing-var this.theArray
    array 1
    dec-type java.lang.Object[]
    rep-type hashcode[]
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable this.theArray[..].getClass()
    var-kind function getClass()
    enclosing-var this.theArray[..]
    array 1
    dec-type java.lang.Class[]
    rep-type java.lang.String[]
    flags synthetic classname
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable this.topOfStack
    var-kind field topOfStack
    enclosing-var this
    dec-type int
    rep-type int
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable x
    var-kind variable 
    dec-type java.lang.Object
    rep-type hashcode
    flags is_param
    comparability 22
  variable x.getClass()
    var-kind function getClass()
    enclosing-var x
    dec-type java.lang.Class
    rep-type java.lang.String
    flags synthetic classname
    comparability 22

ppt DataStructures.StackAr.push(java.lang.Object):::EXIT99
ppt-type subexit
parent parent DataStructures.StackAr:::OBJECT 1
  variable this
    var-kind variable 
    dec-type DataStructures.StackAr
    rep-type hashcode
    flags is_param
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable this.theArray
    var-kind field theArray
    enclosing-var this
    dec-type java.lang.Object[]
    rep-type hashcode
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable this.theArray.getClass()
    var-kind function getClass()
    enclosing-var this.theArray
    dec-type java.lang.Class
    rep-type java.lang.String
    flags synthetic classname
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable this.theArray[..]
    var-kind array 
    enclosing-var this.theArray
    array 1
    dec-type java.lang.Object[]
    rep-type hashcode[]
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable this.theArray[..].getClass()
    var-kind function getClass()
    enclosing-var this.theArray[..]
    array 1
    dec-type java.lang.Class[]
    rep-type java.lang.String[]
    flags synthetic classname
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable this.topOfStack
    var-kind field topOfStack
    enclosing-var this
    dec-type int
    rep-type int
    comparability 22
    parent DataStructures.StackAr:::OBJECT 1
  variable x
    var-kind variable 
    dec-type java.lang.Object
    rep-type hashcode
    flags is_param
    comparability 22
  variable x.getClass()
    var-kind function getClass()
    enclosing-var x
    dec-type java.lang.Class
    rep-type java.lang.String
    flags synthetic classname
    comparability 22

ppt DataStructures.StackAr:::OBJECT
ppt-type object
  variable this
    var-kind variable 
    dec-type DataStructures.StackAr
    rep-type hashcode
    flags is_param
    comparability 22
  variable this.theArray
    var-kind field theArray
    enclosing-var this
    dec-type java.lang.Object[]
    rep-type hashcode
    comparability 22
  variable this.theArray.getClass()
    var-kind function getClass()
    enclosing-var this.theArray
    dec-type java.lang.Class
    rep-type java.lang.String
    flags synthetic classname
    comparability 22
  variable this.theArray[..]
    var-kind array 
    enclosing-var this.theArray
    array 1
    dec-type java.lang.Object[]
    rep-type hashcode[]
    comparability 22
  variable this.theArray[..].getClass()
    var-kind function getClass()
    enclosing-var this.theArray[..]
    array 1
    dec-type java.lang.Class[]
    rep-type java.lang.String[]
    flags synthetic classname
    comparability 22
  variable this.topOfStack
    var-kind field topOfStack
    enclosing-var this
    dec-type int
    rep-type int
    comparability 22


Previous: Example declaration file, Up: Example files   [Contents][Index]

A.5.2 Example data trace file

This is part of file StackArTester.dtrace, which you can create by running the instrumented StackAr.java program (see StackAr example in Daikon User Manual). This excerpt contains only the first two calls to push and the first return from push, along with the associated object program point records; omitted records are indicated by ellipses.

...

StackAr.push(java.lang.Object):::ENTER
this_invocation_nonce
55
x
1217030
1
x.getClass()
"DataStructures.MyInteger"
1
this.theArray
3852104
1
this.theArray.getClass()
"java.lang.Object[]"
1
this.theArray[]
[null]
1
this.theArray[].getClass()
[null]
1
this.topOfStack
-1
1

StackAr:::OBJECT
this.theArray
3852104
1
this.theArray.getClass()
"java.lang.Object[]"
1
this.theArray[]
[null]
1
this.theArray[].getClass()
[null]
1
this.topOfStack
-1
1

...

StackAr.push(java.lang.Object):::EXIT96
this_invocation_nonce
55
x
1217030
1
x.getClass()
"DataStructures.MyInteger"
1
this.theArray
3852104
1
this.theArray.getClass()
"java.lang.Object[]"
1
this.theArray[]
[1217030]
1
this.theArray[].getClass()
["DataStructures.MyInteger"]
1
this.topOfStack
0
1

StackAr:::OBJECT
this.theArray
3852104
1
this.theArray.getClass()
"java.lang.Object[]"
1
this.theArray[]
[1217030]
1
this.theArray[].getClass()
["DataStructures.MyInteger"]
1
this.topOfStack
0
1

...

StackAr.push(java.lang.Object):::ENTER
this_invocation_nonce
94
x
1482257
1
x.getClass()
"DataStructures.StackAr"
1
this.theArray
350965
1
this.theArray.getClass()
"java.lang.Object[]"
1
this.theArray[]
[null]
1
this.theArray[].getClass()
[null]
1
this.topOfStack
-1
1

StackAr:::OBJECT
this.theArray
350965
1
this.theArray.getClass()
"java.lang.Object[]"
1
this.theArray[]
[null]
1
this.theArray[].getClass()
[null]
1
this.topOfStack
-1
1

...

Previous: Example files, Up: File formats   [Contents][Index]

A.6 Version 1 Declarations

This section describes the original version (1.0) of declaration records. These are now obsolete and should not be used.

A declarations file can contain program point declarations, VarComparability declarations, and ListImplementors declarations.


Next: , Up: Version 1 Declarations   [Contents][Index]

A.6.1 V1 Program point declarations

The format of a program point declaration is:

DECLARE
program-point-name
varname1
declared-type1 [# auxiliary-information1]
representation-type1 [= constant-value1]
comparable1
varname2
declared-type2 [# auxiliary-information2]
representation-type2 [= constant-value2]
comparable2
...

Program point information includes:


Next: , Previous: V1 Program point declarations, Up: Version 1 Declarations   [Contents][Index]

A.6.2 Program point name format specification

Instrumenting code creates a .decls file that contains program point names such as:

DataStructures.StackAr.push(java.lang.Object):::ENTER
DataStructures.StackAr.push(java.lang.Object):::EXIT99
PolyCalc.RatNum.RatNum(int, int):::ENTER
PolyCalc.RatNum.RatNum(int, int):::EXIT55
PolyCalc.RatNum.RatNum(int, int):::EXIT67

This section describes the format of these program point names. Someone writing an instrumenter for a new language must be sure to follow this format specification.

A program point name is a string with no tabs or newlines in it. The basic format is ‘topLevel.bottomLevel:::pptInfo’. For the first example given above, the top level of the hierarchy would be DataStructures.StackAr, the bottom level would be push(java.lang.Object), and the program point information would be ENTER.

topLevel may contain any number of periods (‘.’). bottomLevel and pptInfo may not contain any periods. The string ‘:::’ may only appear once.

topLevel and pptInfo are required (i.e., they must be non-empty), as are the period to the right of topLevel and the colons to the left of pptInfo. However, bottomLevel is optional.

By convention, for Java topLevel consists of the class name, and bottomLevel consists of the method name and method signature.

For C, topLevel consists of a filename (or a single period for global functions), and bottomLevel could consist of a function name and signature. More precisely, names of C program points follow these conventions:

For an Input Output Automaton, topLevel consists of an Automaton name and bottomLevel consists of information for a transition state.

By convention, the entry and exit points for a function have names of a special form so that they can be associated with one another. (Currently, those names end with :::ENTER and :::EXIT.) This convention permits Daikon to generate pre-state variables (see Variable names in Daikon User Manual) automatically at procedure exit points, so front ends need not output them explicitly. When there are multiple exit points, then each one should be suffixed by a number (such as a line number, for example, foo::EXIT22). Daikon produces the main (non-numbered) :::EXIT point automatically. All the numbered exits should contain the same set of variables; in general, this means that local variables are not included at exit points. Daikon currently requires that declarations for :::ENTER program points appear before any declarations for matching :::EXIT program points.

Another convention is to have another program point whose bottomLevel is empty and whose pptInfo is OBJECT: for example, StackAr:::OBJECT. This contains the representation invariant (sometimes called the object invariant) of a class. This program point is created automatically by Daikon; it need not appear in a trace file.


Next: , Previous: V1 pptname format, Up: Version 1 Declarations   [Contents][Index]

A.6.3 V1 VarComparability declaration

There is a special VarComparability declaration that controls how the comparability field in program point declarations is interpreted. The default VarComparability is implicit, which means ordinary comparability as described in Program point declarations. (The name implicit is retained for historical reasons.) You can override it as follows:

VarComparability
none

As with all records in Daikon input files, a blank line is required between this record and the next one.


Previous: V1 VarComparability declaration, Up: Version 1 Declarations   [Contents][Index]

A.6.4 V1 ListImplementors declaration

This declaration indicates classes that implement the java.util.List interface, and should be treated as sequences for the purposes of invariant detection. The syntax is as follows:

ListImplementors
<classname1>
<classname2>
...

Each classname is in Java format (for example, java.util.LinkedList).

The --list_type command-line option to Daikon can also be used to specify classes that implement lists; see Options to control invariant detection in Daikon User Manual.


Previous: File formats, Up: Top   [Contents][Index]

General Index

Jump to:   .  
A   B   C   D   E   F   H   I   J   L   M   N   O   P   R   S   T   U   V   W  
Index Entry  Section

.
.decls file: Declarations
.decls file (version 1): Version 1 Declarations
.dtrace file: Data trace records
.jpp files: Compiling Daikon

A
adding new derived variables: New derived variables
adding new invariants: New invariants
adding new output formats: New formatting for invariants
adding new regression tests: Adding regression tests
adding new suppressors: New suppressors
adding new unit tests: Unit testing
adding track logging: Adding track logging

B
branches, in CVS repository: Branches

C
C programs, instrumenting: Instrumenting C programs
changing Daikon: Extending Daikon
comparability, for variables: Variable comparability
comparability, for variables (.decls format version 1): V1 Program point declarations
compiling Daikon: Compiling Daikon
customizing Daikon: Extending Daikon

D
Daikon internals: Daikon internals
data trace format: Data trace records
dataflow hierarchy: Dataflow hierarchy
deallocated pointers: Instrumenting C programs
debugging Daikon: Debugging Daikon
declaration file format: Variable declarations
declaration format: Declarations
derived variable: New derived variables

E
Eclipse: Using Eclipse
efficiency issues: Daikon internals
equality optimization: Equality optimization
extending Daikon: Extending Daikon

F
file formats: File formats
front end, writing: New front ends

H
hashcode type, for variables: Variable declarations
Hg repository: Version control repository
Hg repository: Historical
hierarchy: Dataflow hierarchy

I
instrumenting C programs: Instrumenting C programs
instrumenting Java programs: Example instrumented Java program
invalid values: Instrumenting C programs

J
Java compiler, specifying: Compiling Daikon
Java programs, instrumenting: Example instrumented Java program
javac’ compiler, overriding: Compiling Daikon
Jikes compiler: Compiling Daikon
jpp files: Compiling Daikon

L
logging: Track logging

M
Mercurial repository: Version control repository
Mercurial repository: Historical
missing values for variables, see nonsensical values: Data trace records
modified bit: Data trace records
modifying Daikon: Extending Daikon

N
new derived variables: New derived variables
new invariants: New invariants
new output formats: New formatting for invariants
new regression tests: Adding regression tests
new suppressors: New suppressors
new unit tests: Unit testing
non-checking of invariants: Avoiding work for redundant invariants
non-instantiation of invariants: Avoiding work for redundant invariants
non-printing of invariants: Avoiding work for redundant invariants
nonce, invocation: Data trace records
nonsensical values: Instrumenting C programs
nonsensical values for variables: Data trace records

O
optimizations: Daikon internals
output format, defining new: New formatting for invariants

P
pointer variables, see hashcode type: Variable declarations

R
regression tests: Regression tests
repository: Version control repository
repository: Historical

S
suppressors, adding new: New suppressors

T
testing Daikon: Testing
this_invocation_nonce: Data trace records
track log output: Track log output
track logging: Track logging

U
uninitialized variables: Instrumenting C programs
unit testing: Unit testing
units of measurement, see variable comparability: Variable comparability

V
valid values: Instrumenting C programs
variable comparability: Variable comparability
variable comparability (.decls format version 1): V1 Program point declarations
variable, derived: New derived variables
version control repository: Version control repository
Version control repository: Historical

W
Windows, compiling: Compiling Daikon

Jump to:   .  
A   B   C   D   E   F   H   I   J   L   M   N   O   P   R   S   T   U   V   W