Next: Historical, Previous: Editing, Up: Top [Contents][Index]
This section provides the instructions for publishing a Daikon distribution,
a.k.a. making a release.
If you only want to
create the daikon.tar or daikon.tar.gz file in your own
directory, then simply run make daikon.tar
or make daikon.tar.gz
.
Official releases have even version numbers (e.g., 4.6.4) and intermediate work has odd version numbers (e.g., 4.7.3). This means as you prepare for a release the current version number is probably odd. It will be updated as one of the first steps in the release process. After making the distribution, one of the final steps is to increment the version number again to prepare for subsequent development. This system has the useful side effect of allowing the build and test process to be repeated to fix a problem (steps 2–14 of Distribution steps below) without having to worry about updating or resetting the version number. Another advantage is to reinforce, to people who are working from the version control repository, that they are not using the released version, because the version numbers differ.
The Daikon distribution site is located at http://plse.cs.washington.edu/daikon/ and is served from the UW CSE file system at /cse/web/research/plse/daikon. In order to be able to write to the distribution site, your CSE user id must be a member of the ‘plse_www’ Unix group.
For each of the major steps below, an approximate elapsed time is listed. These timings are up to date as of November 2014. They were measured on a quad x86-64 based machine at 3.4GHz with 16GB of memory (buffalo.cs.washington.edu). Barring any difficulties, the entire process will take at least two hours — but could be much more depending on the number of different platforms on which you test the release.
IMPORTANT:
The default version of the Java JDK on the majority of the CSE
machines is now JDK 8. However, for the time being, we wish to continue
to use JDK 7 for our distribution process; the intention is to maximize
compatibility with our existing code base. You will find a suitable
version (for an AMD64 based machine) at /homes/gws/markro/jdk7.
In addition, if you have used
This should fulfill most of the directory layout requirements noted below. |
Each of the steps below assumes that the current working directory is $DAIKONDIR and that you are using the Bash shell.
• Directory layout: | ||
• Updating dependencies: | ||
• Distribution steps: |
Next: Updating dependencies, Up: Distribution [Contents][Index]
DAIKONDIR
must be set to a clone of
https://github.com/codespecs/daikon.
JAVA_HOME
must be set to your JDK installation.
BIBDIR
must be set to a clone of
https://github.com/mernst/plume-bib.
You can create this clone with the commands:
cd git clone https://github.com/mernst/plume-bib.git bib export BIBDIR=`pwd`/bib
Alternatively, you may set the environment variable BIBDIR
to the location
of the bibliography you wish to use.
Next: Distribution steps, Previous: Directory layout, Up: Distribution [Contents][Index]
set -o pipefail (cd $DAIKONDIR && git pull && git log --branches --not --remotes && git status) (cd $DAIKONDIR/../fjalar && git pull && git log --branches --not --remotes && git status)
Each of the two commands should print exactly 4 lines:
Already up-to-date. On branch master Your branch is up-to-date with 'origin/master'. nothing to commit, working directory clean
[Time: moments]
CHECKERFRAMEWORK
environment
variable to the location of the Checker Framework directory.
[Time: moments]
make -C $DAIKONDIR plume-lib-update cd $DAIKONDIR/plume-lib/java make clean && make && make verify-plume-jar-classfile-version && \ \cp -pf plume.jar ../../java/lib
[Time: 5 min]
(cd $BIBDIR; git pull)
[Time: moments]
The Fjalar tool set (primarily, Kvasir) is built upon, or uses pieces from, two open source projects. The home page for the Valgrind instrumentation framework is http://www.valgrind.org. As of March 2015 we are using version 3.10.1, released in November, 2014. (You can verify this by inspecting the first line of the file $DAIKONDIR/fjalar/valgrind/NEWS.)
The home page for the GNU Binutils (a collection of binary tools, of which we only use readelf) is http://www.gnu.org/software/binutils/. As of April 2015 we are using version 2.25, released in December, 2014. [TODO: How can a reader confirm that this is correct or determine the version on his or her own?]
If a newer version of either of these projects is available, you should update the Fjalar source tree as soon as practical. For details, see the separate document Merging newer versions of Valgrind into Fjalar.
Previous: Updating dependencies, Up: Distribution [Contents][Index]
IMPORTANT:
While the majority of the distribution process is currently run
with JDK 7; we need to ensure that our system runs on JDK 8 as well.
This means you will need to repeat steps 2-5 below in a separate shell
with |
cd $DAIKONDIR make update-dist-version-file
Optionally, override the distribution date (default: today) by
redefining the environment variable TODAY
:
export TODAY='November 1, 2013'
Now, update the appropriate files with the date and version number of the release.
make update-doc-dist-date-and-version
[Time: moments]
make very-clean make rebuild-everything
[Time: 20 min]
make -C $DAIKONDIR/java check-nullness
[Time: 30 min]
make -C $DAIKONDIR/tests clean diffs
[Time: 80 min]
make -C $DAIKONDIR/tests/kvasir-tests clean-all regression-tests
[Time: 20 min]
cd $DAIKONDIR/tests/parser-tests ./run-all
Success is indicated by each of the diff comparisons being identical.
[Time: moments]
cd $DAIKONDIR/doc diff -b -u -s --from-file /cse/web/research/plse/daikon/download/doc *.texinfo
Another good source of information are the repository logs since the last release:
cd $DAIKONDIR DAIKONVERSION=`wget -q http://plse.cs.washington.edu/daikon/download/doc/VERSION \ -O - | xargs echo -n` git log v$DAIKONVERSION..HEAD
In addition, you should run the same command in your Fjalar clone:
cd $DAIKONDIR/fjalar DAIKONVERSION=`wget -q http://plse.cs.washington.edu/daikon/download/doc/VERSION \ -O - | xargs echo -n` git log v$DAIKONVERSION..HEAD
[Time: a few minutes]
cd $DAIKONDIR/doc make spell-check
Any words that may be spelled incorrectly are output to standard out.
@nospellcheck
around it.
[Time: moments]
cd $DAIKONDIR \rm -f `find . -type f -name '*~'` make staging
[Time: 4 min]
make check-for-broken-doc-links
Review check.log and make corrections as appropriate.
In some cases a "403 Forbidden" error is transient, due to the website being down. You can check such links by hand.
If the URL is correct but cannot be checked (for example, because the website prohibits spiders or because the URL redirects to another URL but you prefer to keep the first URL in your document), then you may need to add lines to an appropriate part of file $DAIKONDIR/plume-lib/bin/checklink-args.txt.
In most cases (and for most 404 errors), you should fix the document with the incorrect link. Here is a workflow that may be used to deal with broken links:
Note that you must fix it in the original repository,
not in the drop directories.
This means correcting the file, most likely located beneath either
$DAIKONDIR/doc or $BIBDIR, committing the changes, and
returning to the previous step (make staging
) prior to re-running
the broken link tool.
[Time: 3 min]
make test-staged-dist
[Time: 20 min; runs Checker Framework tests]
mkdir -p ~/tmp cd ~/tmp DAIKONVERSION=`wget -q http://plse.cs.washington.edu/daikon/staging-daikon/download/doc/VERSION \ -O - | xargs echo -n` rm -rf daikon-$DAIKONVERSION.tar.gz daikon-$DAIKONVERSION wget http://plse.cs.washington.edu/daikon/staging-daikon/download/daikon-$DAIKONVERSION.tar.gz tar xzf daikon-$DAIKONVERSION.tar.gz cd daikon-$DAIKONVERSION export JAVA_HOME=/usr/lib/jvm/java export DAIKONDIR=`pwd` source scripts/daikon.bashrc make distribution-check
[Time: 10 min]
The instructions below download the staged release archive, extract the files, set the appropriate environment variables and run the top level Makefile. Finally, a short verification test is run; success is indicated by invariant output being written to the screen.
mkdir -p ~/tmp cd ~/tmp DAIKONVERSION=`wget -q http://plse.cs.washington.edu/daikon/staging-daikon/download/doc/VERSION \ -O - | xargs echo -n` rm -rf daikon-$DAIKONVERSION.tar.gz daikon-$DAIKONVERSION wget http://plse.cs.washington.edu/daikon/staging-daikon/download/daikon-$DAIKONVERSION.tar.gz tar xzf daikon-$DAIKONVERSION.tar.gz cd daikon-$DAIKONVERSION export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 export DAIKONDIR=`pwd` source scripts/daikon.bashrc make distribution-check
[Time: ?? min]
$DAIKONDIR
defined!).
The output from the make command should show a number of lines beginning with "Processed" as dcomp_rt.jar is being built. Next it should indicate that Kvasir was not built because this is not a Linux platform. Then execution will continue with a test that runs Chicory/Daikon on the StackAr example. Success is indicated by invariant output being written to the screen.
mkdir -p ~/tmp cd ~/tmp DAIKONVERSION=`wget -q http://plse.cs.washington.edu/daikon/staging-daikon/download/doc/VERSION \ -O - | xargs echo -n` rm -rf daikon-$DAIKONVERSION.tar.gz daikon-$DAIKONVERSION wget http://plse.cs.washington.edu/daikon/staging-daikon/download/daikon-$DAIKONVERSION.tar.gz tar xzf daikon-$DAIKONVERSION.tar.gz cd daikon-$DAIKONVERSION # Convert OSTYPE from a shell variable to a system environment variable. export OSTYPE export JAVA_HOME="/cygdrive/c/Program Files/Java/jdk1.7.0_45" export DAIKONDIR=`pwd` source scripts/daikon.bashrc make distribution-check
[Time: 10 min]
The output from the make command should show a number of lines beginning with "Processed" as dcomp_rt.jar is being built. Next it should indicate that Kvasir was not built because this is not a Linux platform. Then execution will continue with a test that runs Chicory/Daikon on the StackAr example. Success is indicated by invariant output being written to the screen.
mkdir -p ~/tmp cd ~/tmp DAIKONVERSION=`curl -s http://plse.cs.washington.edu/daikon/staging-daikon/download/doc/VERSION \ | xargs echo -n` rm -rf daikon-$DAIKONVERSION.tar.gz daikon-$DAIKONVERSION curl -O http://plse.cs.washington.edu/daikon/staging-daikon/download/daikon-$DAIKONVERSION.tar.gz # or: wget http://plse.cs.washington.edu/daikon/staging-daikon/download/daikon-$DAIKONVERSION.tar.gz tar xzf daikon-$DAIKONVERSION.tar.gz cd daikon-$DAIKONVERSION # Convert OSTYPE from a shell variable to a system environment variable. export OSTYPE export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home export DAIKONDIR=`pwd` source scripts/daikon.bashrc make distribution-check
[Time: 4 min]
make save-current-release
[Time: moments]
make staging-to-www
[Time: 1 min]
The changeset should only have the change in version number, nothing else, so this should come right after changing daikon/doc/VERSION which should come just before “make staging”.
cd $DAIKONDIR DAIKONVERSION=`cat $DAIKONDIR/doc/VERSION | xargs echo -n` git commit -a -m "Change version to $DAIKONVERSION" git push cd fjalar git commit -a -m "Change version to $DAIKONVERSION" git push
[Time: moments]
cd $DAIKONDIR DAIKONVERSION=`cat $DAIKONDIR/doc/VERSION | xargs echo -n` git tag -a v$DAIKONVERSION git push --tags cd fjalar git tag -a v$DAIKONVERSION git push --tags
[Time: moments]
make update-dist-version-file
If you previously set the environment variable TODAY, you need to reset it:
export -n TODAY
Now, update the appropriate files with the date and version number for ongoing development. (We revert the change to doc/README as we want it to reflect the most recent actual release.)
make update-doc-dist-date-and-version git checkout doc/README
[Time: moments]
make rebuild-everything make quick-test
Success is indicated by invariant output being written to the screen.
[Time: 5 min]
git commit -a -m "Bump version number for ongoing development." git push cd fjalar git commit -a -m "Bump version number for ongoing development." git push
[Time: moments]
Previous: Updating dependencies, Up: Distribution [Contents][Index]