Default Template

[Edited 2015-12-04 to add LSAN, MSAN]

For reference, here's the default Travis configuration that I use to get a selection of builds running against an Autotools-based projects that has a make check target. It includes the base {linux, osx} × {gcc, clang} matrix, together with sanitizer runs (ASAN, UBSAN) and run to generate code coverage information for Coveralls.

language: c
sudo: false
matrix:
    include:
        - os: linux
          compiler: gcc
          env: BUILD_TYPE=normal
        - os: linux
          compiler: clang
          env: BUILD_TYPE=normal
        - os: linux
          compiler: gcc
          env: BUILD_TYPE=coverage
        - os: linux
          compiler: clang
          env: BUILD_TYPE=ubsan
        - os: linux
          compiler: clang
          env: BUILD_TYPE=asan
        - os: linux
          compiler: clang
          env: BUILD_TYPE=lsan
        - os: linux
          compiler: clang
          env: BUILD_TYPE=msan
        - os: osx
          compiler: gcc
          env: BUILD_TYPE=normal
        - os: osx
          compiler: clang
          env: BUILD_TYPE=normal
install:
    - pip install --user 'requests[security]'
    - pip install --user cpp-coveralls
before_script:
    - |
        if [ "$BUILD_TYPE" = "coverage" ]; then
             export CFLAGS="-fprofile-arcs -ftest-coverage"
             export LDFLAGS="-fprofile-arcs -ftest-coverage"
        fi
    - |
         if [ "$BUILD_TYPE" = "asan" ]; then
             export CFLAGS=-fsanitize=address
             export LDFLAGS=-fsanitize=address
         fi
    - |
         if [ "$BUILD_TYPE" = "lsan" ]; then
             export CFLAGS=-fsanitize=leak
             export LDFLAGS=-fsanitize=leak
         fi
    - |
         if [ "$BUILD_TYPE" = "msan" ]; then
             export CFLAGS=-fsanitize=memory
             export LDFLAGS=-fsanitize=memory
         fi
    - |
         if [ "$BUILD_TYPE" = "ubsan" ]; then
             export UBSAN_FLAGS="-fsanitize=undefined"
             export CFLAGS=$UBSAN_FLAGS
             export LDFLAGS=$UBSAN_FLAGS
         fi
script:
    - ./configure && make && make check
    - |
          if [ "$BUILD_TYPE" = "coverage" ]; then
              cpp-coveralls --gcov-options '\-lp'
          fi

Taking Control of the Matrix

The normal way to set up your matrix of Travis build configurations is to specify each of the possible axes (os, compiler, env) as a list, so that Travis builds every possible combination of those options. However, once you start adding custom builds controlled by env values, such as the sanitizer builds described in the previous entry, not all of the possible combinations make sense – for example, you might choose to just run an ASAN build on Linux + Clang, or only generate code coverage output from a single build.

So when your valid build combinations are sparse in the matrix, it can be easier to explicitly include the combinations that are valid, like so:

    matrix:
        include:
            - os: linux
              compiler: gcc
              env: BUILD_TYPE=normal
            - os: linux
              compiler: clang
              env: BUILD_TYPE=normal
            - os: linux
              compiler: gcc
              env: BUILD_TYPE=coverage
            - os: linux
              compiler: clang
              env: BUILD_TYPE=ubsan
            - os: linux
              compiler: clang
              env: BUILD_TYPE=asan
            - os: osx
              compiler: gcc
              env: BUILD_TYPE=normal
            - os: osx
              compiler: clang
              env: BUILD_TYPE=normal
  

Note that it's still possible to use the env.global key to specify an environment variable that should apply across all of the builds (rather than adding a new combination option).

    env:
      global:
        - CPPFLAGS = -Wall
  

One more trick to help with complex build matrices – YAML anchors and aliases can help to cut down repetition in your Travis config. Roughly speaking, adding &name after a key allows later config lines to use *name to repeat the value of the earlier key, either as-is or as the base for other additions:

    matrix:
      include:
        - os: linux
          compiler: clang
          addons:
            apt:
              sources: &common_sources
                - ubuntu-toolchain-r-test
                - llvm-toolchain-precise-3.6
              packages: &common_packages
                - autoconf
                - automake
        - os: linux
          compiler: clang
          env: REAL_CC=clang-3.6 REAL_CXX=clang++-3.6
          addons:
            apt:
              sources: *common_sources
              packages:
              - *common_packages
              - clang-3.6
    …
  

(Thanks to Al for suggesting this tip, and check out the Certificate Transparency Travis config for example of this in action.)

Sanitize All The Things!

[Edited 2015-12-04 to add LSAN]

If your C/C++ project includes run-time tests (you do have tests, don't you?), then it's a really good idea to run the tests under all of the sanitizers that are available from the compiler:

• Address Sanitizer (ASAN, -fsanitize=address): detect out-of-bounds memory accesses
• Memory Sanitizer (MSAN, -fsanitize=memory): detect reads of uninitialized memory.
• Undefined Behaviour Sanitizer (UBSAN, -fsanitize=undefined): detect reads of uninitialized memory.
• Thread Sanitizer (TSAN, -fsanitize=thread): detect data races in multi-threaded programs.
• Leak Sanitizer (LSAN, -fsanitize=leak): detect memory leaks (recent Clang versions enable this by default when ASAN is used)

These tools have been available in Clang for a while, but more recent versions of GCC also include some of them:

Compiler Version	ASAN	MSAN	UBSAN	TSAN	PPA / package
gcc-4.4	N	N	N	N
gcc-4.5	N	N	N	N
gcc-4.6	N	N	N	N
gcc-4.7	N	N	N	N	ubuntu-toolchain-r / gcc-4.7
gcc-4.8	Y	N	N	Y	ubuntu-toolchain-r / gcc-4.8
gcc-5	Y	N	Y	Y	ubuntu-toolchain-r / gcc-5
clang-3.4	Y	Y	Y	Y
clang-3.5	Y	Y	Y	Y	ubuntu-toolchain-r-test, llvm-toolchain-precise-3.5 /clang-3.5*
clang-3.6	Y	Y	Y	Y	ubuntu-toolchain-r-test, llvm-toolchain-precise-3.6 / clang-3.6*
clang-3.7	Y	Y	Y	Y	ubuntu-toolchain-r-test, llvm-toolchain-precise-3.7 / clang-3.7*
clang-3.8	Y	Y	Y	Y	ubuntu-toolchain-r-test, llvm-toolchain-precise / clang-3.8*

*: Only one of the different clang versions can be installed at a time, so you need to pick a single version for your build. (The gcc packages allow multiple gcc-ver binaries to be installed in parallel.)

There are a couple of things to be aware of when pointing the sanitizers at your code:

• MSAN expects that all of the code that makes up your test executable has been compiled with -fsanitize=memory. If it hasn't – for example, if you link in some system-provided library from /usr/lib – then there will be lots of false positive uninitialized-memory-read errors (because memory initialization done by the uninstrumented library is missed). If your project has a lot of external dependencies, this may make it much harder to use MSAN.
• UBSAN can be a little overwhelming when applied to older C codebases, so you might need to replace the overall -fsanitize=undefined with a subset of the individual checks.

Finally, to help confirm that your sanitizer builds are correctly catching errors, here's a test program that has every error under the sun:

#include <stdlib.h>
#include <stdio.h>
#include <pthread.h>

void *inc_x(void *p) {
  int *px = (int*)p;
  (*px)++;
  return NULL;
}

int main() {
  char *p = malloc(10);
  p[5] = '\0';
  if (p[2] == '\0')  /* MSAN: uninitialized memory read */
    printf("found null\n");

  p[11] = '\0'; /* ASAN: heap-buffer-overflow */

  int i = 23;
  i <<= 32;  /* UBSAN: shift overflow */
  char array[3] = "ab";
  printf("one step beyond: %c\n", array[4]); /* UBSAN: index out of bounds*/
  char data[5] = {0x00, 0x01, 0x02, 0x03, 0x04};
  int *pi = (int*)&(data[1]);
  printf("int %08x\n", *pi); /* UBSAN: misaligned address */

  pthread_t t;
  int x = 0;
  printf("x=%d\n", x);
  pthread_create(&t, NULL, inc_x, &x);
  x = 3; /* TSAN: data race */
  pthread_join(t, NULL);
  printf("x=%d\n", x);

  p = NULL; /* LSAN: leak p */
  return 0;
}

When is gcc not gcc?

If you're trying to use Travis to get as diverse a build matrix as possible, here's one thing to watch out for. A Travis config like:

    language: c
    compiler: gcc
    os: osx
  

has an interesting detail in the output:

    $ gcc --version
    Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
    Apple LLVM version 6.0 (clang-600.0.54) (based on LLVM 3.5svn)
    Target: x86_64-apple-darwin13.4.0
    Thread model: posix

Huh? Although we asked for gcc, we're getting something that looks like LLVM! Issue 2423 has the down-low, which is basically that by default OS X has a gcc binary that's a front-end for LLVM. That issue also has the work-around, which is to export CC=gcc-4.8.

---

Which libtool?

Projects that are based around the GNU Autotools will normally use a tool called libtool to abstract away the different tool options that are needed to build software libraries on different platforms.

However, if you're using Travis's support for the OS X platform to build your project, you may encounter a problem: OS X has its own libtool that does something slightly different. As a result, there's an alternate name for the GNU libtool on OS X, namely glibtool; similarly, the companion libtoolize tool is called glibtoolize.

This problem doesn't usually show up when you're building the distributed version of an project, because that will normally ship with a pre-built configure script, and that configure script can generate and use its own ./libtool (normally from an included ltmain.sh file).

However, Travis typically builds things straight from a Git repo, which normally doesn't include the outputs of the autotools process (i.e. the Travis build normally includes a step like ./autogen.sh or autoreconf to generate configure, ltmain.sh and friends). This also means that a missing libtoolize may well show up as the first sign of this problem, since libtoolize is used in this generation process.

If you hit this problem, you may need to fiddle with your Autotools wrapper script so that it copes with either glibtool or libtool. Here's a couple of examples:

• libuv's autogen.sh script.
• c-ares's buildconf script.

See also: Stack Overflow question.

Beware of Absolute Notification Channels

Travis includes lots of different notification mechanisms. However, for popular projects there's a small danger that's not immediately obvious until it's too late.

If you add a notification channel that's directed at an explicit address, then that explicit address will also be present in the .travis.yml file for any forks of your project. That means that any Travis build failures for those forks will also notify the explicit address – so think carefully about whether you want (say) your IRC channel to receive build notifications for everyone's fork of the project.

Extra Packages

The container-based Linux build is much faster than the VM-based system, but it comes with the limitation that no sudo commands are allowed. However, if you're just using sudo for package installation, then the addons section allows package installation from a whitelist without sudo.

Some of the whitelisted packages: aren't in the default sources, so you might also need to add a new sources: entry from the whitelisted sources.

This also makes it important that you check through the "apt" output from your build after you've added a new prerequisite package, because failures to install the packages in the addons section won't fail the build at that step (and the "apt" section of the build log is folded closed by default).

Experimenting with Travis

Travis makes simple things simple, but it can be somewhat fiddly when things get more complicated. This normally means that setting up builds that have external dependencies, or which involve additional tools, involves a fair amount of trial and error. To help with this:

• Set up your build in a temporary branch in your repo, and only copy the changes across into your main branch when the build is fully working. Anyone who syncs to a branch called temp deserves to have that branch deleted out from under them, and confining build changes to a branch like that means the main branch doesn't get polluted by the inevitable sequence of "debug", "tweak config" , "add missing CFLAGS option" commits.
• Test out each change to the build locally before pushing to GitHub and Travis. Given the time taken for a Travis build cycle (particularly once the Americans wake up), it's normally worth checking the build locally to catch typos and misconfigurations faster.
• When setting up the build of an Autotools based project, include a script: line like ./configure $CONFIG_OPTS || tail -1000 config.log, so that you can find out more than a single bit of information when the configure step fails.