QA Tools
                                     aka
                           Nightly Builds made Easy
                                      by
                          Martin Man <mman@swac.cz>
                       ===============================

What is QA Tools
================

QA Tools allow you (the software product maintaner) to automatically generate
regular releases and/or snapshots of your product. Your product can be
distributed as a source code archive (typically .tar.gz) or as a binary
distribution archive (.deb, .rpm, .exe).

QA Tools can be used to generate nighly-built source/binary packages suitable
for continuous integration and testing.

QA Tools can handle very simple product dependencies (read: in order to
build/package product A you first need to build/package product B)

QA Tools work currently with the software packages that have source code
managed using Subversion and that are ./configured && make installed using
autoconf/automake.

QA Tools can help the people who are developing set of dependent crossplatform
products that need to be packaged for different distributions (Debian, Ubuntu,
Fedora Core, Suse, Windows, ...).

You should be familiar with autoconf/automake (particulary `make dist'), RPM
package building, and DEB package building. You should know how to store
source code in SVN (trunk, branches, tags). QA Tools will not do any of those
for you, but will help you to automate certain things and consume less time
doing them.

Naming conventions
==================

QA Tools use the two following terms:

    `Product' is an application that is distributed as one source .tar.gz file
    and one or more binary packages in .rpm, .deb, and .exe format. Example
    product is email application called My-Email-App-1.0.tar.gz

    `Product Suite' is a set of applications, that are distributed as multiple
    dependent source .tar.gz files and multiple binary packages in .rpm, .deb,
    .exe formats. Example Product Suite is My-Email-App-1.0.tar.gz along with
    My-Email-App-GPG-Plugin-1.0.tar.gz and
    My-Email-App-SMIME-Plugin.1.0.tar.gz.

Product Requirements
====================

Product absolutely *must* fullfil the following requirements:

   1) be stored in SVN (Subversion repository)
   2) use autoconf/automake to ./configure && make && make install

[Note that we might support another SCM backend system (CVS, SVK, ...) in the
future]

[Note that we might support another build management system (Ant, Nant, ...)
in the future]

Quick Reference
===============

The following is a quick summary of all available commands (man pages still
missing):

[Note the use of $PRODUCT, $VERSION, $RELEASE variables that are parsed out
from configure.in, debian/changelog, or rpm .spec file, $DATE is the current
date in YYYYMMDD format]

Product related commands
------------------------

    qat-product-snapshot myproduct.product -t

        Generates $PRODUCT-$VERSION+$DATE.tar.gz from SVN trunk of your
        product.

    qat-product-snapshot myproduct.product -b <branch>

        Generates $PRODUCT-$VERSION+$DATE.tar.gz from SVN branch <branch> of
        your product.

    qat-product-release myproduct.product -r <release>

        Generates $PRODUCT-$VERSION.tar.gz from SVN tags <release> of
        your product. Note that $VERSION should usually match <release> here.

    qat-product-install myproduct.product -t

        Installs the tarball generated by qat-product-snapshot -t to $PREFIX

    qat-product-install myproduct.product -b <branch>

        Installs the tarball generated by qat-product-snapshot -b <branch> to
        $PREFIX

    qat-product-install myproduct.product -r <release>

        Installs the tarball generated by qat-product-release -r <release> to
        $PREFIX

    qat-product-cleanup myproduct.product

        Deletes temporary build directories/files left on filesystem after an
        unsuccessful build.

Suite related commands
------------------------

    qat-suite-snapshot myproduct.suite

        Generates $PRODUCT-$VERSION+$DATE.tar.gz from SVN trunk of each and
        every product defined by the myproduct.suite.

    qat-suite-release myproduct.suite

        Generates $PRODUCT-$VERSION.tar.gz from SVN release 'current' of each
        and every product defined by the myproduct.suite.

Distro specific commands
------------------------

    qat-product-snapshot-deb myproduct.product -t

        Generates debian packages for all DEB_DISTROS from SVN trunk of
        packaging related files (read: debian/ subdirectory of your package).

    qat-product-snapshot-rpm myproduct.product -t

        Generates rpm packages for all RPM_DISTROS from SVN trunk of
        packaging related files (read: .spec file of your package).

[Note that you can override DEB_DISTROS, RPM_DISTROS from the commandline in
order to restrict the build to a certain distro only. This is useful for
example for RPM based systems that don't have anything like pbuilder(8).]

SVN related commands
--------------------

Since QA Tools rely heavily on correct structure of your svn repository, there
are also several tools that can help you to mark a branch/release of your
product without invoking svn manually.

    qat-product-svn-release myproduct.product -t

        Creates new release from SVN trunk of your product (by svn copy-ing
        trunk to tags/).

    qat-product-svn-release myproduct.product -b <branch>

        Creates new release from SVN branch <branch> of your product (by svn
        copy-ing that branch to tags/).

    qat-product-svn-branch myproduct.product -t -b <branch>

        Creates new branch from SVN tag of your product (by svn
        copy-ing that branch to branches/). This is typically needed after
        qat-product-svn-release.

Requirements
============

The following software components should be available on your system in order
to install and use QA Tools:

    1) Bourne Shell (as available in every Linux distro)
    2) GNU awk (sometimes available as gawk and not installed by default)
    3) mailx (to have /usr/bin/mail command for build result reporting)
    4) working autoconf/automake
    5) working distro specific package builders (this includes
       /usr/sbin/pbuilder for debian based systems, /usr/bin/rpmbuild for
       rpm based systems, mingw-package for windows)

Installing QA Tools
===================

The magic triple:

$ ./configure && make && sudo make install

should install QA Tools to /usr/local/ directory on your system.

Preparing Product configuration file
====================================

First of all, prepare configuration file for your product. The configuration
file is an ordinary shell script that has to define several variables.

[Note that I am using shell variable expansions here to override some
variables from environment or from other configuration files]

---8<--- [dummy-product.product] ---8<---
# This is a configuration file describing a product that should be
# automatically built by QA Tools.

# what comes in and what comes out
REPOSITORY=${REPOSITORY:="https://lintouch.org/repos/lintouch"}
PRODUCT=${PRODUCT:="dummy-product"}
RESULTDIR="/tmp/${PRODUCT}"

# svn repository layout
LAYOUT="default-svn-layout.conf"

# supported distros
DEB_DISTROS="debian ununtu"
RPM_DISTROS="suse10 fc4"
MINGW_DISTROS="mingw"

# package builders
DEB_BUILDER="default-deb-builder.conf"
RPM_BUILDER="default-rpm-builder.conf"
MINGW_BUILDER="default-mingw-builder.conf"
---8<--- [dummy-product.product] ---8<---

You are expected to describe where the source is code stored, the REPOSITORY,
and PRODUCT variables are used within default-svn-layout.conf file to generate
SVN URLs pointing to trunk, specified branch, and specified release of your
product. Please consult the default-svn-layout.conf file now.

RESULTDIR defines where the resulting source/binary packages should be copied.
The following directories will be created after executing various QA Tools.

    $RESULTDIR/sources/ - will get the .tar.gz sources
    $RESULTDIR/debian/  - will get the .deb packages built for debian
    $RESULTDIR/ubuntu/  - will get the .deb packages built for ubuntu
    $RESULTDIR/suse10/  - will get the .rpm packages built for suse 10
    $RESULTDIR/fc4/     - will get the .rpm packages built for fedora core 4
    $RESULTDIR/mingw/   - will get the .exe installers built for windows
    $RESULTDIR/logs/    - will get the build logs

*_DISTROS define the distributions we support with our binary builds, you can
again look at the default-svn-layout.conf file in order to determine how
packaging related SVN URLs are formed.

*_BUILDER variables define how to build binary packages from given source
package. Please consult the contrib/README file to see some example builders
for .deb and .rpm based systems.

Preparing Product Suite configuration file
==========================================

The product suite merely defines a set of products that should be build one
after each other, while reporting the results (for example via email).

---8<--- [mysuite.suite] ---8<---

# The products to be built/packaged
PRODUCTS="email-app.product email-app-gpg.product email-app-smime-product"

# How to report results
REPORTS="default-email-reporting.conf"

# Where to install the code and where to copy the resulting packages
export PREFIX=/tmp/email-app-prefix
export RESULTDIR="/tmp/email-app

---8<--- [mysuite.suite] ---8<---

PRODUCTS variable contains references to product configuration files that
should be available in the QA Tools share dir (/usr/local/share/qa-tools by
default). Note that I have exported RESULTDIR so that email-app*.product files
can use it instead of defining their own RESULTDIR.

REPORTS defines commands that are used to report success or failure of the
invoked operation. You can see that default-email-reporting.conf uses
/usr/bin/mail to send an email report with some details about the
build/package process.

PREFIX (if defined) will be used to configure and install each of the
PRODUCTS, PREFIX is usually needed in order to satisfy build dependencies of
successively built products.

Generating a daily-build of your product suite
==============================================

Assuming you have the suite configuration file and configuration files for all
the products from such suite, you can use the following commands to generate
source code snapshot and binary packages:

(... on a fc4 system ...)
$ qat-suite-snapshot /usr/local/share/qa-tools/dummy-product.suite
$ DISTROS=fc4 qat-suite-snapshot-rpm \
        /usr/local/share/qa-tools/dummy-product.suite

(... on a suse10 system ...)
$ qat-suite-snapshot /usr/local/share/qa-tools/dummy-product.suite
$ DISTROS=suse10 qat-suite-snapshot-rpm \
        /usr/local/share/qa-tools/dummy-product.suite

(... on a debian system ...)
$ qat-suite-snapshot /usr/local/share/qa-tools/dummy-product.suite
$ DISTROS=debian qat-suite-snapshot-deb \
        /usr/local/share/qa-tools/dummy-product.suite

The commands above will generate:

    1) source code .tar.gz snapshots (with version containing the today's
       date) in the RESULTDIR/sources directory
    2) binary packages (again with version information containing the today's
       date) in the RESULTDIR/fc4, RESULTDIR/suse10, RESULTDIR/debian
       directories

[Note that qat-suite-snapshot-{deb,rpm} requires .tar.gz files that are
generated by qat-suite-snapshot command. The order matters.]

Generating daily builds of your product(s)
==========================================

Note that qat-suite-* commands are only helper scripts to make the invocation
of qat-product-* scripts easier. You can also generate daily build of specific
product using the following commands, but please ensure that the build
dependencies are satisifed:

(... on a fc4 system ...)
    $ qat-product-snapshot /usr/local/share/qa-tools/dummy-product.product
    $ DISTROS=fc4 qat-product-snapshot-rpm \
            /usr/local/share/qa-tools/dummy-product.product

(... on a suse10 system ...)
    $ qat-product-snapshot /usr/local/share/qa-tools/dummy-product.product
    $ DISTROS=suse10 qat-product-snapshot-rpm \
            /usr/local/share/qa-tools/dummy-product.product

(... on a debian system ...)
    $ qat-product-snapshot /usr/local/share/qa-tools/dummy-product.product
    $ DISTROS=debian qat-product-snapshot-deb \
            /usr/local/share/qa-tools/dummy-product.product

[Note that qat-product-snapshot-{deb,rpm} requires .tar.gz files that are
generated by qat-product-snapshot command. The order matters.]

Generating regular release of your product suite
=================================================

At the moment your products are ready to release, which means, that snapshots
of your software compile, install, and run fine, you can use qat-product-svn-*
commands to set up necessary svn copies in the repository.

This usually means 'svn copying' trunk or branch to a specific subdirectory
within tags/ directory. You can use for example the following command:

    $ qat-product-release myproduct.product -t

This will extract the <version> from configure.in of your product and will do
an `svn copy trunk tags/<version>' to remember the release forever.

One of the things that will help us is to have a `current' release of each
product. The current release should be done using `svn copy' for each of your
products:

(... somewhere in the SVN space ...)
    $ cd myproduct/tags
    $ svn delete current && svn ci # if `current' already exists
    $ svn copy 1.0 current && svn ci -m "update current"

Assuming you have done the steps above for each and every product from you
suite, you can release all .tar.gz files using

    $ qat-suite-release /usr/local/share/qa-tools/mysuite.suite

It will take all `current' releases and will generate all .tar.gz archives
suitable to be published on the web.

To get you started
==================

Install qa tools and try to invoke the following commands (on a debian system)

    $ qat-suite-snapshot /usr/local/share/qa-tools/dummy-product.suite
    $ qat-suite-snapshot-deb /usr/local/share/qa-tools/dummy-product.suite

Now look into /tmp/dummy-product to see what is there, from now on, read the
scripts and try to understand them...

TODO: more to be documented, probably in the tutorial way

vim: set et tw=78: