Before getting into the specifics of Ivy, the first question to answer is why use Ivy at all? Why is managing build dependencies important? All the significant Java software I have worked on makes use of third-party libraries such as Hibernate for persisting objects to a relational database or JUnit for unit testing. Both the IDE projects and automated builds for these pieces of software have the common need to deal with these dependencies, most frequently by including them on the classpath for either compilation or at runtime.

These dependencies can be handled manually in an ad-hoc manner. This is what I have done in the past and what I have frequently observed to be done by other teams. But there are significant advantages to using a tool such as Ivy to manage these dependencies:

• Explicit tracking of the versions of the third-party software being used. Too often I have come across projects with a library directory holding a collection of third-party jar files without any information as to the versions being used. Ivy supports defining dependencies explicitly by version number. If dependencies are defined more loosely, such as via a version range (e.g. version 1.0+), then Ivy can generate a report listing the specific versions used. Knowing the version of dependencies is important when trying to resolve issues or make use of the latest features.
• Management of transitive dependencies. If your software depends on third party software that itself has dependencies on other software, Ivy can automatically pull in these transitive dependencies. When multiple dependencies themselves depend on different versions of the same third-party software, Ivy can automatically resolve the conflict. An example will help illustrate this. Let us say that your software depends on two other libraries B and C. B in turn depends on D version 1.0 while C depends on D version 1.1. Ivy will pull in B, C, and D v1.1 and leave out D v1.0.
• Standardized approach to storing dependencies. Ivy uses one or more repositories of software as the source for obtaining dependencies. Each piece of software in these repositories is characterized by a standard set of metadata: the key attributes are organization, module, and revision. Using a public repository that the third-party software you require is updated within makes it trivial to upgrade to a new version: just change the version you depend on, and Ivy can then automatically download the new version. You can also use an enterprise repository to publish your own software into in order to facilitate reuse by others.

There are a number of decisions to be made when using Ivy. In the remainder of this article I provide my experience and recommendations regarding these decisions.

Building Classpaths

There are two main options for building classpaths within your Ant build. The first is to use the Ivy CachePath Ant task to build an Ant path for your project's dependencies that references your local Ivy cache of dependencies. This nicely avoids any duplication of dependency information within your Ant build file. Unfortunately, the same cannot be done for your IDE classpath. Maven's solution is to generate the necessary IDE project files (e.g. Eclipse's .classpath file). The tooling support for Ivy to generate IDE files appears weak (I did not try it myself). Another limitation of this approach is that you will not have the libraries required for your project checked into version control. (You could, however, check an organizational repository into version control separately.)

The second option is to use the Ivy Retrieve Ant task to copy dependencies into your project workspace. You can specify the directory structure and file naming convention to use. I recommend copying the jars into a single lib directory (or one directory per Ivy configuration) and do not include revision information in the file names. This allows the classpath to be easily built in Ant using a simple fileset – e.g. <fileset dir="lib" includes="*.jar"/>. The IDE classpath must be manually maintained but at least keeping files revision-agnostic avoids needing to update the IDE when a dependency is upgraded. Because this lib directory is located within the project you can manage it in version control along with the rest of your project. One drawback of doing so, however, is that this lib directory full of jars now resembles a manual solution to maintaining dependencies so there is a risk that developers unfamiliar with the build or Ivy will directly add libraries to this lib path. I recommend using the retrieve task's option sync="true" to remove anything not specified as a current dependency.

I prefer the second option, partly because I am a fan of putting all project-related artifacts including third-party jars into version control, and partly because it makes it easier to manually manage the classpath within the IDE.

Using Repositories

Ivy requires at least one repository to be defined to obtain dependencies from and provides a number of options for doing so. One option is to use one or more public repositories: third-party repositories available over the Internet. Ivy provides built-in support for two: the public Maven repository, which is probably the largest and best known repository, and IvyRep, which only provides Ivy metadata for dependencies defined elsewhere (typically in the Maven repository). I recommend against using IvyRep: it appears it is no longer updated and I encountered errors resolving dependencies from it. The Maven repository seems like the best place to obtain third-party open source dependencies. You can use the site mvnrepository.com to search for the metadata for the dependencies you are looking for.

The Maven repository does have its share of problems. Dependency metadata does not always match the directory structure or is plain wrong. For large projects like Hibernate or Spring with many dependencies, I was unable to resolve the complete set of their dependencies out of the Maven repository. I have also heard about and observed stability issues: sometimes downloads of dependencies fail.

Since I believe that builds must be reliable and repeatable I feel that relying solely on a public repository is a bad idea. The alternative is an organizational repository, which could be defined for the team or project or spanning the entire enterprise. In either case, Ivy supports a number of options for defining home-built repositories with my favorites being the local file system (typically on a shared network drive) or SFTP/SSH to access a remote file system. Ivy also supports defining a chain of repositories, so you could define an organizational repository first and then a public repository second.

The approach I prefer, however, is to only use an organizational repository for builds and use public repositories solely to install dependencies into the organizational repository. The Ivy Install Ant task can be used to do this. It supports automatically installing transitive dependencies, which is very convenient but comes at a price. By default the install task fails if the dependency you are trying to install already exists in the target repository. This seems like reasonable behavior: you do not want to modify dependencies once they are installed. When combined with transitive dependencies, however, this is no longer useful: it is very likely that modules will have one or more common transitive dependencies, so when trying to install the second module, it will fail because it tries to transitively install a dependency that has already been installed. The workaround is to use the overwrite="true" option. I would prefer instead to have an option on the install task to ignore dependencies that are already installed.

When specifying a repository you must define the pattern used for the directory structure and file naming convention. My preferred pattern is: ${ivy-repository.dir}/[organisation]/[module]/[revision]/[artifact](-[classifier])-[revision].[ext]. Making the revision into a sub-directory allows Ivy to use atomic publishing. Specifying the optional classifier was required for certain dependencies that define, for example, source code and javadoc artifacts as well as the compiled code – without the classifier specified, all three artifacts resolve incorrectly to the same file (which Ivy fortunately identifies as an error).

Defining Dependencies

When defining dependencies for a module I believe that the general goal should be to use the minimum set of dependencies that are required. To do this effectively requires that you use configurations, which are basically defined sets of dependencies for a given module. Typical configurations include compile, runtime, and test. If you do not define the configuration(s) you want from a third-party module then Ivy defaults to pulling in all dependencies. For a third-party library like Spring which defines a large number of optional dependencies (in a configuration appropriately called "optional") this will result in pulling in a large number of jars that you simply do not require to compile or even run your code.

Should you explicitly define all your dependencies or automatically pull in transitive dependencies? One of the whole points of using a dependency management system like Ivy is to handle transitive dependencies, so I am a fan of using them. If you want to be aware of all the dependencies that are pulled in, you can produce a report using the Ivy Report Ant task that lists all the resolved dependencies for a module.

Conclusion

If you use Ant for your builds then I highly recommend using Ivy to manage dependencies. For an example Ant+Ivy build check out the Java Examples project available for download from the Software page.

By default Ant writes informational messages to the console. While this is usually sufficient for a successful build, more detail is often useful in the case of failed builds. Relying on the console output for information about the build has limitations – there might be too much output to fix in the console's buffer, or you might clear or close the console. The solution is to write Ant's output to a log file. This can be done using the <record> task, which allows you to specify a log level of "verbose" or "debug" to provide more details than the standard "info" log level. This task should be the first one performed by the build for any target to ensure that a log is always produced. To accomplish this, I put the <record> task at the start of a target named "init" which I ensure is the first dependency of every other target. An example is given below:

<target name="init">
  <record name="build.log" loglevel="verbose" append="false"/>
</target>

Normally I prefer to use the "verbose" level for the log file, as the "debug" level provides so much additional detail that it is only really useful for debugging bad behavior within Ant code. Notice that I specified the option append="false" which ensures that at the start of the build any existing log file is deleted and a new one created. This makes it easier to see what happens for an individual execution of the build and avoids having the log become too large. This can be a problem, however, if you need to refer back to the log from a prior build to resolve problems or for auditing purposes. I have had situations where after a build fails I look at the build log for the problem, run some other ant target (such as clean) before I have finished resolving the problem, then realize that I have wiped the record of the problem.

One solution to this issue is to create a separate log file for each execution of the build. This can be done by using a build timestamp as part of the log file name, as shown in the below example:

<target name="init">
  <tstamp>
    <format property="timestamp" pattern="yyyy-MM-dd_HH-mm-ss"/>
  </tstamp>
  <property name="build.log.dir" location="${basedir}/buildlogs"/>
  <mkdir dir="${build.log.dir}"/>
  <property name="build.log.filename" value="build_${timestamp}.log"/>
  <record name="${build.log.dir}/${build.log.filename}"
    loglevel="verbose" append="false"/>
  <echo message="Build logged to ${build.log.filename}"/>
</target>

The log files are written to a buildlogs subdirectory to avoid cluttering the main directory. For some reason the <record> task requires an absolute path when specifying a directory, so the build.log.dir property is prefixed with ${basedir}. After setting up the logging, a message is written to the console displaying the log filename used to make it easier to look up the log file if a problem occurs. This approach will cause log files to accumulate within the log directory. If this becomes a problem, you could add some Ant logic to delete log files older than a certain date. I will leave the implementation of this as an exercise to the reader. Hint: use a <delete> task with a <fileset> that contains a <date> selector.

Ant ships with two relevant tasks: <scp> and <sshexec>. Both of these tasks require the third-party library Java Secure Channel (Jsch) to provide a Java implementation of the SSH protocol. The jsch.jar file should be placed in the /lib directory of your Ant installation. While these tasks support both the SSH 1 and SSH 2 protocols, SSH 2 is preferred because it is much more secure. Use of SSH 2, however, requires Ant version 1.7.0 or greater.

Using the <scp> task appears at first glance to be quite similar to the <ftp> task. Here is an example:

<scp remoteTodir="${user}@${server}:${target.dir}"
  password="${password}"
  trust="yes"
  sftp="true">
    <fileset dir="${source.dir}">
      <include name="**/*"/>
    </fileset>
</scp>

This transfers files from ${source.dir} on your machine running the Ant build to the directory ${target.dir} on the machine ${server}. Despite the similarity to the <ftp> task, there are several significant differences:

• SCP transfers files as binary copies only and has no support for the ASCII transfer mode of FTP. This means that transferring text files between operating systems (Windows, Unix, Mac) can cause problems as the line ending characters vary across the three operating systems. The solution is to use the <fixcrlf> task to convert the line endings of text files before or after they are transferred.
• SCP transfers all the files in the fileset every time, whereas the FTP task supports transferring only newer files.
• SCP provides no mechanism for setting the file access permissions of newly created or existing files. On UNIX operating systems, SCP grants read & write access to the file owner and no permissions for the group or the world (other). These restrictive permissions are seldom appropriate – executable files should have the execute flag set and often the group needs read & execute permission. The solution I use to fix the permissions is to run a script on the target server using the <sshexec> task after the <scp> task to fix the permissions on the files & directories. This script is maintained as part of the code base and transferred to the target server via SCP along with the rest of the code. Below is an example of using <sshexec> to do this.

<sshexec host="${server}"
  username="${user}"
  password="${password}"
  trust="yes"
  command="perl -W ${target.dir}/bin/post_deploy_configuration.pl"
/>

One capability that the <scp> and <sshexec> tasks have that the <ftp> task lacks is the ability to use public key authentication instead of passwords. One significant advantages of using keys instead of passwords for an automated build is that keys do not expire, whereas passwords are often required to expire regularly. Keys are also much more secure in situations when the server or the network between the build machine and the server is at risk of being compromised.

The downside to using public key authentication is that some up-front work is required to set up and install the keys. The necessary steps are:

• Create a new key pair, either using the SSH software on your server or using an utility such as PuTTYgen. You should end up with a file holding the public key, a file holding the private key, and a passphrase for accessing the private key.
• Ensure the private key is stored in the openssh format, which is what the Jsch library used by the <scp> and <sshexec> tasks requires. PuTTYgen, for example, by default stores private keys in PuTTY's format but can convert a key to openssh format.
• Install the public key on the target server under the user id being used by the build. This involves copying the public key file into a specially-named subdirectory of the user's home directory and adding an entry listing the public key file to a special file within this subdirectory. The name of the subdirectory and special file depends on the specific operating system and specific version of SSH being used.
• Make the private key available to the build machine.
• Change your build.xml to include the following attributes for the <scp> and <sshexec> tasks instead of the password attribute: keyfile="${key.file}" passphrase="${passphrase}". Just like with passwords, do not embed the passphrase directly within build.xml. I store it in a property file which along with the private key is stored in a secured non-public location available to the build machine.

The <sshexec> task is limited to executing a single command on the server. The SSHTools project provides a more powerful <ssh> Ant task that supports automating an interactive remote session. Below is an example:

<ssh host="${server}"
  username="${user}"
  password="${password}"
  verifyhost="false"
>
  <shell term="vt100">
    <write>whoami</write>
    <read timeout="50000">${user}</read>
    <write>chmod -R u+rwx,g+rx ${target.dir}</write>
    <write>echo Done</write>
    <read timeout="50000">Done</read>
  </shell term="vt100">
</ssh>

In my previous article on Designing for Deployability, I wrote about generating files per environment as one approach to handling environmental differences. EnvGen is designed to allow you to easily use this approach.

You can read more about EnvGen in the EnvGen Release Documentation.

ANT provides a FTP task. It requires the installation of two external libraries into ANT's \lib directory: Jakarta ORO and Jakarta Commons Net. The FTP task connects to a remote server via the FTP protocol and provides all the basic FTP operations: send files, get files, delete files, create directories, delete directories, and change permissions. For the full details on how to use the FTP task, see the ANT manual's entry on the FTP task.

For my website, I use the FTP task to transfer my server logs to my local workstation for analysis. Here is an example (using ANT version 1.6.5):

<ftp action="recv"
  server="ftp.server"
  remotedir="/logs"
  userid="my-userid"
  password="my-password"
  verbose="yes"
>
  <fileset dir="${log.dir}">
      <include name="*.log"/>
  </fileset>
</ftp>

This task transfers the files named *.log from the directory /logs of the server ftp.server to the ${log.dir} directory on your local machine.

At work, I usually use the FTP task for deploying application changes - sending a set of files built on my local workstation or development server to the test or production server. Here is an example:

<ftp action="send"
  server="ftp.server"
  userid="my-userid"
  password="my-password"
  remotedir="/incoming"
  depends="yes"
  binary="no"
  chmod="755"
>
  <fileset dir="${ftp.source.dir}">
      <include name="**/*"/>
  </fileset>
</ftp>

This task transfers all the files within the local directory ${ftp.source.dir} to the directory /incoming of the server ftp.server. Files in sub-directories are also transferred into corresponding sub-directories on the remote server, and the sub-directories are created if they do not already exist. Only new or changed files are actually transferred (depends="yes"). The files are transferred in ASCII mode (binary="no"). Assuming the remote server runs UNIX, the permissions of the files are set to 755 (chmod="755").

Using the FTP task to deploy a set of files has a number of limitations, especially if these files are organized into a hierarchy of directories. While the FTP task will automatically create sub-directories as required, I have not found a way to ensure that the required directory permissions are assigned for new directories (this assumes the remote server runs UNIX). In particular, the chmod attribute is only assigned to new files, not new directories. The only workaround I have found is to execute a script on the server that assigns the necessary directory permissions. Another limitation is that the FTP task will not create empty directories. If you want to create a particular directory structure on the remote server, then each directory you want created must have at least a single file in it. The workaround is to create a dummy file to ensure the directory is created.

Deleting obsolete files and directories on the remote server is also difficult for a couple of reasons. The FTP task does provide all the basic FTP operations, including deleting files and directories. But each is handled as a separate action which requires a separate invocation of the FTP task, instead of performing all the operations within a single FTP session. This is a minor inconvenience. The more significant limitation is in determining how to delete obsolete files and directories. The development build usually assembles a set of files to transfer to the server without caring about the individual files or directories - it just assembles a directory structure. Therefore, if certain files or directories are deleted or renamed, this just results in a new set of files without the build process knowing which are to be deleted. Therefore, automating the deletion of these obsolete files by listing each one is quite difficult. An easier approach would be to simply delete all the files and directories deployed to the server and then re-deploy the new set. This solution also has its problems. First, this requires recreating all the sub-directories, which as I explained above leads to the problem of ensuring their permissions are properly configured. Second, the directory structure into which files are deployed may include files created by server processes which cannot or should not be deleted, such as server logs or application data files. The workaround I use to deal with obsolete files and directories is to simple leave them in place - they typically have no impact on the operation of the system. But I am on the lookout for a better solution to this problem - if you know of any, please let me know via a comment.

Despite the limitations, I have been pleased with my experiences using ANT to automate FTP tasks. I encourage you to give it a try.

Update June, 2009: I have had questions about what version of the ORO and Commons Net libraries to use, as not all versions are compatible. I have had success with Ant 1.7, Commons Net 1.4.0, and Jakarta ORO 2.0.8.