Ivy is a tool for managing build dependencies within the Ant build tool. In this article I share my experiences using Ivy and provide recommendations on what to do and not do. This article is not a tutorial on using Ivy: for that see the Ivy documentation.
Before getting into the specifics of Ivy, the first question to answer is why use Ivy at all? Why is managing build dependencies important? All the significant Java software I have worked on makes use of third-party libraries such as Hibernate for persisting objects to a relational database or JUnit for unit testing. Both the IDE projects and automated builds for these pieces of software have the common need to deal with these dependencies, most frequently by including them on the classpath for either compilation or at runtime.
These dependencies can be handled manually in an ad-hoc manner. This is what I have done in the past and what I have frequently observed to be done by other teams. But there are significant advantages to using a tool such as Ivy to manage these dependencies:
- Explicit tracking of the versions of the third-party software being used. Too often I have come across projects with a library directory holding a collection of third-party jar files without any information as to the versions being used. Ivy supports defining dependencies explicitly by version number. If dependencies are defined more loosely, such as via a version range (e.g. version 1.0+), then Ivy can generate a report listing the specific versions used. Knowing the version of dependencies is important when trying to resolve issues or make use of the latest features.
- Management of transitive dependencies. If your software depends on third party software that itself has dependencies on other software, Ivy can automatically pull in these transitive dependencies. When multiple dependencies themselves depend on different versions of the same third-party software, Ivy can automatically resolve the conflict. An example will help illustrate this. Let us say that your software depends on two other libraries B and C. B in turn depends on D version 1.0 while C depends on D version 1.1. Ivy will pull in B, C, and D v1.1 and leave out D v1.0.
- Standardized approach to storing dependencies. Ivy uses one or more repositories of software as the source for obtaining dependencies. Each piece of software in these repositories is characterized by a standard set of metadata: the key attributes are organization, module, and revision. Using a public repository that the third-party software you require is updated within makes it trivial to upgrade to a new version: just change the version you depend on, and Ivy can then automatically download the new version. You can also use an enterprise repository to publish your own software into in order to facilitate reuse by others.
There are a number of decisions to be made when using Ivy. In the remainder of this article I provide my experience and recommendations regarding these decisions.
Building Classpaths
There are two main options for building classpaths within your Ant build. The first is to use the Ivy CachePath Ant task to build an Ant path for your project's dependencies that references your local Ivy cache of dependencies. This nicely avoids any duplication of dependency information within your Ant build file. Unfortunately, the same cannot be done for your IDE classpath. Maven's solution is to generate the necessary IDE project files (e.g. Eclipse's .classpath file). The tooling support for Ivy to generate IDE files appears weak (I did not try it myself). Another limitation of this approach is that you will not have the libraries required for your project checked into version control. (You could, however, check an organizational repository into version control separately.)
The second option is to use the Ivy Retrieve Ant task to copy dependencies into your project workspace. You can specify the directory structure and file naming convention to use. I recommend copying the jars into a single lib directory (or one directory per Ivy configuration) and do not include revision information in the file names. This allows the classpath to be easily built in Ant using a simple fileset – e.g. <fileset dir="lib" includes="*.jar"/>. The IDE classpath must be manually maintained but at least keeping files revision-agnostic avoids needing to update the IDE when a dependency is upgraded. Because this lib directory is located within the project you can manage it in version control along with the rest of your project. One drawback of doing so, however, is that this lib directory full of jars now resembles a manual solution to maintaining dependencies so there is a risk that developers unfamiliar with the build or Ivy will directly add libraries to this lib path. I recommend using the retrieve task's option sync="true" to remove anything not specified as a current dependency.
I prefer the second option, partly because I am a fan of putting all project-related artifacts including third-party jars into version control, and partly because it makes it easier to manually manage the classpath within the IDE.
Using Repositories
Ivy requires at least one repository to be defined to obtain dependencies from and provides a number of options for doing so. One option is to use one or more public repositories: third-party repositories available over the Internet. Ivy provides built-in support for two: the public Maven repository, which is probably the largest and best known repository, and IvyRep, which only provides Ivy metadata for dependencies defined elsewhere (typically in the Maven repository). I recommend against using IvyRep: it appears it is no longer updated and I encountered errors resolving dependencies from it. The Maven repository seems like the best place to obtain third-party open source dependencies. You can use the site mvnrepository.com to search for the metadata for the dependencies you are looking for.
The Maven repository does have its share of problems. Dependency metadata does not always match the directory structure or is plain wrong. For large projects like Hibernate or Spring with many dependencies, I was unable to resolve the complete set of their dependencies out of the Maven repository. I have also heard about and observed stability issues: sometimes downloads of dependencies fail.
Since I believe that builds must be reliable and repeatable I feel that relying solely on a public repository is a bad idea. The alternative is an organizational repository, which could be defined for the team or project or spanning the entire enterprise. In either case, Ivy supports a number of options for defining home-built repositories with my favorites being the local file system (typically on a shared network drive) or SFTP/SSH to access a remote file system. Ivy also supports defining a chain of repositories, so you could define an organizational repository first and then a public repository second.
The approach I prefer, however, is to only use an organizational repository for builds and use public repositories solely to install dependencies into the organizational repository. The Ivy Install Ant task can be used to do this. It supports automatically installing transitive dependencies, which is very convenient but comes at a price. By default the install task fails if the dependency you are trying to install already exists in the target repository. This seems like reasonable behavior: you do not want to modify dependencies once they are installed. When combined with transitive dependencies, however, this is no longer useful: it is very likely that modules will have one or more common transitive dependencies, so when trying to install the second module, it will fail because it tries to transitively install a dependency that has already been installed. The workaround is to use the overwrite="true" option. I would prefer instead to have an option on the install task to ignore dependencies that are already installed.
When specifying a repository you must define the pattern used for the directory structure and file naming convention. My preferred pattern is: ${ivy-repository.dir}/[organisation]/[module]/[revision]/[artifact](-[classifier])-[revision].[ext]. Making the revision into a sub-directory allows Ivy to use atomic publishing. Specifying the optional classifier was required for certain dependencies that define, for example, source code and javadoc artifacts as well as the compiled code – without the classifier specified, all three artifacts resolve incorrectly to the same file (which Ivy fortunately identifies as an error).
Defining Dependencies
When defining dependencies for a module I believe that the general goal should be to use the minimum set of dependencies that are required. To do this effectively requires that you use configurations, which are basically defined sets of dependencies for a given module. Typical configurations include compile, runtime, and test. If you do not define the configuration(s) you want from a third-party module then Ivy defaults to pulling in all dependencies. For a third-party library like Spring which defines a large number of optional dependencies (in a configuration appropriately called "optional") this will result in pulling in a large number of jars that you simply do not require to compile or even run your code.
Should you explicitly define all your dependencies or automatically pull in transitive dependencies? One of the whole points of using a dependency management system like Ivy is to handle transitive dependencies, so I am a fan of using them. If you want to be aware of all the dependencies that are pulled in, you can produce a report using the Ivy Report Ant task that lists all the resolved dependencies for a module.
Conclusion
If you use Ant for your builds then I highly recommend using Ivy to manage dependencies. For an example Ant+Ivy build check out the Java Examples project available for download from the Software page.
If you find this article helpful, please make a donation.
