# EnvGen - Environment-Specific File Generator

Overview | User's Manual | Change History

## Overview

EnvGen is an Ant task for generating different versions of the same file parameterized for different environments (i.e. development, test, and production). File generation is done using FreeMarker, a template engine with a full-featured templating language.

You specify environment-specific properties in a CSV file (comma separated value spreadsheet). EnvGen functions similar to Ant's Copy task: you run EnvGen against one or more source filesets into a target directory. Rather than simply copying each file over, EnvGen generates a version of the target file for each environment specified in the properties CSV file. Usually the target directory or filename is parameterized by environment so that EnvGen writes each version to a different physical file.

### Why Use EnvGen?

EnvGen is most useful when the technology being used does not allow environment differences to be parameterized. Frequent examples of this that I have come across are database tablespace or other storage settings in DDL, and mainframe code where namespaces are typically used to provide separate environments. While EnvGen is a surprisingly powerful tool, I recommend only using it when you absolutely need to. Generation adds an extra level of abstaction and complexity that is best avoided if possible. Most code can be parameterized for environmental differences without resorting to generation.

EnvGen is essentially an Ant task that wraps the FreeMarker template engine. While FreeMarker is a great template engine, I found it difficult to use directly because it does not provide any useful Ant tasks. The FreeMarker community has addressed this by creating FMPP, a wrapper for FreeMarker that makes it easier to use and includes an Ant task. The main different between FMPP and EnvGen is that FMPP is a general-purpose text file preprocessor tool with a broad set of features, while EnvGen is focused entirely on environment-specific file generation and makes this single task much easier to do than FMPP.

### Requirements

EnvGen requires Java Runtime Environment (JRE) version 5+ and Ant 1.7+.

### Legal

EnvGen is copyright © 2007-2011 by Basil Vandegriend. The software is licensed for use under the terms of the GNU General Public License (GPL).

The software is provided for use as-is, and is not warranted to be defect-free or to meet your requirements.

### Download

The latest version of EnvGen can be downloaded from www.basilv.com/psd/software.

The source code is available from GitHub at basilv/EnvGen.

### Feedback

Feedback concerning this software is appreciated, and can be provided on the Software Feedback page. I would appreciate hearing if you find the software useful and make use of it regularly.

## User's Manual

### Overview

The basic steps to use EnvGen are as follows:

1. Install Ant.
2. Install EnvGen.
3. Define the EnvGen task within the Ant build.
4. Create the environment properties file.
5. Create the template file(s).
6. Invoke the EnvGen task within the Ant build.

The `example` directory included with EnvGen can be used as a starting point for using EnvGen and is referenced within this manual.

The last section of this manual covers more advanced topics regarding the use of EnvGen.

### Installing EnvGen

To install EnvGen simply unzip the `EnvGen-1.X.zip` file into an appropriate directory. The contents of the zip file will be contained within a root directory `EnvGen-1.X`.

The contents of an EnvGen release (zip file) includes the following files and directories:

- `EnvGen-with-dependencies.jar` - Jar file containing EnvGen class files plus all dependent libraries.
- `dependencies-list.txt` - Text file listing the dependent libraries and their versions used to develop and test EnvGen. These libraries are included in the `EnvGen-with-dependencies.jar` file.
- `EnvGen.jar` - Jar file containing only EnvGen class files. Use this jar when you want to use a different version of one of the dependent libraries.
- `/example` - This directory contains a simple example of how to use EnvGen which is referenced in this manual.
- `/src` - Source code for EnvGen.
- API Documentation - JavaDoc API documentation for EnvGen.
- `/build/build.xml` - Ant build file for EnvGen.

### Defining the EnvGen task

To use the EnvGen Ant task as part of an Ant build, it must first be defined within that build file. The easiest way to do this is to use `<taskdef>` and specify the `EnvGen-with-dependencies.jar` jar file for the classpath as per the following excerpt taken from the example `build.xml`:

    <taskdef name="envgen"
     classname="com.basilv.envgen.EnvGenTask"
     classpath="${envgen.dir}/EnvGen-with-dependencies.jar"
    />

If you want to use a different version of one of the dependent libraries such as FreeMarker, then you can specify a classpath that includes `EnvGen.jar` plus the desired versions of all the required libraries.

### Creating the Environment Properties File

Environment-specific settings are specified in an environment properties file which is a CSV-formatted spreadsheet. Each row corresponds to a single property or a comment. For a property, the first column contains the name of the property. Each subsequent column contains the value of the property for a single environment. A row whose first column is empty or contains text starting with '#' or '//' is considered to be a comment which causes that line to be ignored.

The example environment properties file `envProperties.csv` is listed below (without the comments). It defines three properties named "env", "server", and "database" for three environments.

env | devl | test | prod
server | intel-010-d | intel-021-t | intel-025-p
database | app-d1 | app-t1 | app-p1

### Creating the Template File(s)

You must create one or more template files to supply to EnvGen to use as the basis for generating the environment-specific files based on the properties you defined in the environment properties file. The template files can be any type of text file. Since EnvGen uses FreeMarker as the template engine, you need to use the FreeMarker Template Language within each template file to specify property substitutions or to use more advanced logic.

Most commonly you will want to substitute properties defined in the environment properties file into the template file. This is done using the syntax `${<propertyname>}`. For example, given the "env" property defined in the example environment property file above, a line in the template file consisting of `"Environment = ${env}"` will be transformed into the output `"Environment = devl"` for the first environment.

See the example template file `environment-summary.txt.ftl`. The contents of this file are reproduced below:

    Summary for Environment "${env}"
    ------------------------------

    Build number ${buildNumber}

    <#if env="devl">
    This is development!
    </#if>

    Server: ${server}
    Database: ${database}

### Invoking the EnvGen Task

EnvGen is invoked from within an Ant build like any other Ant task. This section describes how to invoke EnvGen as an Ant task.

#### Parameters

The attributes supported by the EnvGen task are:

Attribute | Description | Required?
--- | --- | ---
envPropertiesFile | The CSV file containing the environment-specific properties. | Yes
destDir | The destination directory to write the generated files to. This directory can be parameterized in which case the actual directory path will be generated for each environment. | Yes
stripFileExtension | Strip the right-most file extension (from the last period right) from the generated filename. For example "test.txt.ftl" would be changed to "test.txt". | No; defaults to false
overwrite | Overwrite existing files even if the destination file is up to date. even if the destination files are newer than both the source file and the environment properties file. | No; defaults to false
diffToUpdate | By default (`overwrite = "false"` and `diffToUpdate = "false"`), each destination file is only generated if it is older than either the source file or the environment properties file. When diffToUpdate is set to true, then the generation is done and the destination file is updated only if the generated content is different from the existing content. This is useful when the FreeMarker `<#include>` directive is used since a change to an included file will not trigger a regeneration if the source file was not changed. The content comparison ignores differences in line endings, in case the `<fixcrlf>` task is executed against EnvGen's output. | No; defaults to false

#### Parameter Specified as Nested Elements

The nested elements supported by the EnvGen task are:

##### Source

Each `<source>` is an implicit fileset used to select groups of files to generate.

##### Transform

Transforms are a feature of FreeMarker that allow for the programmatic transformation of a block within a template file. A transform is implemented as a Java class and must be registered with the FreeMarker engine via this element. The `<transform>` element has the following attributes:

Attribute | Description | Required?
--- | --- | ---
name | The name of the variable storing the transform, used within the template file to invoke it. The name must correspond to a java identifier: in particular, it cannot contain periods or dashes. | Yes
class | The name of the transform class. This class must be accessible on the classpath. | Yes

EnvGen comes with the following built-in templates:

Template Class | Description
--- | ---
`com.basilv.envgen.MainframeFileFormatTransform` | Transforms input into a format acceptable for mainframes. Tabs are converted to 4 spaces. Whitespace is trimmed from the end of lines. The transform will fail with an error message if any line is longer than 75 characters.
`com.basilv.envgen.SkipGenerationTransform` | Skips the generation of the target file, deleting it if it previously existed. This is useful when you are generating multiple files for multiple environments and certain files are not needed in certain environments. This transform is only useful within conditional logic like FreeMarker's `<#if>` directive in order to allow the file to still be generated for certain environments.

##### SharedVariable

Shared variables are a feature of FreeMarker that allow for a variable to be defined once up front and be available for use within all template files. The `<sharedVariable>` element has the following attributes:

Attribute | Description | Required?
--- | --- | ---
name | The name of the variable, used within the template file to access it. The name must correspond to a java identifier: in particular, it cannot contain periods or dashes. | Yes
value | The value of the variable. | Yes

#### Example

The sample invocation below is taken from the example `build.xml` provided as part of the release.

    <envgen destdir="${dist.dir}/$${env}"
    	envPropertiesFile="envProperties.csv"
    	overwrite="false"
    	stripFileExtension="true"
    >
    	<source dir="${source.dir}"/>
    	<transform name="mainframeFileFormat" class="com.basilv.envgen.MainframeFileFormatTransform"/>
    	<sharedVariable name="buildNumber" value="100"/>
    </envgen>

This generates files from the `${source.dir}` directory for each environment specified in the `envProperties.csv` file and puts them in the target directory based on the environment property `env`. If the ant property `${dist.dir}` is set to `/dist` and the environment properties file specifies one value of `env` as `devl`, then the target directory for this environment would be `/dist/devl`.

### Advanced Topics

This section covers more advanced topics when using EnvGen.

#### Using FreeMarker's <#include> Directive

When using FreeMarker's `<#include>` directive, the path you specify to the file must be relative to the base directory of the fileset the file is in. For an example, consider the following directory structure:

    /src
      common.txt
      /resources
        common-resources.txt
        properties.txt.ftl

EnvGen is run against the root `/src` directory. The `properties.txt.ftl` file can include `common.txt` and `common-resources.txt` via the following include directives:

    <#include "common.txt">
    <#include "resources/common-resources.txt">

One limitation of EnvGen when using `<#include>` is that EnvGen will not rebuild target files when only included files are modified. For the above example, if only `common.txt` is changed, then EnvGen will not regenerate `properties.txt`. There are several workarounds for this limitation. The easiest workaround is to set the attribute `diffToUpdate = "true"`, which will not rely on timestamps to determine whether to regenerate, but instead do the generation and then only update the target file if the generated content has changed. Other workarounds include using Ant's `<dependset>` task, a touch file, or to set `override = "true"`.

## Change History

Minor releases have version numbers of the form A.B.C and only contain defect fixes or documentation improvements. Major releases have version numbers of the form A.B and contain new features or feature enhancements.

### Version 1.4 - December, 2011

- Upgrade to Java 5 compatiblity. This version of EnvGen will no longer work with Java 1.4.
- Migrate source code repository to GitHub at basilv/EnvGen. All known defects are now listed there.
- Upgraded to FreeMarker 2.3.18.
- Fixed defect: Allow comments without commas in first row of csv file.

### Version 1.3.1 - January, 2009

- Upgraded to FreeMarker 2.3.15.
- Known Defects: A defect has been reported against EnvGen 1.3 using FreeMarker's include directive on Unix systems in a template file located in a subdirectory of the root directory. FreeMarker cannot resolve the include and will display an error message containing the following "Caused by: freemarker.template.TemplateException: Error reading included file ...".

### Version 1.3 - August, 2007

- Added the `com.basilv.envgen.SkipGenerationTemplate` template, which allows you to avoid generating a file for one or more environments.

### Version 1.2.1 - June, 2007

- Improved error messages reported by Ant by including FreeMarker error text.

### Version 1.2 - May, 2007

- Upgraded to FreeMarker 2.3.10
- Fixed a defect: the use of FreeMarker's `<#include>` directive did not work when including files in other directories. The <#include> directive now can be used to include any file within the same fileset. The file path you specify must be relative to the fileset base directory.
- Added the `diffToUpdate` attribute to make it easier to use FreeMarker's `<#include>` directive.

### Version 1.1 - April, 2007

- Added support for comments and blank lines within the environment properties file
- Fixed compilation to allow EnvGen release to run under Java 1.4
- Minor documentation fixes

### Version 1.0 - February, 2007

- Initial public release.