Fetching from Repositories

The infrastructure offered by PDE Build provides steps to fetch the source code and pre-built plug-ins from a repository as part of the build process (see the eclipse.fetch task and the properties to control fetching). Two possibilities are available: let PDE build automatically fetch the source, or fetch the source code manually.

First step to automation, the map file

Given the identifier of a feature or a plug-in that needs to be built, the build infrastructure is able to fetch from a repository all the included features and plug-ins.

In order to know where to get things from, PDE uses map files. A map file is a java property file which maps feature and plug-in ids to a location and a tag in a repository. The format of map files is open and but the key part is fixed by PDE Build:

<elementType>@<elementID>[,<elementVersion>] = <repo specific content>

where elementType is one of bundle, feature, plugin or fragment, and elementId and elementVersion give the specifics of the element.

Map file entry for CVS

The format of a map file entry to fetch content from CVS is the following:

<key> = CVS, [,args]

where args is a comma-separated list of key/value pairs described below:

tag, the CVS tag to check out;
cvsRoot, a CVS connection string;
path, an optional CVS module and path where to get the folder containing the element. By default the element name is used as module name;
prebuilt, an optional boolean indicating if the element downloaded is prebuilt. By default this is false;
cvsPassFile, an option path to a cvs pass file;
password, an optional password to use to connect to CVS;

Example:

plugin@com.ibm.icu,3.6.1=CVS,tag=v200704191630,cvsRoot=:pserver:anonymous@dev.eclipse.org:/cvsroot/tools,path=org.eclipse.orbit/com.ibm.icu/bin,prebuilt=true

Old style Map Entries

In Eclipse 3.2 and earlier, the map file format for CVS was:

<elementType>@<elementID> = CVS, <TAG>, <CVSROOT>[,<PASSWORD>[,<PATH>[,<CVSPASSFILE>]]]

If any of the optional arguments aren't specified, then the commas are still required to delimit any following arguments. This format should be considered deprecated and the key/value format is prefered.

Map file entry for Ant GET

The format of a map file entry to fetch content from any URL supported by Ant GET is the following:

<key> = GET, <url> [,args]

where url is the url to retrieve the data from and args is an optional comma-separated list of key/value pairs described below:

unpack, set to true the indicate that the downloaded element should be unzipped;
verbose, set to true to show verbose progress information when downloading. The default value is false;
ignoreerrors, set to true to log errors during the transport. The default value is false;
usetimestamp, conditionally download a file based on the timestamp of the local copy (for http URLs only) when set to true. The default value is false;
username, the username for "BASIC" http authentication;
password, the password to perform the download.

Example:

plugin@com.ibm.icu,3.4.5=GET,https://download.eclipse.org/tools/orbit/downloads/drops/S200705301823/bundles/com.ibm.icu_3.4.5.jar, unpack=true

Map file entry for a p2 fetch

A new extension has been added to PDE/Build that enables users to fetch artifacts from p2 repositories. You can now add an entry to your map file for build input and PDE/Build will retrieve the artifact and include it in the build. The map file entry needs to include the id and version of the installable unit (IU), and the URI of the repository where the IU can be found. The metadata and artifact repositories are assumed to be co-located.

The format of a map file entry to fetch content via p2 is as follows:

<elementType>@<elementID> = p2IU, id=<id>, version=<version>, repository=<repository>

id, the identifier of the Installable Unit;
version, the version of the Installable (optional). If not specified, the highest version available will be fetched.
repository, the location (URI) of the repository where the IU will be found.

Fetching features with p2

In the Helios release, features can be fetched with p2 by specifying the Installable Unit representing the feature jar. See the feature metadata page for details on the structure of p2 metadata for features.

Examples:

plugin@my.bundle.id,1.0.0=p2IU,id=my.bundle.id,version=1.0.0,repository=http://example.eclipse.org/repo
feature@my.feature.id = p2IU,id=my.feature.id.feature.jar, version=1.0.0,repository=http://example.eclipse.org/repo

Map file entry for other repositories

PDE Build provides an extension point where fetch script generators for different repositories can be plugged in. When your eclipse install contains a bundle that provides a script generator extension for a different type of repository, you can use that repository by specifying it in your map file entries and adhering to the format specified by the extension provider.

Setting up for CVS source fetching

To set up fetching your source from CVS, you should do the following:

Make sure you have a cvs.exe on your system search path. (Download CVS from here).
Create a folder called maps in the build directory.
Create a file with extension .map in the maps folder.
Fill in this map file with all the elements that need to be fetched from a repository.
Edit the build.properties file from the configuration folder and comment out the line skipFetch=true. If that property is set, then PDE build will skip the entire fetch phase. If you want the tag specified in the map files to be ignored you can set the property fetchTag to a specific value. This is useful when doing a nightly build from HEAD instead of from the tagged versions.

Getting the map files from CVS

Map files are usually stored in a repository. PDE Build offers default infrastructure to get those map files from a CVS repository. To enable this function change the following properties in your configuration's build.properties from your configuration directory:

skipMaps: Comment out this property. If this property is set, then PDE build will not fetch the map files.
mapsRepo = :pserver:anonymous@example.com/path/to/repo. The CVS repository from which to get the map files.
mapsRoot = /path/to/maps. The path in the CVS repository to the directory containing the map files.
mapsCheckoutTag = HEAD. The CVS tag to use to checkout the map files.

Fetching the map files from CVS occurs during the Pre-Build phase of the build. Scripts to fetch all the features and plug-ins included in the feature you are building will be generated and run during the fetch phase of the build.

Automatic fetching from other repositories

If you are automatically fetching your map files from the repository, you will need to copy the customTarget.xml file from org.eclipse.pde.build/templates/headless-build into your configuration directory. The target getMapFiles is used to fetch the map files, this should be modified to fetch from your repository.

Getting source code manually

You can write custom fetch targets to retrieve your plug-ins and features which can be invoked from the preSetup or postSetup targets in the customTargets.xml file. Features and plug-ins should be fetched to ${buildDirectory}/features and ${buildDirectory}/plugins respectively.