Downloading Eclipse plug-in update sites for offline installation

A plug-in that I want to install provides an update site for installation. However, the Eclipse installation that I want to install it to is on a machine that is not connected to the Internet. Is there a way for me to access the site (HTTP, FTP, etc.) to download the files in it for offline installation?

Eclipse offers a way of mirroring these sites automatically, either through the command line or through ant tasks.

Mirror based on p2 information

$eclipse_home/eclipse -application org.eclipse.equinox.p2.artifact.repository.mirrorApplication -source $1 -destination $2
$eclipse_home/eclipse -application org.eclipse.equinox.p2.metadata.repository.mirrorApplication -source $1 -destination $2

Reference: Equinox p2 repository mirroring

Mirror based on site.xml information

java -jar $eclipse_home/plugins/org.eclipse.equinox.launcher_*.jar -application org.eclipse.update.core.standaloneUpdate -command mirror -from $from -to $to

Reference: Running the update manager from the command line

You can follow the evolution of these scripts in my script repository.

After struggling with mirroring for some time, I realized that it is much easier (at least, for me) to use "wget" instead.

In short:

1. Download the site:

   wget --recursive --no-parent http://url.of/updatesite

2. Take the content of the downloaded update site and move it to your offline environment

3. Add your offline update site directory to the list of available software sites via "Local..." button.

You can read about it with more details here.

Most Eclipse plug-ins can be installed without the Eclipse updater, by copying the required JARs available at the update site, into the plugins and features directories of the Eclipse installation.

In certain cases, it is necessary to start Eclipse with a -clean switch.

Here's an example of how to do this for the m2eclipse plugin:

1. The m2eclipse update site is http://m2eclipse.sonatype.org/update. Identifying the list of JARs required is the first step. This is normally found in the site.xml file; in this case, you'll find it in http://m2eclipse.sonatype.org/update/site.xml.
2. Filter the list of JARs to be downloaded to the version of the plugin that you intend to use. This can be determined by the version attribute for each "feature". If you are lucky, you'll find the description of the JAR in the category node.
3. Note the url attribute of each JAR that needs to be downloaded. This will include the subdirectory on the server where the JAR is available, and also the directory in the Eclipse installation where they need to be placed.

PS: This method is a bit hackish, but it is based off the site-map reference. But do refer to the update

Update

I haven't attempted this, but you can create a local mirror site, from where everyone else can pickup the Eclipse plug-ins. In Galileo, this can be done by running the Eclipse updater in the standalone mode via the mirror command.

You can mirror p2 sites using Ant tasks:

<target name="springide">
    <echo>springide</echo> 
	<p2.mirror verbose="true">
		<repository location="${REPO_HOME}/springide" name="springide" append="true"/>
		<source>
			<repository location="http://springide.org/updatesite" />
		</source>
		<iu id="Core / Spring IDE" version="" />
		<iu id="Extensions / Spring IDE" version="" />
		<iu id="Integrations / Spring IDE" version="" />
		<iu id="Resources / Spring IDE" version="" />
	</p2.mirror>
</target>

or findbugs:

<target name="findbugs">

<echo>findbugs</echo>
	<p2.mirror verbose="true">
		<repository location="${REPO_HOME}/findbugs" name="findbugs" append="true"/>
		<source>
			<repository location="http://findbugs.cs.umd.edu/eclipse/" />
		</source>
		<iu id="edu.umd.cs.findbugs.plugin.eclipse.feature.group" version="" />
	</p2.mirror>
</target>

In order for this to work you have to run ant tasks in the same JVM as eclipse.

You can find IU ID:s by opening "Software Updates" and copy it from there. In Eclipse 3.5 there should be a More... button, in 3.4 you have to click on properties button.

Eclipse plugins usually depend on other plugins. It's kind of hard to trace the dependencies. It's better to download all dependencies using update site once, and you can distribute to other Eclipse dropins. For Eclipse 3.4 or newer, you can use dropins which is an Eclipse feature. This way you don’t have to install plugin from update site every time you have to re-install your Eclipse. Read on http://michsan.web.id/content/how-install-eclipse-plugins-offline

If you can't see the web, I'll give you some description

Prepare directory for external plugins

Create special directory to hold our beloved plugins, e.g. in /home/ichsan/eclipse-dropins we will install Maven plugin: m2eclipse.

mkdir /home/ichsan/eclipse-dropins

For now on, we'll call this directory as DROPINS

Preparing sandbox

Next, by using Git we'll create an Eclipse sandbox. The point is to install one plugin on a fresh Eclipse. Instead of installing fresh Eclipse every time we want to install a new plugin, we'd better use Git to create new branch of fresh Eclipse.

First, extract/install new Eclipse to a directory e.g. /home/ichsan/eclipse-sandbox (so that we'll find /home/ichsan/eclipse-sandbox/eclipse.ini). We call the directory as ECLIPSE_SANDBOX.

Next, commit the fresh installation. This step should be done once only.

cd $ECLIPSE_SANDBOX
git init
git add .
git commit -am "Fresh Eclipse"

Install plugin on sandbox

Now is the interesting part. Supposed we have to install m2eclipse plugin. We will install this on new Git branch so that the master branch will stay clean or stay intact.

cd $ECLIPSE_SANDBOX
git checkout -b "m2eclipse"

Now, we start the Eclipse of the ECLIPSE_SANDBOX and download the plugin. Once we're done, we shut the Eclipse down and check what new directories or files have been created (using Git). Remember that, we only care about new plugins and features directories and the contents inside them. So, we won't copy the rest into dropins.

# Prepare the m2eclipse plugin directories
mkdir -p $DROPINS/m2eclipse/eclipse/plugins
mkdir -p $DROPINS/m2eclipse/eclipse/features
 
cd $ECLIPSE_SANDBOX
for f in $(git status | sed "s/#\t//g" | grep -P "^plugins" ); do cp -R $f $DROPINS/m2eclipse/eclipse/plugins; done
for f in $(git status | sed "s/#\t//g" | grep -P "^features"); do cp -R $f $DROPINS/m2eclipse/eclipse/features; done
 
# Make the directory read only
chmod -R -w $DROPINS/m2eclipse
 
# Commit changes
git add .
git add -u
git commit -am "M2Eclipse plugin installed"

 
# Back to master branch to make Eclipse clean again and ready for other plugin installations
git checkout master
Installing the plugin

Just copy the directory of DROPINS/m2eclipse into ECLIPSE_HOME/dropins or create a symbolic link. And we're done!

cd $ECLIPSE_HOME/dropins ln -s $DROPINS/m2eclipse

Another way is to backup differences between fresh-Eclipse commit and after-plugin-installation commit.

for i in `git diff hashFreshEclipse hashPluginInstall --name-only`;do 
  if [ -f $i ]; then
    tar -r -f m2e-android.tar $i
  fi
done
gzip m2e-android.tar

You can get it from here https://repository.sonatype.org/content/repositories/forge-sites/m2e/1.3.0/N/1.3.0.20121023-1108/

Download all files drilling down into plugins and features. Store in a directory on your machine keeping everything in the same directory structure. Move it to a folder on your dev machine.

In Eclipse go to Help | Intall New Software... Hit the 'Add' button Hit the 'Local...' button Browse for the directory you dropped the files into. Follow the on screen instructions.

I found that the p2 mirrorApplication did not work very well on some sites and mirrored duplicate artifacts (both the pack200 and jar version). The b3 Aggregator worked much better and made it much easier to customize my update site. See the manual at: https://wiki.eclipse.org/Eclipse_b3/aggregator/manual for install instructions and details.

The basic steps I used were:

• File > New > Other... > b3 > Repository Aggregation.
• Select the Aggregation node and set the Build Root to the folder where you want the aggregated repo to go.
• Add the platforms you care about using R-click the Aggregation node > New Child > Configuration.
• Add one Validation Set, normally set its Label to main.
• If you don't need any additional bundles from the main eclipse repository (ie: http://download.eclipse.org/releases/mars), you can add it to the Validation Set as a Validation Repository, but typically you will need to add it as a Mapped Repository under a Contribution so dependencies and/or additional eclipse features can be downloaded.
• For each plugin, add a Contribution with a Label, and under that at least one Mapped Repository with a Location.
• Add Custom Category nodes to the Aggregation if you want; give them each a unique Description, Identifier, and Label.
• Expand the Repository Browser node at the bottom > expand the repo url > expand the Installable Units > Look at the features under Categories (if present) and/or under Features.
• R-click the features you want and Add to Custom Category (if you are using those), otherwise Add to parent mapped repository as > Mapped Feature.
• R-click any node > Build Aggregation.

Important Note: if you don't map any features from a given repository, that entire repository will be mirrored (the latest versions of all bundles I think, not actually everything in the repo).

I fortunately didn't have to mess with Exclusion Rules or Valid Configuration Rules, which seem to make things more complicated. However, exclusion rules may be necessary if the repositories contain bundles that have dependency conflicts, in which case one or more conflicting bundles need to be excluded.

While b3 Aggregator normally only downloads the latest version of every feature you have mapped (and it's dependencies), if you repeatedly use Build Aggregation over time as new versions are released, the obsolete versions will accumulate in your aggregation. You could use Clean then Build Aggregation but this means you will have to redownload everything. Instead, simply add another .b3aggr aggregation file, set the Build Root, add your Configurations, and add one Contribution with a Mapped Repository using the path to your local mirror's final directory. Don't map any features or create any categories. Then Build Aggregation and only the latest versions from your mirror will be aggregated!

I just faced this issue and resolved it by following the instructions from this guide. In summary, run the following commands on your terminal in Eclipse folder:

1. eclipsec.exe -application org.eclipse.equinox.p2.metadata.repository.mirrorApplication -source $1 -destination $2

2. eclipsec.exe -application org.eclipse.equinox.p2.artifact.repository.mirrorApplication -source $1 -destination $2

Where $1 refers to the URL to the online repository and $2 refers to the path to the local folder. E.g. to download Vrapper on my Windows Desktop:

$1 = http://vrapper.sourceforge.net/update-site/stable/

$2 = C:/Users/foo/Desktop

Transfer the folder to the machine without Internet connection. Then, launch Eclipse -> Help > Install new software. Specify the local repository (i.e. the folder that you've just transferred) for the installation. That should work.