How to clean old dependencies from maven repositories?

JavaMavenDependenciesRepositoryDependency Management

Java Problem Overview


I have too many files in .m2 folder where maven stores downloaded dependencies. Is there a way to clean all old dependencies? For example, if there is a dependency with 3 different versions: 1, 2 and 3, after cleaning there must be only 3rd. How I can do it for all dependencies in .m2 folder?

Java Solutions


Solution 1 - Java

If you are on Unix, you could use the access time of the files in there. Just enable access time for your filesystem, then run a clean build of all your projects you would like to keep dependencies for and then do something like this (UNTESTED!):

find ~/.m2 -amin +5 -iname '*.pom' | while read pom; do parent=`dirname "$pom"`; rm -Rf "$parent"; done

This will find all *.pom files which have last been accessed more than 5 minutes ago (assuming you started your builds max 5 minutes ago) and delete their directories.

Add "echo " before the rm to do a 'dry-run'.

Solution 2 - Java

Short answer - Deleted .m2 folder in {user.home}. E.g. in windows 10 user home is C:\Users\user1. Re-build your project using mvn clean package. Only those dependencies would remain, which are required by the projects.

Long Answer - .m2 folder is just like a normal folder and the content of the folder is built from different projects. I think there is no way to figure out automatically that which library is "old". In fact old is a vague word. There could be so many reasons when a previous version of a library is used in a project, hence determining which one is unused is not possible.

All you could do, is to delete the .m2 folder and re-build all of your projects and then the folder would automatically build with all the required library.

If you are concern about only a particular version of a library to be used in all the projects; it is important that the project's pom should also update to latest version. i.e. if different POMs refer different versions of the library, all will get downloaded in .m2.

Solution 3 - Java

Given a POM file for a maven project you can remove all its dependencies in the local repository (by default ~/.m2/respository) using the Apache Maven Dependency Plugin.

It includes the dependency:purge-local-repository functionality that removes the project dependencies from the local repository, and optionally re-resolve them.

To clean the local dependencies you just have to used the optional parameter reResolve and set it to false since it is set to true by default.

This command line call should work:

mvn dependency:purge-local-repository -DreResolve=false

Solution 4 - Java

  1. Download all actual dependencies of your projects

    find your-projects-dir -name pom.xml -exec mvn -f '{}' dependency:resolve
    
  2. Move your local maven repository to temporary location

    mv ~/.m2 ~/saved-m2
    
  3. Rename all files maven-metadata-central.xml* from saved repository into maven-metadata.xml*

    find . -type f -name "maven-metadata-central.xml*" -exec rename -v -- 's/-central//' '{}' \;
    
  4. To setup the modified copy of the local repository as a mirror, create the directory ~/.m2 and the file ~/.m2/settings.xml with the following content (replacing user with your username):

    <settings>
     <mirrors>
      <mirror>
       <id>mycentral</id>
       <name>My Central</name>
       <url>file:/home/user/saved-m2/</url>
       <mirrorOf>central</mirrorOf>
      </mirror>
     </mirrors>
    </settings>
    
  5. Resolve your projects dependencies again:

    find your-projects-dir -name pom.xml -exec mvn -f '{}' dependency:resolve
    
  6. Now you have local maven repository with minimal of necessary artifacts. Remove local mirror from config file and from file system.

Solution 5 - Java

I came up with a utility and hosted on GitHub to clean old versions of libraries in the local Maven repository. The utility, on its default execution removes all older versions of artifacts leaving only the latest ones. Optionally, it can remove all snapshots, sources, javadocs, and also groups or artifacts can be forced / excluded in this process. This cross platform also supports date based removal based on last access / download dates.

https://github.com/techpavan/mvn-repo-cleaner

Solution 6 - Java

It's been more than 6 years since this question was asked, but I still didn't find any tool to satisfactorily clean up my repository. So I wrote one myself in Python to get rid of old local artefacts. Maybe it will be useful for someone else also:

repo-cleaner.py:

from os.path import isdir
from os import listdir
import shutil
import semver

import Constants

# Change to True to get a log of what will be removed
dry_run = False


def check_and_clean(path):
    files = listdir(path)
    only_files = True
    for index, file in enumerate(files):
        if isdir('/'.join([path, file])):
            only_files = False
        else:
            files[index] = None
    if only_files:
        return

    directories = [d for d in files if d is not None]
    latest_version = check_if_versions(directories)
    if latest_version is None:
        for directory in directories:
            check_and_clean('/'.join([path, directory]))
    elif len(directories) == 1:
        return
    else:
        print('Update ' + path.split(Constants.m2_path)[1])
        for directory in directories:
            if directory == latest_version:
                continue
            print(directory + ' (Has newer version: ' + latest_version + ')')
            if not dry_run:
                shutil.rmtree('/'.join([path, directory]))


def check_if_versions(directories):
    if len(directories) == 0:
        return None
    latest_version = ''
    for directory in directories:
        try:
            current_version = semver.VersionInfo.parse(directory)
        except ValueError:
            return None
        if latest_version == '':
            latest_version = directory
        if current_version.compare(latest_version) > 0:
            latest_version = directory
    return latest_version


if __name__ == '__main__':
    check_and_clean(Constants.m2_path)

Constants.py (edit to point to your own local Maven repo):

# Paths
m2_path = '/home/jb/.m2/repository/'

Make sure that you have Python 3.6+ installed and that the semver package has been installed into your global environment or venv (use pip install semver if missing).

Run the script with python repo-cleaner.py.

It recursively searches within the local Maven repository you configured (normally ~/.m2/repository) and if it finds a catalog where different versions reside it removes all of them but the newest.

Say you have the following tree somewhere in your local Maven repo:

.
└── antlr
    ├── 2.7.2
    │   ├── antlr-2.7.2.jar
    │   ├── antlr-2.7.2.jar.sha1
    │   ├── antlr-2.7.2.pom
    │   ├── antlr-2.7.2.pom.sha1
    │   └── _remote.repositories
    └── 2.7.7
        ├── antlr-2.7.7.jar
        ├── antlr-2.7.7.jar.sha1
        ├── antlr-2.7.7.pom
        ├── antlr-2.7.7.pom.sha1
        └── _remote.repositories

Then the script removes version 2.7.2 of antlr and what is left is:

.
└── antlr
    └── 2.7.7
        ├── antlr-2.7.7.jar
        ├── antlr-2.7.7.jar.sha1
        ├── antlr-2.7.7.pom
        ├── antlr-2.7.7.pom.sha1
        └── _remote.repositories

Any old versions, even ones that you actively use, will be removed. It can easily be restored with Maven (or other tools that manage dependencies).

You can get a log of what is going to be removed without actually removing it by setting dry_run = True. The output will look like this:

    update /org/projectlombok/lombok
    1.18.2 (newer version: 1.18.6)
    1.16.20 (newer version: 1.18.6)

This means that versions 1.16.20 and 1.18.2 of lombok will be removed and 1.18.6 will be left untouched.

The latest version of the above files can be found on my github.

Solution 7 - Java

I wanted to remove old dependencies from my Maven repository as well. I thought about just running Florian's answer, but I wanted something that I could run over and over without remembering a long linux snippet, and I wanted something with a little bit of configurability -- more of a program, less of a chain of unix commands, so I took the base idea and made it into a (relatively small) Ruby program, which removes old dependencies based on their last access time.

It doesn't remove "old versions" but since you might actually have two different active projects with two different versions of a dependency, that wouldn't have done what I wanted anyway. Instead, like Florian's answer, it removes dependencies that haven't been accessed recently.

If you want to try it out, you can:

  1. Visit the GitHub repository
  2. Clone the repository, or download the source
  3. Optionally inspect the code to make sure it's not malicious
  4. Run bin/mvnclean

There are options to override the default Maven repository, ignore files, set the threshold date, but you can read those in the README on GitHub.

I'll probably package it as a Ruby gem at some point after I've done a little more work on it, which will simplify matters (gem install mvnclean; mvnclean) if you already have Ruby installed and operational.

Solution 8 - Java

Just clean every content under .m2-->repository folder.When you build project all dependencies load here.

In your case may be your project earlier was using old version of any dependency and now version is upgraded.So better clean .m2 folder and build your project with mvn clean install.

Now dependencies with latest version modules will be downloaded in this folder.

Solution 9 - Java

I did spend some hours looking at this problem and to the answers, many of them rely on the atime (which is the last access time on UNIX systems), which is an unreliable solution for two reasons:

  1. Most UNIX systems (including Linux and macOS) update the atime irregularly at best, and that is for a reason: a complete implementation of atime would imply the whole file system would be slowed down by having to update (i.e., write to the disk) the atime every time a file is read, moreover having a such an extreme number of updates would very rapidly wear out the modern, high performance SSD drives
  2. On a CI/CD environment, the VM that's used to build your Maven project will have its Maven repository restored from a shared storage, which in turn will make the atime get set to a "recent" value

I hence created a Maven repository cleaner and made it available on https://github.com/alitokmen/maven-repository-cleaner/. The bash maven-repository-cleaner.sh script has one function, cleanDirectory, which is a recursive function looping through the ~/.m2/repository/ and does the following:

  • When the subdirectory is not a version number, it digs into that subdirectory for analysis
  • When a directory has subdirectories which appear to be version numbers, it only deletes all lower versions

In practice, if you have a hierarchy such as:

  • artifact-group
    • artifact-name
      • 1.8
      • 1.10
      • 1.2

... maven-repository-cleaner.sh script will:

  1. Navigate to artifact-group
  2. In artifact-group, navigate to artifact-name
  3. In artifact-name, delete the subfolders 1.8 and 1.2, as 1.10 is superior to both 1.2 and 1.8

This is hence very similar to the solutions Andronicus and Pavan Kumar have provided, the difference is that this one is written as a Shell script. To run the tool on your CI/CD platform (or any other form of UNIX system), simply use the below three lines, either at the beginning or at the end of the build:

wget https://raw.githubusercontent.com/alitokmen/maven-repository-cleaner/main/maven-repository-cleaner.sh
chmod +x maven-repository-cleaner.sh
./maven-repository-cleaner.sh

Solution 10 - Java

You need to copy the dependency you need for project. Having these in hand please clear all the <dependency> tag embedded into <dependencies> tag from POM.XML file in your project.

After saving the file you will not see Maven Dependencies in your Libraries. Then please paste those <dependency> you have copied earlier.

The required jars will be automatically downloaded by Maven, you can see that too in the generated Maven Dependencies Libraries after saving the file.

Thanks.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionCherryView Question on Stackoverflow
Solution 1 - JavaFlorianView Answer on Stackoverflow
Solution 2 - JavaGyanendra DwivediView Answer on Stackoverflow
Solution 3 - JavaJuanjo MarronView Answer on Stackoverflow
Solution 4 - JavaSergey B.View Answer on Stackoverflow
Solution 5 - JavaPavan KumarView Answer on Stackoverflow
Solution 6 - JavaAndronicusView Answer on Stackoverflow
Solution 7 - JavaGeoffrey WisemanView Answer on Stackoverflow
Solution 8 - JavaAmandeep Singh BhatiaView Answer on Stackoverflow
Solution 9 - JavaalitokmenView Answer on Stackoverflow
Solution 10 - JavaShubhasish BhuniaView Answer on Stackoverflow