How to clean old dependencies from maven repositories?
JavaMavenDependenciesRepositoryDependency ManagementJava Problem Overview
I have too many files in .m2 folder where maven stores downloaded dependencies. Is there a way to clean all old dependencies? For example, if there is a dependency with 3 different versions: 1, 2 and 3, after cleaning there must be only 3rd. How I can do it for all dependencies in .m2 folder?
Java Solutions
Solution 1 - Java
If you are on Unix, you could use the access time of the files in there. Just enable access time for your filesystem, then run a clean build of all your projects you would like to keep dependencies for and then do something like this (UNTESTED!):
find ~/.m2 -amin +5 -iname '*.pom' | while read pom; do parent=`dirname "$pom"`; rm -Rf "$parent"; done
This will find all *.pom files which have last been accessed more than 5 minutes ago (assuming you started your builds max 5 minutes ago) and delete their directories.
Add "echo " before the rm to do a 'dry-run'.
Solution 2 - Java
Short answer -
Deleted .m2 folder in {user.home}
. E.g. in windows 10 user home is C:\Users\user1
. Re-build your project using mvn clean package
. Only those dependencies would remain, which are required by the projects.
Long Answer - .m2 folder is just like a normal folder and the content of the folder is built from different projects. I think there is no way to figure out automatically that which library is "old". In fact old is a vague word. There could be so many reasons when a previous version of a library is used in a project, hence determining which one is unused is not possible.
All you could do, is to delete the .m2 folder and re-build all of your projects and then the folder would automatically build with all the required library.
If you are concern about only a particular version of a library to be used in all the projects; it is important that the project's pom should also update to latest version. i.e. if different POMs refer different versions of the library, all will get downloaded in .m2.
Solution 3 - Java
Given a POM file for a maven project you can remove all its dependencies in the local repository (by default ~/.m2/respository) using the Apache Maven Dependency Plugin.
It includes the dependency:purge-local-repository
functionality that removes the project dependencies from the local repository, and optionally re-resolve them.
To clean the local dependencies you just have to used the optional parameter reResolve and set it to false since it is set to true by default.
This command line call should work:
mvn dependency:purge-local-repository -DreResolve=false
Solution 4 - Java
-
Download all actual dependencies of your projects
find your-projects-dir -name pom.xml -exec mvn -f '{}' dependency:resolve
-
Move your local maven repository to temporary location
mv ~/.m2 ~/saved-m2
-
Rename all files maven-metadata-central.xml* from saved repository into maven-metadata.xml*
find . -type f -name "maven-metadata-central.xml*" -exec rename -v -- 's/-central//' '{}' \;
-
To setup the modified copy of the local repository as a mirror, create the directory ~/.m2 and the file ~/.m2/settings.xml with the following content (replacing user with your username):
<settings> <mirrors> <mirror> <id>mycentral</id> <name>My Central</name> <url>file:/home/user/saved-m2/</url> <mirrorOf>central</mirrorOf> </mirror> </mirrors> </settings>
-
Resolve your projects dependencies again:
find your-projects-dir -name pom.xml -exec mvn -f '{}' dependency:resolve
-
Now you have local maven repository with minimal of necessary artifacts. Remove local mirror from config file and from file system.
Solution 5 - Java
I came up with a utility and hosted on GitHub to clean old versions of libraries in the local Maven repository. The utility, on its default execution removes all older versions of artifacts leaving only the latest ones. Optionally, it can remove all snapshots, sources, javadocs, and also groups or artifacts can be forced / excluded in this process. This cross platform also supports date based removal based on last access / download dates.
Solution 6 - Java
It's been more than 6 years since this question was asked, but I still didn't find any tool to satisfactorily clean up my repository. So I wrote one myself in Python to get rid of old local artefacts. Maybe it will be useful for someone else also:
repo-cleaner.py
:
from os.path import isdir
from os import listdir
import shutil
import semver
import Constants
# Change to True to get a log of what will be removed
dry_run = False
def check_and_clean(path):
files = listdir(path)
only_files = True
for index, file in enumerate(files):
if isdir('/'.join([path, file])):
only_files = False
else:
files[index] = None
if only_files:
return
directories = [d for d in files if d is not None]
latest_version = check_if_versions(directories)
if latest_version is None:
for directory in directories:
check_and_clean('/'.join([path, directory]))
elif len(directories) == 1:
return
else:
print('Update ' + path.split(Constants.m2_path)[1])
for directory in directories:
if directory == latest_version:
continue
print(directory + ' (Has newer version: ' + latest_version + ')')
if not dry_run:
shutil.rmtree('/'.join([path, directory]))
def check_if_versions(directories):
if len(directories) == 0:
return None
latest_version = ''
for directory in directories:
try:
current_version = semver.VersionInfo.parse(directory)
except ValueError:
return None
if latest_version == '':
latest_version = directory
if current_version.compare(latest_version) > 0:
latest_version = directory
return latest_version
if __name__ == '__main__':
check_and_clean(Constants.m2_path)
Constants.py
(edit to point to your own local Maven repo):
# Paths
m2_path = '/home/jb/.m2/repository/'
Make sure that you have Python 3.6+ installed and that the semver package has been installed into your global environment or venv
(use pip install semver
if missing).
Run the script with python repo-cleaner.py
.
It recursively searches within the local Maven repository you configured (normally ~/.m2/repository
) and if it finds a catalog where different versions reside it removes all of them but the newest.
Say you have the following tree somewhere in your local Maven repo:
.
└── antlr
├── 2.7.2
│ ├── antlr-2.7.2.jar
│ ├── antlr-2.7.2.jar.sha1
│ ├── antlr-2.7.2.pom
│ ├── antlr-2.7.2.pom.sha1
│ └── _remote.repositories
└── 2.7.7
├── antlr-2.7.7.jar
├── antlr-2.7.7.jar.sha1
├── antlr-2.7.7.pom
├── antlr-2.7.7.pom.sha1
└── _remote.repositories
Then the script removes version 2.7.2 of antlr
and what is left is:
.
└── antlr
└── 2.7.7
├── antlr-2.7.7.jar
├── antlr-2.7.7.jar.sha1
├── antlr-2.7.7.pom
├── antlr-2.7.7.pom.sha1
└── _remote.repositories
Any old versions, even ones that you actively use, will be removed. It can easily be restored with Maven (or other tools that manage dependencies).
You can get a log of what is going to be removed without actually removing it by setting dry_run = True
. The output will look like this:
update /org/projectlombok/lombok
1.18.2 (newer version: 1.18.6)
1.16.20 (newer version: 1.18.6)
This means that versions 1.16.20 and 1.18.2 of lombok will be removed and 1.18.6 will be left untouched.
The latest version of the above files can be found on my github.
Solution 7 - Java
I wanted to remove old dependencies from my Maven repository as well. I thought about just running Florian's answer, but I wanted something that I could run over and over without remembering a long linux snippet, and I wanted something with a little bit of configurability -- more of a program, less of a chain of unix commands, so I took the base idea and made it into a (relatively small) Ruby program, which removes old dependencies based on their last access time.
It doesn't remove "old versions" but since you might actually have two different active projects with two different versions of a dependency, that wouldn't have done what I wanted anyway. Instead, like Florian's answer, it removes dependencies that haven't been accessed recently.
If you want to try it out, you can:
- Visit the GitHub repository
- Clone the repository, or download the source
- Optionally inspect the code to make sure it's not malicious
- Run
bin/mvnclean
There are options to override the default Maven repository, ignore files, set the threshold date, but you can read those in the README on GitHub.
I'll probably package it as a Ruby gem at some point after I've done a little more work on it, which will simplify matters (gem install mvnclean; mvnclean
) if you already have Ruby installed and operational.
Solution 8 - Java
Just clean every content under .m2-->repository folder.When you build project all dependencies load here.
In your case may be your project earlier was using old version of any dependency and now version is upgraded.So better clean .m2 folder and build your project with mvn clean install.
Now dependencies with latest version modules will be downloaded in this folder.
Solution 9 - Java
I did spend some hours looking at this problem and to the answers, many of them rely on the atime
(which is the last access time on UNIX systems), which is an unreliable solution for two reasons:
- Most UNIX systems (including Linux and macOS) update the
atime
irregularly at best, and that is for a reason: a complete implementation ofatime
would imply the whole file system would be slowed down by having to update (i.e., write to the disk) theatime
every time a file is read, moreover having a such an extreme number of updates would very rapidly wear out the modern, high performance SSD drives - On a CI/CD environment, the VM that's used to build your Maven project will have its Maven repository restored from a shared storage, which in turn will make the
atime
get set to a "recent" value
I hence created a Maven repository cleaner and made it available on https://github.com/alitokmen/maven-repository-cleaner/. The bash maven-repository-cleaner.sh
script has one function, cleanDirectory
, which is a recursive function looping through the ~/.m2/repository/
and does the following:
- When the subdirectory is not a version number, it digs into that subdirectory for analysis
- When a directory has subdirectories which appear to be version numbers, it only deletes all lower versions
In practice, if you have a hierarchy such as:
artifact-group
artifact-name
1.8
1.10
1.2
... maven-repository-cleaner.sh
script will:
- Navigate to
artifact-group
- In
artifact-group
, navigate toartifact-name
- In
artifact-name
, delete the subfolders1.8
and1.2
, as1.10
is superior to both1.2
and1.8
This is hence very similar to the solutions Andronicus and Pavan Kumar have provided, the difference is that this one is written as a Shell script. To run the tool on your CI/CD platform (or any other form of UNIX system), simply use the below three lines, either at the beginning or at the end of the build:
wget https://raw.githubusercontent.com/alitokmen/maven-repository-cleaner/main/maven-repository-cleaner.sh
chmod +x maven-repository-cleaner.sh
./maven-repository-cleaner.sh
Solution 10 - Java
You need to copy the dependency you need for project.
Having these in hand please clear all the <dependency>
tag embedded into <dependencies>
tag
from POM.XML file in your project.
After saving the file you will not see Maven Dependencies in your Libraries
.
Then please paste those <dependency>
you have copied earlier.
The required jars will be automatically downloaded by Maven, you can see that too in
the generated Maven Dependencies Libraries
after saving the file.
Thanks.