Can I delete container images from Google Cloud Storage artifacts bucket?

Google App-EngineGoogle Cloud-PlatformGoogle Cloud-StorageGoogle Cloud-Build

Google App-Engine Problem Overview


I have a Google App Engine app, which connects to Google Cloud Storage.

I noticed that the amount of data stored was unreasonably high (4.01 GB, when it should be 100MB or so).

So, I looked at how much each bucket was storing, and I found that there was an automatically created bucket called us.artificats. that was taking up most of the space.

I looked inside, and all it has is one folder: containers/images/.

From what I've Googled, it seems like these images come from Google Cloud Build.

My question is, can I delete them without compromising my entire application?

Google App-Engine Solutions


Solution 1 - Google App-Engine

I have solved this problem by applying a deletion rule. Here's how to do it:

  1. Open the project in Google Cloud console
  2. Open the storage management (search for "Storage" for example).
  3. In the Browser tab, select the container us.artifacts....
  4. Now, open the Lifecycle section. You should see something like:

enter image description here

  1. Click on Add a rule and provide the following conditions:
    1. In the action, select Delete object
    2. In the conditions, select Age and enter for example 3 days
  2. Click on create to confirm the creation

Now all objects older than 3 days will be automatically deleted. It might take a few minutes for this new rule to be applied by Google Cloud.

Solution 2 - Google App-Engine

For those of you seeing this later on, I ended up deleting the folder, and everything was fine.

When I ran Google Cloud Build again, it added items back into the bucket, which I had to delete later on.

As @HarshitG mentioned, this can be set up to happen automatically via deletion rules in cloud storage. As for myself, I added a deletion step to my deployment GitHub action.

Solution 3 - Google App-Engine

Here is the reference to the documentation. https://cloud.google.com/appengine/docs/standard/go/testing-and-deploying-your-app#managing_build_images

> Built container images are stored in the app-engine folder in Container Registry. You can download these images to keep or run elsewhere. Once deployment is complete, App Engine no longer needs the container images. Note that they are not automatically deleted, so to avoid reaching your storage quota, you can safely delete any images you don't need. For more information about managing images in Container Registry, see the Container Registry documentation.

This can be automated by adding a Lifecycle rules like @HarshitG mentioned. enter image description here

Solution 4 - Google App-Engine

Same issue. Thanks for the update, Caleb.

I'm having the same issue, but I don't have an App running; I just have:

  • Firebase Auth
  • Firestore
  • Firebase Functions
  • Cloud Storage

Not sure why I have 4GB stored in those containers, and I'm not sure if I should delete them or if that would break my functions.

UPDATE: I deleted the container folder and all still works. Not sure if those are backups or whatnot, but I can't find anything online or in the docs. I will post here if something happens. As soon as a cloud function ran, the folder had 33 files again.

Solution 5 - Google App-Engine

You can add a trigger to your lifecycle rules on console.cloud.google.com.

  1. Access the bucket with artifacts (default is "us.artifacts.yourAppName.appspot.com")
  2. Go to "Life cycle".
  3. Click on "Add a rule".
  4. Check "Object delete" and press "Continue".
  5. Check the filter to delete the bucket, I chose "age" and selected three days as the number of auto delete old elements (after element has 3 days of life it's auto deleted).
  6. Click on "Create" and the rule is working now, then you do not need to visit every day to clean the bucket.

Solution 6 - Google App-Engine

I recommend against setting up a lifecycle rules on your Storage buckets. It will likely lead to breaking subsequent updates to the function (as described in https://stackoverflow.com/questions/67338215/cloud-function-build-error-failed-to-get-os-from-config-file-for-image)

If you are interested in cleaning up container images, you should instead delete container images stored in Google Cloud Registry https://console.cloud.google.com/gcr. Deleting container images on GCR repos will automatically cleanup objects stored in your Cloud Storage.

https://issuetracker.google.com/issues/186832976 has relevant information from a Google Cloud Functions engineer.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionCaleb H.View Question on Stackoverflow
Solution 1 - Google App-EngineHarshitGView Answer on Stackoverflow
Solution 2 - Google App-EngineCaleb H.View Answer on Stackoverflow
Solution 3 - Google App-EnginesamchuangView Answer on Stackoverflow
Solution 4 - Google App-EngineEric CrescioniView Answer on Stackoverflow
Solution 5 - Google App-EngineDjongaNelson LontrowskiView Answer on Stackoverflow
Solution 6 - Google App-EngineDaniel LView Answer on Stackoverflow