How do I identify the Google Cloud Storage URI from my Google Developers Console?

Google Cloud-PlatformGoogle BigqueryGoogle Cloud-Storage

Google Cloud-Platform Problem Overview


When I attempt load data into BigQuery from Google Cloud Storage it asks for the Google Cloud Storage URI (gs://). I have reviewed all of your online support as well as stackoverflow and cannot find a way to identify the URL for my uploaded data via the browser based Google Developers Console. The only way I see to find the URL is via gsutil and I have not been able to get gsutil to work on my machine.

Is there a way to determine the URL via the browser based Google Developers Console?

Google Cloud-Platform Solutions


Solution 1 - Google Cloud-Platform

The path should be gs://<bucket_name>/<file_path_inside_bucket>.

Solution 2 - Google Cloud-Platform

To answer this question more information is needed. Did you already load your data into GCS?

If not, the easiest would be to go to the project console, click on project, and Storage -> Cloud Storage -> Storage browser.

You can create buckets there and upload files to the bucket.

Then the files will be found at gs://<bucket_name>/<file_path_inside_bucket> as @nmore says.

Solution 3 - Google Cloud-Platform

Couldn't find a direct way to get the url. But found an indirect way and below are the steps:

  1. Go to GCS
  2. Go into the folder in which the file has been uploaded
  3. Click on the three dots at the right end of your file's row
  4. Click rename
  5. Click on gsutil equivalent link
  6. Copy the url alone

Solution 4 - Google Cloud-Platform

Follow the following steps :

  1. Go to GCS
  2. Go into the folder in which the file has been uploaded
  3. On the top you can see overview option
  4. You can see there will be Link URL and link for GSUtil

Solution 5 - Google Cloud-Platform

Retrieving the Google Cloud Storage URI To create an external table using a Google Cloud Storage data source, you must provide the Cloud Storage URI.

The Cloud Storage URI comprises your bucket name and your object (filename). For example, if the Cloud Storage bucket is named mybucket and the data file is named myfile.csv, the bucket URI would be gs://mybucket/myfile.csv. If your data is separated into multiple files you can use a wildcard in the URI. For more information, see Cloud Storage Request URIs.

BigQuery does not support source URIs that include multiple consecutive slashes after the initial double slash. Cloud Storage object names can contain multiple consecutive slash ("/") characters. However, BigQuery converts multiple consecutives slashes into a single slash. For example, the following source URI, though valid in Cloud Storage, does not work in BigQuery: gs://[BUCKET]/my//object//name.

To retrieve the Cloud Storage URI:

Open the Cloud Storage web UI.

CLOUD STORAGE WEB UI

Browse to the location of the object (file) that contains the source data.

At the top of the Cloud Storage web UI, note the path to the object. To compose the URI, replace gs://[BUCKET]/[FILE] with the appropriate path, for example, gs://mybucket/myfile.json. [BUCKET] is the Cloud Storage bucket name and [FILE] is the name of the object (file) containing the data.

Solution 6 - Google Cloud-Platform

If you need help on subdirectories, check this out on https://cloud.google.com/storage/docs/gsutil/addlhelp/HowSubdirectoriesWork

And https://cloud.google.com/storage/images/gsutil-subdirectories-thumb.png, if you need to see how gsutil provides a hierarchical view of objects in a bucket.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionKellyView Question on Stackoverflow
Solution 1 - Google Cloud-PlatformnmoreView Answer on Stackoverflow
Solution 2 - Google Cloud-PlatformFelipe HoffaView Answer on Stackoverflow
Solution 3 - Google Cloud-PlatformshobanaView Answer on Stackoverflow
Solution 4 - Google Cloud-PlatformAshish KumarView Answer on Stackoverflow
Solution 5 - Google Cloud-PlatformGenesis_GDKView Answer on Stackoverflow
Solution 6 - Google Cloud-PlatformDerek ChiaView Answer on Stackoverflow