Go, go get, go install, local packages, and version control

I am having trouble understanding the workflow for creating a go project that has local packages.

Say I create a new project, using git for version control, which has a main.go file and a tools.go file which will be in package utils. So I have a directory structure such as this:

/myproject/
   main.go
   utils/
     tools.go

main.go looks like this:

package main
import "./utils"
func main() {
    utils.DoSomthing()
}

and tools.go looks like this:

package utils;
func DoSomething() {
}

Everything works just fine locally, using go build and go run. But this is being hosted on github, and I'd like to be able to have others use the go get command to install it. So the local package import must be changed to use the format "github.com/user/project/utils", which works, except now I have two copies of the source code, and the real problem is that the copy with the git history has an import that makes use of the downloaded copy. So if I'm working on the copy with the git history, any changes made to tools.go will go unnoticed, because it will be using the downloaded copy.

So I'm wondering if someone can explain the right way of using go get, version control, and package imports within the same project.

I've just written a short step-by-step guide on how I am using the new go tool and github.com. You might find it useful:

1. Setup your GOPATH

You can set the environment variable GOPATH to any directory you like. If you have larger projects, it's probably a good idea to create a different GOPATH for each of them. I would recommend this approach especially for the deployment, so that updating a library for project A doesn't break project B which might require an earlier version of the very same library.

Also note that you can set your GOPATH to a list of directories, delimited by colons. So you might have a GOPATH containing all commonly used packages, and separate GOPATHS for each project with additonal packages or different versions of existing packages.

But unless your are working on a lot of different Go projects simultaneously, its probably enough to have just a single GOPATH locally. So, let's create one:

mkdir $HOME/gopath

Then you need to set two environment variables to tell the go tool where it can find existing Go packages and where it should install new ones. It's probably best to add the following two lines to your ~/.bashrc or ~/.profile (and do not forget to reload your .bashrc afterwards).

export GOPATH="$HOME/gopath"
export PATH="$GOPATH/bin:$PATH"

2. Create a new project

If you want to create a new Go project which should be hosted at github.com later, you should create this project under $GOPATH/src/github.com/myname/myproject. It's important that the path matches the URL of the github.com repo, because the go tool will follow the same convention. So, let's create the project root and initialize a new git repository there:

mkdir -p $GOPATH/src/github.com/myname/myproject
cd $GOPATH/src/github.com/myname/myproject
git init

Because I do not like to type such long paths, I normally create symbolic links for the projects I am currently working on in my home folder:

ln -s $GOPATH/src/github.com/myname/myproject ~/myproject

3. Write your application

Start coding and don't forget to git add and git commit your files. Also, do not use relative imports like import "./utils" for sub-packages. They are currently undocumented and shouldn't be used at all, because they won't work with the go tool. Use imports like github.com/myname/myproject/utils instead.

4. Publish your project

Create a new repository at github.com, upload your SSH public key if you haven't done that before and push your changes to the remote repository:

git remote add origin [email protected]:myname/myproject.git
git push origin master

5. Continue working on your project

If you have set the GOPATH in your .bashrc and if you have created a symlink to your project in your home folder, you can just type cd myproject/ and edit some files there. Afterwards, you can commit the changes using git commit -a and send them to github.com by doing a git push.

You probably don't want two copies of the source. Following How to Write Go Code, you should have a path where you do your Go development, lets say "godev", and under that, a "src" directory, and under that, your "github.com/user/project" and "github.com/user/project/utils". (I agree, it seems a little rigid, but the advantage explained to us is freedom from make files.) Ditch the myproject, this is where you will do your work.

You will have GOPATH set to godev at least, but you will probably want your GOPATH to start with a path for external packages that are not yours. For example the GOPATH I use is <my place on the file system>/goext:<my place on the file system>/godev.

You are right that your import in main.go should now read "github.com/user/project/utils.

Don't worry about go get or any of the go commands overwriting your files or messing up version control. Through GOPATH, they see where you are working and they know how version control works.

If you want to keep your code in local version repository, just put your code in GOPATH.

GOPATH accepts multiple paths. eg. on linux

GOPATH=$HOME/go:$HOME/prj/foo

So, you could go get 3rd party packages installed in $HOME/go/src/... And, you could keep your code controlled in $HOME/prj/foo/src.

ref: go help gopath

lot of people say should use absolutlly path to construct project struct and import :

import "github.com/user/utils"

but this may limit your project within a single repo (github). I would like to use relative path instead :

  import "./utils"

I havn't find a way to do that, I raise a question here: https://stackoverflow.com/questions/51413252/how-to-oraganize-go-project-packages-if-it-will-hosted-on-different-reposgithub

After looking for this answer in different posts and Go documentation, with Go 1.11 there has been released a new approach to handle packages and is using Go Modules.

Read the whole post series about Go Modules here.

Go Modules also takes care of version control as well