multiple go projects and sharing a vendor directory (in go before 1.11)
up vote
1
down vote
favorite
I have started learning go (1.7.4) and have a project which currently produces two executables. I have a directory structure as below following the standard go layout:
GOPATH=`pwd`
bin
src/
src/<project1>
src/<project1>/vendor
src/<project1>/glide.yaml
src/<project2>
src/<project2>/vendor
src/<project2>/glide.yaml
pkg/
Project 1 and project 2 share a lot of dependencies.
Is there a way to share the vendor directory between project1 and project2 and still pin the versions to ensure reproducible builds?
I don't want to duplicate the glide.yaml and vendor directories for each project as it bloats the build and violates DRY.
The pkg directory is the obvious the way to do this but unlike vendor I don't have a dependency manager tool like glide to ensure a specific version is used (see also my related question).
A possibly related issue is how this project is organised. I believe in go it would be more conventional for each project sub-directory to map to a single github repository. However, for my project I want to build at least two executables. I realise you can do this by using different package names but it confuses go and glide. I wrestled with getting this to work under a single project and decided/discovered it was easier to use the standard go layout and work two levels up. For example, an advantage is that "go build" etc. in the subdirectories just works without having to name the package. I can also have my build, test and package machinery at the top level operate on all projects and keep my go environment separate from any others.
The programs are not complex enough to warrant separate git repositories (even as submodules). If there is a pattern that makes this work it might render my original question moot.
go version-control
add a comment |
up vote
1
down vote
favorite
I have started learning go (1.7.4) and have a project which currently produces two executables. I have a directory structure as below following the standard go layout:
GOPATH=`pwd`
bin
src/
src/<project1>
src/<project1>/vendor
src/<project1>/glide.yaml
src/<project2>
src/<project2>/vendor
src/<project2>/glide.yaml
pkg/
Project 1 and project 2 share a lot of dependencies.
Is there a way to share the vendor directory between project1 and project2 and still pin the versions to ensure reproducible builds?
I don't want to duplicate the glide.yaml and vendor directories for each project as it bloats the build and violates DRY.
The pkg directory is the obvious the way to do this but unlike vendor I don't have a dependency manager tool like glide to ensure a specific version is used (see also my related question).
A possibly related issue is how this project is organised. I believe in go it would be more conventional for each project sub-directory to map to a single github repository. However, for my project I want to build at least two executables. I realise you can do this by using different package names but it confuses go and glide. I wrestled with getting this to work under a single project and decided/discovered it was easier to use the standard go layout and work two levels up. For example, an advantage is that "go build" etc. in the subdirectories just works without having to name the package. I can also have my build, test and package machinery at the top level operate on all projects and keep my go environment separate from any others.
The programs are not complex enough to warrant separate git repositories (even as submodules). If there is a pattern that makes this work it might render my original question moot.
go version-control
See also stackoverflow.com/questions/53086136/… which covers the same problem solution for installable commands written in go. Has answer for both go 1.11 with modules and earlier versions.
– Bruce Adams
Nov 8 at 11:31
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I have started learning go (1.7.4) and have a project which currently produces two executables. I have a directory structure as below following the standard go layout:
GOPATH=`pwd`
bin
src/
src/<project1>
src/<project1>/vendor
src/<project1>/glide.yaml
src/<project2>
src/<project2>/vendor
src/<project2>/glide.yaml
pkg/
Project 1 and project 2 share a lot of dependencies.
Is there a way to share the vendor directory between project1 and project2 and still pin the versions to ensure reproducible builds?
I don't want to duplicate the glide.yaml and vendor directories for each project as it bloats the build and violates DRY.
The pkg directory is the obvious the way to do this but unlike vendor I don't have a dependency manager tool like glide to ensure a specific version is used (see also my related question).
A possibly related issue is how this project is organised. I believe in go it would be more conventional for each project sub-directory to map to a single github repository. However, for my project I want to build at least two executables. I realise you can do this by using different package names but it confuses go and glide. I wrestled with getting this to work under a single project and decided/discovered it was easier to use the standard go layout and work two levels up. For example, an advantage is that "go build" etc. in the subdirectories just works without having to name the package. I can also have my build, test and package machinery at the top level operate on all projects and keep my go environment separate from any others.
The programs are not complex enough to warrant separate git repositories (even as submodules). If there is a pattern that makes this work it might render my original question moot.
go version-control
I have started learning go (1.7.4) and have a project which currently produces two executables. I have a directory structure as below following the standard go layout:
GOPATH=`pwd`
bin
src/
src/<project1>
src/<project1>/vendor
src/<project1>/glide.yaml
src/<project2>
src/<project2>/vendor
src/<project2>/glide.yaml
pkg/
Project 1 and project 2 share a lot of dependencies.
Is there a way to share the vendor directory between project1 and project2 and still pin the versions to ensure reproducible builds?
I don't want to duplicate the glide.yaml and vendor directories for each project as it bloats the build and violates DRY.
The pkg directory is the obvious the way to do this but unlike vendor I don't have a dependency manager tool like glide to ensure a specific version is used (see also my related question).
A possibly related issue is how this project is organised. I believe in go it would be more conventional for each project sub-directory to map to a single github repository. However, for my project I want to build at least two executables. I realise you can do this by using different package names but it confuses go and glide. I wrestled with getting this to work under a single project and decided/discovered it was easier to use the standard go layout and work two levels up. For example, an advantage is that "go build" etc. in the subdirectories just works without having to name the package. I can also have my build, test and package machinery at the top level operate on all projects and keep my go environment separate from any others.
The programs are not complex enough to warrant separate git repositories (even as submodules). If there is a pattern that makes this work it might render my original question moot.
go version-control
go version-control
edited Nov 8 at 11:26
asked Oct 31 at 15:17
Bruce Adams
1,5501332
1,5501332
See also stackoverflow.com/questions/53086136/… which covers the same problem solution for installable commands written in go. Has answer for both go 1.11 with modules and earlier versions.
– Bruce Adams
Nov 8 at 11:31
add a comment |
See also stackoverflow.com/questions/53086136/… which covers the same problem solution for installable commands written in go. Has answer for both go 1.11 with modules and earlier versions.
– Bruce Adams
Nov 8 at 11:31
See also stackoverflow.com/questions/53086136/… which covers the same problem solution for installable commands written in go. Has answer for both go 1.11 with modules and earlier versions.
– Bruce Adams
Nov 8 at 11:31
See also stackoverflow.com/questions/53086136/… which covers the same problem solution for installable commands written in go. Has answer for both go 1.11 with modules and earlier versions.
– Bruce Adams
Nov 8 at 11:31
add a comment |
2 Answers
2
active
oldest
votes
up vote
1
down vote
accepted
It should be possible to have a shared vendor directory. The way I am doing it involves Go 1.11 and the new Go feature called modules. But I am pretty sure it should work with vendor and tools like glide and dep. To use dep/glide your directory structure might looks like this
- src
- projects
- project1
- project2
- vendor
- Glide.yaml
And you can build it either from the projects folder using go build -o p1 project1/*.go
or from individual project folder using go build
The same structure, but outside of GOPATH will work for Go 1.11 modules. You would have to set the GO111MODULE variable to "on" or "auto". Mind you that go modules store dependencies in some other location and download them automatically during the build process when needed.
Note: glide github page recommends switching to dep as the more official tool
Edit: Just tested it with dep. It works for me.
both glide and dep are obsoleted by modules. However I'm using go 1.7.4 as provided with Debian 9. dep might work but is offically supported only for 1.9 above.
– Bruce Adams
Oct 31 at 17:30
Thanks, I will try your solution and accept if it works for me.
– Bruce Adams
Oct 31 at 17:30
Yes, modules are preferred, but I believe dep is the more "official" tool for Go < 1.11. If you are free to upgrade, you could use snap to get the latest Go.
– Mad Wombat
Oct 31 at 17:32
That seems to work and answers another question I had. Using -o for out of source builds. The trick seems to be the /*.go if you try to be clever go doesn't like it, it wants to be clever for you.
– Bruce Adams
Nov 1 at 10:07
1
In production you don't want a constantly moving target. In general I try to use the versions of packages the OS vendor provides for stability. Debian and RedHat are both a little on the conservative side but 1.8 was only released late in 2016. In this case I'm copying colleagues who've recently introduced go.
– Bruce Adams
Nov 1 at 15:23
|
show 2 more comments
up vote
1
down vote
I recommend look at new vendoring system - https://github.com/golang/go/wiki/Modules
It allows you to fix versions of packages used:
module github.com/my/thing
require (
github.com/some/dependency v1.2.3
github.com/another/dependency/v4 v4.0.0
)
That is correct for go 1.11 which I have now switched to as it is a better solution than depending on glide and retool. I had to accept the other answer however as my question refers to an earlier version of go.
– Bruce Adams
Nov 8 at 11:25
Yes, that's fair. Besides the question, I recommend to start learning from the fresh versions - they have big improvements. And you will not waste time relearning. I've had spent some time investigating modules after classic GOPATH.
– Eugene Lisitsky
Nov 9 at 19:54
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
It should be possible to have a shared vendor directory. The way I am doing it involves Go 1.11 and the new Go feature called modules. But I am pretty sure it should work with vendor and tools like glide and dep. To use dep/glide your directory structure might looks like this
- src
- projects
- project1
- project2
- vendor
- Glide.yaml
And you can build it either from the projects folder using go build -o p1 project1/*.go
or from individual project folder using go build
The same structure, but outside of GOPATH will work for Go 1.11 modules. You would have to set the GO111MODULE variable to "on" or "auto". Mind you that go modules store dependencies in some other location and download them automatically during the build process when needed.
Note: glide github page recommends switching to dep as the more official tool
Edit: Just tested it with dep. It works for me.
both glide and dep are obsoleted by modules. However I'm using go 1.7.4 as provided with Debian 9. dep might work but is offically supported only for 1.9 above.
– Bruce Adams
Oct 31 at 17:30
Thanks, I will try your solution and accept if it works for me.
– Bruce Adams
Oct 31 at 17:30
Yes, modules are preferred, but I believe dep is the more "official" tool for Go < 1.11. If you are free to upgrade, you could use snap to get the latest Go.
– Mad Wombat
Oct 31 at 17:32
That seems to work and answers another question I had. Using -o for out of source builds. The trick seems to be the /*.go if you try to be clever go doesn't like it, it wants to be clever for you.
– Bruce Adams
Nov 1 at 10:07
1
In production you don't want a constantly moving target. In general I try to use the versions of packages the OS vendor provides for stability. Debian and RedHat are both a little on the conservative side but 1.8 was only released late in 2016. In this case I'm copying colleagues who've recently introduced go.
– Bruce Adams
Nov 1 at 15:23
|
show 2 more comments
up vote
1
down vote
accepted
It should be possible to have a shared vendor directory. The way I am doing it involves Go 1.11 and the new Go feature called modules. But I am pretty sure it should work with vendor and tools like glide and dep. To use dep/glide your directory structure might looks like this
- src
- projects
- project1
- project2
- vendor
- Glide.yaml
And you can build it either from the projects folder using go build -o p1 project1/*.go
or from individual project folder using go build
The same structure, but outside of GOPATH will work for Go 1.11 modules. You would have to set the GO111MODULE variable to "on" or "auto". Mind you that go modules store dependencies in some other location and download them automatically during the build process when needed.
Note: glide github page recommends switching to dep as the more official tool
Edit: Just tested it with dep. It works for me.
both glide and dep are obsoleted by modules. However I'm using go 1.7.4 as provided with Debian 9. dep might work but is offically supported only for 1.9 above.
– Bruce Adams
Oct 31 at 17:30
Thanks, I will try your solution and accept if it works for me.
– Bruce Adams
Oct 31 at 17:30
Yes, modules are preferred, but I believe dep is the more "official" tool for Go < 1.11. If you are free to upgrade, you could use snap to get the latest Go.
– Mad Wombat
Oct 31 at 17:32
That seems to work and answers another question I had. Using -o for out of source builds. The trick seems to be the /*.go if you try to be clever go doesn't like it, it wants to be clever for you.
– Bruce Adams
Nov 1 at 10:07
1
In production you don't want a constantly moving target. In general I try to use the versions of packages the OS vendor provides for stability. Debian and RedHat are both a little on the conservative side but 1.8 was only released late in 2016. In this case I'm copying colleagues who've recently introduced go.
– Bruce Adams
Nov 1 at 15:23
|
show 2 more comments
up vote
1
down vote
accepted
up vote
1
down vote
accepted
It should be possible to have a shared vendor directory. The way I am doing it involves Go 1.11 and the new Go feature called modules. But I am pretty sure it should work with vendor and tools like glide and dep. To use dep/glide your directory structure might looks like this
- src
- projects
- project1
- project2
- vendor
- Glide.yaml
And you can build it either from the projects folder using go build -o p1 project1/*.go
or from individual project folder using go build
The same structure, but outside of GOPATH will work for Go 1.11 modules. You would have to set the GO111MODULE variable to "on" or "auto". Mind you that go modules store dependencies in some other location and download them automatically during the build process when needed.
Note: glide github page recommends switching to dep as the more official tool
Edit: Just tested it with dep. It works for me.
It should be possible to have a shared vendor directory. The way I am doing it involves Go 1.11 and the new Go feature called modules. But I am pretty sure it should work with vendor and tools like glide and dep. To use dep/glide your directory structure might looks like this
- src
- projects
- project1
- project2
- vendor
- Glide.yaml
And you can build it either from the projects folder using go build -o p1 project1/*.go
or from individual project folder using go build
The same structure, but outside of GOPATH will work for Go 1.11 modules. You would have to set the GO111MODULE variable to "on" or "auto". Mind you that go modules store dependencies in some other location and download them automatically during the build process when needed.
Note: glide github page recommends switching to dep as the more official tool
Edit: Just tested it with dep. It works for me.
answered Oct 31 at 17:23
Mad Wombat
6,24843262
6,24843262
both glide and dep are obsoleted by modules. However I'm using go 1.7.4 as provided with Debian 9. dep might work but is offically supported only for 1.9 above.
– Bruce Adams
Oct 31 at 17:30
Thanks, I will try your solution and accept if it works for me.
– Bruce Adams
Oct 31 at 17:30
Yes, modules are preferred, but I believe dep is the more "official" tool for Go < 1.11. If you are free to upgrade, you could use snap to get the latest Go.
– Mad Wombat
Oct 31 at 17:32
That seems to work and answers another question I had. Using -o for out of source builds. The trick seems to be the /*.go if you try to be clever go doesn't like it, it wants to be clever for you.
– Bruce Adams
Nov 1 at 10:07
1
In production you don't want a constantly moving target. In general I try to use the versions of packages the OS vendor provides for stability. Debian and RedHat are both a little on the conservative side but 1.8 was only released late in 2016. In this case I'm copying colleagues who've recently introduced go.
– Bruce Adams
Nov 1 at 15:23
|
show 2 more comments
both glide and dep are obsoleted by modules. However I'm using go 1.7.4 as provided with Debian 9. dep might work but is offically supported only for 1.9 above.
– Bruce Adams
Oct 31 at 17:30
Thanks, I will try your solution and accept if it works for me.
– Bruce Adams
Oct 31 at 17:30
Yes, modules are preferred, but I believe dep is the more "official" tool for Go < 1.11. If you are free to upgrade, you could use snap to get the latest Go.
– Mad Wombat
Oct 31 at 17:32
That seems to work and answers another question I had. Using -o for out of source builds. The trick seems to be the /*.go if you try to be clever go doesn't like it, it wants to be clever for you.
– Bruce Adams
Nov 1 at 10:07
1
In production you don't want a constantly moving target. In general I try to use the versions of packages the OS vendor provides for stability. Debian and RedHat are both a little on the conservative side but 1.8 was only released late in 2016. In this case I'm copying colleagues who've recently introduced go.
– Bruce Adams
Nov 1 at 15:23
both glide and dep are obsoleted by modules. However I'm using go 1.7.4 as provided with Debian 9. dep might work but is offically supported only for 1.9 above.
– Bruce Adams
Oct 31 at 17:30
both glide and dep are obsoleted by modules. However I'm using go 1.7.4 as provided with Debian 9. dep might work but is offically supported only for 1.9 above.
– Bruce Adams
Oct 31 at 17:30
Thanks, I will try your solution and accept if it works for me.
– Bruce Adams
Oct 31 at 17:30
Thanks, I will try your solution and accept if it works for me.
– Bruce Adams
Oct 31 at 17:30
Yes, modules are preferred, but I believe dep is the more "official" tool for Go < 1.11. If you are free to upgrade, you could use snap to get the latest Go.
– Mad Wombat
Oct 31 at 17:32
Yes, modules are preferred, but I believe dep is the more "official" tool for Go < 1.11. If you are free to upgrade, you could use snap to get the latest Go.
– Mad Wombat
Oct 31 at 17:32
That seems to work and answers another question I had. Using -o for out of source builds. The trick seems to be the /*.go if you try to be clever go doesn't like it, it wants to be clever for you.
– Bruce Adams
Nov 1 at 10:07
That seems to work and answers another question I had. Using -o for out of source builds. The trick seems to be the /*.go if you try to be clever go doesn't like it, it wants to be clever for you.
– Bruce Adams
Nov 1 at 10:07
1
1
In production you don't want a constantly moving target. In general I try to use the versions of packages the OS vendor provides for stability. Debian and RedHat are both a little on the conservative side but 1.8 was only released late in 2016. In this case I'm copying colleagues who've recently introduced go.
– Bruce Adams
Nov 1 at 15:23
In production you don't want a constantly moving target. In general I try to use the versions of packages the OS vendor provides for stability. Debian and RedHat are both a little on the conservative side but 1.8 was only released late in 2016. In this case I'm copying colleagues who've recently introduced go.
– Bruce Adams
Nov 1 at 15:23
|
show 2 more comments
up vote
1
down vote
I recommend look at new vendoring system - https://github.com/golang/go/wiki/Modules
It allows you to fix versions of packages used:
module github.com/my/thing
require (
github.com/some/dependency v1.2.3
github.com/another/dependency/v4 v4.0.0
)
That is correct for go 1.11 which I have now switched to as it is a better solution than depending on glide and retool. I had to accept the other answer however as my question refers to an earlier version of go.
– Bruce Adams
Nov 8 at 11:25
Yes, that's fair. Besides the question, I recommend to start learning from the fresh versions - they have big improvements. And you will not waste time relearning. I've had spent some time investigating modules after classic GOPATH.
– Eugene Lisitsky
Nov 9 at 19:54
add a comment |
up vote
1
down vote
I recommend look at new vendoring system - https://github.com/golang/go/wiki/Modules
It allows you to fix versions of packages used:
module github.com/my/thing
require (
github.com/some/dependency v1.2.3
github.com/another/dependency/v4 v4.0.0
)
That is correct for go 1.11 which I have now switched to as it is a better solution than depending on glide and retool. I had to accept the other answer however as my question refers to an earlier version of go.
– Bruce Adams
Nov 8 at 11:25
Yes, that's fair. Besides the question, I recommend to start learning from the fresh versions - they have big improvements. And you will not waste time relearning. I've had spent some time investigating modules after classic GOPATH.
– Eugene Lisitsky
Nov 9 at 19:54
add a comment |
up vote
1
down vote
up vote
1
down vote
I recommend look at new vendoring system - https://github.com/golang/go/wiki/Modules
It allows you to fix versions of packages used:
module github.com/my/thing
require (
github.com/some/dependency v1.2.3
github.com/another/dependency/v4 v4.0.0
)
I recommend look at new vendoring system - https://github.com/golang/go/wiki/Modules
It allows you to fix versions of packages used:
module github.com/my/thing
require (
github.com/some/dependency v1.2.3
github.com/another/dependency/v4 v4.0.0
)
answered Oct 31 at 20:45
Eugene Lisitsky
7,12221644
7,12221644
That is correct for go 1.11 which I have now switched to as it is a better solution than depending on glide and retool. I had to accept the other answer however as my question refers to an earlier version of go.
– Bruce Adams
Nov 8 at 11:25
Yes, that's fair. Besides the question, I recommend to start learning from the fresh versions - they have big improvements. And you will not waste time relearning. I've had spent some time investigating modules after classic GOPATH.
– Eugene Lisitsky
Nov 9 at 19:54
add a comment |
That is correct for go 1.11 which I have now switched to as it is a better solution than depending on glide and retool. I had to accept the other answer however as my question refers to an earlier version of go.
– Bruce Adams
Nov 8 at 11:25
Yes, that's fair. Besides the question, I recommend to start learning from the fresh versions - they have big improvements. And you will not waste time relearning. I've had spent some time investigating modules after classic GOPATH.
– Eugene Lisitsky
Nov 9 at 19:54
That is correct for go 1.11 which I have now switched to as it is a better solution than depending on glide and retool. I had to accept the other answer however as my question refers to an earlier version of go.
– Bruce Adams
Nov 8 at 11:25
That is correct for go 1.11 which I have now switched to as it is a better solution than depending on glide and retool. I had to accept the other answer however as my question refers to an earlier version of go.
– Bruce Adams
Nov 8 at 11:25
Yes, that's fair. Besides the question, I recommend to start learning from the fresh versions - they have big improvements. And you will not waste time relearning. I've had spent some time investigating modules after classic GOPATH.
– Eugene Lisitsky
Nov 9 at 19:54
Yes, that's fair. Besides the question, I recommend to start learning from the fresh versions - they have big improvements. And you will not waste time relearning. I've had spent some time investigating modules after classic GOPATH.
– Eugene Lisitsky
Nov 9 at 19:54
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53086744%2fmultiple-go-projects-and-sharing-a-vendor-directory-in-go-before-1-11%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
See also stackoverflow.com/questions/53086136/… which covers the same problem solution for installable commands written in go. Has answer for both go 1.11 with modules and earlier versions.
– Bruce Adams
Nov 8 at 11:31