What is the best practice for "Copy Local" and with project references?

I have a large c# solution file (~100 projects), and I am trying to improve build times. I think that "Copy Local" is wasteful in many cases for us, but I am wondering about best practices.

In our .sln, we have application A depending on assembly B which depends on assembly C. In our case, there are dozens of "B" and a handful of "C". Since these are all included in the .sln, we're using project references. All assemblies currently build into $(SolutionDir)/Debug (or Release).

By default, Visual Studio marks these project references as "Copy Local", which results in every "C" being copied into $(SolutionDir)/Debug once for every "B" that builds. This seems wasteful. What can go wrong if I just turn "Copy Local" off? What do other people with large systems do?

FOLLOWUP:

Lots of responses suggest breaking up the build into smaller .sln files... In the example above, I would build the foundation classes "C" first, followed by the bulk of the modules "B", and then a few applications, "A". In this model, I need to have non-project references to C from B. The problem I run into there is that "Debug" or "Release" gets baked into the hint path and I wind up building my Release builds of "B" against debug builds of "C".

For those of you that split the build up into multiple .sln files, how do you manage this problem?

In a previous project I worked with one big solution with project references and bumped into a performance problem as well. The solution was three fold:

1. Always set the Copy Local property to false and enforce this via a custom msbuild step

2. Set the output directory for each project to the same directory (preferably relative to $(SolutionDir)

3. The default cs targets that get shipped with the framework calculate the set of references to be copied to the output directory of the project currently being built. Since this requires calculating a transitive closure under the 'References' relation this can become VERY costly. My workaround for this was to redefine the GetCopyToOutputDirectoryItems target in a common targets file (eg. Common.targets ) that's imported in every project after the import of the Microsoft.CSharp.targets. Resulting in every project file to look like the following:

   <Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
     <PropertyGroup>
       ... snip ...
     </ItemGroup>
     <Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
     <Import Project="[relative path to Common.targets]" />
     <!-- To modify your build process, add your task inside one of the targets below and uncomment it. 
          Other similar extension points exist, see Microsoft.Common.targets.
     <Target Name="BeforeBuild">
     </Target>
     <Target Name="AfterBuild">
     </Target>
     -->
   </Project>
   

This reduced our build time at a given time from a couple of hours (mostly due to memory constraints), to a couple of minutes.

The redefined GetCopyToOutputDirectoryItems can be created by copying the lines 2,438–2,450 and 2,474–2,524 from C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Microsoft.Common.targets into Common.targets.

For completeness the resulting target definition then becomes:

<!-- This is a modified version of the Microsoft.Common.targets
     version of this target it does not include transitively
     referenced projects. Since this leads to enormous memory
     consumption and is not needed since we use the single
     output directory strategy.
============================================================
                    GetCopyToOutputDirectoryItems

Get all project items that may need to be transferred to the
output directory.
============================================================ -->
<Target
    Name="GetCopyToOutputDirectoryItems"
    Outputs="@(AllItemsFullPathWithTargetPath)"
    DependsOnTargets="AssignTargetPaths;_SplitProjectReferencesByFileExistence">

    <!-- Get items from this project last so that they will be copied last. -->
    <CreateItem
        Include="@(ContentWithTargetPath->'%(FullPath)')"
        Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(ContentWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(ContentWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(_EmbeddedResourceWithTargetPath->'%(FullPath)')"
        Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_EmbeddedResourceWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(Compile->'%(FullPath)')"
        Condition="'%(Compile.CopyToOutputDirectory)'=='Always' or '%(Compile.CopyToOutputDirectory)'=='PreserveNewest'">
        <Output TaskParameter="Include" ItemName="_CompileItemsToCopy"/>
    </CreateItem>
    <AssignTargetPath Files="@(_CompileItemsToCopy)" RootFolder="$(MSBuildProjectDirectory)">
        <Output TaskParameter="AssignedFiles" ItemName="_CompileItemsToCopyWithTargetPath" />
    </AssignTargetPath>
    <CreateItem Include="@(_CompileItemsToCopyWithTargetPath)">
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_CompileItemsToCopyWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_CompileItemsToCopyWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>

    <CreateItem
        Include="@(_NoneWithTargetPath->'%(FullPath)')"
        Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='Always' or '%(_NoneWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"
            >
        <Output TaskParameter="Include" ItemName="AllItemsFullPathWithTargetPath"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectoryAlways"
                Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='Always'"/>
        <Output TaskParameter="Include" ItemName="_SourceItemsToCopyToOutputDirectory"
                Condition="'%(_NoneWithTargetPath.CopyToOutputDirectory)'=='PreserveNewest'"/>
    </CreateItem>
</Target>

With this workaround in place I found it workable to have as much as > 120 projects in one solution, this has the main benefit that the build order of the projects can still be determined by VS instead of doing that by hand by splitting up your solution.

I'll suggest you to read Patric Smacchia's articles on that subject :

• Partitioning Your Code Base Through .NET Assemblies and Visual Studio Projects --> Should every Visual Studio project really be in its own assembly? And what does 'Copy Local=True' really mean?
• Lessons learned from the NUnit code base --> The VisualStudio Project Reference + Copy Local true option is evil!)
• Analyzing the code base of CruiseControl.NET --> Bad usage of Copy Local Reference Assembly option set to True)

> CC.Net VS projects rely on the copy local reference assembly option set to true. [...] Not only this increase significantly the compilation time (x3 in the case of NUnit), but also it messes up your working environment. Last but not least, doing so introduces the risk for versioning potential problems. Btw, NDepend will emit a warning if it founds 2 assemblies in 2 different directories with the same name, but not the same content or version.

> The right thing to do is to define 2 directories $RootDir$\bin\Debug and $RootDir$\bin\Release, and configure your VisualStudio projects to emit assemblies in these directories. All project references should reference assemblies in the Debug directory.

You could also read this article to help you reduce your projects number and improve your compilation time.

I suggest having copy local = false for almost all projects except the one that is at the top of the dependency tree. And for all the references in the one at the top set copy local = true. I see many people suggesting sharing an output directory; I think this is a horrible idea based on experience. If your startup project holds references to a dll that any other project holds a reference to you will at some point experience an access\sharing violation even if copy local = false on everything and your build will fail. This issue is very annoying and hard to track down. I completely suggest staying away from a shard output directory and instead of having the project at the top of the dependency chain write the needed assemblies to the corresponding folder. If you don't have a project at the "top," then I would suggest a post-build copy to get everything in the right place. Also, I would try and keep in mind the ease of debugging. Any exe projects I still leave copy local=true so the F5 debugging experience will work.

You are correct. CopyLocal will absolutely kill your build times. If you have a large source tree then you should disable CopyLocal. Unfortunately it not as easy as it should be to disable it cleanly. I have answered this exact question about disabling CopyLocal at How do I override CopyLocal (Private) setting for references in .NET from MSBUILD. Check it out. As well as Best practices for large solutions in Visual Studio (2008).

Here is some more info on CopyLocal as I see it.

CopyLocal was implemented really to support local debugging. When you prepare your application for packaging and deployment you should build your projects to the same output folder and make sure you have all the references you need there.

I have written about how to deal with building large source trees in the article MSBuild: Best Practices For Creating Reliable Builds, Part 2.

In my opinion, having a solution with 100 projects is a BIG mistake. You could probably split your solution in valid logical small units, thus simplifying both maintenance and builds.

I am surprised no one has mentioned using hardlinks. Instead of copying the files, it creates a hardlink to the original file. This saves disk space as well as greatly speeding up build. This can enabled on the command line with the following properties:

/p:CreateHardLinksForAdditionalFilesIfPossible=true;CreateHardLinksForCopyAdditionalFilesIfPossible=true;CreateHardLinksForCopyFilesToOutputDirectoryIfPossible=true;CreateHardLinksForCopyLocalIfPossible=true;CreateHardLinksForPublishFilesIfPossible=true

You can also add this to a central import file so that all your projects can also get this benefit.

If you got the dependency structure defined via project references or via solution level dependencies it's safe to turn of "Copy Local" I would even say that it's a best practice todo so since that will let you use MSBuild 3.5 to run your build in parallel (via /maxcpucount) without diffrent processes tripping over each other when trying to copy referenced assemblies.

our "best practise" is to avoid solutions with many projects. We have a directory named "matrix" with current versions of assemblies, and all references are from this directory. If you change some project and you can say "now the change is complete" you can copy the assembly into the "matrix" directory. So all projects that depends on this assembly will have the current(=latest) version.

If you have few projects in solution, the build process is much faster.

You can automate the "copy assembly to matrix directory" step using visual studio macros or with "menu -> tools -> external tools...".

You don't need to change CopyLocal values. All you need to do is predefine a common $(OutputPath) for all projects in the solution and preset $(UseCommonOutputDirectory) to true. See this: http://blogs.msdn.com/b/kirillosenkov/archive/2015/04/04/using-a-common-intermediate-and-output-directory-for-your-solution.aspx

Set CopyLocal=false will reduce build time, but can cause different issues during deployment.

There are many scenarios, when you need to have Copy Local’ left to True, e.g.

• Top-level projects,
• Second-level dependencies,
• DLLs called by reflection

The possible issues described in SO questions
"https://stackoverflow.com/questions/602765/when-should-copy-local-be-set-to-true-and-when-should-it-not";,
"https://stackoverflow.com/questions/1091853/unable-to-load-one-or-more-of-the-requested-types-retrieve-the-loaderexceptions/6200173#6200173"
and  aaron-stainback's answer for this question.

My experience with setting CopyLocal=false was NOT successful. See my blog post "Do NOT Change "Copy Local” project references to false, unless understand subsequences."

The time to solve the issues overweight the benefits of setting copyLocal=false.

I tend to build to a common directory (e.g. ..\bin), so I can create small test solutions.

You can try to use a folder where all assemblies that are shared between projects will be copied, then make an DEVPATH environment variable and set

<developmentMode developerInstallation="true" />

in machine.config file on each developer's workstation. The only thing you need to do is to copy any new version in your folder where DEVPATH variable points.

Also divide your solution into few smaller solutions if possible.

This may not be best pratice, but this is how I work.

I noticed that Managed C++ dumps all of its binaries into $(SolutionDir)/'DebugOrRelease'. So I dumped all my C# projects there too. I also turned off the "Copy Local" of all references to projects in the solution. I had noticable build time improvement in my small 10 project solution. This solution is a mixture of C#, managed C++, native C++, C# webservice, and installer projects.

Maybe something is broken, but since this is the only way I work, I do not notice it.

It would be interesting to find out what I am breaking.

Usually, you only need to Copy Local if you want your project using the DLL that is in your Bin vs. what is somewhere else (the GAC, other projects, etc.)

I would tend to agree with the other folks that you should also try, if at all possible, to break up that solution.

You can also use Configuration Manager to make yourself different build configurations within that one solution that will only build given sets of projects.

It would seem odd if all 100 projects relied on one another, so you should be able to either break it up or use Configuration Manager to help yourself out.

You can have your projects references pointing to the debug versions of the dlls. Than on your msbuild script, you can set the /p:Configuration=Release, thus you will have a release version of your application and all satellite assemblies.

If you want to have a central place to reference a DLL using copy local false will fail without the GAC unless you do this.

http://nbaked.wordpress.com/2010/03/28/gac-alternative/

If the reference is not contained within the GAC, we must set the Copy Local to true so that the application will work, if we are sure that the reference will be preinstalled in the GAC then it can be set to false.

Well, I certainly don't know how the problems works out, but i had contact with a build solution that helped itself in such that all created files where put on an ramdisk with the help of symbolic links.

• c:\solution folder\bin -> ramdisk r:\solution folder\bin\

• c:\solution folder\obj -> ramdisk r:\solution folder\obj\

• You can also tell additionally the visual studio which temp directory it can use for the build.

Actually that wasn't all what it did. But it really hit my understanding of performance.
100% processor use and a huge project in under 3 Minute with all dependencies.