Code Build Optimisation Part 4 – Incremental Linking and the Search for the Holy Grail.

Original Author: Andy Firth

So you’ve done your full rebuild, waited out your now much shorter build time while talking with colleagues over coffee… but wait…you forgot to make that 1 edit to test your function… there done…you see your file building… its done and BAM  “Linking…” time to go get more coffee… this baby is going to take a while.

Typically on small programs linking is measured in mere seconds (often less) and is largely unnoticeable. However on larger projects with millions of lines of code, heavy template usage and optimizations such as inline functions and weak references (declspec selectany) your symbol count goes off the charts. Combo this with the probable use of libraries to compartmentalize your code and the likelihood that you’re compiling more than 1 target at a time (dll’s, tools, runtime). The linker is likely doing a significant amount of the work for you AFTER compiling your single file. As an example for me within our current code @ Bungie a single file change in the code I usually work on can cost as much as 10mins in linking (many targets). If i opt to build a single target (say the runtime) then this is reduced to ~2 mins. This is without any linker optimization such as Link Time Code Generation which is known to heavily increase Link time.

In my experience I’d say most of my changes (and most of the changes my colleagues make) before doing a compile involve a single file. More often than not its simpler than that: a single line. So the iteration is something like

  • make change
  • hit compile
    • 1-2 seconds of compile
    • 1-2 mins to link single runtime target.
  • load game
  • test
  • repeat
Now Loading the game is always going to be a problem*, but but the build itself is completely dominated by linking. If only that step could be faster.

Incremental Linking

Incremental linking sets up an ilk file. The first time you link with /INCREMENTAL enabled  it will generate a database of the symbols within the Target executable and the location that supplied them within object files of said executable. This file is used in subsequent links to cross reference a changed object file (the single file you changed) with final executable allowing the linker to “patch” the symbols that are different with the new version. The Patch is achieved using a code thunk so the resultant code can be slower. The recommendation is to do a full rebuild before doing any performance testing.

Incremental linking is on for link.exe by default. New Projects generated with MSVC will explicitly disable it for Release however. I would assume that most if not all of your simple programs are using it in debug. Sadly the most common “working” case is the one that needs it least; small programs have short link times. In my experience the more complex your project the less likely incremental linking is working for you. I have spoken with many engineers over the years about incremental linking and almost all believe it to be a flawed feature of MSVC. Sometimes it works, sometimes it doesn’t. When it doesn’t work it actually increases link times and therefore, for many engineers it isn’t worth wrangling with.

I experienced the same, sometimes it works and its fantastic. Sometimes it simply doesn’t… most of the time it tells you a reason.

If your project uses any linker options that alter the symbols the linker uses or explicitly lists the order in which those symbols are used then internally the linker will disable incremental linking (the MSDN docs explicitly mention this).

  • /ORDER (forcing comdat order): 
  • /OPT:ICF (comdat folding): MSDN definition is somewhat verbose but it boils down to: a library is a container for a set of objects. Link resolves external references by searching first in libraries specified then default libs.