enable the /MP MSVC compiler argument for parallel compilation

[imrichardcole]

As detailed here:

https://learn.microsoft.com/en-us/cpp/build/reference/mp-build-with-multiple-processes?view=msvc-170

This enables parallel compilation. The build generator can already parallelise at the project level i.e -j10 for example, but this enables parallel compilation.

On my machine, times without this change:

Debug = 32.02s
Release = 34.99s

And times with:

Debug = 11.30s
Release = 12.88s

[LebedevRI]

Does the build generator not produce one compiler invocation per source file?
https://learn.microsoft.com/en-us/cpp/build/reference/mp-build-with-multiple-processes?view=msvc-170#number-of-processes

[imrichardcole]

@LebedevRI it won't spawn more than there are source files, but /MP on it's own is driven by the number of logical cores on the host machine:

Or you can use the default, which is equal to the number of effective processors in a computer, if you omit the processMax argument.

It can be limited by adding a number e.g /MP2. What we normally do with this setting is work out the number of cores on the machine, then subtract 2. I can implement that here if required.

[LebedevRI]

My question is: is that generator effectively always in Unity build / jumbo build mode?
I don't understand why this would help otherwise.

But i'll leave this for @dmah42. It's probably fine i guess.

[imrichardcole]

An example might be easier.

Assume we have a VS solution with three projects, each one builds an exe and there are no dependencies between them. If we pass -j3 at the cmake build stage, then MSBuild will attempt to build all three projects in parallel.

If each project has 10 cpp files, without /MP each source will be built in a serial fashion i.e we'd use 3 cores. By specifying /MP you can then enable parallel compilation. So if we had a machine with 30 cores, in theory the above example would use all of them (3 projects x 10 source files each)

Hope that clarifies?

[LebedevRI]

I see. Sounds like the solution is to use a better generator (ninja?).

[dmah42]

ninja might be a solution but if folks want to use msbuild this seems like a healthy improvement.

[imrichardcole]

You raise an interesting point about ninja. By using a tracing python package to visualize .ninja_log into a flame graph, it still seems to be serial:

I'll take a look at this later today as I had also assumed /MP was redundant when using ninja.

[LebedevRI]

Huh, that sounds weird. ninja really isn't serial even if you don't specify -j<N>.