Hi, sorry I'm not very familiar with build systems (CMake and make in this case).
I was wondering how I could debug this issue? I'd like to compile code more efficiently.
Please note, this is for a job and so the codebase existed long before I started working.
#Makefile compiles with 1 job but not more than 4
umbral pawn
rich stagBOT
still pine
-j1 makes it run with 1 job, -j4 with 4 jobs and no -j option also with 1 job. Look at the arguments given to make and you should find the option there. If multiple options are set it takes the last one.
umbral pawn
`-j1` makes it run with 1 job, `-j4` with 4 jobs and no `-j` option also with 1 ...
If I run it with make it compiles successfully, running it with make -j4 results in compilation errors. Is there any way to debug this?
I understand that the -j argument provides the number of jobs, I'm asking how do I get it to utilise my cpu horsepower
still pine
-v exists for verbose output. Maybe even -vvv, I forgot if make reacts to that with more output, but worth a try.
I don't think there is a real way to debug this besides looking at the rules and spotting the error. The error will be that something depends on something but you didn't specify that.
umbral pawn
I don't think there is a real way to debug this besides looking at the rules and...
What do you mean by something depends on something?
Like, wouldn't it be automatically handled?
if something depends on something else wouldn't that be compiled first?
still pine
A depends on B which depends on C, but you didn't specify that. If you run with -j1 it will do A, then B, then C and it just works. If you use -j4 it will run A, B and C in parallel, but B and C fail because the results of A don't exist yet.
umbral pawn
A depends on B which depends on C, but you didn't specify that. If you run with ...
Ahh I see, so it doesn't figure out the dependencies and try to resolve them?
still pine
Like, wouldn't it be automatically handled?
No. Make doesn't magically know dependencies. A makefile mainly consists of specifications on what depends on what.
umbral pawn
Is it solvable?
still pine
CMake is somewhat smarter and will be able to look at files and what they include to figure out the dependency graph. Make is not that smart, you need to specify it. There is some gcc command to dump dependencies of a file.
umbral pawn
Is there a modern practice to avoid such an issue?
still pine
Is it solvable?
I'm almost tempted to say no. I have never been able to write a correct makefile. You have a file that produces 2 outputs and make just refuses to understand that. Others claim to have succeeded.
still pine
Is there a modern practice to avoid such an issue?
CMake handles all of that automatically as long as you don't, say, want to include protobuf or something.
umbral pawn
CMake handles all of that automatically as long as you don't, say, want to inclu...
protobuf?
some form of prototyping?
umbral pawn
CMake handles all of that automatically as long as you don't, say, want to inclu...
So I run cmake inside of the build folder, and then make, to compile the codebase
However, when specifying say, 4 or 8 jobs, I definitely get a compilation error
still pine
protobuf?
No, a serializer. You specify a .proto file that defined messages and the proto compiler spits out a fitting .cpp and .h that implement generating and parsing the messages. Maybe search in combination with "google" because they made it.
A practical hack to just be able to start working is to just compile again. Some of the jobs failed to missing dependencies, others didn't have missing dependencies and finished, so now there are more things available. Keep compiling until it works. This will bite you in the ass later when you change something and it still has the old object file and your changed had no effect. Then you debug for hours, change something random and it takes the new dependency and cry in despair. I did.
Having a build system that compiles the things that need compiling at the correct time seems to just be too difficult a task.
umbral pawn
A practical hack to just be able to start working is to just compile again. Some...
So just keep compiling with a high number of jobs and this will resolve the issue itself? (not actually resolve)
still pine
So I run cmake inside of the build folder, and then make, to compile the codebas...
Normally you have an IDE and press F5 or something and it does all the things. Save all files, invoke CMake, invoke make and start the executable under the debugger. I would not recommend doing this manually.
still pine
So just keep compiling with a high number of jobs and this will resolve the issu...
More jobs make it worse.
umbral pawn
I use vscode to debug but still compile via terminal
umbral pawn
More jobs make it worse.
Why is that?
still pine
Because it exposes more missing dependencies.
More jobs = more concurrency = more chances something isn't done yet.
umbral pawn
But more jobs = higher effficiency, no?
still pine
Sure
still pine
I use vscode to debug but still compile via terminal
I have reasonably good experience with Qt Creator under both Windows and Linux. If you can't get it to work in vsc maybe try that.
Tell it to open the cmakelists.txt file directly and it should do all the things.
umbral pawn
Is it bad to compile via the command line?
I still use git via the command line, although I've seen colleagues use extensions, I can't understand why
still pine
It's just an unnecessary alt-tab, up, enter, alt-tab when you could have F5 instead. Otherwise it's more or less the same.
umbral pawn
Ah you mean in terms of just having a build job
still pine
Qt Creator understand git pull, push, merge, rebase. That's about it. Anything more complicated I need to use command line too.
umbral pawn
Like, the end command is going to be the same? (building)
I thought the IDE would automatically figure out how many jobs to send to make, for instance
Or something like that
still pine
In my experience they default to use as many jobs as you have cores.
umbral pawn
Ahh I see
And if you have a 12th gen intel chip, how many jobs should you be specifying?
still pine
Which is "fun" when cores * build processes > available RAM.
umbral pawn
Which is "fun" when cores * build processes > available RAM.
32 gb is holding on for now
umbral pawn
I understand, but do you count p-cores and e-cores equally?
The 12th gen intel chips have p-cores and e-cores, as you're aware
Which number do you give? The sum of the two?
still pine
I have no idea. I would guess the sum.
umbral pawn
I have no idea. I would guess the sum.
Okay, thanks for all the useful information and advice!
How do I close this?
still pine
You can do !solved even though we didn't really solve anything
umbral pawn
!solved