Okay so usually you'll actually put the base colours on top of the greyscale shading, which is the opposite of what you would normally do if you're shading directly with colour. Then set the base colour to a different blend mode. Typically that will be multiply or overlay.
It looks like you're using just a normal mode here which wouldn't work since it will just hide whatever is below it. You want the 2 layers to interact, not cover each other