Start with one neutral and one cloth that has a little weight. Cream, oat, and charcoal cover most days. Olive is the accent that still reads as quiet. If a new piece only works with one other thing you own, leave it.
The aim is fewer decisions in the morning. Lay out the shirt, the trouser, and the layer you will actually reach for. If that combination still looks like itself by evening, it belongs in the wardrobe.