Let's see, how did I rationalize the addition of minerals in the past...
Extract has had about 80% of the water removed for liquid (100% for dried). So if 3ish pounds comes in about a half gallon container, figure that represents about 2 gallons of [reconstituted] water. If you're using DME, every 3 pounds represents about 2.5 gallons...not exact numbers here, but round-about.
So lets say your recipe calls for 6 pounds of DME. That six pounds is "mineralizing" your whole 5 gallon batch, by my reckoning. LME might fall slightly short, but probably not enough to worry about.
My primary concern with adding minerals or not adding minerals is not for taste so much as yeast health. So I may not use salts in the future, but I'll most certainly be adding yeast nutrients.