I live in a neighborhood with an HOA, and am honestly willing to put up with their BS because I have heard the horror stories from those who don't live in an HOA protected area. Clearly, you are in compliance to what they actually want - modifications to the property that don't detract from the surroundings. Your work has been incredibly well done, and it seems to match the existing structure as best as possible. I'd say they didn't even consider bringing the hammer down on you, because you are trying to do things with some consideration to how it looks to both you and your neighbors.
Showing up with spray paint and not a lawyer is the best thing you could have done. If you had been willing to play ball and they still refused your request, then that would have indicated you are actually dealing with small people - not just neighbors trying to protect the aesthetics of the area. I don't think you'll have any more problems, and I'm excited to see the finished product. I know you are as well.