Yeah, a lack of O2 basically leads to less yeast growth, which results in stressed yeast and off-flavors. Depending on the yeast strain and style of beer you can sometimes get away with it, but its something to consider for improving your beers. How strong are your beers normally? For most normal gravity beers (~5-6% ABV), you can get by just fine with using regular air rather than oxygen. I just use an aquarium pump with a filter for the air hose, and its definitely a cheaper alternative to a full oxygen setup. (You can also shake/stir if you've got the arm strength and the confidence to take enough time to do it adequately.) I'll admit that I also use it for stronger beers and with a good-sized starter I haven't noticed too much in the way of off-flavors, but its something that I'm also planning on adding here at some point.
Edit: Another downside to less oxygen is that due to the lower level of yeast, you can get stuck/stalled fermentations and lower levels of attenuation, which can lead to beer that's overly sweet.