I think you've gotten a lot of great suggestions so far.
As everyone has mentioned, there are a lot of great stater kits out there from places like Northern Brewer. I got my starter kit from my local homebrew supply store.
When deciding on a kit, I think it might help to ask yourself a few questions.
As mentioned before, the most popular starter kits seem to be either 1 gallon or 5 gallon. How much space you have to brew, and especially ferment, can help you determine which one of these may be better for you. Bear in mind that you'll want as temperature controlled environment as you can get for fermenting - least a space that stays relatively cool in your house with even temperatures (like a closet in the middle of your house). If you're like many of us, you may want to step up to better temperature control at some point, be it a swamp cooler or one of many other options, and space will be a consideration there too.
The other thing to perhaps consider is that there isn't much difference in the amount of time it takes to brew and ferment 1 gallon of beer vs. 5 gallons of beer. Do you want more beer for the same amount of time and effort invested?
Then again, there are brewers who want to try different recipes all of the time, and don't want 5 gallons (about 2 cases) of one style before moving on to the next style. These brewers prefer one gallon batches because they move on to the next brew more quickly and don't mind the extra time investment.
When I asked myself these questions, it became pretty clear that the 5 gallon kit was the way to go for me personally, but everyone is different.