I always decant off my break material when making a starter. I invariably get a little bit of it into the flask, but I don't sweat it. I leave as much behind as I can then just carry on with my process. Not sure if it affects the starter one way or another. But my starters have always taken off in a few hours with canned wort.