Considering that grain getting into wort must have occurred more than once in the history of the world, I figure that if it were deadly dangerous someone would have warned us about it by now and there'd be numerous articles about how to avoid this catastrophe.
Unless any-one says otherwise, I'd assume it's fine. Heck, if you can drop a *screwdriver* into wort, I think some grain should be fine.