Interestin Scenario

My buddy and I scored at a liquidation of a bar/grill that the owner had been brewing in. We got about 17lbs of Willamette hops, about 80lbs of Victory malt, 80 of caramel malt, 80lbs of chocolate and 50-75lbs of red wheat oats. All for about $120.
This will be my first venture into all grain. I realize these are all considered specialty grains. I would love to make an almost all willamette IPA with these grains . Does anyone have any suggestions as to what the ratio of grains could be?

You're planning on a 5 gallon batch, yes? You might be able to formulate an interesting Black IPA with those grains. You'd need a base malt, Pale Ale or two row for an IPA would be best. You could probably get away with 10-15% victory and 4-8% chocolate, maybe a little more considering the grains you have and 4-8% red wheat oats with the remaining being base malt. I'm just supposing here. As for hops, that's entirely up to you, just keep in mind that most IPAs have a BU:GU of about .90

I am making a 5 gal batch. The black ipa sounds cool. Why though do I need the pale malt and not just go heavy in the victory?

Victory will contribute fermentable sugars, though not as much as a base malt. I suppose if you use enough of it you could easily hit an acceptable OG with your other malts. That being said, you will have an extremely toasty and biscuity flavor if you're using victory as a base malt, it might be overwhelmingly so. That being said, home brewing is all about experimenting, it may make an interesting brew, it's just the normal grist ratio for victory is about 5-15% in your average batch.

Victory does not have enough diastasic power to convert it self you need a base malt to get proper conversion. Also victory is a roasted malt a little goes a long way.