I was walking through the building and my boss came up behind me put his hands on my shoulders gave me a small shove(not a nudge, but a shove) and told me to "walk faster you need to act like you got someplace to go!" ( I wanted to give him an elbow to the chest,but I resisted) I didn't say anything I just started looking at the ground and purposely walked slower, because I didn't feel that I was walking that slowly at the time anyways. He's been known to act like this to everybody not just me, but this was the first time he's actually put his hands on me while trying to chew me out, I'm not sure if he's ever put his hands on somebody else before. I'm not really planning to pursue any legal action, but would like to know if him putting his hands on me and giving me a shove is enough reason to take legal action just incase this type of behavior was to happen again?

It's certainly very unprofessional. If it rises to the level of assault, your recourse is to file a complaint with the police.