I just got rid of a roommate who spent their time doing illegal activites such as having drinking parties with minors, pot smoking and of course breaking the noise rules of the apartment by blasting bass going into the early hours of the morning. I am wondering does this look bad on my part even after I have removed him from the lease and does this effect any renter credit or history?

if so, is there any action to take against him to repair any problems on my renters credit?

You are free of the roommate, so that is good. It does not sound like you should have any liability on your credit but it depends on what your arrangement was with the roommate- like if you were on lease alone and the person paid you their share of rent, or if both of you were on the lease and there was a joint and several liability clause meaning the other is responsible for full rent if one leaves or does not pay.
Best answer that can be provided without seeing the actual lease as that governs this, and also any lease with this past roommate.

My roommate was added onto the lease a month after I first moved in, when he was kicked out, we was removed from the lease. Little items like damage to my apartment while he was moving, hasn't returned the keys which is making me change the dead bolt.

Odds are that you have joint and several liability for damage to the unit. i.e., you'll have to pay for any damage if the room-mate does not.

Your recourse is to sue the room-mate to recover.