Having loaded up every weight of bullet you talked about, I prefer the 180gr being pushed down the tube by 6.7gr. of Power Pistol giving me around 980fps or 176PF. With a little extra powder you can easily hit a PF of180+. The lighter bullets all shot fine but there was a much more noticeable recoil, more of a crack with the lighter bullets and a thump with the heavier bullets. For punching paper at the range I've gone to 6.0gr PP.
The small difference in the PF is probably only on paper. I don't think if you were to shoot someone they are going to shake it off because your PF was down a few points!
Looking to the government for good guidance on anything will more often than not get you in trouble.