Notice your elbows-forward/chest-up position when unracking -- exemplary. Prior to rep 1, however, this position degrades significantly. This laxity is making it harder to push the bar backwards into your face, creating the moment arm between bar and shoulders that you're feeling on the grindy reps.
A quick fix for this might be to press closer to the rack, that is, take less steps backwards. A half step works for me. A slightly higher eye gaze may also help reinforce the hips-forward, chest-up position.
Also, I wouldn't be surprised if the bar is inching toward your fingers during the set, as you finish rep 5 with a clean-like rack position. Ensure that you're squeezing the piss out of the bar until it's back on the hooks.