The convention in strength training is total weight: all the plates + the bar (+ the collar where the collar is significant and/or for those who like to calibrate exactly). So you count all of it. Not just the plates. Not just the plates on one side.
"Add 5 lbs" means + 5 to the weight being lifted.
Simple, right? But here's the catch: sometimes people will say "add a 5." In this case it means add a 5-lb to each side. This is why you should always double check how your bar is loaded when working with other people. They can hear you wrong, misinterpret, grab a 5-kg instead of a 5-lb plate or simply be math-challenged.