You actually answered your own question.
Ron asked a great follow-up question because a specific answer would require telling us which specific polisher you're using. Since you posted this in a forum group for DA Polishers I'll assume, (something I hate to do), that you're using a DA Polisher.
IF that's then let me re-word your answer to be more accurate as to the benefit. You wrote,
you are right I am using a DA polisher
A smaller pad won't develop more power, it will just be able to use the existing power more efficiently.
Assuming you're using a DA Polisher which is a tool with a
Free Rotating Spindle Assembly, then a smaller pad will enable the this type of tool to better be able to keep the pad rotating which means it will be better able to remove small particles of paint, which is how you remove below surface defects like swirls and scratches.
This is especially true with first generation DA Polishers. With the new 2nd generation polishers like all the DA Polishes Autogeek currently offers, it's less of a problem because the new generation DA Polishers are ALL better at keeping pads rotating for correction work, even 6.5" and 7.0" pads.
would the 5.5 inch pad with the first generation DA polisher make it easier to remove defects than with a 6.5" pad?
Does that help?
yes.
Which polisher do you own?
I own a G110
Free Rotating Spindle Assemblies