the thing about physics is that it is so broad, it is impossible to understand every concept completely, and they don't expect you to anyway, but if you want to UNDERSTAND, then you have to do your own research, keep asking questions etc.
with chem, there is understanding, calculations, essay style questions, logic everything. its kinda like bio and physics combined :s if that makes sense.
if you're interested in chem and have a good logical sense, i advice you to do chemistry.
imo unlesss you LOVEEE physics, chem>physics.
hope that helps =]