IMO seems a bit silly to drop to 3 if you don't have anything lined up or can cope with the work load. Your putting yourself behind early, and if you do get an internship/grad offer, your going to force yourself to overload because you haven't got a summer session to fall back on.
Maybe wait until your 3rd/Final year before doing it. If you don't get an internship, take a unit over summer so you can do 3 subjects in semester 1 of your final year (when grad applications and interviews occur). If you do get an internship, then you know your going to finish on time without overloading.
My grad offer is conditional on me graduating by a certain date. There were two other interns I know had to turn down their offers because they would not complete by that date. Basically, they'll have to do grad applications all over again at the start of next year during their last semester. If they don't get mid year intake, that could mean 6+ months without income because they can't get Youth Allowance and may not have a job anywhere else.
But of course it's up to you.