A few of my friends were getting 40s-50s and maybe even lower than that, and they weren't happy...so they dropped. They love maths now. They're topping the whole grade (for General Maths). I don't know what marks they get though...probably 80s.
It boils down to this: If you're not willing to put in MORE effort to get into the 80s+ mark range, maybe get a maths tutor, practice, practice, practice...and you really hate your subject for whatever reason you have, drop. However, if you want to keep fighting and ARE willing to improve and really buckle down, then I believe you can improve enough to get a decent mark in the HSC. However, it depends on your work load, what subjects you do, whether doing mathematics is required for what you want to do (or what you might want to do), etc.
Our teachers told us that if you didn't get atleast 50% for the yearlies then you should drop down, because then if anything you'll get 70% MAX for HSC or something like that.
If you're really concerned and feel like giving up on the Mathematics course, change to General. It'll be pretty easy for you to top the grade if you weren't even failing Mathematics.