Just took a look at my first year maths notes (their only use now is to prop up my laptop and give it better circulation), here's what you'll learn to do with matrices in Semester 1:
* Basic matrix operations (as you mentioned -> add, multiply, inverse, transpose, determinants)
* Using matrices as representations of linear systems (basically, just google "Gaussian elimination" and learn that (it's easy) and you'll be WAY ahead of everyone)
Later, in Semester 2, you learn something called "Linear Algebra" which is basically just a bunch more stuff to do with matrices (hence why learning them well is a good thing to do).
Finally, I assume the HSC taught vectors, but just in case it didn't, also pick up vectors (basically 1-column matrices) and a couple of things to do with them (dot product, cross product, lengths, etc.).
It's all really easy stuff but because there's so much information you need to learn, it all gangs up on you quite quickly - the stuff above makes up less than a quarter of the course, but form the basis (excellent, a linear algebra pun) of the rest of the material so it's much easier to pick up once you know the basic stuff.