I've never bought Acrylics... so I wouldn't know.
But this goes for all paints... The grade of paint is important. If you buy student quality you will have a cheap in price but cheap in colour. The pigments will be made from dyes that can fade, etc... but you can buy Artist quality which will give you the best use of the paint and will be resilient, however the price can be quite high. You can choose between Student and Artist quality for all paints, including Acrylics. The Artist quality Acrylics aren't that bad all and at times they can give the same strength in colour as oils do.
A tip in choosing the quantity of the paints is to figure out the colours you will use the most. White usually goes quickly so you might have to stock up on that, but for some colours you might find yourself using hardly any of it. So I recommend buying slightly more White than the rest, have red, blue and yellow at the same quantity and keep your ammount of black slightly lower. Doing this will stop you buying too much of a unneeded colour. I don't know the mLs of Acrylics so I can't be that specific. So for Acrylic I would recommend you getting White, Black, Red, Blue and Yellow. But check if its a 'warm' or 'cool' version, and pick which will suit you. There are some affordable Artist quality paints around that won't break your wallet.
For Oils... the colours you want are anything really besides buying 'hues'. If you want a range in tones, hues will not give you it. The basic 3 I use are Titanium White, Burnt Umber and Prussian Blue. Mixing Prussian Blue and Burnt Umber will give you a equivilant to black. Best not to buy black for black, cause it will absorb everything. You can use a colour like Ivory Black, etc... but only for really rare touches. Also if you are tempted to buy a Ptahlo colour, don't they are super permaneant... not subtle at all. If you don't want your painting to develope a strange tinge then don't buy Ptahlo anything. For colours the best is to decide on either warm or cool colours and pick a red, blue and yellow that suits you. Other colours like Green and Orange are easy to create, colours you can buy like Viridian aren't that great as its best to mix your ternary colours. However Violet can be a problem as thats pretty hard to get right.
For Canvas... the price is based on three things. The quality of the wooden frame, canvas material and the primer they use to coat it. The cheaper grade are ok for Oil paintings but for Acrylics they are risky. Cheaper canvas will be a weaker than the expensive and you run the risk of piercing the canvas. Probably won't happen but you never know... So the thing to test it out is to hold the canvas to the light with your hand behind it, the harder to see your hand, the better. It will help show the strength of the material and primer. You can buy different fabrics for you canvas, but best to just use the standard surface, especially if you are using Acrylics.
For Watercolours... its up to personal preference, pick what colours you need. But here are the standard ones I use; Payne's Gray, Vermillion, Raw and Burnt Sienna, Raw and Burnt Umber, Aureolin Yellow, Winsor Blue, Antwerp Blue and occassionally French Ultramarine. As watercolours tend to be quite messy when mixing with other colours, you should only mix two colours... anymore and the resulting colour will be muddy and impure.
There are essentially two types of Watercolour aka Aquarelle paint, your standard watercolour or the other called Gauche (there is water based oils but don't worry about them, cause they are very hard to use). Watercolour is more translucid and will be even more so with water, while Gauche is more of a watercolour and acrylic hybrid... so the colour when used with water will still be vibrant and not toned down as much. A little Gauche goes a long way.
Watercolour Supports are very varied. You can get Watercolour Canvas, which will require at times lots of paint but has the upside of being able to rub out your mistakes with water. However the primer can start to resist the watercolour (which is really bad) and the canvas is expensive. Whilst the most common option is paper. The cost is determined on the material, surface and the heaviness. For material, its best to get the cotton based one, for surface it depends. If you are doing detailed work then the "smooth" surface is best but if you are doing landscapes and ones with less detail then "rough" is better. "Medium" is really a compromise between the two but isn't really tailored for detail, just finer landscapes. Heaviness is worked out on the GSM (grams per square metre) of the paper. You will find numbers like 180 GSM, 200GSM, 240 GSM, etc... GSM is really just a indicator of the absorbing quality of the paper. The higher the GSM the more water the paper will hold, whether this is good or bad depends on the individual. You will also notice the thickness of the paper, but really this can be misleading, so don't worry too much about it.
Sorry I got carried away, thats all you need to consider I think. Don't have to read it all, just the sections that concern the medium you are using.
As for places that are affordable... it really depends. I always use this independant store in Hornsby that also has a student card feature which takes off 10% or so, which helps.
