School isnt just about academic education. Alot of it is education about real world stuff, if you are only getting your "life education" from only one person who has lived through it, how accurate is that going to be? Learning from 20-30 or more adults is far better in this regard.
Socially, most homeschool kids are weird. It isnt their fault, but they havent had the sort life socialising with peers that normal kids take for granted and so they have trouble fitting in. My mums best friend had 4 kids that we played with sometimes, all nice kids and bright, but they had trouble socialising, they did and said weird things and they were a bit arrogant because all they have been told their whole lives is that they are fantastic and bright, instead of being brought back to earth by other kids in a school.
The other thing is that i would say the vast majority of home schooling parents dont do it because they think they can provide a better education, they do it because they are a fundamentalist wackjob that even most of the religious population would find weird and they want to instill those crazy beliefs in their kids. Also they want to "protect" their kids from the world because it is a scary place with people who want to kill you, rape you and bring you down. What a fantastic view to hold and pass on to your kids.
The fact is that even if your kid is exceptionally bright, and you are a genius and a fantastic teacher, well versed in multiple fields of study and can provide your child with a top rate [and unbiased] academic education[and honestly, how many homeschooler parents can say thats them?], it still probably wont be the best choice because your kids will be sorely lacking in real world and social skills. They are missing out on the best part of school.