I definitely do not have an objection to fiction! I read both fiction and non-fiction and enjoy both immensely.
I'd say people have an idea about said fiction when they see someone reading it - I'm sure 50 shades of grey and twilight don't help this notion.
But personally, I love both and...