I have read very little since leaving school, and read as little as I could scrape by with for school. I'm not poor at reading, though, as I was actually a voracious reader as a child. However, this inclination died down in adolescence for reasons I still don't know. Here is my experience with the OP.
Fiction is what I read the least of - I haven't finished even one fictional book in a few years now. Again, I don't understand what the aversion is truly about, which is why I'm putting my fractured stabs at it out here.
For one thing, reading a story is an emotional commitment. If I put down a book when I am in the middle of it, the mood of the unresolved conflicts will color the rest of my day. This leads me to be choosy to the point of avoidant tendencies when approaching fiction. Perhaps this is an emotional regulation issue. Or hyperfocusing - switching gears between activities has always been an ordeal for me. Due to this, the restriction to popular fiction doesn't apply to me - it would be the very last thing I'd invest this kind of energy into.
Another difficulty is feeling like I've gotten nothing done if I spend my time reading fiction. That it is impractical. When I close the book, I don't see what I have to show for it. Nothing new has been created, for all those hours. But then again, I hardly do anything more productive with my days anyway - I've come to the conclusion that although I am not literally a substance of gambling addict, there is very little that I do, ever, without an element of self-medication to it these days. Unless my material survival depends on it, I don't ever finish things - I just suck a hit of something out of their novel colors and leave them behind for the next thing, unfinished and dry. It's disappointing to realize I don't know what the something is, but that's the truth - and books do not have it. Everything is a distraction from something else. Books are perhaps the most concrete example, as between loving them and forgetting them, I spent time in an intermediate stage where I picked up a ton of books I didn't finish or, in the case of some, even start. Eventually, I stopped trusting myself with these purchases at all.
If one thing can be distilled from my anecdote, it's that the books aren't the one with the problem.
Occasionally I read nonfiction, but am very picky about that - with a lot of those books, I feel like they would be 80% shorter if the irrelevant information and anecdotes were taken out. It has to be extremely to the point and relevant to a pressing personal issue or interest. In a way, I guess textbooks are my ideal nonfiction reading.