1. The person in question might impart that he never cared much for that writer's works.
2. He or she may not have even heard of that writer.
3. The bothersome attendees might besiege him for autographs.
4. During the autograph session, no one might show up.
Fear is a valuable thing. It keeps us from running after armed robbers wearing only slippers and a thong. Fear sometimes gets us to the doctor's office, hopefully in time to catch that nasty affliction in the bud. In fiction, fear is primo character motivation and reader tingler. Fear, however, is not a very good career, marketing, or social counselor.
I went to my first SF convention in 1978 at the invitation of IASFM editor, George H. Scithers, who thought it would be good to meet my future readers (at that time, I only had one short story, "The Tryouts," that had been published). If the editor who had, at that point, bought all of the stories I'd sold (about seven then) hadn't invited me to the con, I wouldn't have gone. The whole idea terrified me. What if no one had read my story? What if everyone had read it and couldn't wait to dump on me? When I got there, I found that writers and fans were issued the same style of nametag. There was nothing on it that said I was a writer!
How will they find me? But do I want them to find me? No! Yes! No!
Well, the panic subsided, I learned a bit of humility, made some friends who still like me, met a lot of readers only a few of whom had read my story, got pumped up, and came away charged up with story ideas and enthusiasm that lasted until the next con I attended where my batteries were recharged.
At NorthAmericon in St. Louis, Editor Scithers convinced me to do a Q&A talk on writing as a program item. There were three hundred or so attendees, the session went on for four hours, and was I ever charged up. From that session were the origins of my writing instructional, Science-Fiction Writers Workshop-I, as well as a love of doing talks on writing and writing workshops at conventions. In later years, this evolved into my online writing course, The Write Stuff, which became my how-to book of the same name. There are many stories and novels I've written that can, in whole or in part, trace their origins to conversations I've had or panels I've been on or attended at conventions.
If you haven't tried a genre convention, please do so. Besides joining the most interesting part of the human race, cons are a well of ideas and enthusiasm, you get to meet editors, agents, and your fellow writers, and most important of all, you get to meet your current and potential readers. You say you only write mysteries? Most of those who attend the science fiction and fantasy convention Readercon in Burlington, Massachusettes are also big mystery readers. Just saying.
It's getting to know you, and you getting to know them. Yes, it helps sales, but that's not really the most important part. You will discover entire new rooms within yourself simply packed with exciting things to do, to think, and to write. It's a lot more fun than hiding in your home office 24-7.