I've learned feedback is really invaluable. But I've also learned that getting good feedback is a problem entirely on my shoulders -- not on my testers.
I like to think of feedback as rare, magic water in a lake somewhere. You're thirsty and this magic water quenches any thirst no matter how much of it you drink. The problem is that if you try to gather it yourself, it goes away entirely.
The only way to get the water is to convince people to bring it to you. Some people bring it in a nice, clean cup. Most bring it in whatever they have lying around. And every now and again people bring it in an unwashed glass with stuff floating around.
I had a rogue-like that I needed tested. For those that don't know, a rogue-like is a game based on an game called Rogue where you only have one life and if you die that character gets deleted (among other things). Many times the testers would say, "Why don't you add more lives?"
It never worked out to just say "Well, it's a rogue-like and that's how it's supposed to be." Every time I attempted to give in to my ego and start defending some of my design choices, the feedback either stopped or became muddied.
I started getting better feedback when I tried not to defend anything. I had to nod and say, "Yeah, that's a good idea -- I'll think about it more" a lot. Sometimes I got actual problems that I fixed later. Every once in a while I had something I really needed to think about.
Some problems arose because some of my testers knew too much and others didn't. Testing another game, I got complaints about things I thought were basic -- "How do I fire a laser?" It was very easy, I told them, all you have to do is tap the screen. But people kept asking similar questions.
I had an epiphany -- they didn't know because nothing told them. And this went both ways. It didn't matter how fundamental having one life is or how basic firing the laser is. If no one told them how were they supposed to know? That's when I realized this is the kind of feedback I needed.
I wanted to know exactly what the player didn't understand or couldn't figure out. The best feedback I got was when I completely dropped the players into the deep-end with only the game's description and kept my mouth shut.
The best feedback are the complaints you would have heard if you didn't test everything out properly beforehand.