Only if you can actively/consciously* consider it.
* Which does, as you imply, make the word "qualia" a circular reference.
I fail to see how it could have a circular reference.
The sentence "This sentence is false" would have a circular reference if we could think of it as referring to itself. But sentences don't refer to anything by themselves. Instead, we have to do all the work of referring. We have somehow to decide for ourselves and to imagine what it is we mean the sentence to refer to. So, the sentence "This sentence is false" does not refer to itself because sentences are not the kind of things capable of referring in the first place. And then the question becomes whether such a sentence can make any sense to you as somehow referring to itself. So if there is any circularity it's in the conception some people choose to have.
Qualia are the quality of your experience. So there has to be experience for there to be qualia. Knowing any quale is just another expression to say that you are conscious of it. However, the term "consciousness" is ambiguous, so the notion of being actively conscious only brings unnecessary ambiguity to the issue. The merit of the term "qualia" is precisely not to assume any uncertain ontology. Qualia are just what you know because they are what you experience. I don't see any circularity in that.
I also see no harm in believing that qualia may exist outside our experience of them. In fact, that's obviously what we do when we assume that other people are also experiencing qualia. So if you're not actively conscious of being cold, it might be that some part of your brain is nonetheless conscious of it, and it also might be that something is experiencing the quale of being cold even if you're not yourself conscious of it. Again, no circularity in that. What would be wrong, however, would be to pretend that we know that it's the case. We may believe it, we don't know it.
EB