There are an endless number of studies like this, where they expose subjects to some kind of stimuli and then test their capacity to remember, learn, calculate or cooperate etc, and from the quantitative data, some form of scientific conclusion is drawn, and this data is then considered to be more or less trustworthy.
Fine. I have no problem with this, quite the contrary, I can think of a hundred similar studies I would like to see performed in this way.
It’s just that I think we are missing many pieces of knowledge that are quite readily attainable through development of the ability to observ one’s own thought processes, to actually study psychology using one self as the study object. That is not science in the general sense, but at the very least it can serve as a basis for dreaming up ideas for statistical examination – and I think this is how it is done many times. Many ideas might start with a person thinking ”this is how it works for me, let’s test it on a lot of people”.
The fact that something has not yet been tested on sufficiently many people though, is of course not a proof that a particular claim or hypothesis is not useful. Not relying on something on the basis that it is not properly studied yet, is perhaps a good strategy many times, but not always. I shall have to get back to this at a later time.