Psi Experiments website announces the launch of its third psi experiment

I feel bad about never finishing and sending you my discussion about your experimental design. I've been completely snowed under at work. The problem, I think, lies not so much conscious efforts to memorize as unconscious recognition leading to some success in the task. Things like the precise registration of the edge of the card with the pixel positions can influence the look of the image. Furthermore, anyone determined to break this protocol can do so pretty easily -- same pixels in the image means the same card.

Because feedback is the "statistics" over 20 trials -- not trial by trial feedback -- it will take a lot of runs to accumulate enough information, whether consciously or unconsciously, to significantly bias the results of a run.

This experiment is not rigorous -- but unlike the first experiment it is, I think, interesting. You cannot eliminate these effects -- and so cannot say psi occurred -- but the alternative explanations are of low enough probability (not tiny, by any means, but fairly low) that its reasonable, and rational to speculate that if psi exists and has the characteristics found in other, more rigorous, experiments then there is some reason to suppose that a positive result might be psi operating.

Nothing to convince a skeptic (after all Bryan has already stated elsewhere that thousands of successful, highly rigorous experiments are irrelevant since some experiments have failed to show an effect; if those experiments won't convince a fairly reasonable skeptic like Bryan, there is little chance that anything you do would convince the kind of "skeptic" I most frequently run into), but something interesting.

We are in the area of Bayesian statistics or "subjective probability". The statement "...explanations are of low enough probability" is a totally meaningless one in traditional (frequentist) probability theory. You can attach whatever probability you want to the alternatives (deliberate or unconscious use of information about repeated targets) given the assumption that psi exists (which is what I was speaking of). That assumption is necessary since this experiment cannot be treated as evidential of psi.

My thinking was that given the subtlety of the information transmission and the apparent difficulty of deliberately decoding it there was a reasonable chance that effects might be due to psi if we assume that psi occurs and has the characteristics previously demonstrated. The question is how likely (a "Bayesian prior") is it that someone would put the large effort necessary to significantly distort the overall results by cheating? Even if it ends up fairly high, say .75 cheat, .25 psi (remember this is contingent on an assumption that psi exists and is, though small, not uncommon under these circumstances) then we can still say things like "If this is psi then it is interesting that ...". In the first experiment this was not possible because the probability of call bias overwhelming any psi effects (under the same assumption) was so high as to make contingent statements about psi effects meaningless.

However, this is no longer, I'm afraid, my stand on this experiment. I have found an easy method to cheat on this test not requiring very many runs but producing a significantly high hit rate than.

As for your question "Is this the only case where flaws were dismissed for being of low probability...?" I'm not sure what you mean. There is always the possibility in any experiment in any field whatsoever that an experiment might have been marred by very low probability flaws. For example, any experiment's results (positive or negative) can be attributed to an elaborate conspiracy by a powerful outside organization intent on producing false results, who break into the lab or the equipment supply room and tamper with the equipment, samples or whatever. Those low probability, unknown, special reasons are why we do replications (when replications are actually done -- much rarer in real science than in textbooks).

Here though, the probability is much too high (even in my original evaluation) to admit the results as evidence for the existence of psi. Given reasonable prior doubt about the existence of psi, any deviation from chance expectation in this experiment could only be seen as evidence of conscious or unconscious sensory contamination.

When I talk about the experimental evidence for psi I am speaking of a body of highly rigorous experiments that are consistent with the psi hypothesis -- i.e., that falsify the conventional assumptions about the physical isolatability of information.

As to quoting you. Sorry. I thought you were very clear, and multiple times. I didn't think my summary would be the least bit controversial, even to you. When I have time (I'm way behind in my day job) I will find one of the statements (and context) and quote it.

"'Parapsychology experiments have been shown to have, despite the greater difficulties they face, roughly the same inter-experimental consistency (i.e., replicability) as main-stream particle physics'. Been shown has it? Gee, was it shown in a journal of 'main-stream particle physics'"?

Yes and no. There is no more reason to demand that parapsychology results be publishied in main-stream particle physics journals than that biology results are. Its easy to dismiss a scientific result when your stand is "a failure to reject the results is sufficient evidence of incompetence (gullibility)." It makes things so much easier -- you don't have to bother actually doing any critical examination of the evidence.

Anyway, the Particle Data Group, an international collabrative of high energy physicists, each year publishes "The Review of Particle Physics" (previously the "Review of Particle Properties"), in which the summarize the current state of knowledge of particles, most especially their measured properties. It is published in collaboration with major physics journals. It is very mainstream, in fact it is the most heavily cited source in high energy physics.

In order to derive their statistical summary of the best estimates of the properties of particles they must evaluate and reconcile the many studies that bear on each property. To do so, they have developed procedures that describe the consistency of the body of literature in question so that they can reject those experiments that are inconsistent with the overall body of work. The results are surprisingly inconsistent and a fair amount of data is simply thrown out.

When those same exact techniques are applied to parapsychological experiments (by viewing, for example, ESP experiments as, essentially, measuring the rate of guessing) the consistency measures are found to be slightly better than in particle physics.

That is just a fact -- your opinion, or for that matter, mine is irrelevant. The foundation of all science is observed data. You can stand there all you want and say "The Earth is no warmer than it ever was" and the facts remain the same.

Of course, we can have disagreements about how to interpret the facts -- you could show, for example, why the analysis of consistency in parapsychology is deceptively high or that the apparent level of inconsistency in particle physics is somehow different from that in parapsychology. But unless you deal with the facts, and actually demonstrate a real difficulty in applying the analysis you are not making a rational argument -- just stating your prior belief.

"Did I not 'actually identify the flaws' right here?"

No, you did not. Using slightly more specific terms for "experimental flaws" does not identify the flaws. If you want to blame, for example, "inadequate controls" you need to identify in what way the controls are inadequate, that is, you need to show how the controls actually used do not rule out conventional explanations. Simply stating with only "proof by vehement assertion" that there must be something wrong with the controls is worthless and meaningless.

Watch, "Anthrogenic climate change is unproven because there is something wrong with the models it uses. That's proven since the details of the different models' predictions differ even if they all are wildly inconsistent with there being no meaningful effect of human activity on the climate". Ta da, I have just eliminated all the evidence for "Global Warning" in a stroke, without having to bother doing the work of examining the evidence and finding anything wrong with it. Never mind that the variation seen in the models is precisely what is expected given our current state of knowledge. I just need to bang the table and say that if the climatologists knew what they were doing all the models would agree completely -- the existence of any uncertainty proves that everything is compeltely uncertain and nothing is known. Anthrogenic climate change is a myth (I trust noone will quote me out of context on that nonesense).

"If I don't conform to your ideas of what a skeptic is and what skeptics do, well, we'll both just live with that."

I suppose so. You do conform to my ideas of, for lack of a better term I call "organized Skepticism", which is a social group who have adopted the label and postive reputation of philosophical skepticism (and generally firmly believe that its justified) but see no reason to adopt its requirements. They tend to sit around in magazines like SI and groups like sci.skptic and assure each other that since they are Skeptics that whatever they believe must be the product of critical thinking so they don't have to bother actually doing any.

Terms that others have used is "pseudo-skepticism", scientism, True Disbelievers, and simply Skepticism with a capital S. None of these capture the group dynamics which I think is essential to understanding the phenomenon.

Bryan, I "pretended to" nothing, and that you have decided to stoop to a personal attack on my integretity, its pretty clear that you are getting desperate. This is a classic pattern in debates between Skeptics and parapsychologists. The Skeptic keeps presenting their heartfelt beliefs bolstered by cherry-picked quotes, while the parapsychologist keeps presenting facts and logic and eventually the Skeptic ends up making unsubstantiated claims of lying, fraud and stupidity against the parapsychologist. Did I say unsubstantiated -- not unsubstantiated at all, its all substantiated by the fact that the alternative is that the Skeptic would otherwise be simply wrong about something he/she have utter and complete faith in.

My claim was that particle physics, specifically the measurement of the fundamental properties of particles (e.g., mass of the muon, the speed of the photon -- until that got set as a fixed quantity against which other quantities are measured, or the mass of the electron nuetrino) shows a high degree of inconsistency as determined by the particle physics community itself. That's a fact -- but it is a fact that conflicts with your own simplistic, sanitized view of how REAL SCIENCE works. Instead of checking the facts you attack me -- now there is rational, critical thinking for you.

As for the other, there is a term outside of logic for the rhetorical falacy you committed there, its called "bait and switch" (in logic the fallacy is called "equivocation"). We were talking about the corpus of parapsychological evidence and I pointed out that you were attacking it as flawed without any attempt of showing how it was flawed -- just making a broad unsubstantiated claim. You came back with a statement that you had specifically identified flaws. When I pointed out that you had not, you came back with "Oh, I wasn't talking about that, I was talking about something else entirely -- these informal, amateur experiments that we all agree were flawed. How foolish you are to think that I was talking about what we had been discussing."

You have not shown that I am a "pretender." What you have shown is that I should have known better than to treat someone who is interested in scoring points as if he were someone who, however much prior bias he has, is interested in a discussion that might lead to discovering some truth. My mistake was to treat this as a cooperative discussion rather than an adversarial one, and to share my thoughts with you after the few spurts of 10 or 15 minutes of work that I could devote to this.

You claim to be competent in statistics, yet you make a terrific offer "I'll offer up to $5000 in prize money as long as I don't actually have to risk more than $20 of it" (If you were competent, you would no that that is what the .05 level means in connection with your requirement that your expected loss under the null hypothesis could not exceed $1). So which is it: 1) You don't even meet Intro to Stats levels of competence in statistics; 2) This was deliberate Flim-Flam; 3) Sometimes when one shoots from the hip one says something that was wrong.

I'm not interested in sparring -- come back when you can show some manners and I may continue the discussion.

"As I wrote before, my theory is that psi does not exist, and that theory makes specific falsifiable predictions that should hold every time."

Once again, this is not the facts. Your theory makes specific falsifiable predictions that should hold every time. Psi says that those predictions can be falsified -- that sometimes predictions made on the basis of your theories are wrong. This has proven to be the case not just occassionally (all that is required) but frequently when enough care and attention are made.

You rejoin that the meer falsification of your beliefs should not count as a reason to set them aside. That there might be a vast conspiracy of parapsychologists so clumsy and stupid that they keep churning out flawed experiments but so skilled and bright that they have managed to hide these flaws from hundreds of rabid reviewers and observers over a century and more.

You are welcome to your belief -- but it ain't science. Doesn't it bother you that you use exactly the same arguments as those who claim no connection between smoking and lung cancer, no connection between HIV and AIDS, no connection between human activity and climate change, etc.? Anyone who wants to deny clear-cut scientific evidence has to use the same tired, invalid arguments.

Yet of the two of us, you are the one applying special pleading in your case against parapsychology. Sometimes you bother to justify the special pleading as not special at all, sometimes you don't. Real science in any field is not the simple, open and shut case you make it out to be.

And yes, those connections are demonstrable, just as the claims of parapsychology are. Those who oppose acceptance of those connections use precisely the same arguments you do: a statistical deviation from chance doesn't prove anything, some experiments that have been performed are have not stood on there own as unambiguous proof and a few have appeared to contradict the rest, those qualified to conduct the tests tend to concur with the "wrong" opinion and are therefore clearly biased and their results should be ignored, if it was true we would be in complete control of it (specifically used by opponents of the HIV/AIDS connection).

Of course, you know the difference: you are right and support truth while those other people are wrong and use those arguments to support lies.

If you are using "repeatability" in a loose, non-technical sense, then we are in complete agreement. If you are attempting to use it in the technical sense, then you are flat out wrong -- perfect repeatability is not now, nor has it ever been an evidential requirement. When available it is a powerful experimental tool, but it not a requirement. What is a requirement is replicability not repeatability.

Replicability means that experiments can be independently replicated with enough consistency to exclude it being a fluke. When one experiment replicates another it produces results in agreement with the first. Experiments are said to be "conceptual replication" when they produce consistent results using substantially different experimental procedures.

In the real world sometimes experiments contain some subtle flaw in its conduct or circumstances, whatever care is taken. For that reason experiments should be replicated before much reliance is put in them. Actually most experiments in most fields are never precisely replicated and a surprisingly many uncontroversial experiments enter into the "canon" without ever having been even conceptually replicatable. That is undesirable but is an unfortunate consequence of the fact that simply doing the same thing as someone else and getting the result that everyone expects is not real easy to get funding for and almost impossible to get published. But that doesn't apply to parapsychology where replications are routinely done and generally published.

There is no logical or practical requirement that every attempted replication succeed. It is the fact that well conducted experiments sometimes fail that creates the requitement for replication in the first place. What percentage of experiments fail to replicate the overall results naturally vary depending on what you are doing. A rate of about 30% is common -- for example in particle physics. Parapsychology generally does a bit better than that.

Repeatability on the other hand is the quality of being able to routinely produce the same results every time. It adds no evidential power. It is never the less immensely desirable -- the holy grail of experimental science. Why? Because it makes experiments that tell you about the phenomenon so much easier. Parapsychology along with many other fields has not succeeded in that -- although the auto-ganzfeld showed signs that it was close to being developed to the point that it qualified.

So you have confused "repeatability" which parapsychology doesn't have (nor does the active parst of, e.g., particle physics), with replicability which parapsychology does have. I am not engaging special pleading -- when you demand that parapsychology shows perfect repeatability unlike any other field and with no logical justification you are.

Just check some real sources in the practical philosophy or sociology of science, or check out the Particle Data Groups publications, or just talk to some real experimental physics. However much you loudly assert otherwise the half page in your high school physics text on "the philosophy of science" is not all there is. The real world of experimental science is much more complex and interesting. Your ignorance of it does not constitute an effective argument against parapsychology.

You continue to confuse replicability and repeatability. But ignoring that, you confuse how some experimenters out of frustrations have used the phrase "actively evasive" for psi." It has nothing to do with whether the experiments are open to the possibility that psi exits or not ("sheep" vs "goats" is a distinction made about groups of subjects in an experiment, it is not applicable to individuals -- including experimenters -- outside that context; your use of it as a term of disparagement amounts to ad hominim).

N-Rays: precisely -- N-Ray experiments met the criteria of replicability, though weakly. There was a disagreement about the reasons for those results. Critics didn't just say "there must be something wrong with those results -- the only experiments that count are the ones that are consistent with my beliefs". They identified a plausible explanation for those results other than literal rays and showed that the differentiation between successful and unsuccessful experiments was due to that explanation. That's called science as opposed to just saying "there must be something wrong with psi (or smoking/cancer or CO2/climate change or species variability/evolution etc)" which is faith.

Please cite these experiments that "consistently fail under skeptical controls." The history of parapsychology is pretty much taking even pretty extreme controls suggested by skeptics and showing that they had no effect on the results. Generally, of course, the controls on parapsychology experiments are tighter than some pontificating Skeptic says would be required.

I'm "still on that" because up to now you haven't said anything to even attempt to counter it except to deny the clear-cut facts. Now, on the other hand you are claiming that the labels of the discipline that we choose to apply to the experimental field is somehow important to the validity of the evidence. Apparently measurements of the mass of the muon is valid despite having the same or lower repeatability (repeatablility is a continuum by the way, not a dichotomy) as psi experiments because other experimenters who experiment with the existence of beta rays have the same label (particle physicist). That is a far more "paranormal" view of the universe than I'm willing to swallow, myself.

Re: High school text book -- I'm not arguing with your misapplication of the sheep/goat effect (the interesting but not terribly surprising experimental result that subjects score in conformance with how their beliefs motivate them) nor the speculation on the part of a few parapsychologists that psi is "actively evasive" (which has nothing to do with the belief of the experimenter). I was using that to describe the level of sophistication (very naive) of the philosophy of science you espouse so assertively.

As to falsifiability -- you continue to ignore my previous statements on that. Your beliefs (my old beliefs) have been thoroughly and completely falsified. Your insistence that evidence (not theory) contrary to your beliefs be "falsifiable" (whatever that means about evidence) puts your beliefs outside the range of falsifiable (i.e., scientific) hypothesis.

If you say, "There is no such thing as the sun" and I point up on a clear day and say "there it is, unmistakably". You cannot say "sometimes it is night and sometimes it is overcast and you can't point to the sun, since you don't accept that those failures your hypothesis is unfalsifiable and the times there is this big, intensely bright glowing thing in the sky must be due to some unidentified error and my belief that there is no such thing as the sun or anything like it is beyond question."

I'm not going to repeat this again. Unless you have something new to say about falsifiable I'm going to ignore your "proofs by vehement assertion."

"'You continue to confuse replicability and repeatability'

Years ago I did, but not here."

So you are saying that your continued use of the latter as if it were the former is what? ... deliberate deception?

-------------

You claimed a "consistent pattern" but instead of citing a survey you cite two isolated cases -- which don't even support your case. I'm not so stupid as to think that parapsychology is unique in the sciences in having no flawed experiments -- but you haven't even identified them.

Gellar was investigated at SRI. Putthoff and Targ allowed him to do some demonstrations, which they identified as mostly, clearly faked. While unconvinced by the remainder they felt that it was worth the effort to conduct an experiment under tight controls. They conducted the experiment and got positive results. They published the results including a caveat about the likelihood that Gellar would stoop to cheating but they saw no way that there controls would allow that.

Persei Diaconis responded that he had attended an informal demonstration by Gellar (with neither P or T present) and since the controls were inadequate then -- and that he thought that magicians tricks were being used in the demo that this somehow proved the inadequacy of P & T's controls. Bizarre logic but widely cited by organized Skepticism. One criticism of the P & T controls were published -- based on the assumption that since they did not explicitly say that they did something obviously necessary that they must have done something obviously wrong. P & T demonstrated from their records (including film) that they had done the obviously correct thing.

I would not put this high on my list strong evidence for psi (the evidential strength of any results with "extraordinary subjects" goes way down in my book, and I think most parapsychologists agree with me) but it does not illustrate your point. There were no controls put forward by critics that their single experiment lacked and there was certainly no attempt by them or anyone else to repeat the experiment with stronger controls at which point the effect seen disappeared.


Project Alpha -- no experiment at all and it didn't involve parapsychologists except as critics. Two physicists without any training in parapsychology were given a wad of money to study psi. The amazing Randi sent in some young magicians to fool them. His "Alpha Boys" did some demonstrations for the physicists who thought that there might be something worth testing. Randi unsolicited made some suggestions about controls at about the same time that the physicists brought film and descriptions to the annual Parapsychological Association meeting asking for confirmation that this was something of interest. The response was really negative -- I'm sorry to say even rude -- but it was made clear that there really was no evidence that anything real was taking place and were given some controls to include in the further demonstrations. About the same time Randi put forward some (but not all) of the same suggestions. When the parapsychologists controls were used the "Alpha Boys" stopped being able to perform and so no experiment was ever conducted.

When it was clear that the "Alpha Boys" were no longer accomplishing anything, Randi waited until the experimenters were out of the country at a physics conference and called a press conference where he revealed "all." He later gave a reward to the experimenters for having the intelligence to follow "his" suggestions.

Randi's account never says anything that contradicts this but leaves people a very different impression.

You're 0 for 2 in proving the existence of a consistent pattern.

I might be able to dig up a few cases of skeptics finding problems in parapsychological experiments that then proved to be the source of the results. The vast majority of criticisms though, as in any field, comes from parapsychologists. Most Skeptics criticism has, however, turned out to be either completely invalid or failed to explain the results.

-----------

I think you misunderstand, though I've said it clearly enough. Particle physics certainly has consistent results. For the vast majority of contemporary work on the properties of particles -- take your pick, the mass of any of the particles, their lifetime, etc. -- the consistency is gotten by removing the "outliers" amounting to about 30% of the studies. Parapsychology also needs to remove a (somewhat smaller) percentage of outliers to get a consistent measurement. The difference is that in particle physics there is not a vehement group of people who insist that the only acceptable value is one outside the range of demonstrated by the consistent majority of experiments.

Oh, there is another difference. Parapsychological journals have a policy of accepting articles for publication on the basis of the experimental design before the experiment is conducted so that even "unsuccessful" experiments get published. This is not true in particle physics.

If you know about this major group of current research in particle physics with virtually no failed experiments please post the reference here. Also inform the Particle Data Group -- they would be thrilled.

-----------------

Are you claiming that parapsychology needs to be the only experimental field that requires no skills and talent on the part of its researchers? It was Blackmore's foolishness that she was convinced that the only skill needed was a positive belief.

When we are talking about encouraging people to perform well at what appears to be a delicate task a positive attitude does seem to be a reasonable requirement. At least its felt that way in psychology (note -- no "para"). Clearly someone who manages to communicate that they would be rather displeased if there was a positive result can't expect the same response as those who are able to be honestly encouraging. The "experimenter effect" has been well demonstrated in psychology -- subjects tend to give experimenters what they want whether or not the experimenter has consciously communicated that to the subjects. Dealing with human subjects -- in any field -- requires certain hard to define skills in human interaction.

But this is not confined to psychology, sociology, anthropology, etc.

In physics, I think it was Fermi who was rather famous for screwing up any experiment he was involved with. To the extent that physicists used to joke whenever they started getting weird results that "Fermi must have entered the building".

-----------------

I've shown you the "sun" over and over again. You have replied that you will only believe it if it shows up when and where you want it to. That's not science.

I'm sure that you are sincere in your current belief that the demo would convince you. But I find it hard to believe that when "push comes to shove" that you, who so easily has dismissed thousands of large, well controlled experiments without ever looking at them, would look at this outcome and not just shrug and say "son-of-a-gun a 1 in 20 chance" or "I musta missed something."

Furthermore, if you did become convinced then that would be just one more person who was no longer a skeptic. By your logic that should have no conceivable influence on anyone else. It is that the experiment you personally designed convinces you. Science is about collective evidence not personal revelation. Why should I waste my precious time (a great deal of time and effort to be done properly) for your personal edification?

And no, you don't have to repeat yourself. I got the message -- you want personal proof not scientific evidence.