The Girl With X-Ray Eyes

imageNatasha Demkina, a young girl living in Saransk, Russia, began to receive a lot of media attention around the middle of last month. It started with an article in Pravda, which hailed her as the 'Girl with X-ray vision'. You see, Natasha possesses the unusual ability to peer through human flesh and spot diseases and injuries that are lurking unseen within people's bodies. Or, at least, this is what Pravda claimed. It didn't take long for more newspapers to catch onto the story. The British Sun has been the most relentless about pursuing it. They've actually flown Natasha to London and are now parading her around like some kind of weird curiosity. Does Natasha really have x-ray eyes? Well, I doubt it. But I'm sure The Sun is going to milk this for all it's worth.

Health/Medicine

Posted on Tue Feb 03, 2004



Comments

I commend you, Fomalhaut (a well-known star in the Southern night sky - I take it that you are into astronomy) on your computer programming skills, but your statistical reasoning needs to go one step further. In Statistical testing, we begin with the presumption that there is nothing of interest in the data. Here that refers to Natasha guessing. We then ask, what is the chance of getting a result as good as the one actually found or better by coincidence? Here that is the probability of getting 4 or more right, i.e. (70+21+1)/5040, which is 1 in 55. The reason we include the 'or better' is that by guessing Natasha could conceivably have gotten 5 or more right, though this would be far less likely.
Posted by Miland Joshi  on  Thu Feb 17, 2005  at  02:44 AM
Fomalhaut, the easy way to see the fault in your reasoning is this: Imagine that she had looked at 1000 people, not 7. Then the probability of any one single result (correct out of 1000) would be small. Hence a probability less than 5% would prove nothing, as we'd expect results better than random guessing about half the time, and all of these would individually have a probability of < 5%. Hence claiming 95% confidence would be plain silly. Looking at the probability of results as good or better than she achieves solves this problem.

Cheers,

Ross-c
Posted by Ross-c  on  Thu Feb 17, 2005  at  06:30 AM
Thanks for the comments. I think see the point of the 4-or-more bit now.

But I'm puzzled that Andrew Skolnick, posting here on 9 Dec 04, quotes Professor Ray Hyman responses to Puck's first (wrong) estimate of the probability:

"The second mistake this critic makes is to use the probability for getting exactly four correct matches. The number that is relevant for our test is the probability of getting four or more correct matches. Contrary to this persons assertion, the probability of getting exactly four matches in our test is .01533 and not 1/840 (.0012) as he claims. The relevant probability is the probability of getting four or more correct matches which is .01899 (rounded to .02 or 1 in 50)."

Now, by my calculations, the probability of getting exactly four matches is 70/5040 or 0.01389, not .01533. And the probability of getting four or more is 0.01818, not .01899. These are slight differences, but differences all the same. So how did they get their figures? By using a formula out of some book, it seems. What is this formula, and how was it derived?
Posted by fomalhaut  on  Thu Feb 17, 2005  at  01:10 PM
Fomalhaut, I wrote a quick program to calculate the probabilities, and I get exactly the numbers you do. 40/5040 for exactly 4 right out of 7, and 92/5040 for 4 or more right out of 7.

I wondered if the original calculations were done using the binomial distribution (which would have been a silly thing to do, but I gave it a go). Those numbers are much further off. So, I have no anwer as to where their numbers came from.
I'd guess that they made a calculation error, but could be wrong.

Cheers,

Ross-c
Posted by Ross-c  on  Thu Feb 17, 2005  at  02:31 PM
Ooops! That 40/5040 was a typo. It was 70/5040.

Cheers,

Ross-c
Posted by Ross-c  on  Thu Feb 17, 2005  at  02:52 PM
Ross, I'm glad we got the same answers. Perhaps the explanation for the discrepancy between our figures and CSMMH's lies in Andrew Skolnick's 9 Dec 04 post, from which I've already quoted, but which includes these remarks:


"The problem we are dealing with is known as the matching problem. The mathematics for calculating the correct odds is not self evident. Indeed, it is very complicated. I painstakingly worked out the correct probabilities using the formulae in Frederick Mosteller's Fifty Challenging Problems in Probability With Solutions. I believe this is still available from Dover Books. The critic might find it useful to carefully follow the argument in this book. My other source was Hoel, P.G., Port, S.C., and Stone, C.J. (1971). Introduction to Probability Theory. This latter source provides some useful approximations for those who do not have the patience to calculate the exact probabilities. Richard Wiseman was able to check my probability calculations using tables provided by the Journal of the Society for Psychical Research. Our probabilities agreed."

So perhaps the figures they produced were "useful approximations" rather than "exact probabilities" for a "very complicated" problem. And the problem is indeed very complicated - or at least extremely tedious - if attempted manually. But it is quite easy to solve using a simple computer program (as you found out). This rather suggests that the probabilities were calculated using methods appropriate to a pre-computer era, and which gave approximate rather than exact answers. The 1971 book on probability cited is some 35 years old, and thus dates from an era when computers were monstrous beasts that lived in university mathematics departments, and so it was most likely written for use by people who did not have access to such computers.

I'm neither a mathematician nor a statistician, but it seems to me that people who are working with probability and statistics really ought to be able to calculate their figures with an accuracy appropriate to the era in which they live. I find myself slightly raising an eyebrow that CSMMH apparently lacks the computing skill to correctly calculate probabilities (even if in this case it is relatively inconsequential). And I find myself wondering what else they can't do.
Posted by fomalhaut  on  Thu Feb 17, 2005  at  05:50 PM
Hmmm.... Interesting. I didn't use any formulae to calculate the probability, my program just enumerated all possiblities, and counted the matches for each one. It would be possible for both of us to agree on incorrect results if we both made the same incorrect assumption.

On the page:

http://www.ds.unifi.it/VL/VL_EN/urn/urn6.html

the matching problem is discussed, and the formulae given for calculating the probabilities of matches. I wrote another program to calculate the probabilities of various matches for the 7 people on the tv show. I get ...

ross@home$ ./matching
For k=0 b is 1854
For k=1 b is 1855
For k=2 b is 924
For k=3 b is 315
For k=4 b is 70
For k=5 b is 21
For k=6 b is 0
For k=7 b is 1
ross@home$

Or, 70/5040 for k=4, and 92/5040 for k >= 4. I.e. the same as both of us got with our previous programs.

The page mentions a "Poisson approximation", but this is for very large matching problems, not 7. Also, even if the problem had been solved by hand, it wouldn't be that difficult. So, there's probably more to this than it appears.

Though, in the end, the differences in the probabilities are very small.

Cheers,

Ross-c
Posted by Ross-c  on  Fri Feb 18, 2005  at  02:22 PM
Hi Cranky Media Guy

You said: It's painfully obvious that you have drawn your conclusion without any actual EVIDENCE and that you are not going to let any EVIDENCE change your mind.

I don't agree with what you say. If you look at her credentials you can see that she followed several studies and trainings. With some of them you can "earn" a official grade and with some of them not. Some of them are universal studies and some of them are not. When I read the credentials I understood that the grades in Atmospheric Physics, Theology and Philosophy are official.

Now, someone here told me that the universities where she got the grades in Theology and Philosophy are not regocnized in some states. He showed me some sites. I didn't really understand those sites but I think he told the truth by saying that these universities aren't regocnized by some states.

My opinion about this case is that only because a school is not regocnized by some states doesn't mean that you don't learn anything during your studie at that university. As far as I know, after reading his post, there are only a few states that don't recognize the universities. So most states DO recognize that universities. Because of that I think she has the right to say that she holds a doctorate in Philosopy an Theology. As far as I know from the most states do.
Posted by Marlon  on  Sat Feb 19, 2005  at  07:16 AM
Hi Ross-c

You seem quite reasonable to me. I also find it very fair that you say science isn't perfect...

About the experiment.I don't thinkt this is a fair, and an accurate experiment to test for the ability of seers to see the auras, and therefore that the aura exists.

The seers could make appointments with eachother.

For example thay can say...If the person has blond hair and wears glasses, we say that the person his aura is torned on the left side. If the person is a man and is bold, his aura is red, with a lot of grey and green. Ore they could say, if the person has brown eyes his aura is very large and bright...

And so they could make many appointments.
And if no one nows this, it seems to be that they ARE able to see aura's.

So, this test would be worhtless if you want to prove that some people can see aura's and that the aura exists.

I hope I have made my point claar about this

Marlon
Posted by Marlon  on  Sat Feb 19, 2005  at  11:37 AM
Marlon,

First, can I apologise for getting your name wrong in one of my answers. The font I was reading this in at work wasn't that clear.

OK: About your answer concerning the experimental design. Yes, the experiment is flawed as it would permit the seers to cheat. However, it's actually much worse than that.

In the initial stages where the seers are agreeing on everything and checking out each other's abilities, there is the possibility that they could agree to cheat. But, a far more subtle problem is that the seers might adjust their decisions to be consistent without realising it. Hence, there would be no concious cheating, but an unconcious cheating. I think it's important to consider the possibility that this unconcious agreement is a potential cause for consistency.

I believe that it's problems like these that cause a lot of the problems between "scientific" and "alternative" types. Imagine this scenario: the three seers spend quite some time together, and without realising it, start to synchronise their predictions. "hmmm... both A and B say the aura is detached here, is there something that I'm missing ...". They perform the experiment, and consistency is found. Then, someone accusses them of coming to an agreement ahead of time, i.e. cheating. The seers know that they haven't (conciously) cheated, and hence their beliefs in their ability to see the aura are strengthened even further, as they're being told that they "must have cheated", but they know that they didn't. Hence they are now *really sure* that they have the power. That such implicit effects can occur seems reasonable given the research into the necessity for double blind (not just single blind) experiments.

In terms of science being imperfect. I work at a uni myself both teaching and researching. About two years ago I decided to "upgrade" my knowledge of stats and experimental design. Hence reading books such as "Methodological errors in medical research". What I'm slowing coming to understand is how poor people's abilities are in designing and interpreting experiments in quite a number of fields. Problems in medical research are better known than most, because of the importance of medical research being correct. Even then I've just been reading how hospitals have been using medicines to prevent heart attacks which the research suggests will increase the survival rate. Except that it isn't, and poor research is quoted as the most likely cause. In other sciences, there is much less pressure to ensure that research is actually accurate, and I personally believe that there are "fields" of science where much, if not most, published research is unreliable.

I'm not claiming that I'm good at stats (yet). But, in my field, I seem to be in the situation where I know that I'm not yet really good enough, but I'm better than (it seems) many other people. This is scary.

Cheers,

Ross-c
Posted by Ross-c  on  Sun Feb 20, 2005  at  02:50 AM
Hi Ross-c

I realy don't matter you got my name wrong. That can happen to anyone.

I find the things you have written very interesting. I can learn something from you. I will read it again. Hopefully we can all learn something from eachother.

Thank you,
Marlon
Posted by Marlon  on  Sun Feb 20, 2005  at  09:32 AM
JoeSixpack, we were required by the producer/director to bend over backwards as much as possible to accomodate Natasha Demkina's requests (as long as we didn't allow her to circumvent the main controls of our test). To accomodate her, we told her that she could take as long as she needed. We NEVER imagined that she would take more than four hours! It was a miracle that the subjects put up with it for so long. I was afraid that one or more would have had to leave. They were volunteers and remarkably patient and generous with their time. Natasha takes about 10 minutes to give a person a reading of virtually every organ in their body. So you can imagine why we were surprised that she took more than 4 hours to find just six clearly defined organ targets when told what to look for and where to look in the body. As for her use of a cell phone to send text messages, we never imagined she'd do that and we were not looking for it. She did it during breaks. Did she use the phone to have someone look up information about any of the target medical conditions? We do not know. As for your doing as well as Natasha, don't underestimate her. She's had more than 6 years of practice. You can read the test protocols on the Commission for Scientific Medicine and Mental Health's web site: http://www.csmmh.org/demkina BTW, please by me a lottery ticket.
Posted by askolnick  on  Sun Feb 20, 2005  at  11:15 AM
Marlon says:

"You ("skepps") say that the " believers" are avoiding evidence etc. but in much cases (and also in this case) I notice that the "skepps" are avoiding things!"

Yes, we try to avoid false statements of facts and making irrational arguments.

She says, "You only respond on the "lacks" in our stories. You don't see the story as a whole and don't look at our whole experience."

A premise based on false facts is not a true premise. A belief based on error is in error. A belief based on a lie is a lie. Marlon's solution for avoiding reality is to look past contradictory facts in order to embrace the conclusion she wants to believe.

Marlon continues: "That's the problem with the whole world. A lot of times we are focused too much on details. Focussing on details is oke but not at te cost of our picture of the whole. In my opinion everything would change if we would concentrate more on the whole."

By "details" Marlon means "facts." When facts get in their way of our beliefs, she says we should look past those troubling facts and just embrace the beliefs.

She then offers us this falsehood: "If we give then thousend reasons why we believe or know that the paranormal excist, you find that reason that is the 'weakest' in your opinion."

Marlon hasn't been listening. Skeptics have not been asking for a "thousend reasons," they've been asking for at least ONE sound piece of evidence to prove the existance of the paranormal. Not thousands. Even just one piece of evidence that can stand up to rational examination and be verified would do it. What we get instead are thousands of flimsy claims, testimonials, pseudoscientific experiments, outright fraud, covered with a myriad of excuses and obfuscations. What we want Marlon is ONE. Not thousands. Just one. Give us your best. We don't want you're "weakest." Give us one irrefutable piece of convincing evidence.

Marlon does not understand why we're asking for this, because she believes the paranormal is real. For her, belief comes first, and evidence is just the icing you put on the cake. "I am sure it excists," she says. And for her, that's all that counts.

Sorry, Marlon, that's not good enough. If we stuck with beliefs instead of facts and reason, we would still be fighting smallpox by casting out demons and burning witches instead of finding out the true cause of the illness and then eliminating it from the world.

Marlon says, "Reality has everything to do with experience. In kind of a way everything you experience is reality. We shouldn't focus so much on whats real and what's not."

I remember "Son of Sam" (David Berkowitz) who experienced voices from his neighbor's dog commanding him to shoot and kill young women and their boyfriends who were necking in their cars. Wow! I guess those voices must be real afterall. In this case, I think we need to set David Berkowitz free.

All in favor say, "Aye! I'm crazy too."
Posted by askolnick  on  Sun Feb 20, 2005  at  11:50 AM
Tali Karoola says that the CSMMH-CSICOP investigators called Natasha Demkina "deluded." That's not true. We did not call her "deluded."

Cranky Media Guy, Marlon doesn't even have a pair of deuces in her "hand." But she believes she's holding the winning hand and, in her view of reality, a good belief will trump any real hand of cards. It is because she wishes it to be. She makes her own reality simply by believing it so. Don't you wish you could do that?
Posted by askolnick  on  Sun Feb 20, 2005  at  12:03 PM
Marlon said, "Now, someone here told me that the universities where she got the grades in Theology and Philosophy are not regocnized in some states. He showed me some sites."

That's not what I said. I said those "universities" are non-accredited diploma mills and the degrees they offer are bogus.

And she says, "I didn't really understand those sites but I think he told the truth by saying that these universities aren't regocnized by some states."

The only reason she doesn't want to "understand those sites" (which are simply lists of schools offering bogus degrees that are posted on Michigan and Oregon state web sites) is because they contradict her "reality." Marlon is never interested in any truths if they disagree with her beliefs. And it comforts her to believe that her teacher has doctorate degrees in theology and in philosophy and that the B.A. degrees her teacher's school offers are also real. Any honest, rational person would be troubled to learn that their teacher's credentials are bogus. But Marlon is never troubled by facts, when she's got her beliefs to keep her happy.
Posted by askolnick  on  Sun Feb 20, 2005  at  12:32 PM
askolnick said:

"Cranky Media Guy, Marlon doesn't even have a pair of deuces in her "hand." But she believes she's holding the winning hand and, in her view of reality, a good belief will trump any real hand of cards. It is because she wishes it to be. She makes her own reality simply by believing it so. Don't you wish you could do that?"

Yes and no. Yeah, it's tempting at times to wish that I could believe in stuff just because I want it to be real. After all, I had 12 years of Catholic school; I was taught a LOT about things that don't have any basis in fact. I confess that there's a (small) part of me that might wish that I could just turn off the section of my brain that says, "Oh yeah? Show me!"

The thing is, though, that I just don't want to be a "mark." I see so much in contemporary life that's based on lying to the public at large. I see intelligent people actively working at suspension of belief so that they can rationalize their political convictions. It's sad and, I believe, dangerous for society to ignore reality. That way lies madness and collapse. Nazi Germany was based, in large part, on unsupportable beliefs.

Ever seen the play, "Rhinoceros?" The protagonist, whose name I forget, finds everyone around him turning into a rhino (I'm told it's supposed to be a metaphor for Nazism). Part of him wants to conform but he just doesn't have it in him to be a rhino too. I don't mean to sound pompous, but sometimes I feel like that.

My local Fox TV affiliate lead their 10 o'clock news the other night with a "story" about a local "psychic" who said that a missing woman was buried in a shallow grave in a wooded area! Uh, you know any murdered people buried in a DEEP grave in the middle of town? This is especially sad when you know that a majority of Americans say that they get all or most of their news from local TV newscasts. I just don't want to be one of the rhinos who believe in "psychics."
Posted by crankymediaguy  on  Sun Feb 20, 2005  at  02:27 PM
I can sympathise with people's need to believe. I used to have a favourite cough medicine, which seemed to work better than others. Hence, any cold and cough, and out I went to buy it. Then, it's reported that proper investigation of cough medicines revealed that they were all useless. So, now when I have a cough, there's nothing to do. Still, I found myself tempted when I last had a cough, just so that I could do *something*.

Cheers,

Ross-c
Posted by Ross-c  on  Mon Feb 21, 2005  at  03:35 AM
Askolnick, I see you have returned to the thread.

Ross-c and I discussed last week some of your earlier posts concerned with calculating the probablities of the matching problem. We concluded that the probabilities had been slightly miscalculated.

Are you able to state exactly how the probabilities were calculated?
Posted by fomalhaut  on  Mon Feb 21, 2005  at  07:53 AM
Fomalhaut, I would like to point out that I thought that either the probabilities were incorrectly calculated, OR, that both you and I had made the same incorrect assumption. That is possible, and maybe Askolnick could discuss what model s/he used.

In any case, the variations in numbers are small. I think it's important not to get too mired in the details and lose sight of the big picture.

Cheers,

Ross-c
Posted by Ross-c  on  Mon Feb 21, 2005  at  08:40 AM
I don't think we miscalculated the probabilities, Ross. Several other people have reached the same figures as us by different computing methods.

I think we simply used accurate computing methods that the testers did not. It's fairly clear, from Slotnik's earlier postings, that they used another less accurate approximation to arrive at their numbers.

The only question is: what was that approximation method? And why did they use that rather than compute the figures accurately?
Posted by fomalhaut  on  Mon Feb 21, 2005  at  10:39 AM
Oops. That should be "Skolnik" above.

And I think the little details are part of the big picture.
Posted by fomalhaut  on  Mon Feb 21, 2005  at  10:46 AM
Formalhut, I don't understand why you're obsessed with this non-problem. The source of the odds we used was explained to you. One of your previous posts answered this question. The odds for the test were worked out by Prof. Ray Hyman and Prof. Richard Wiseman. Here is what Prof. Hyman wrote -- and you quoted:

"The problem we are dealing with is known as the matching problem. The mathematics for calculating the correct odds is not self evident. Indeed, it is very complicated. I painstakingly worked out the correct probabilities using the formulae in Frederick Mosteller's Fifty Challenging Problems in Probability With Solutions. I believe this is still available from Dover Books. The critic might find it useful to carefully follow the argument in this book. My other source was Hoel, P.G., Port, S.C., and Stone, C.J. (1971). Introduction to Probability Theory. This latter source provides some useful approximations for those who do not have the patience to calculate the exact probabilities. Richard Wiseman was able to check my probability calculations using tables provided by the Journal of the Society for Psychical Research. Our probabilities agreed."

What earthly reason are you obsessed with the minor difference between the approximate odds Profs. Hyman and Wiseman calculated using tables published in books and the more precise odds you calculated by brute force using a computer? A difference that makes no difference is no difference. Would you critize someone for using 3.14 instead of 3.141159 to calculate the circumference of his swimming pool?

If as you say the little details are part of the big picture, why should anyone trust YOUR comments. Not only did you mispell my name, you mispelled it again when correcting your error! It's Skolnick.
Posted by askolnick  on  Tue Feb 22, 2005  at  09:00 AM
Excuse my typo. I meant to type "3.14159".
Posted by askolnick  on  Tue Feb 22, 2005  at  10:20 AM
askolnick, looks like you're the one with pi on his face. 😉
Posted by JoeSixpack  on  Tue Feb 22, 2005  at  10:37 AM
Good one, Joe.
Posted by askolnick  on  Tue Feb 22, 2005  at  01:02 PM
The joke was begging to be made. I couldn't stop myself.
Posted by JoeSixpack  on  Tue Feb 22, 2005  at  01:41 PM
O.K. But next time you get that urge again, stop, take a deep breath, and count to pi.
Posted by askolnick  on  Tue Feb 22, 2005  at  01:44 PM
"take a deep breath, and count to pi."


I only counted to e.
Posted by JoeSixpack  on  Tue Feb 22, 2005  at  01:54 PM
A Skolnick wrote: "What earthly reason are you obsessed with the minor difference between the approximate odds Profs. Hyman and Wiseman calculated using tables published in books and the more precise odds you calculated by brute force using a computer?"

I wouldn't call it a 'brute force' method. It's the method outlined in your earlier postings, but dismissed as 'unwieldy' (which it is, performed manually). I think Puck upthread used a brute force method - of generating 10 million answers and scoring them - in order to arrive at answers less accurate than mine and Ross's.

But in answer to your question, I guess I'm puzzled why approximate probabilities were calculated using what appear to be 35-year-old pre-computing methods, rather than accurately calculated with a computer. Are the professors unaware of such computing solutions? It doesn't take long to write the program.

Natasha's was a high profile case, with a TV programme to go with it. Pretty big bucks, I'm sure. In those circumstances it would appear appropriate to have allocated a few bucks to get some bullet-proof mathematics - particularly for a test that principally depended upon calculating probabilities. Or put it this way: I think that an an extraordinary case demands extraordinarily accurate mathematics.

A Skolnick wrote: "A difference that makes no difference is no difference. Would you critize someone for using 3.14 instead of 3.141159 to calculate the circumference of his swimming pool?"

Well, I might criticize.

But why, if "a difference that makes no difference is no difference", did you bother to correct your slightly incorrect value for pi? If it made no difference, why correct it?

A Skolnick wrote: "If as you say the little details are part of the big picture, why should anyone trust YOUR comments."

There's no need for anyone to trust me. It's not me who's just spent thousands of bucks running a televised high profile test, without having bothered to accurately work out the fundamental probabilities involved.

It's like someone all dressed up smartly for their wedding, but with a gravy stain on their necktie. Sure, it's a minor detail. No part of the big picture. Most likely nobody, least of all the mother-in-law, will notice.
Posted by fomalhaut  on  Tue Feb 22, 2005  at  07:42 PM
Hi everyone. This is part 1 out of 2!

It's true that the difference in probabilities is very small, and does not affect the result. It is easy for scientists to know that the difference in probabilities is very small, but this is beyond the understanding of much of the general public. All they know is that someone is pointing out that "the scientists got it wrong" in some small detail, and this is something for proponents of pseudoscience to start working on. (Piltdown man is still quoted by creationists). Any excuse that can be pounced on to allow them to maintain their beliefs seems enough. To debunk pseudoscience, the debunking needs to be more than just "good enough" but damn near perfect.

I am concerned about the use of tables that gave an approximate result. The alternative method (I posted a link earlier on) which calculates the probabilities exactly would be easy to complete by hand by anyone using a calculator with a factorial button. Mathematically, the differences to be a non-issue, I would still like to know why tables were used. It allows people to start quibbling about details, and quibbling about details has a risk of burying the main point.

I would like to ask whether the people who designed the experiment are aware of (and the need for) such things as p-value adjustment, and the situations where bayesian statistics suggests that the confidence suggested by "standard" statistics are unreasonably high. I personally think that there is grave risk that debunking is going to be counter-productive for the following reason: No matter how good the statistical experiment, there is always a non-zero chance of the wrong result occurring by chance. Natasha could have been 100% correct by random chance. If the paranormal is tested over and over and over again, then eventually one trial will come up supporting the paranormal. And when that happens, the single result will be trumpeted far and wide by paranormalists, and the other experiments conveniently forgotten about. This can happen even with properly designed and analysed experiments.
Posted by Ross-c  on  Wed Feb 23, 2005  at  04:59 AM
Part 2 of 2

In medicine the standard level of support required is 95% confidence. Even if that confidence was exactly correct, and (say) the hypothesis that homeopathic remedies have no more effect than a placebo is true, we'd still expect one in 20 trials to support homeopathy having an effect, of which about one in 40 would show improved effect over placebo.

We'd expect one in 40 trials to support homeopathy. Experimental or design flaws could raise the probability of a "significant" result (misconduct such as incorrect experiment design and/or analysis, to outright fraud), so we'd expect more than one in 40 trials to support homeopathy. Add the tendancy to report "positive" results more often than "negative" results, and we'd expect there to be fair proportion of published studies supporting homeopathy, even if there is no effect over and above placebo. If people select only those publications that support homeopathy, then you get a long list of publications saying that homeopathy works!

I believe that it's important for society to make sure that rubbish claims are exposed for what they are. But, doing so is extremely difficult, with little chance of making much of a benefit ("I don't care what those scientists say, rubbing powdered rats testicles on my tummy cured my diahorrea within days"). Debunking has to be done with as few flaws as humanly possible, or opponents have an excuse to try and argue away the results of the debunking so that they can go on doing what they are doing and/or believing what they are believing.

I would be prepared to volunteer *some* time for checking the design of experiments intended to evaluate, erm, "paranormal claims", where there is no profit motive. I wouldn't put myself forward as the sole person, but I could at least do some double checking.

Cheers,

Ross-c
Posted by Ross-c  on  Wed Feb 23, 2005  at  04:59 AM
Ross wrote: "It is easy for scientists to know that the difference in probabilities is very small, but this is beyond the understanding of much of the general public. All they know is that someone is pointing out that "the scientists got it wrong" in some small detail, and this is something for proponents of pseudoscience to start working on. (Piltdown man is still quoted by creationists). Any excuse that can be pounced on to allow them to maintain their beliefs seems enough. To debunk pseudoscience, the debunking needs to be more than just "good enough" but damn near perfect."

Hey, I've already said that I'm not a mathematician. I'm a member of the general public. So it was a member of the general public that pointed out that "the scientists got it wrong" in a small detail by calculating the approximate rather than exact probabilities.

But I didn't do that in order to give ammunition to proponents of pseudoscience. I did it because I don't accept that science is always unconditionally right, and everything else is rubbish, pseudoscience, and wrong. I don't think the world is divided into a high priesthood of clear-eyed, knowledgeable scientists and a mass of dumb, ignorant, and gullible proles.

And also I don't think this is an an either-or matter of either Natasha has X-ray vision or she's a a blatant fraud. From what I saw she seemed quite impressive, but I didn't believe for a moment that she actually had X-ray vision. I simply thought that she was probably a sensitive, sympathetic, highly intelligent, and probably well-read girl who had something of a gift for looking people over and detecting signs of infirmity or disease. There are probably thousands, if not millions, of doctors who take one look at someone and see the symptoms of disease X or Y or Z. Some people are just better at doing some things than others. Some people play chess very well, some people can perform astonishing feats of mental arithmetic, and some people have extraordinary memories, and so on and on and on. And in many cases, this is a natural aptitude. There's nothing paranormal about this. It doesn't threaten the foundations of science that this is so. After watching the programme, I half felt that the whole issue had been hyped up by both sides into a titanic collision between science and superstition, when it was actually nothing of the sort.

If it had been down to me, I guess I would have presented Natasha with several hundred people, all suffering from some known disorder (but unknown to Natasha), and seen how many she got right. There wouldn't have been a "pass mark". There would have been an assessment ranging from "hopeless" to "excellent", and let statisticians decide what scores translated to which. If she did very well, I'd have thought no more about it, and maybe have recommended that she become a doctor or something.

And that, funnily enough, is exactly what she's now doing at Moscow university.
Posted by fomalhaut  on  Wed Feb 23, 2005  at  08:58 AM
Hi people,

I
Posted by Marlon  on  Wed Feb 23, 2005  at  09:13 AM
Part 2


I said: "You ("skepps") say that the " believers" are avoiding evidence etc. but in much cases (and also in this case) I notice that the "skepps" are avoiding things!"

Askolnick said: Yes, we try to avoid false statements of facts and making irrational arguments.

Ok, I get the point. But who says that your definition of something being irrational is true. Something being irrational ore not isn
Posted by Marlon  on  Wed Feb 23, 2005  at  09:14 AM
Part 3

You said: Marlon does not understand why we're asking for this, because she believes the paranormal is real. For her, belief comes first, and evidence is just the icing you put on the cake. "I am sure it exists," she says. And for her, that's all that counts.

I admit that I am sure the paranormal exists. That means that it is just as real and useful as the physical reality. So when you use my definition of the word fact (so fact ore not has to do with experience) you can say that the paranormal is a fact for me. What does happen is that
Posted by Marlon  on  Wed Feb 23, 2005  at  09:15 AM
Part 4

Askolnick says: Marlon hasn't been listening. Sceptics have not been asking for a "thousand reasons," they've been asking for at least ONE sound piece of evidence to prove the existence of the paranormal. Not thousands. Even just one piece of evidence that can stand up to rational examination and be verified would do it. What we get instead are thousands of flimsy claims, testimonials, pseudoscientific experiments, outright fraud, covered with a myriad of excuses and obfuscations. What we want Marlon is ONE. Not thousands. Just one. Give us your best. We don't want you're "weakest." Give us one irrefutable piece of convincing evidence.

Ok. I understand. You want one evidence. I can
Posted by Marlon  on  Wed Feb 23, 2005  at  09:16 AM
Part 5

Ok, my final worth on this is something I have said before. It all has to do with experience. It really has to do with faith, with surrendering. Without faith in each other we are nowhere. You surrender to science
Posted by Marlon  on  Wed Feb 23, 2005  at  09:17 AM
Hi Formalhaut, you are wrong about Puck's statistical calculations. They were way off. You still claim to be puzzled that Profs. Hyman and Wiseman would use published tables to approximate the odds for this matching problem rather than a computer to caculate the odds more precisely. The simple answer is that more precise numbers were NOT needed because we were going to round the odds off to 2 decimal places anyway!!! Your argument makes no more sense than an obsessive-compulsive mathematician who would criticize an architect for using 3.142 to calculate the circumference of a circular dome rather than calculating pi out to 100 places! A difference that makes no difference is no difference. We use maps all the time to find our way around -- despite the fact that they are imprecise representations of the actual landscape! Profs. Hyman and Wiseman used published tables to calculate the odds. You may not agree, but their calculated odds were good enough for our purposes. Seeking higher precision for no reason, I think, is a sign of a compulsive disorder. You may like to argue exactly how many angels can dance on a head of a pin, but I'd rather debate whether or not angels exist.
Posted by askolnick  on  Wed Feb 23, 2005  at  09:26 AM
Marlon wrote too much to reply to for the time I have, so I'll just comment on one thing he wrote:

"Several decades ago it wasn
Posted by askolnick  on  Wed Feb 23, 2005  at  09:42 AM
Rocc-c said;

"It allows people to start quibbling about details, and quibbling about details has a risk of burying the main point"

Ross, the fatal flaw in your logic is that you think that if enough uncertainty is removed from the test, the pseudoscience believers will have nothing to hide behind. Nothing could be further from the truth. They will always find a fig leaf for their naked emperor. There will always be some bit of minutia for them to quibble over and take issue with. If not this slight discrepancy over probability, then the test conditions, or the "negative energy from the sceptics". These charlatins and frauds are always good at passing the "tests" their believers give, but when a sceptic wants to test them, they become prima donnas, whining and complaining over every protocol untill the experiment is fuzzy enough for them to dodge any real judgement.
Posted by JoeSixpack  on  Wed Feb 23, 2005  at  11:06 AM
fomalhaut said;

"I don't accept that science is always unconditionally right"

Science can be neither right or wrong. Science is a method of discerning truth(and has proven to be the very best method throughout history). Scientists, on the other hand can be wrong, but that is usually because they don't follow scientific method properly.

You also said;

"...I didn't believe for a moment that she actually had X-ray vision. I simply thought that she was probably a sensitive, sympathetic, highly intelligent, and probably well-read girl who had something of a gift for looking people over and detecting signs of infirmity or disease."

fomalhaut, I have a knack for being able to identify within a hundred miles or so where a person grew up based on their regional accent. I picked up this skill when I was in the US navy because I met people from all over the US, and because I was a good listener. There is nothing paranormal about this, as many people have this "gift". I would be a liar, however, if I were to tell people I knew where they grew up based on their "aura". This girl is either lying or delusional because she is claiming to use supernatural powers when she is using good 'ol empathy and smarts. That's the issue here.
Posted by JoeSixpack  on  Wed Feb 23, 2005  at  11:30 AM
I think Joe Sixpack's points are all solid and right on target. To add to his point about the test not convincing believers: We had no expectation of doing so. We knew they would have many quibbles they could cling to because our test was only a preliminary examination to see if we found was evidence that would justify a more carefully controlled study of Natasha Demkina's claims. The test and the rest of our investigation yielded no evidence that would warrant further study. What Natasha appears to be doing is simply providing people with medically-focused "cold readings."

Those educated enough to know how science works know that it's up to the claimant to prove a claim. It's not up to others to disprove it. We were willing to examine the evidence she could provide. We did so and we found that it does not support her claim that she can see organs and tissues inside of people's bodies.

I especially like Joe's comment on science not being right or wrong. To put it another way, science is nothing but a flashlight that we can use to help us find our way in the dark. We can use the flashlight to find things that help us. We can use it to find things that hurt us. And some people, usually believers in the supernatural, try to use the flashlight of science to hunt for ghosts, goblins, and the other mythical entities. If they look long enough, believers always seem to "find" them in the shadows. A curious thing though: these supernatural entities never seem to appear in the actual beam of the flashlight, so that the rest of us will also believe. They prefer to lurk safely in the shadows of people's imagination and delusions. Funny about that.
Posted by askolnick  on  Wed Feb 23, 2005  at  12:41 PM
Joe Sixpack:

Actually I did think (and say, perhaps not clearly enough) that true believers will always find some sort of a fig leaf to hide behind. See my comment on powdered rats testicles. However, there may be some more sensible people who might be prepared to consider both sides. It's for these people that the scientific approach needs to be watertight.

The true believers will always find something to quibble on. But, the more watertight the science, the less justified the quibbling. And, the less justified the quibbling, the more likely it is that third parties will conclude that the quibbling is unfounded.

Cheers,

Ross-c
Posted by Ross-c  on  Wed Feb 23, 2005  at  02:21 PM
Ross-c said;

"Actually I did think (and say, perhaps not clearly enough) that true believers will always find some sort of a fig leaf to hide behind."

I understood your point very well, I think maybe it's me that wasn't so clear. Sorry.

I was speaking of her defenders ability to overwhelm the "...more sensible people who might be prepared to consider both sides... " with either irrelevent quibbles over protocol or outright denial of the validity of the test.

The "sensible people" you hope to sway are often ignorant of things like the significance of a small discrepancy of probability and are bewildered by the ammount and complexity of the arguments the frauds are able to make. Quite frankly, they have too many other things that need their attention during the day, so they are likely to "keep an open mind" to the possibility of pseudoscientific claims.

The charlitan has the advantage no matter how water-tight the experiment because it's easier to spread manure than it is to clean it up.

I spelled your name right this time.
Posted by JoeSixpack  on  Wed Feb 23, 2005  at  03:17 PM
a)

Skolnick wrote: "you are wrong about Puck's statistical calculations. They were way off."

Wrong. They weren't way off. They were exactly right.

The exact probabilities are:

0 - 1854/5040 = 36.79%
1 - 1855/5040 = 36.81%
2 - 924/5040 = 18.33%
3 - 315/5040 = 6.25%
4 - 70/5040 = 1.38%
5 - 21/5040 = 0.42%
6 - 0/5040 = 0.00%
7 - 1/5040 = 0.02%

These were the (correct) probabilities posted by Puck T Benson, on Thu Dec 09, 2004 at 10:58 PM:

0 36.79%
1 36.81%
2 18.33%
3 6.25%
4 1.38%
5 0.42%
6 0.00%
7 0.02%

which you promptly declared to be wrong, but "not as grossly wrong" as in his previous "attack" - when they had been wrong.

b)

Skolnick wrote: "You still claim to be puzzled that Profs. Hyman and Wiseman would use published tables to approximate the odds for this matching problem rather than a computer to caculate the odds more precisely. The simple answer is that more precise numbers were NOT needed because we were going to round the odds off to 2 decimal places anyway!!!"

Wrong again.

On Thu Dec 09, 2004 at 06:02 PM, Skolnick wrote: "the probability of getting exactly four matches in our test is .01533 and not 1/840 (.0012) as he claims. The relevant probability is the probability of getting four or more correct matches which is .01899 (rounded to .02 or 1 in 50)."

The probability of four matches is 0.0138, not 0.0153. Rounding these to two decimal places gives 0.14 and 0.15. Not the same. Also the probability of getting four or more right isn't 0.01899, but 0.01825, but which does luckily happen to round to 0.02.

c)

Skolnick wrote: "Your argument makes no more sense than an obsessive-compulsive mathematician who would criticize an architect for using 3.142 to calculate the circumference of a circular dome rather than calculating pi out to 100 places!"

"Obsessive-compulsive" is some psychological term which has no place in a discussion of mathematics. Mathematicians are usually concerned to get accurate results. This isn't a psychological defect.
Posted by fomalhaut  on  Wed Feb 23, 2005  at  03:47 PM
JoeSixpack wrote: "Science is a method of discerning truth (and has proven to be the very best method throughout history)."

I don't think science ever "discerns truth", but only at best gradually approximates towards it. The best Ptolemaic science of 500 years ago had the sun going round the earth. Copernicus had the earth going round the sun. Kepler discovered their orbits were elliptical. Newton showed how gravitational forces resulted in elliptical orbits. And so on. Views are in constant transition. Today's science is always being disproved or shown to be limited or restricted by tomorrow's science.

JoeSixpack wrote: "This girl is either lying or delusional because she is claiming to use supernatural powers when she is using good 'ol empathy and smarts. That's the issue here."

But Natasha Demkina hasn't claimed 'supernatural' powers, but instead some different way of "seeing". I think that when a doctor looks at a patient, or a mathematician at a set of figures, they are "seeing" them in a different way than most people see. It's not supernatural. And perhaps that's all that Natasha means - she looks at people differently.

I don't see that this should have been a question of whether Natasha "sees" in some unusual way like "X-ray vision", but how good she is or isn't at diagnosing medical conditions. If she did well at that, only then would one move on to discovering how she does it - which probably is something like "good ol' empathy and smarts". First things first.

As it is, I'm not sure what this test has shown at all. The more I think about it, the more it looks like a way of making a provocative documentary which will sell well, and more about sales than science.
Posted by fomalhaut  on  Wed Feb 23, 2005  at  04:31 PM
Where to begin, fomalhaut? You start of with a critique of science, saying;

"I don't think science ever "discerns truth", but only at best gradually approximates towards it."

I got news for you, Every useful modern convenience is the direct result of men and women who used science and the scientific method. That computer you type at, the vaccinations that kept you from getting whooping cough as an infant, the aeroplane you fly in to go visit family far away, the light bulb you have sitting over your desk, penicillin, MRI machines, the telephone, and on and on.

What have the psychics done for us over the past 500 years? Let's see, there's the daily horoscope in the newspaper (for entertainment only), lot's of ex post facto "predictions", and lot's of completely bogus claims that don't stand up under the slightest scrutiny. In a word, nothing.

So if you want to claim, "science doesn't have all the answers", you better come up with something else that DOES have some useful answers to back that up with.

Next, you claim that;

"But Natasha Demkina hasn't claimed 'supernatural' powers, but instead some different way of "seeing"."

That's where you're wrong. She claims to be able to see inside her "patient". Here's what she says in two seperate Sun articles;

"I can see inside the human body"

and in a seperate interview;

" Describing her power, Natasha said bluntly:
Posted by JoeSixpack  on  Wed Feb 23, 2005  at  10:16 PM
Formalhaut, you are wrong, wrong, wrong, as well as obsessive-compulsive. Puck T Benson's original calculations were way off. After being corrected, he posted the ones you cite. In fact, your obsessive-compulsiveness over the miniscule differences in odds makes me wonder whether, after being embarrassed, Puck adopted a new screen name and returned here to keep arguing about the meaningless differences in the odds calculations. I wouldn't be surprised. It certainly would explain your obsession with a difference that is so minor and irrelevant.

Incredibly, you state: "The probability of four matches is 0.0138, not 0.0153. Rounding these to two decimal places gives 0.14 and 0.15."

Helllloooo! Do you ever bother to read what you write? Rounding 0.0138 and 0.0153 to two decimal places gives us 0.01 and 0.02 -- NOT 0.14 and 0.15. You keep making such a fuss because the odds we cited for the test were a miniscule 0.00074 off from the odds you calculated using a better method. And here you are, making calculations that are off by 1400% and 750%! Obviously, you're not the right person to be giving others "math lessons."

And you are so very wrong: Science certainly does concern truth, as Joe wrote. The examples you give to prove otherwise show exactly the opposite of what you want to prove.

As you said,"Copernicus had the earth going round the sun." Contrary to what you would like us to believe, this is as true today as it was when Copernicus first announced his theory. Science has discovered nothing in the past 5 centuries to change that truth. And scientific discoveries are highly unlikely to ever find that the earth does not travel around the sun.

And you are so wrong about what Natasha Demkina claims, it makes me wonder if you bothered to learn anything about her before coming here to enlighten us with false information.

Natasha claims to have a supernatural power to see organs and tissues inside of people's bodies. And that's the claim we tested.

We expected there would be people like you who would try to defend Natasha by obfuscations and arguing that we tested the wrong thing -- that Natasha never said she can actually see inside of people's bodies. Sorry, Fonmalhaut, but I'm here to stop such obfuscations and deception.
Posted by askolnick  on  Wed Feb 23, 2005  at  10:25 PM
In fact, Formalhaut, your obsessions and the style and content of your writing are so similar to "Puck T. Benson's," I believe that I was right when I predicted that he would slink away and return using a new screen name after his foolish mistatements were exposed and discredited. We never heard from Puck again. Now here you are, sounding exactly like him, harping on the very same points like a cracked record.
Posted by askolnick  on  Thu Feb 24, 2005  at  06:23 AM
JoeSixPack wrote: "I got news for you, Every useful modern convenience is the direct result of men and women who used science and the scientific method. That computer you type at, the vaccinations that kept you from getting whooping cough as an infant, the aeroplane you fly in to go visit family far away, the light bulb you have sitting over your desk, penicillin, MRI machines, the telephone, and on and on."

You're talking about technology rather than science. And most of technology is produced by inventors and angineers, not scientists. Sometimes engineers use science, sometimes they don't. For example, the Wright brothers weren't scientists, but inventors. The scientists showed up later. Same with early steam engines, all produced by enterprising engineers. Thermodynamic physics came along when they wanted to improve engine efficiency. For the most part, scientists don't invent anything. They simply open doors for enterprising engineers. And, equally often, enterprising engineers oopen doors for scientists.

And I don't give a hang about psychics. I'm not interested in them. I didn't see Natasha Demkina as a psychic, but simply as someone who seemed to be quite good at diagnosing diseases. I took her claims of "seeing" into human bodies as a figure of speech. The word "to see" has many meanings. Like when someone explains something to me, and I say, "I see what you mean." Well, of course I don't literally "see" anything. Do you see what I mean. No, you probably don't, because you take the word "see" literally. I don't believe that Natasha quite literally sees inside human bodies.

But if that is actually what she's claiming, then I have no interest in her. That's definitely paranormal. And I'm not interested in the paranormal. You don't seem to have noticed that I'm not actually a believer in the paranormal and the supernatural. I guess you can't "see" it.
Posted by fomalhaut  on  Thu Feb 24, 2005  at  06:32 AM
Comments: Page 2 of 15 pages  < 1 2 3 4 >  Last ›
Commenting is not available in this channel entry.