About Me

When not at work with students, I spend my time in my room either reading, calculating something using pen and paper, or using a computer. I read almost anything: from the pornographic to the profound, although my main interests are mathematics and physics. "When I get a little money I buy books; and if any is left I buy food and clothes." -Erasmus

Sunday, December 12, 2010

Unmasking charlatans with fake scientific credentials

One of the downsides of having the internet is that anyone can post all sorts of claims. If you happen to believe in strange claims, (such as miracle cures, ways to make a thousand dollars every hour, etc.) there will be websites just for you. I think that this is hard on the elderly, since a lot of them have not developed enough filters to separate the true from the crap.

If you go to the websites of charlatans (I have a certain "doctor" in mind), you will find lots of things. If the charlatan claims to be an expert, then he will have to produce a C.V., and that is the best place to start. Do not read his claims; instead, check out his credentials.

The C. V. should contain where he had his schooling, his listed refereed publications, his work experience, and other information that should support his claim of expertise. A charlatan will pad his C. V. because he must make an effort to look like an expert. A real expert's C. V. will have all of the above and more, so on the surface, it will be hard to distinguish the real from the fake.

This is where google comes in; if he claims to have received an outstanding research fellow award from a Newton Hall Foundation, then type "Newton Hall Foundation" in the google search field. Include the quotation marks to indicate that google should search for the whole phrase. Without the quotation marks, it will search for each of the keywords, and the webpages produced may contain only two out of three, or one out of three of the keywords.

If you try doing it, you will (as of today) generate only two hits from google. (If I happen to be unlucky, it may rise to three after this posting.) From the hits I generate, I can easily conclude that the said foundation exists only in the imagination of the charlatan. An imaginary honorary degree should raise warning bells. If he can lie about this, then he may be lying about other things as well.

If an Oxford College of Applied Science is listed, anyone familiar with the Oxford and Cambridge university system will know that a College in Oxbridge is more like a dorm than what we think of as a college in my country. A College is not a degree-awarding body in Oxbridge. (In the Harry Potter universe, Hogwarts would be analogous to Oxford University, while the houses such as Gryffindor and Slytherin would be analogous to the colleges). So an honorary degree from an "Oxford College of Applied Science" is also a warning signal for the knowledgeable. A simple google search with keywords "Oxford College of Applied Science" will produce only two lists of webpages.

Another test is to look for scientific publications. It's not enough to claim that clinical studies have been made. Anyone can claim that. Instead look at the research papers that he cites. Do the said papers actually exist? Does the journal actually exist? Use google to check. If the journal exists, another filter is to check if it's in the list of academic journals compiled by ISI (Institute for Scientific Information), a citation indexing service. You can also check out the papers cited; if the paper has been refereed and is being read by other experts, then it should be cited in other academic papers.

Articles from newspapers (even major broadsheets!) are insufficient. Many journalists do not have scientific training, and I suspect that many of them do not understand the difference between scientific papers from crap. Of the two hits produced by a search for "Newton Hall Foundation" (with quotation marks), the first one is from the website of that imaginative doctor, while the second one is from a major broadsheet. The journalist involved should have at least verified that the so-called foundation actually exists!

Why the emphasis on refereed articles in scientific journals? A scientific paper must go through a process. After the paper is written and submitted to a journal editor, it must go through peer-review. The paper is read by other experts known to the journal editor, and the claims are checked. Only after the paper is approved by the referees will the paper be published.

Although the process may still allow crap to go through, it does reduce the amount. Referees are often overworked, and it is difficult to replicate everything, especially if the paper reports experimental results. A large number of citations by other scientists should increase your confidence that the paper is correct. (Unless, of course, it is cited by papers on scientific misconduct.)

If the paper's results are replicated by many other groups, then your confidence in the correctness of the paper should rise. Conversely, if other groups are unable to replicate the results, then you should doubt the results. This is the mechanism through which scientific misconduct can be discovered; fabricated data will be impossible to reproduce. As an example, Schon's papers were eventually retracted because other researchers could not reproduce his work. (Schon was eventually found guilty of scientific misconduct.)

Testimonials are the least reliable basis for scientific belief. Memory is faulty, and a lot of people are unable to distinguish between causation and correlation. There are a lot of reasons for doubting testimonials, too numerous to go into here. (Maybe a separate blog post?) For example, religious belief is based on the most part on testimonials; if testimonials were reliable, there would be no multiplicity of religions in the world.

So check and recheck. There is a lot of crap on the net; one should always upgrade one's bullshit detector. A defective bullshit detector is dangerous to your health.

No comments: