Sunday, August 14, 2011

Was there really a man on the moon? Are you sure?

Some weeks ago, the tree octopus made headlines again. If you had never heard of this creature before, don’t worry, it is an internet hoax used for classes on information literacy. It is easy enough to laugh about the naiveté of students believing in the tree octopus. Or people believing in spaghetti trees for that matter. Scientists in particular are obliged to carefully check all facts they use in their arguments. But in reality, none of us can check all the facts all the time. A lot of what we know is based on trust and an ethereal skill called ‘common sense.’ We’re born trusting adults tell us the truth – about the binky fairy. Most of us grow up adding a healthy dose of skepticism to any new information, but we still rely heavily on trusted sources and the belief that few people are willfully evil. What happened to that in the age of the internet?

When I write a paper, I usually make an effort to check that the references I am citing do actually show what they claim, at least to some level. Sometimes, digging out the roots of a citation tree holds spaghetti surprises. But especially when it comes to experiments, fact checking comes to a quick halt because it would simply take too much time putting under scrutiny each and everything. And then peer review has its shortcomings. In my daily news reading however I am far less careful. After all, I’m not being paid for it and I have better things to do than figuring out if every story I read (Can you really get stuck on an airplane’s vacuum toilet?) is true. Most of the time it doesn’t actually matter because, you see, urban legends are entertaining even if not true. And, well, don’t flush while you s it.

I think of myself as a very average person, so I guess that most of you use similar recipes as I to roughly estimate a trust-value of some online recource. The rule of thumb that I use is based on two simple questions: 1) How much effort would one have to make to fake this piece of information in the present form, and 2) How evil would one have to be.

How much effort would one have to make to put up a website about a non-existing animal? Well, you have to invest the time to write the text, get a domain, and upload it. I.e. not so very much. How evil do you have to be? For the purpose of teaching internet literacy, somebody probably believed he was being good. Trust-value of the tree-octopus: Nil. How much effort do you have to make to fake some governmental website? Some. And it’s probably illegal too, so does require some evil. How much effort would you have to make to fake the moon landing?

Of course such truth-value estimates have large error-bars. Faking somebody else’s writing style for example can be quite difficult (if it wasn’t I’d be writing like Jonathan Franzen), but depends on that writing style to begin with. If you’ve never registered a domain before you might vastly overestimate the effort it takes. And how difficult is it really to convince some billion people the Earth is round? (Well, almost.) Or to convince them some omniscient being is watching over them and taking note every time they think about somebody else’s underwear? There you go. (And Bielefeld, btw, doesn’t exist either.)

The trustworthiness of Wikipedia is a question with more than academic value. For better or worse, Wikipedia has become a daily source of reference for hundreds of millions of people. Its credibility comes from its articles being scrutinized by millions of eyes. Yet, it is very difficult to know how many and which people did indeed check some piece of information, and how much they were influenced by the already existent entry. The English Wikipedia site thus, very reasonably, has a policy that information needs to have a source. Reasonable as that may sound, it has its shortcoming, a point that was made very well in a recent NYT article by Noam Cohen who reports on a criticism by Achal Prabhala, an Indian advisor to the Wikimedia foundation.

There is arguably information about the real world that is not (yet?) to be found in any published sources. Think of something trivial like good places in your neighborhood to find blackberries (the fruit)1. More interesting, Prabhala offered the example of a children’s game played in some parts of India, and its Wikipedia article in the local language, Malayalam. Though the game is known by about 40 millions of people, there is no peer reviewed publication on it. So what would have constituted a valid reference for the English version of the website? What counts as a trusted source? Do videos count? Do the authors of the Wikipedia article have to random sample and analyze sources with the same care as a scientific publication would require? It seems then, the information age necessitates some rethinking of what constitutes a trusted source other than published works. Prabhala says:
“If we don’t have a more generous and expansive citation policy, the current one will prove to be a massive roadblock that you literally can’t get past. There is a very finite amount of citable material, which means a very finite number of articles, and there will be no more.”

Stefan remarked dryly they could just add a reference to Ind. J. Anth. Cult. [in Malayalam], and nobody would raise an eyebrow. Among physicists this is, tongue-in-cheek, known as “proof by reference to inaccessible literature” (typically to some obscure Russian journal in the early 1950s). The point is, asking for references is useless if nobody checks even the existence of these references. Most journals do now have software that checks reference lists for accuracy and at the same time for existence. The same software will inevitably spit out a warning if you’re trying to reference a living review.

But to come back to Wikipedia: It strikes me as a philosophical conundrum, a reference work that insists on external references. Not only because some of these references may just not exist, but because with a continuously updated work, one can create circular references. Take as an example the paper “Moisture induced electron traps and hysteresis in pentacene-based organic thin-film transistors” by Gong Gu and Michael G. Kane, Appl. Phys. Lett. 92, 053305 (2008). (Sounds seriously scientific, doesn’t it?) Reference [13] cites Wikipedia as a source on fluorescent lamps. There is a paper published in J. Phys. B that cites Wikipedia as a source for the double-slit experiment, and a PRL that cites the Wikipedia entry on the rainbow. Taemin Kim Park found a total of 139 citations to Wikipedia in the fields of Physics and Astronomy in the Scopus database as of January 20112.

That citation of Wikipedia itself would not be a problem. But the vast majority of people who cite websites do not add the date on which they retrieved the site. More disturbingly, the book “World Wide Mind” that I read recently, had a few “references” to essays by mentioning they can easily be found searching for [keywords], totally oblivious to the fact that the results of this search changes by the day, depends on the person searching, and that websites move or vanish. (Proof by Google?)

While the risk for citation loops increases with frequently updated sources, it is not an entirely new phenomenon. A long practiced variant of the “proof by reference” is citing one’s own “forthcoming paper” (quite common if page restrictions don’t allow further elaboration), but in this forthcoming paper - if it comes forth - one references the earlier paper. After ten or so self-referencing papers one claims the problem solved and anybody who searches for the answer will give up in frustration. (See also: Proof by mutual reference.)

Maybe the Wikipedia entry on the octopus hoax is a hoax?

Take away message: References in the age of the internet are moving targets and tracing back citations can be tricky. Restricting oneselves to published works only leaves out a lot of information. Citation loops by referencing frequently updated websites can create alternate realities. But don’t worry, somewhere in the level 5 multiverse it’s as real as, say, the moon landing.

Have you cited or would you cite a Wikipedia article in a scientific publication? If you did, did you add a date?



1 And why isn't there a website where one can enter locations of fruit trees and bushes that nobody seems to harvest? Because where we live a lot of blackberries, cherries, plums, peas, and apples are just rotting away. It’s a shame, really.
2 From Park's paper, it is not clear how many of these articles citing Wikipedia were also about Wikipedia. The examples I mentioned were dug out by Stefan.

31 comments:

  1. Because where we live a lot of blackberries, cherries, plums, peas, and apples are just rotting away. It’s a shame, really.

    Darn right, there's so much starvation in the world, and it's terrible when some resources are wasted in one location when they could be applied to others. We need better social engineering.

    Wikipedia

    All good comments Bee, especially re NOT giving the date of the reference. Given that these entries are living, even if the date were given, how would the reader that truly looks up the reference know what that entry said on the day it was referenced? Go back in time and look? Wiki-referencing is kinda dumb.

    As an example of a Wiki reference that is NOT complete (but important IMO for research I'd like to do), click on ===> Solutions of the Einstein field equations. Look at all the sections (toward the bottom) that say "This section requires expansion."

    In a year, hopefully, those will be filled in. But what if someone with a crazy theory (that sounds all science-y) references it today? In a year, there may be some contradiction.

    There are many other examples.

    I trust the Mathematics articles at Wiki 98%, and The Physics ones are almost as good, say 96%. Everything else should be taken with a grain of salt, so, I would rate a paper rather low and suspicious if Wiki is referenced. My two cents.

    ReplyDelete
  2. Hi Bee.
    As a regular reader of your blog, I am mostly interested in the philosopical implications of physics.
    If you assume reality to be the opposite of fiction, you will sure get into trouble...
    Two of the fundamental problems which hides behind your amusing insights were laid out most clearly in the classical publication from Willard Van Orman Quine, "Two Dogmas of Empiricism", (see http://www.ditext.com/quine/quine.html ).
    My recommendation to deal with the state of truth of facts in the internet (and elsewhere) is not to try to find a foundation to it. As a practical solution for the citation problem in the internet, maybe some modification of the Google Pagerank algorithm will serve best to define what is real.

    ReplyDelete
  3. . Or people believing in spaghetti trees for that matter.

    Everybody knows, that spaghetti are not the primary product. What You get from trees are the precursors of macaroni, With the aid of a coring drill the spaghetti are cut out of that thik noodle, leaving the hollow macaroni as a residue.
    Georg

    ReplyDelete
  4. Hi Donnerstilzchen,

    Reality is what you believe it is. Some beliefs are more useful than others. I care what people are made to believe because I believe it affects what others will believe. If that makes sense. I don't want people to be unhappy because they believe the wrong thing (and fell off the edge of the earth). Pagerank ranks popularity, which imho is not useful to that end. Thanks for the reference, I'll look at this, though I tend to mix up words that end on ism. Best,

    B.

    ReplyDelete
  5. There are many obvious pros and cons to fast and easy wiki referencing as opposed to slow and methodical journal referencing (like gentleman's agreements vs legal documents). It is not clear to me which one will get us to the ultimate truth statistically more quickly. The correct answer almost definitely changes on a case by case basis.

    ReplyDelete
  6. I basically hate wikipedia, though I use it in a pinch. I would have to disagree with Steven on physics and math topics in wiki. The problem there, as Bee aludes to, is the use of references. They are used in a fetishistic way in which the facts are laid out with references to those facts, but with very poor or nonexistant reasoning from the initial premise to the ultimate resolution. If you don't understand the concept before reading the article you certainly won't understand after reading it. Mostly you'll just be able to fake understanding better than your neighbor who didn't read it. Society already has way too much of this encouragement of fake knowledge - with references.

    I should say that really good use of references is employed by commenters here, where reading the references employed doesn't just add a patina of professionalism but actually makes you understand what Bee or the commenter is trying to say. There is way too fucking much fake sophistication in much of wikis articles. Needlessly.

    ReplyDelete
  7. Hi Steven,

    "Given that these entries are living, even if the date were given, how would the reader that truly looks up the reference know what that entry said on the day it was referenced? Go back in time and look? Wiki-referencing is kinda dumb."

    For all I know, Wikipedia keeps track of all changes, so you could indeed go back and look if you have a date. Also, you might find some pages in the Google cache, though your selection of dates would be limited. Best,

    B.

    ReplyDelete
  8. You can download a full history of Wikipedia edits, and see what an article looked like on a particular day. In the spirit of this article, I include a "reference" on how to do this here....



    http://en.wikipedia.org/wiki/Wikipedia:Database_download

    Hmmm, this reference comes from Wikipedia, but I think it is a formal Wikipedia article written by Wikipedia people themselves. So is this a valid or unsupported resource? Ugh, my head is spinning.

    ReplyDelete
  9. http://arXiv.org/abs/physics/0205089

    Table at end of paper. This is not a quality citation.

    Wikipedia is the beginning of a search. It concentrates and illuminates but does not necessarily inform.

    http://en.wikipedia.org/wiki/Neutron_star#Properties

    The original text was crap when Uncle Al put its numbers into other equations. An e-mail chat with Lattimer put things right - with an equation plus references. Now... arxiv:1108.1859 Somebody else can update the update, or not.

    Learned tomes celebrate Michelangelo's masterful use of light and shadow on the Sistine Chapel ceiling - before centuries of candle soot were removed to reveal comic book-bright images. DSM I, II, III, IV, V; economics - thoughts that maim and kill large numbers of people are routinely revealed to be Tinkerbell's butt dust.

    Question authority vs. real world before citing it. Look how that got Galileo... crushed.

    ReplyDelete
  10. Al, I have an interesting story to tell about your first reference there. I was always interested in alternative propulsion prospects and Dr Aquino was my first foray into that world around 2002. Mostly it's just a picaresque, embarrassing, farcical tale as is typical of a person just getting his feet wet in something.

    I don't hold any respect for his theory though I did for a short time. Unfortunately, not knowing my ass from a hole in the ground back then I emailed him that I admired his ideas. Next thing I know he is using my email as one of his references! Holy shit. Next thing you know I'm on a mailing list of movers and shakers in the alternative propulsion field. You can't make this stuff up. Actually you could but nobody would believe you.

    ReplyDelete
  11. Hi Al,

    ... before centuries of candle soot were removed to reveal comic book-bright images.

    What are you saying, our ever enigmatic and damn near impossible to decipher therefore very entertaining Mensan Uncle and resident Chemist, hmm? Did Michelangelo's Dad paint a sandbox for him as a boy in primary colors? Nothing wrong with that, eh mate? My Dad did. Yours, too?

    DSM I, II, III, IV, V; economics - thoughts that maim and kill large numbers of people are routinely revealed to be Tinkerbell's butt dust.

    I knew somebody would someday out-funny you, and I'm not surprised that person would turn out to be you, yourself. Dayam, that's just great prose. If you want to something to be done right, do it yourself hellsyeah. (Tautologies can never be proven wrong.)

    Question authority vs. real world before citing it. Look how that got Galileo... crushed.

    Naw, Professor Galilei of U. Piza (Mathematics chair, impressive) and U. Padua was only placed under house arrest. Look at it this way: the local ladies feel sorry for you, come over to hear you rant about crazy Cardinals who accepted your data but not your interpretations of same, and cook you great homemade spaghetti. Think of all the money he saved on not having to take them out to a restaurant! :-)

    ReplyDelete
  12. Hi Bee,

    I would agree the important aspect of evaluating truth rests with the individual in respect to what tools and techniques they’ve cultivated for such purpose. However what I consider the most important of all, which needs to supersede those you’ve listed here respective of critical thinking is the final sieve its run through, with that being reason as it relates to logic.

    For instance you call yourself average when it comes to evaluating things of common truth, yet I would challenge that as having you as either being extremely modest or you have too high of an opinion of what the mean happens to be. That is my experience tells me most people don’t have much of an interest in critical thinking and if it weren’t for the concerted efforts of those that do and care to I think you’d find many more would come to believe in tree dwelling octopi and spaghetti trees. So I would contend methods are one thing, while finding reason to care about them quite another.

    “Good sense is, of all things among men, the most equally distributed; for everyone thinks himself so abundantly provided with it, that those even who are the most difficult to satisfy in everything else, do not usually desire a larger measure of this quality than they already possess. And in this it is not likely that all are mistaken the conviction is rather to be held as testifying that the power of judging aright and of distinguishing truth from error, which is properly what is called good sense or reason, is by nature equal in all men; and that the diversity of our opinions, consequently, does not arise from some being endowed with a larger share of reason than others, but solely from this, that we conduct our thoughts along different ways, and do not fix our attention on the same objects. For to be possessed of a vigorous mind is not enough; the prime requisite is rightly to apply it. The greatest minds, as they are capable of the highest excellences, are open likewise to the greatest aberrations; and those who travel very slowly may yet make far greater progress, provided they keep always to the straight road, than those who, while they run, forsake it.”

    -René Descartes, “Discourse On The Method”, Part 1 , page 4 (1637)

    Best,

    Phil

    ReplyDelete
  13. Hi Eric,

    Holy shit. Next thing you know I'm on a mailing list of movers and shakers in the alternative propulsion field. You can't make this stuff up. Actually you could but nobody would believe you.

    I believe you, Eric, you turned me on to Assange and for that I'm eternally grateful. Hey, I have an alternative propulsion theory, although like most of them we have to thank Robert L. Forward. Click here to see it. DISCLAIMER: Only Eric is allowed to click there, nobody else, as this website has a clear comment rule against advertising your own blog.

    ReplyDelete
  14. Steven,
    Interesting idea. Of course there are many technical problems. But the biggest problem of a non-technical nature is the commitment cost in money, and especially time.

    It would still probably take hundreds, maybe thousands of years to get to another star system. In the meantime technology marches on back on Earth. Wouldn't it be demoralizing to be fifty years into your intergenerational voyage when a sporty little flying saucer zips up to your ship. Out pops a human being with greetings from Earth which he just left 3 hours ago. I sure wouldn't want to risk possibly many generations of travel time in the hope that mpre advanced propulsion systems don't come around in the meantime.

    Maybe it could be done sooner. I'm not sure what you are using as far as a time schedule.

    ReplyDelete
  15. Hi Eric,

    Yes, I'd agree that Wikipedia sucks to the end of understanding a derivation. But I don't think it's really the purpose of an encyclopedia to teach. For what I am concerned, Wikipedia is much too detailed already on some math and physics derivations, which reduces its usefulness for looking up something. See, if I really want to learn something I don't turn to Wikipedia anyway, I read a textbook or a review article, or something that's coherent and well written (and, yes, trustworthy). What I want from an encyclopedia is just to spit out the most relevant facts. Ie, the so-and-so equation, who derived it when, what it's used for, and where I can find a proof if I don't know it but need it. Best,

    B.

    ReplyDelete
  16. This comment has been removed by the author.

    ReplyDelete
  17. Actually, if you got up a decent percentage of the speed of light using your system it wouldn't take that long in ship's time. But you would still leave knowing everyone back on earth would be dead if you ever chose to return. This would be opposed to a propulsion system that manipulated space itself around the exterior of the ship. In such a situation it is conceivable one wouldn't necessarily increase the gravitational and inertial mass (they can't be separated) of the ship at relativistic velocities. It would be like lowering the inertial viscosity of space in the direction you are traveling. That is my interest. There are quite a few ideas out there about how this might be accomplished- none very developed ( as far as we know).

    ReplyDelete
  18. Bee, agreed. Even as an encyclopedia I wish Wiki was better. Sorry about going off topic on my old haunt. Old habits die hard.

    ReplyDelete
  19. Hi Bee,

    This is a good one, so rather than flying pigs it’s penguins; reminds me of what my mother once told me :-)

    “If grandma had wheels she’d be a trolley car.”

    Best,

    Phil

    ReplyDelete
  20. Hi Bee,

    Then to be perfectly fair there was a time they said that your namesake shouldn’t be able to fly either and yet they do :-)

    Best,

    Phil

    ReplyDelete
  21. I think we had talked about this before as well.

    This is not to say anyone has to do anything they don't want to do. Scientist, layperson with knowledge that others may not have.

    Accessibility to good information.

    If one could be happy to say that there is a viable method with which to give all people accessibility to this what would we do as a society to ensure that this can happen?

    Our contribution to clearing away the information that is old and has now been advanced? How perception in history had paved the way for?

    I have embraced this endeavor from the standpoint that one could help another.....so in your mind....reading......some other linked location of a information source you can trust?

    What would that be if you have said basically wiki is useless? We had talked about improving it? How so?

    I have always appreciated the style of posting that you and Stefan have gone into, the sharing of your research. I have always appreciated those that would bring clarity to wiki.

    Best,

    ReplyDelete
  22. Hi Eric,

    You wrote:

    Actually, if you got up a decent percentage of the speed of light using your system it wouldn't take that long in ship's time.

    And what percentage would that be? I'll ballpark 10% , so it would realistically take 40 years to get to Alpha Centauri A and Alpha Centauri B and that wascally wabbit interloper Proxima Centauri who is just passing by. The reason is: Galactic wind. I don't care how big of a cometary iceball you have on the front of your ship or on it's rear end when you decelerate, all sorts of bad things happen when speed increases beyond 0.10 c. A little hydrogen atom sitting relatively still becomes a bullet. This is why you physicists with your spherical cows need us engineers. We'll bring ya down to earth, or, you know, Alpha C. Someday. Not in my lifetime, but the planning starts now.

    Hey Eric, ever go to the weblog Centauri Dreams run by Paul Gilster? Amazing place. The Tau Ceti Foundation as well.

    But you would still leave knowing everyone back on earth would be dead if you ever chose to return.

    You will never return. 100 years to Tau Ceti which should be the first star we check out. Say your goodbyes and split like a banana. Look at it this way: You won't have to be subjected to Fox News anymore, and that's a plus!

    Lara and Gloria have a decent chance of living to be 110 years old. Their children to 130. Bee's and Stefan's great-great-grandchildren to 180 and the great-great-greats to 400 or 500 years old, so in the future, yah, MAYbe they could return, but who'd want to? What would be the point?

    ReplyDelete
  23. Steven,
    I have lots of ideas and explanations for some of my thinking, much of which goes against conventional wisdom. However, I'm trying to use sheer willpower to not hijack this post. Bee, you don't know how much! I hope sometime you will dedicate some blog time to new forms of propulsion so all of us don't feel a conversational form of coitus interruptus.

    ReplyDelete
  24. Back on-topic then.

    Have you cited or would you cite a Wikipedia article in a scientific publication?

    No. Then again I haven't written a paper since 1979, and Wikipedia didn't exist then.

    If you did, did you add a date?

    I wouldn't, but if I lacked self-respect and didn't care about my reputation and thus did then yes, I would.

    ReplyDelete
  25. I was recently reading a book (A Great Leap Forward) on total factor productivity in the 1930s in the US, and the citations for a lot of the tables of numbers were US government websites, usually the BLS or BEA, and all citations provided a URL and the date downloaded. Given the recent downward GDP revisions, a download date is pretty important.

    ReplyDelete
  26. Hi Bee,

    somehow my first post didn´t show up in your blog. This does not matter too much, since it gives me a reason to sharpen my argument a little more.
    You stated that you care about what people believe. Then you should take into account that individual belief systems (like human minds) do not optimize truth values in first place. Since most physical experiments show minor consequences than falling off the edge of the world, it is more important for people to have beliefs which are socially acceptable or just make them happy.
    The Pagerank algorithm analyses the structure of the links between web pages, so popularity is a indirect result, but not the only one. I am with you if you say reality is what you believe it is. Now what about truth?
    I still think that the pagerank algorithm can tell us important facts about people´s believe system structure. It can reveal fatal developments at an early stage, keeping us safely at the middle of our planet disk.

    ReplyDelete
  27. Hi Donnerstilzchen,

    Sorry, your comment was stuck in the spam queue which I hadn't noticed (thus my reply). If this happens, please either send me an email or post another comment saying the first didn't go through. (Make sure the 2nd comment is explicitly addressed to me, otherwise I might not read it.) More later,

    B.

    ReplyDelete
  28. Hi Donnerstilzchen,

    I don't know what 'truth' is (other than in a colloquial sense). The problem with pagerank is that it's too undifferentiated. People do for example link to pages they find particularly ugly or wrong. Best,

    B.

    ReplyDelete
  29. To answer your topic question, please see the last section in the Wikipedia article "Moon landing". That's my parsonally-written text, along with most of the other text in the article. I quite deliberately put the link to "empirical" in there. Don't trust references, trust data.

    ReplyDelete

COMMENTS ON THIS BLOG ARE PERMANENTLY CLOSED. You can join the discussion on Patreon.

Note: Only a member of this blog may post a comment.