Thursday, May 19, 2016

The Holy Grail of Crackpot Filtering: How the arXiv decides what’s science – and what’s not.

Where do we draw the boundary between science and pseudoscience? It’s is a question philosophers have debated for as long as there’s been science – and last time I looked they hadn’t made much progress. When you ask a sociologist their answer is normally a variant of: Science is what scientists do. So what do scientists do?

You might have heard that scientists use what’s called the scientific method, a virtuous cycle of generating and testing hypotheses which supposedly separates the good ideas from the bad ones. But that’s only part of the story because it doesn’t tell you where the hypotheses come from to begin with.

Science doesn’t operate with randomly generated hypotheses for the same reason natural selection doesn’t work with randomly generated genetic codes: it would be highly inefficient and any attempt to optimize the outcome would be doomed to fail. What we do instead is heavily filtering hypotheses, and then we consider only those which are small mutations of ideas that have previously worked. Scientists like to be surprised, but not too much.

Indeed, if you look at the scientific enterprise today, almost all of its institutionalized procedures are methods not for testing hypotheses, but for filtering hypotheses: Degrees, peer reviews, scientific guidelines, reproduction studies, measures for statistical significance, and community quality standards. Even the use of personal recommendations works to that end. In theoretical physics in particular the prevailing quality standard is that theories need to be formulated in mathematical terms. All these are requirements which have evolved over the last two centuries – and they have proved to work very well. It’s only smart to use them.

But the business of hypotheses filtering is a tricky one and it doesn’t proceed by written rules. It is a method that has developed through social demarcation, and as such it has its pitfalls. Humans are prone to social biases and every once in a while an idea get dismissed not because it’s bad, but because it lacks community support. And there is no telling how often this happens because these are the stories we never get to hear.

It isn’t news that scientists lock shoulders to defend their territory and use technical terms like fraternities use secret handshakes. It thus shouldn’t come as a surprise that an electronic archive which caters to the scientific community would develop software to emulate the community’s filters. And that is, in a nutshell, basically what the arXiv is doing.

In an interesting recent paper, Luis Reyes-Galindo had a look at the arXiv moderators and their reliance on automated filters:


In the attempt to develop an algorithm that would sort papers into arXiv categories automatically, thereby supporting arXiv moderators to decide when a submission needs to be reclassified, it turned out that papers which scientists would mark down as “crackpottery” showed up as not classifiable or stood out by language significantly different from that in the published literature. According to Paul Ginsparg, who developed the arXiv more than 20 years ago:
“The first thing I noticed was that every once in a while the classifier would spit something out as ‘I don't know what category this is’ and you’d look at it and it would be what we’re calling this fringe stuff. That quite surprised me. How can this classifier that was tuned to figure out category be seemingly detecting quality?

“[Outliers] also show up in the stop word distribution, even if the stop words are just catching the style and not the content! They’re writing in a style which is deviating, in a way. [...]

“What it’s saying is that people who go through a certain training and who read these articles and who write these articles learn to write in a very specific language. This language, this mode of writing and the frequency with which they use terms and in conjunctions and all of the rest is very characteristic to people who have a certain training. The people from outside that community are just not emulating that. They don’t come from the same training and so this thing shows up in ways you wouldn’t necessarily guess. They’re combining two willy-nilly subjects from different fields and so that gets spit out.”
It doesn’t surprise me much – you can see this happening in comment sections all over the place: The “insiders” can immediately tell who is an “outsider.” Often it doesn’t take more than a sentence or two, an odd expression, a term used in the wrong context, a phrase that nobody in the field would ever use. It is only consequential that with smart software you can tell insiders from outsiders even more efficiently than humans. According to Ginsparg:
“We've actually had submissions to arXiv that are not spotted by the moderators but are spotted by the automated programme [...] All I was trying to do is build a simple text classifier and inadvertently I built what I call The Holy Grail of Crackpot Filtering.”
Trying to speak in the code of a group you haven’t been part of at least for some time is pretty much impossible, much like it’s impossible to fake the accent of a city you haven’t lived in for some while. Such in-group and out-group demarcation is subject of much study in sociology, not specifically the sociology of science, but generally. Scientists are human and of course in-group and out-group behavior also shapes their profession, even though they like to deny it as if they were superhuman think-machines.

What is interesting about this paper is that, for the first time, it openly discusses how the process of filtering happens. It’s software that literally encodes the hidden rules that physicists use to sort out cranks. For what I can tell, the arXiv filters work reasonably well, otherwise there would be much complaint in the community. But the vast majority of researchers in the field are quite satisfied with what the arXiv is doing, meaning the arXiv filters match their own judgement.

There are exceptions of course. I have heard some stories of people who were working on new approaches that fell between the stools and were flagged as potential crackpottery. The cases that I know of could eventually be resolved, but that might tell you more about the people I know than about the way such issues typically end.

Personally, I have never had a problem with the arXiv moderation. I had a paper reclassified from gen-ph to gr-qc once by a well-meaning moderator, which is how I learned that gen-ph is the dump for borderline crackpottery. (How would I have known? I don’t read gen-ph. I was just assuming someone reads it.)

I don’t so much have an issue with what gets filtered on the arXiv, what bothers me much more is what does not get filtered and hence, implicitly, gets approval by the community. I am very sympathetic to the concerns of John The-End-Of-Science Horgan that scientists don’t clean enough on their own doorsteps. There is no “invisible hand” that corrects scientists if they go astray. We have to do this ourselves. In-group behavior can greatly misdirect science because, given sufficiently many people, even fruitless research can become self-supportive. No filter that is derived from the community’s own judgement will do anything about this.

It’s about time that scientists start paying attention to social behavior in their community. It can, and sometimes does, affect objective judgement. Ignoring or flagging what doesn’t fit into pre-existing categories is one such social problem that can stand in the way of progress.

In a 2013 paper published in Science, a group of researchers quantified the likeliness of combinations of topics in citation lists and studied the cross-correlation with the probability of the paper becoming a “hit” (meaning in the upper 5th percentile of citation scores). They found that having previously unlikely combinations in the quoted literature is positively correlated with the later impact of a paper. They also note that the fraction of papers with such ‘unconventional’ combinations has decreased from 3.54% in the 1980s to 2.67% in the 1990, “indicating a persistent and prominent tendency for high conventionality.”

Conventional science isn’t bad science. But we also need unconventional science, and we should be careful to not assign the label “crackpottery” too quickly. If science is what scientists do, scientists should pay some attention to the science of what they do.

104 comments:

  1. It would be interesting to pump some of Einstein's early papers through the crackpot filter, to see what it would say.

    ReplyDelete
  2. Dear Sabine,

    I am happy your blog is not on arXiv...:)

    Should I guess that if the young Einstein was coming now with his first paper on relativity he would be filtered-out? If I remind correctly this paper comes with "new manners of speaking".

    How can something really new come with the same semantics as the older stuff? Or are physicists so certain of what they know at conceptual level that anything else is crackpotry? Are we damned to eternal cut and paste?

    Nice post, thanks
    J.

    ReplyDelete
  3. I agree, how many people thought Einstein was a crackpot when he suggested time was not universal?
    But the new idea must be broader and deeper than previous one,a better explanation.

    ReplyDelete
  4. I don't think Einstein was ever regarded a crackpot. He was very much part of the community at his time and "spoke the language" just fine. Bringing up the myth that Einstein was an outsider is itself one of the things that marks an outsider.

    Einstein's papers would almost certainly not pass the arXiv filter, but this has very little to do with Einstein and very much with the changes in the style of scientific writing generally (not to mention language changes generally). To begin with, in Einstein's days a big part of the scientific literature was in German.

    ReplyDelete
  5. Hello Sabine,

    I like your well balanced post.

    I think it is inevitable that filtering papers will render a more conventional ensemble, and I am not against that. The 'incrowd' has specific skills which are relevant.

    What I would like to see is more initiatives such as Fqxi, where the 'unconventional language' does not get filtered out so easily, and content is individually assessed, sometimes leading to discovery of different relevant skills. Such systems can exist side by side, no problem, let there be a mutual influence.

    Best regards,

    Noa

    ReplyDelete
  6. Not a scientist but the same sorts of filtering issues apply to publishing in philosophy. As result talented outsiders are being ignored for the wrong reasons. There may be little an "amateur physicist" can do these days but that should not be the case in philosophy

    ReplyDelete
  7. "It doesn’t surprise me much – you can see this happening in comment sections all over the place: The “insiders” can immediately tell who is an “outsider.” Often it doesn’t take more than a sentence or two, an odd expression, a term used in the wrong context, a phrase that nobody in the field would ever use."

    As a long-time user of usenet, I find that I can recognize familiar posters by reading just half of the first line.

    ReplyDelete
  8. "I don’t so much have an issue with what gets filtered on the arXiv"

    You might if you, or someone you care about, gets filtered out when they shouldn't.

    "what bothers me much more is what does not get filtered and hence, implicitly, gets approval by the community."

    I think it is better to be cautious and err on the side of letting too much through. Even with reputable refereed journals, sometimes crackpottery slips through. I don't think that this is tantamount to "approval by the community". People just ignore such papers, don't cite them, and so on.

    ReplyDelete
  9. Einstein was very much a part of the physics community, and was actually quite conservative (in regard to physics---otherwise, not so much). What has changed is that this is more difficult today without an institutional affiliation, mainly because back then the community was much smaller and even if one didn't know everyone else, one could still read all the literature. (As late as the 1930s, Feynman read the entire Physical Review.)

    ReplyDelete
  10. "Where do we draw the boundary between science and pseudoscience?" Observation. Good ideas need only be testable. Fix the problem, mourn the dead, repeat.

    General relativity perfectly predicts but excludes quantization. Non-classical gravitation is untestable. Rigorous derivation cannot be internally falsified. Observation outside physics' postulates contradicts accepted theory it assaults - crackpottery! the business of hypotheses is guarded stagnation.

    "natural selection doesn’t work with randomly generated genetic codes" A million failed reactions are a combinatorial library - intensely commercialized, doi:10.13040/IJPSR.0975-8232.4(7).2502-16; suddenly embraced by metallurgy, doi:10.1038/533306a Ginsparg's quotation is damning. Green's function removes geometric chirality from physics, a part-per-trillion bad move.

    http://www.combichemistry.com/review.html
    Test everything.

    ReplyDelete
  11. I don't know how easy it would be to automate, but John Baez's crackpot index is a pretty efficient filter.

    ReplyDelete
  12. Phillip,

    I think this is a misunderstanding. When I say that I am more concerned about what does not get filtered, I don't mean that this is an issue that should be taken on by filtering (and certainly not by the arXiv). I instead mean to say it's an issue that bothers me much more and that no filter will ever solve. I agree that we should err on the side of caution.

    Sure, the arXiv filter would bother me more if it would affect more people I know - but that isn't presently the case. And in the absence of any data on the issue, this gives me little reason to worry except for the general issue of "unknown unknows" that I'd rather there wouldn't be. I know about a handful of scientists who have encountered problems, but for all I know these were eventually resolved somehow.

    ReplyDelete
  13. " I am very sympathetic to the concerns of John The-End-Of-Science Horgan that scientists don’t clean enough on their own doorsteps."

    My impression of his recent remarks is that he is throwing the baby out with the bathwater. Sure, there is no invisible hand, and science is self-correcting, which means that scientists have to do the correcting. But blasting skeptics by saying that they go after low-hanging fruit is wrong on at least two grounds. First, not all of skepticism is concerned with easy targets; there are skeptics who go after hard targets. (Maybe fewer but, hey, they are hard. As Feynman said to a journalist: "“Listen, buddy, if I could explain it to you in a minute, it wouldn't be worth the Nobel Prize.") He seems to know little about the current skeptic scene. Second, things like homoeopathy and mistrust in vaccines might be easy targets from a scientific point of view, but they kill thousands of people each year, so obviously there is a need to continue to work on this front.

    ReplyDelete
  14. It disturbs me that these filtering processes - whether intentionally or not - effectively prevent "alternative" theories, technology or whatever being subjected to open and, more importantly, unbiased examination.
    Just as peer review will always tend to maintain the status quo, because acceptance of disruptive ideas would pull the rug out from under long established dogma (and tenures), the dumping of papers evaluated by some algorithm on the basis of "not invented here" language into some unread catch-all filing backwater unofficially called "crackpottery" stands a high risk of consigning perfectly valid work to oblivion.
    Also, as someone accustomed to interdisciplinary working (which appears to be becoming the rule rather than the exception), I believe that trying to force square pegs into round holes is inherently wrong.
    There is no easy solution to this problem; but on the whole I think it would be better to publish everything and be damned, letting the readers judge submissions purely on the merit of their content.

    ReplyDelete
  15. "Bringing up the myth that Einstein was an outsider is itself one of the things that marks an outsider." - I like that a lot. That always drives me nuts. I heard Lee Smolin doing this repeatedly in an interview, so maybe that makes him an honorary outsider hehe.

    ReplyDelete
  16. Hi Bee, I don't quite agree with you on Einstein's early papers.
    Although he was very much part of the physics community, I think most historians agree that the language and layout of Einstein's 1905 paper on electrodynamics was quite different to the works of his contemporaries, and quite a surprise.
    Because AE argues from first principles, the paper is full of phrases like "If I say 'that train arrives here at 7 o' clock' I mean something like this: the pointing of the small hand of my watch to 7 and the arrival of the train are simultaneous events" (Einstein 1905 p3).
    Instead of the dense mathematical equations of the physics literature of the late 19th century, there is a great deal of argument like this from simple imagined scenarios. Indeed the descriptions throughout of different observers observing their watches attracted some ridicule at first, according to historians like John Stachel and John North.
    Regards, Cormac

    ReplyDelete
  17. There's an image of Einstein coming out of nowhere with five groundbreaking papers in 1905. But he didn't come out of nowhere, he had written five papers in Annalen der Physik on thermodynamics from 1901 to 1904, and written over a dozen reviews of other papers in 1905 before his first "Annus Mirabilis" paper was published. He wasn't a world leader, but he wasn't nobody either.

    ReplyDelete
  18. When I look at the arXiv dailies, its a bit depressing. Perhaps once a week in my topics comes a paper that is not so completely narrowly focused as to be unusable for anyone but the author and 5 other people. Then there is 20 mins on a paper only to find its kinda weak. Its all about publish or perish and that creates an almost unusable arXiv.

    Much better would be open moderation (instead of in secret), with communities upvoting what they want to bubble up. Reddit has a great system for that. Crackpots can have their own corners, important work from other fields automatically shows up in your feed, and comments are auto filtered based on your personal and your communities likes.

    One could even have arXiv papers come down in number, and let anyone put anything on the 'pre arXiv' server, which would only get promoted on certain conditions - their peer impact level or another score like it should be included even its boring but 'needed to be done' work.

    Thought provoking papers that are 'certainly wrong' have just as much right to be on arXiv as papers that are "small mutations of ideas that have previously worked". We are not going to get anywhere if all we do is small mutations. But these small papers need to be 99% of arXiv. But not 99.99999% - which seems to be the goal of the auto magic filters.

    ReplyDelete
  19. Coraifeartaigh,

    You say you don't agree with me, but I didn't actually say anything that disagrees with you. I merely said that Einstein wasn't an outsider. How much Einstein's writing style stood out at the time is probably a question we'll never be able to answer quantitatively. Yes, he worked a lot with 'thought experiments' which was rather unusual at the time. But then the writing back then was generally more conversational than it is today - eg scientists would frequently reply to each others papers.

    Having said that though, I don't actually know what the arXiv algorithm picks up on, but I doubt that it would be ticked off by specifics in personal writing. To begin with, that would flag all personal historical accounts, of which you find quite a lot on the arXiv. There are also quite a lot of people who write up talks and these tend to be distinctively different from research papers. So, I don't think it's the personal writing style we're referring to.

    ReplyDelete
  20. Tom,

    A voting algorithm or comments would only amplify social streamlining. Think about this again. The benefit of having a software scanning submissions is that at least it treats everyone the same. Any criteria that rely on personal connections or popularity of the author will make in-group behavior considerably worse. What you'd get quickly would be a rich-get-richer trend.

    ReplyDelete
  21. Hi Bee, just an addendum you and you readers might find interesting:
    In 'Albert Einstein: Beitrag Fur Sein Lebensbild', Einstein's sister claims that AE submitted a paper on relativity as his PhD. "But the thing didn't seem right to the leading professors, as the wholly unknown author paid no heed to authority figures, even attacked them! So the work was rejected and the candidate was compelled to write another, more harmless work, on the basis of which he obtained the title of doctor Philosophiae"(Winteler-Einstein 1924 p23-24; also discussed in (CPAE VOl 2 p 266).
    Admittedly, this was back in 1902, but if his sister is correct, the incident gives an interesting insight into the perception of Einstein, and his relativity theory, in those years.
    Regards, Cormac

    ReplyDelete
  22. coraifeartaigh,

    Just an addendum, not specifically about Einstein, but relevant nevertheless. The label "crackpottery" isn't so much about being right or wrong, but about living up to the scientific standard. Scientists are wrong all the time.

    ReplyDelete
  23. Hi Sabine, thank you for a very good summary of the Horae paper. We're soon publishing another paper on closely related topics, including new data on filter efficiency provided by Paul Ginsparg. It should be on the arXiv soon, but for now here it is on academia.edu: https://www.academia.edu/25486531/A_note_concerning_Primary_Source_Knowledge

    ReplyDelete
  24. By chance, this appeared today: http://www.snopes.com/einstein-rejection-letter/

    ReplyDelete
  25. Whoops, my second comments seems to have got lost, I suspect I clicked the wrong box, apologies

    As regards Einstein, it is admittedly a tangential issue - but on the 'Einstein the outsider' legend, I sometimes think that physicists are so tired of the popular notion that Einstein's 1905 papers came out of nowhere and shocked the world, that we sometimes overreact and forget that there is a small element of truth in the story.

    If one looks at other papers in volume 17 of Ann. der Physik (or previous volumes), Einstein's paper on electrodynamics really does look quite strange: there is an enormous amount of what appears to be long-winded philosophical reasoning on the nature of time, accompanied by a few simple-looking equations.
    According to his sister's memoir, Einstein was very relieved that the paper was published in the Annalen, but extremely disappointed at the reaction to the paper (or lack of) in later volumes; years later, he commented that it was a lucky thing that Max Planck took notice.
    What I'm getting at is whether Einstein was seen as an outsider or not in 1905 is not quite as clear cut as I for one had thought. John Stachel has a good discussion of this in the notes to CPAE volume 2.
    Kind regards, Cormac

    ReplyDelete
  26. Speaking of crackpots, one of the premises of "Intelligent Design" is that human intelligence and design work are done by magic and are therefore evidence of a magical being who designed this whole universe. I have claimed the opposite, that human intelligence and design work are done by the evolutionary algorithm, and therefore are more evidence for evolution (and the Theory of). Thanks very much for what I take as a concurring opinion (although that is not the main point of your post), which I intend to cite as evidence that I may not be a crackpot.

    ReplyDelete
  27. It's pretty bad for people who think outside the box. Having had problems with arxiv's refusal to cross-list to the most reasonable sections, I've written a number of items on the subject, one published in Nature:
    Vital resource should be open to all physicists; also re my unfruitful correspondence with arxiv people, and regarding the unfortunate consequences of automatic filtering.

    ReplyDelete
  28. Thanks for the interesting article. One disadvantage of kicking out papers that take terminology from different fields is that it would exclude papers where the article sees how work done in one field could have applications to another field.

    The example was mentioned above of Einstein's papers on relativity. Perhaps his papers on special relativity would not have been kicked out, but general relativity? How many papers on gravity discussed non-Euclidean geometry at that time. Zero?

    Bob Clark

    ReplyDelete
  29. Does anyone know of a valuable paper written by an outsider in the last 50 years within any area of physics? I know of a handful, but they are almost universally written by x-PhDs or people who are actively working with private labs.

    Also perhaps a few exceedingly gifted highschool students who might be mentored by an insider, but that's about it.

    Of the crackpots that I have met, there have been a few that I would classify as 'almost' being able to write something valuable, but that lack the support structure of academia where a capable friend could tell them 'no, neat idea but this can't work b/c of this unpublished lore'

    ReplyDelete
  30. Haelfix,

    Dyson springs to mind, but it's also the only example I know. I don't think you can make a contribution in physics today without having a PhD or an equivalent education (title or not). The title really isn't the relevant thing, it's all the information that you need to absorb somehow. And if you take the time to do that, you could as well make a PhD.

    I know a strange story of a guy who, when he was up for tenure, discovered that his PhD title was actually invalid. So technically he's written stacks of papers without a PhD but I don't think that counts... (He ended up having to quickly produce a thesis and "graduated" in record time briefly before being tenured.)

    I know quite a lot of people with a PhD in one field who have made contributions in another field (the physics -> biology and physics -> economy crossover being the most common). In most cases I think they could probably make a PhD in that field, but it's not relevant for them.

    ReplyDelete
  31. JimV's comment: 'Speaking of crackpots, one of the premises of "Intelligent Design" is that human intelligence and design work are done by magic and are therefore evidence of a magical being who designed this whole universe' manifests a common misconception, the confusion of Intelligent Design and Creationism. ID does not presume some magical being, but instead looks at the evidence and attempts to assess scientifically whether it is consistent with the traditional explanations such as Darwinism. This mere examination of the evidence arouses extraordinarily strong reactions from people almost religiously(!) attached to the view that materialistic explanations suffice. ID in itself is neutral as to what the explanation might be for any discrepancy, though naturally its adherents might have their own views on the matter.

    A query for you Jim: let's assume that your mechanism works, and let's refer it by the secular term The Force. Might individuals be able to establish some kind of information-exchanging connection with The Force, in the same sort of way that people interact with other people, and religious people say they interact with God?

    ReplyDelete
  32. @Haelfix The Mpemba (African high school kid) effect and furious theory (nine arXiv papers) following re the Pioneer anomaly. The Allais (economist) effect violates gravitation models. So? Euclid suffers non-Euclidean geometries. Newton bobbled c, h, and k_B (and demonstrably the Equivalence Principle on Principia's first page). Physics' mirror-symmetric universe excludes Big Bang baryogenesis. Curve-fitted theory is a counterfeit.

    http://phys.org/news/2010-03-mpemba-effect-hot-faster-cold.html

    http://www.science-frontiers.com/sf074/sf074a05.htm
    http://science.nasa.gov/science-news/science-at-nasa/1999/ast06aug99_1/
    http://briankoberlein.com/2015/03/22/the-pendulum-of-truth/

    ReplyDelete
  33. See https://arxiv.org/abs/1403.7377 and note page 9:

    2.1.2 Tests of local Lorentz invariance
    Although special relativity itself never benefited from the kind of “crucial” experiments, such as the perihelion advance of Mercury and the deflection of light, that contributed so much to the initial acceptance of GR and to the fame of Einstein, the steady accumulation of experimental support, together with the successful merger of special relativity with quantum mechanics, led to its acceptance by mainstream physicists by the late 1920s...


    It was 25 years before special relativity was accepted by mainstream physicists. If Einstein had his annus mirabilis today, one suspects his papers wouldn't be on the arXiv.

    ReplyDelete
  34. Maybe someday AI can pickup on the language of trained physicists and get past the filtering.

    A lot of science papers reminds one of the song "Little Boxes" by Malvina Reynolds.
    Lyrics here; http://people.wku.edu/charles.smith/MALVINA/mr094.htm

    Youtube version here: https://www.youtube.com/watch?v=2_2lGkEU4Xs

    ReplyDelete
  35. I was struck by your comment: "..even fruitless research can become self-supportive." Can you possibly suggest examples? :-) I know of at least one.

    ReplyDelete
  36. John Loop,

    Every fad in high energy physics that has since vanished falls into this category.

    ReplyDelete
  37. "And the children go to summer camp
    And then to the university,
    Where they are put in boxes
    And they come out all the same."

    Of course, some few have much more ticky tacky than others. They get to make make the rules. And the Skinner boxes.

    ReplyDelete
  38. Sabine... But at least they vanished? Somebody proved them wrong, or found contrary evidence. Some don't vanish, they become religion. I think the present AGW "sciences" now fit this bill. They even have "heretics" and they are trying to burn them at the stake, so to speak.

    ReplyDelete
  39. Re John Loop's "But at least they vanished? Somebody proved them wrong, or found contrary evidence. Some don't vanish, they become religion", interestingly enough the reverse can happen. In 2002 the arxiv's gloss on cold fusion in response to a submission was this: "Unfortunately the policy here is that the material posted on the arXiv at least in principle be publishable in conventional journals. We regret if these resources are too conservative for your needs, but there are other more open internet fora available for such purposes." But now there are 10 papers that have LENR in their abstracts (and others with 'cold fusion', but the situation in that case is complicated by the fact that there is a second use for this term). The change may well be related to the use of the algorithm, which has perhaps picked up the fact that at this time papers on cold fusion/LENR are publishable in 'conventional journals'.

    ReplyDelete
  40. "If one looks at other papers in volume 17 of Ann. der Physik (or previous volumes), Einstein's paper on electrodynamics really does look quite strange: there is an enormous amount of what appears to be long-winded philosophical reasoning on the nature of time, accompanied by a few simple-looking equations."

    There are two issues here. One is the question whether Einstein was "an outsider" in the sense of today's "independent researcher". He was not, not by any stretch. True, he didn't have an academic job for a while, but, due to the small size of the community, this wasn't as big a barrier as it is today. He read all the literature (which most "independent researchers" do not) and many physicists knew him.

    Another question is whether the style of his papers was somewhat unorthodox. Yes, they were. But this doesn't really have much to do with being an outsider in the normal sense of the term. Yes, most outsiders' papers are unorthodox, but an unorthodox (heterodox?) paper is not necessarily the work of an outsider.

    With regard to Dyson: certainly not an outsider. You can't be more part of the establishment in physics than being at Princeton. True, he doesn't have a doctorate (neither did Thomas Gold, and there are a few others), but equating the lack of a doctorate with being an outsider is itself a mark of being an outsider.

    On the other hand, Brian Josephson (see above) has not only a doctorate but also a Nobel Prize and is definitely an outsider (these days---he didn't used to be).

    ReplyDelete
  41. @Brian Josephson

    1) "Cold fusion" anomalies are true.
    2) "Cold fusion" failures (e.g., MITI's exhaustive endeavor) are true.
    3) Subtract. Physics and engineering disappear, leaving chemistry.

    Metallic lithium plates out given high amperage density. Pd-Li low alloys are stable to air and water, affording a Pd-Li-D ternary alloy electron-dense as an electrochemical cell's cathode. Deuterium cold fusion is induced nuclear proximity then coulomb barrier tunneling. Add some titanium (re titanium hydride).

    ReplyDelete
  42. Can people please stick to the point? I can't see what Al's comment has to do with the issue under discussion.

    ReplyDelete
  43. @Brian Josephson "papers on cold fusion" Cold fusion papers are defective because theory misinterprets observations. Consider OPERA superluminal muon neutrinos. "Theory needs no passport" is Aristotelian. Galileo and Popper demand conflicting observations reconcile, then publishable as a rectifying proposal's reduction to practice.

    Problems are solved within boundary conditions that create them. Einstein rederiving Newton via Maxwell into SR was SOP. His conclusions were Officially unbearable. I apologize for chemistry bearing on physics. Now you apologize for discarding chemistry bearing on physics.

    ReplyDelete
  44. " I can't see what Al's comment has to do with the issue under discussion."

    Well, B.J. and I agree here. His comments are often tangential at best.

    ReplyDelete
  45. "It was 25 years before special relativity was accepted by mainstream physicists. If Einstein had his annus mirabilis today, one suspects his papers wouldn't be on the arXiv."

    The accepted it but there wasn't much utility for it, and no one was willing to give up Newtonian Mechanics. Still aren't. Newton remains dominant to this day. The vast majority of calculations are performed in Newton's theory.

    ReplyDelete
  46. I wonder if arXiv, (or any similar venue), would consider publishing reproducible results of an amateur investigating a phenomena that professional's deem improbable, or impossible, and thus haven't bothered to even examine in a laboratory setting?

    ReplyDelete
  47. "Although special relativity itself never benefited from the kind of “crucial” experiments, such as the perihelion advance of Mercury and the deflection of light, that contributed so much to the initial acceptance of GR and to the fame of Einstein, the steady accumulation of experimental support, together with the successful merger of special relativity with quantum mechanics, led to its acceptance by mainstream physicists by the late 1920s. "

    It was the Red Shift that sealed the deal. The cosmological landscape Hubble painted was impossible to make sense of at that time. Until someone got around to the new radical theories by Einstein. And the rest is history but it was arguably the greatest most serendipitous coincidence of all time.

    It was that event of two unrelated scientific pursuits, one empirically led, the other abstract theoretical, coming together as the single one and only viable explanatory pairing.

    It was the mind-blowingly perfect fit that ends with full scientific acceptance at consensus the fastest and most absolute in scientific history. Which may ye prove the greatest scientific wrong-turning in history.

    ReplyDelete
  48. Brian Josephson, re "the confusion of Intelligent Design and Creationism".

    Your contention was presented to a judge and after reviewing evidence and expert testimony the judge issued a legally-binding (in the USA) conclusion that ID is in fact a stealth form of creationism. Do an online search for "Dover Kitzmiller Trial" for details.

    (I regret having raised this off-topic issue and will refrain from further comments on it.)

    ReplyDelete
  49. Perhaps arXiv should have a dedicated section labeled "Pseudoscience", in which the rejected papers are dumped. That way at least everyone gets a fair hearing. Should it be later determined that a paper placed in the Pseudoscience repository meets the criteria of good science, then it could be reassigned to a legitimate category.

    ReplyDelete
  50. Off topic: The spelling rule I learned in college was that there is no "judge" in the word "judgment." Because the common mistake, which is shared in this blog post is to spell the word incorrectly as "judgement."

    ReplyDelete
  51. David,

    perhaps the incredibly huge amount of papers on ArXiv which are science-fiction could be reassigned to where they stand. That would clean the place... and maybe slow the run for publication.

    J.

    ReplyDelete
  52. "I wonder if arXiv, (or any similar venue), would consider publishing reproducible results of an amateur investigating a phenomena that professional's deem improbable, or impossible, and thus haven't bothered to even examine in a laboratory setting?"

    arXiv isn't refereed, but it has moderators. Extraordinary claims require extraordinary evidence, but arXiv doesn't have the resources to verify the evidence.

    Submit it to a journal. If it is rejected, they will say why, and, at least if it is a good journal, it won't be just a short boilerplate rejection. After it has been accepted by a journal, you should be able to get it onto arXiv.

    ReplyDelete
  53. "Perhaps arXiv should have a dedicated section labeled "Pseudoscience", in which the rejected papers are dumped. That way at least everyone gets a fair hearing."

    We already have that. It's called viXra. No serious scientist reads stuff there.

    "Should it be later determined that a paper placed in the Pseudoscience repository meets the criteria of good science, then it could be reassigned to a legitimate category."

    Determined by whom?

    ReplyDelete
  54. The spelling rule I learned in college was that there is no "judge" in the word "judgment." Because the common mistake, which is shared in this blog post is to spell the word incorrectly as "judgement."

    Both forms exist; without the judge is American:

    "In Great Britain and many of its former colonies, “judgement” is still the correct spelling, but ever since Noah Webster decreed the first E superfluous, Americans have omitted it. Many of Webster’s crotchets have faded away (each year fewer people use the spelling “theater,” for instance), but even the producers of Terminator 2: Judgment Day chose the traditional American spelling. If you write “judgement” you should also write “colour.” "

    Source: http://public.wsu.edu/~brians/errors/judgement.html

    Think about it: it is much more logical if judge is in judgement. Ditto with acknowledgement.

    ReplyDelete
  55. @JimV: it is indeed a pity that this subject was brought up, but I'd like to make a few points here. The case wasn't about whether ID is a form of creationism, but about whether a particular book should be taught in local schools, and the arguments centred on something rather different, viz. whether ID is scientific or not. The problem we have here is that different arguments in favour differ in how compelling they are, and what sceptics tend to do is to pick on the worst of the arguments and treat them as representative. You shouldn't trust 'experts' holding strong views, as Miller does. For what it's worth, here are some items to consider:

    1) since the judgments of judges have been brought up, it is perhaps of interest to note that Lord Mackay of Clashfern, former Lord Chancellor (and incidentally an Honorary Fellow of my college), set up a meeting to inform selected people of the arguments in favour of ID. He obviously has different views to yours

    2) I asked the speaker at this event, Cambridge Ph.D. Stephen Meyer, if he was a creationist and he replied that he wasn't. I must say that this hardly surprised me, but I thought I should ask, for the record.

    ReplyDelete
  56. The lecture is available here, in case anyone is interested. And there's also a report of the event.

    ReplyDelete
  57. Just to be clear, ID as normally used in the US is almost exclusively used by creationists as an attempted back-door to get creationism taught in schools. There are, though, non-creationists who echo at least part of the intelligent-design argument: one often runs across this with people who think that we are living in a simulation, or that the universe evolved to be favourable to life, and so on.

    ReplyDelete
  58. Brian, I think the issue of ID is matter for philosophers. And I don't mean that as a put-down. One could speak of what criteria are required for an agent to be "intelligent" or what it means for an artifact to be "designed" -- and test those hypotheses against what occurs naturally.

    That's a trap, though -- ID allows no naturally occurring events. Though Stephen Meyer may not believe the world was created 6,000 years ago or that dinosaurs co-existed with humans, he is indeed a creationist.

    As it is, we know that almost anything that can be explained by the act of an intelligent agent, can be duplicated by complex systems science operating over multiple scales in a time dependent framework. If Nature is the agent, Spinoza's philosophy applies. Ask Meyer if he is a Spinozist. I seriously doubt it.

    Giving up common ancestry affects not just evolution; it destroys our whole idea of science as a rationalist enterprise.

    ReplyDelete
  59. @Phillip Helbig: I'm actually pursuing "extraordinary evidence" related to the "extraordinary claims" of others. If nothing shows up I probably won't even bother to try to publish the experimental procedure and results, not even in viXra.org, of which I was recently informed of.

    ReplyDelete
  60. It occurs to me that the central problem presented by Karl Popper -- the demarcation problem of science from non-science -- still has a ways to go.

    Pushing the questionable research into a category of "pseudoscience" only co-opts the scientific enterprise for ideas that don't merit it, as they don't often deal with objective knowledge.

    Philosophy is as much in need for improvement of its logical methods as physics needs to improve its mathematical methods.

    ReplyDelete
  61. Well, the thing I find most interesting, when it comes to the Intelligent Design argument, is, those who counter against it, like JimV above, more often than not invoke the "evolutionary algorithm" defense; but if the "evolutionary algorithm" isn't intelligent, just what the hell is? There seems to be a rather good reason that the "evolutionary algorithm" shows up, usually (if not always) in a fundamental role, in virtually every machine learning platform in existence (I say virtually because my knowledge is constrained to the immediate surroundings); that reason is, primarily, because it puts the learn in learning. And to my way of thinking, any design incapable of learning is certainly not the most intelligent design!

    Take for instance the Busy Beaver Turing Machines. These Busy little Beavers create a plethora of interesting patterns before they finally meet their match and halt, and the Busiest of the Busy are certainly intelligent designs, i.e. they reflect the intelligence of their creators, but they are certainly not the most intelligent design in the set of all pattern generating computational devices! And I would confidently conjecture that the most intelligent design in the set of all pattern generating computational devices, let's call it Maxwell's Demon, would rely quite heavily on some learning mechanism, most probably an "evolutionary algorithm."

    Also, to clarify a bit, when I stated previously that Strong Emergence is equivalent to the Buddhist concept of Anatman, I was thinking of Strong Emergence in the sense of John Wheeler's Law without Law, i.e. even the so-called fundamental laws emerge. To my way of thinking, to view the fundamental laws in any other way is to erroneously reify (Deify?) those laws . . . and Mathematics in general. This would seem to require some form of Platonic realm and some method of activating, i.e. bringing into being, the forms in that realm. If that's not Intelligent Design I don't know what is!

    To my way of thinking, mathematics is simply a method of knowledge acquisition. It's probably one of the better methods but it's only one of many. I view what Buddhists call conventional reality as the set of all patterns, validated by consensus, in our perception and Mathematics is really just the study of patterns so it seems natural that Mathematics should be an effective tool of knowledge acquisition but I don't reify that knowledge. The differential equation describing simple harmonic motion has a number of approximate actualizations in our perceptual apparatus, i.e. springs, pendulums, certain electrical circuits, but the pattern it describes has no existence without that equation or those physical actualizations.

    ReplyDelete
  62. Of course to a Buddhist, such as myself, nothing in conventional reality has intrinsic existence, rather, it all emerges do to the play of the Buddha Bodies; it's an epi-phenomena in the sense of Hofstadter. And this play has never been born and shall never die, i.e. it has no beginning or end, it is of infinite duration. To a Buddhist, such as myself, ultimate reality is what you experience between physical death and physical rebirth; it's referred to as emptiness because it is a state of pure symmetry, pre-distinction, pre-individual, non-dual; it is primordial awareness! You, meaning anyone, can access this primordial state via meditation; scientists, including Harvard cardiologist, Herbert Benson (is he an outsider?), have studied yogis in meditative states who have lowered their metabolism by as much as 70%; those yogis are experiencing the primordial state. This is why they're called mystics; they experientially understand the mystery!

    Quite frankly, I've never understood why "insider" scientists are so intent on limiting themselves to a subset of all that can be known. Imagine if mathematicians did the same? Have you ever read the book Flatland? The Second Law doesn't apply in this case; Maxwell's Demon is injecting energy into the system and the patterns are accumulating; how long can "insider" scientists ignore them and still be accepted as scientists? Something tells me we are going to find out . . .

    ReplyDelete
  63. Re the various new points:

    @Philip: it may well be that real creationists use ID to support their position, but that doesn't imply that ID is creationism, though it may well have contributed to the widespread confusion in regard to this point.

    @t h ray: a related misconception: creationism doesn't allow naturally occurring events but ID does allow for them, in fact I believe people working on ID accept the existence of evolution, but the issue is whether the conventional view is the whole story. It's a question of probabilities and we don't have a real calculation, in the absence of which it is an act of faith that current science provides the whole explanation.

    @Wes H: I believe ID people don't think the even the usual evolutionary algorithms in whatever form will suffice to explain everythinh. My own view is that 'the trouble with physics' is going to be sorted out before too long and that will imply acceptance of ID as part of it.

    ReplyDelete
  64. "those who counter against it, like JimV above, more often than not invoke the "evolutionary algorithm" defense; but if the "evolutionary algorithm" isn't intelligent, just what the hell is?"

    To clarify, in my case I argue that "intelligence" is in fact that evolutionary algorithm in action:

    1) Generation of some things (in biological evolution, genetic mutations; in intelligence, ideas) naturally, often randomly (e.g, random firing of neurons in dreams and musings).

    2) Selection mechanisms which filter out the less worthwhile things (e.g., natural selection; peer review; survival of the fittest in the marketplace, for commercial designs). (As with biological evolution there is also "neutral evolution" in which ideas which are neither very good nor very bad become fixed in a population by luck.)

    3) Some form or forms of memory to pass along the results of 2) for future use and further development (in biological evolution, DNA).

    As a design engineer with 35 years of experience designing complex machines (steam and gas turbines which involve thermodynamics, fluid dynamics, metallurgy, stress and strain, vibration analysis, etc.) I can testify that human "design" certainly proceeds by trial and error according to the above algorithm. I can cite lots of examples of this and supporting statements from experts (Einstein, Bohr, Edison, etc.) if asked. We have all seen automobiles, phones, computers, etc. evolve in our lifetimes: design is evolutionary.

    I think "intelligence" is also, that is, it produces ideas and makes progress by turning the crank of the above algorithm (using 70+ billion neurons in the most powerful nanotech super-computers yet known on Earth).

    Right or wrong, it seems to me that people who support or claim to do research in "intelligent design" should start by researching how human intelligence and design actually work. Until they can explain these things without invoking magic, what have they explained? And if in fact the I and D in ID are just more evolution in action, what is left of their argument?

    (Apologies for commenting again. I itched to rebut comments about the Dover trial above but left it to people to read the online sources for themselves. However, it appeared from the above quote that I had not made my position clear about the evolutionary algorithm.)

    ReplyDelete
  65. I agree with you Brian, that it is an act of faith to accept that current science provides the whole explanation ... of what? The origin of life? Neither science nor ID can even say what 'life' is.

    Who claims that science provides the 'whole explanation' of anything? It only explains what can be explained, and what can be explained is that which can be defined.

    That's what makes the demarcation problem so important. There's no profit in mixing science and philosophy, if we want to maintain science as a rationalist enterprise.

    Tom

    ReplyDelete
  66. @JimV: I agree with their being something along the lines of your 'evolutionary algorithm'. But there is the issue that I raised above, whether this 'Force' might be equivalent to what some experience as God, impling a kind of unification of science and religion, rather than the two being opposed to each other as commonly assumed.

    ReplyDelete
  67. Brian, I appreciate the subtle and delicate position you have taken. On the other hand, I see the potential to exploit this opening by religious demagogues for their own gain, in opposition to science.

    If by 'unifying' science and spirituality, you mean the rational spirituality of Spinoza, why not just say that?

    One can't fault your logic:

    "Logically, the subtler experiences of mysticism can also be interpreted as being a part of the workings of intelligence in nature (which characterization can, in principle, also include the divinei intelligence). Because of our consciousness we can have direct knowledge of these processes that is not accessible to the usual means of investigation of science and, as previously noted, it is illogical to rule out this kind of knowledge while accepting the use of mathematics."

    This is not a view shared by religionists for whom science and mathematics is subordinate to a singular Supreme Being.

    ReplyDelete
  68. @th: so you would halt progress in physics because religious demagogues might like it? But cutting the ground from under the feet of atheists would be a good outcome.

    No, it isn't the rational spirituality of Spinoza, but much more similar conventional science.

    Those religionists who are unfamiliar with meditative practices will indeed not get the point.

    ReplyDelete
  69. Brian, to be sure, you have thought more and written more on this subject than most. Just to cut the ground from under atheists, though, seems to me a goal unworthy of science -- as would be the mollifying demagogues.

    No, I wouldn't want to halt progress in physics for any reason. However, if you include mathematics research with meditative practices, they surely will not get the point. This is an art reserved for a very, very few who "do mathematics" -- make conjectures, prove theorems -- and excludes the great majority of those who consume mathematics, though they may call themselves mathematicians.

    I am at a loss to perceive any operational difference between that rationalist spirituality and the philosophy of Spinoza. Mathematics is demonstrably objective, and not in the category of what Einstein called the "merely personal" that might come of one's personal meditation.

    Kurt Godel said, “I believe that the most fruitful principle for gaining an overall view of the possible world-views will be to divide them up according to the degree and the manner of their affinity to or, respectively, turning away from metaphysics (or religion). In this way we immediately obtain a division into two groups: scepticism, materialism and positivism stand on one side, spiritualism, idealism and theology on the other. We also at once see degrees of difference in this sequence, in that scepticism stands even farther away from theology than does materialism, while on the other hand idealism, e.g., in its pantheistic form, is a weakened form of theology in the proper sense.”

    ReplyDelete
  70. @th: you misunderstand me. That article was not the theory.

    ReplyDelete
  71. "I believe ID people don't think that even the usual evolutionary algorithms in whatever form will suffice to explain everything."

    I'm the guy who posted the NON comment on Ben Goertzel's Rupert Sheldrake blog post:

    http://multiverseaccordingtoben.blogspot.com/2014/09/morphic-fields-memory-psi-unified.html

    You don't have to argue very hard to convince me that there is more to evolution than evolution!

    By the way, I read the book Sync which, as you may know, contains a nice little account of your debate with Bardeen. Unlike Helbig and the other Cabalists, I greatly appreciate your scientific integrity and the scientific integrity of those like you: Will Tiller; Rupert Sheldrake; Dean Radin; Ben Goertzel; Nick Herbert . . . I don't know if you read Herbert's blog but he recently passed the torch to a young Brazilian physicist, Gabriel Guerrer:

    http://quantumtantra.blogspot.com/

    Dr. Guerrer is in the process of replicating Dean Radin's two-slit experiment:

    http://deanradin.blogspot.com/2014/03/psychophysical-interactions-with-double.html

    This is what I meant above when I said the patterns keep accumulating. I subscribe to a number of newsletters, the Tiller Institute, the Mind and Life Institute, the Institute of Noetic Sciences, etc., and they are all steadily cranking out high-quality experiments. And to bring this back to the subject of this blog post, I just find it absolutely absurd that scientists of your own and Will Tiller's caliber can apply the same finely honed scientific methods and critical analysis that you apply to enable distinguished careers in the "insider" Cabal, to subjects deemed "lunatic fringe" by Mr. Ginsparg's algorithm and all of a sudden you're shut out of the arxiv and labeled "outsiders." And absolutely absurd is being overly polite!

    I already know what the problem with science is; it's the same problem afflicting mainstream religion: an aversion to the truth! It's pretty telling when an "insider" (I think Lee Smolin is an insider but I should probably check with Helbig) such as Smolin begins his book Three Roads with the appeal, "If we could all just be honest . . ." I have never understood why some people have such an aversion to the truth. Actually, it's a pathology, I think, which is explained rather well by Goertzel's Complex Systems model of mind . . .

    @thray

    "Mathematics is demonstrably objective"

    What the hell are you talking about!?! What mathematicians do is constrain the Universe of Logical Discourse with a set of axioms and then study the patterns that emerge do to that set of axioms. Those axioms are completely subjective! Hell, the logic underlying those axioms is completely subjective; if there's some objective "Master" logic that you're aware of I certainly wish you would direct my attention to it! And then there's the fact that we originally abstracted mathematics from the world around us; it's not like we were born with the Hindu-Arabic base ten system fresh in our minds! There were quite a few different Numeric systems in use before the Hindu-Arabic system, some of which didn't even contain a signifier for zero! You should read Gilbert Simondon's fine paper:

    http://www.columbia.edu/cu/arts/vad/critical_issues_on_art/Simondon.PDF

    And I'm wondering if you could possibly clarify this statement, "However, if you include mathematics research with meditative practices, they surely will not get the point." I guess I don't understand what "point" you are referring to . . .

    ReplyDelete
  72. @thray,

    "However, if you include mathematics research with meditative practices, they surely will not get the point."

    I understand what you're saying; you're referring to this quote:

    "Because of our consciousness we can have direct knowledge of these processes that is not accessible to the usual means of investigation of science and, as previously noted, it is illogical to rule out this kind of knowledge while accepting the use of mathematics."

    I somewhat agree with you on this one; even though mathematics is itself a meditation, most mathematicians would probably not validate meditative experience based on their experience with mathematics. Mathematics is purely a mental exercise while meditative practice, properly executed, results in a psycho-physiological experience commonly referred to as the Chandalini Awakening. It is also known to transform the brain:

    https://www.youtube.com/watch?v=m8rRzTtP7Tc

    Sometimes math stresses me out, to the point I can't even do yoga, but yoga and meditation never stress me out, quite the opposite rather. But anyway, I still say mathematics is not at all objective:

    http://abcnews.go.com/2020/autistic-savant-daniel-tammet-solves-problems-blink-eye/story?id=10759598

    ReplyDelete
  73. Brian, I'm happy to expose my own prejudices. Rationalism holds theory as primary, so if I haven't dug deep enough to understand the correspondence between the theory of ID and evolutionary history, I beg to be set straight.

    If the correspondence falls outside rational boundaries, however, I would have a hard time following the argument.

    Is there an argument, in fact, that preserves science as a rationalist enterprise while making room for anti-realistic theories? Isn't there a point where antirealism becomes anti-rational?

    ReplyDelete
  74. Wes, you may believe that mathematics is not objective; in that case, you will find it hard to apply a mathematical theory or technique to a physical problem.

    As a language art, mathematics is "bigger" than physics; it has the facility for capturing this world and another -- mathematical -- world, that may or may not "exist." It is the terms of existence we objectify, an artist's medium.

    That is why I am interested in Professor Josephson's theory.

    ReplyDelete
  75. @th: in science, contrary to your position, observation is primary, not theory. If observation and theory don't agree then it is theory that has to make way (unless the observation can be shown to be in error). If you choose to define rationalism in a way that contradicts scientific practice, then we need to part company. Discussion of the remainder of your contribution is perhaps best left to philosophers, unless you can make your intent clear to ordinary mortals.

    ReplyDelete
  76. @th: you can see a version of the theory before a lot of details were sorted out in my Imperial College Maths Workshop on Time lecture. Subsequent to that talk someone with whom I was discussing it drew my attention to the work of Karen Barad. Her subtle concepts such as intra-action have allowed details as to mechanism to be filled in, but you'll have to wait for those details. But basically it's all about the existence of a kind of order well known to humanists but which physicists haven't as yet got round to thinking about.

    ReplyDelete
    Replies
    1. Great comment. My research is all about Barad and I use Hossenfelder's work as well.

      Delete
  77. Brian, that may be Aristotle's idea of science -- so how how do you answer the induction problem of Hume?

    Conclusions based on observation make no closed logical judgement: what would cosmic background radiation be without the theory of the big bang? Noise without meaning.

    And this in no way contradicts scientific practice. So far as I know, Karl Popper was the first philosopher of science to take Hume's problem seriously -- and solve it. Results must be interpreted in context of a theory, and only in that context. Popper appropriated Tarski's correspondence principle of truth to define his framework for objective knowledge -- an "unending quest." That correspondence is what 'rational' means.

    I don't want to create bad feelings, though, so this can be the end of it if you please.

    ReplyDelete
  78. @th: You seem to have little experience of real science. What have you got to say about superconductivity for example, which for more than 40 years had no valid theory to accompany it and was purely the observation that the resistance of certain metals became zero at low temperatures? No-one said we should ignore this observation until there is a theory that we can do a Popperian act upon; it would indeed have been irrational to do that. And cosmic background radiation would have been hailed as in important discovery even had this discovery preceded the idea of the big bang.

    This discussion is indeed rather proving a waste of time and I may not respond further.

    ReplyDelete
  79. Phillip Heilbeg:
    As I pointed out earlier, to a historian of science it is much too simplistic to baldly claim that 'Einstein was by no stretch an outsider in 1905', especially without reference to any evidence to back the claim. (It is also simplistic to claim the opposite).
    For example, AE himself commented later that he had limited access to publications while at Berne, and limited interaction with other physicists (some historians suggest the latter may even have been an advantage).
    In some ways, AE was very much an outsider in these years (see the comments of Professor Kleiner and Professor Minkowski for example) -in other ways not so much. History is never quite that simple.
    Regards, Cormac

    ReplyDelete
  80. Brian, no one is arguing that observation plays no role. It is conclusion by induction that is at issue.

    I realize that "real science" to an experimenter is identical to doing experiments. I don't know any way of interpreting those experimental results other than theory.

    ReplyDelete
  81. Hi Mr Josephson,
    You say "No-one said we should ignore this observation until there is a theory that we can do a Popperian act upon; it would indeed have been irrational to do that"

    Well that's right because there was an empirical foothold as you describe. But I was sure that superconductivity was theoretically available from the fundamental description of temperature, as 'vibration' of atoms.
    ----> cooling temperature gets less vibration, and less vibration in a metallic crystal lattice widens the channel for electrons significantly. I'd be surprised if that took 40 years.

    ReplyDelete
  82. " Results must be interpreted in context of a theory"

    Sure, but Popper elsewhere demonstrated that all actions are 'theory-laden'. Theories were always in play, just not always explicit while theorist not necessarily consciously aware.

    Something that faults Popperians is the way that point they make (that you did) about theory encourages misinterpretation by the way it is presented. People go away visualizing a finalized robust theory.

    ReplyDelete
  83. "you may believe that mathematics is not objective; in that case, you will find it hard to apply a mathematical theory or technique to a physical problem."

    It would seem to me that you conflate objectivity with inter-subjectivity; they are two distinct things! The reality we are embedded in is an inter-subjective construct and mathematics is an inter-subjective practice; both of these are grounded in information - pattern. The fact that a number of subjects in a shared environment or activity reach consensus about that environment or activity is no escape from subjectivity, rather, it's simply an extension of that subjectivity. I mean, if it's all objective then why the hell can't we all agree on an interpretation of Quantum Mechanics? Surely one and only one interprets THE objective mathematical object isomorphic to THE objective reality!

    "It is the terms of existence we objectify, an artist's medium."

    When one begins an mathematical exercise the primitive terms are left undefined, their meaning induced in the process of individuation. You're trying to tell me that that induction is objective!?! Did you even bother to read the autistic-savant article from abcnews.com I linked to above? Obviously it would seem as though you did not. Quite honestly, I would love to see the Mathematical Universe the way that fellow does!

    To be quite honest, there's Theorems accepted and utilized by the mathematical community which I just don't accept; I can demonstrate counter-examples for all of them! They're not Theorems, rather, they're wishful thinking but yet they're utilized because they make things possible which otherwise would not be! And they seem to hold in the vast majority of cases (well, some of them anyway). But there's absolutely NOTHING objective about that! In my opinion, every Theorem, Corollary, and Lemma, starting with first-order logic or any other logic you, as a subject, may wish to utilize, should come with a p-value indicating the probability that it holds in the general case; that would give the "term" fuzzy math a whole new interpretation!

    ReplyDelete
  84. th ray,

    Listen, perhaps I'm being a bit harsh with mathematics and the mathematical community; I'm not even a member being almost completely self-taught and understanding a limited subset. But even so, I love mathematics, you know; it's a wonderful and elegant art form, of course, and extremely useful when applied. I don't mean to be cantankerous towards you or anyone else . . . well, maybe Phil Helbig. But he and I have a bit of a history . . .

    With regards,
    Wes Hansen

    ReplyDelete
  85. I started recently to follow your debate. If you have enough endurance and enough spare time you may read the following linked paper: “Space, this great unknown”, which in fact addresses a variety of issues and some of those you have talked about. It deals in part with nowadays deep lack of realism in theoretical physics.

    https://www.researchgate.net/publication/301585930_Space_this_great_unknown

    ReplyDelete
  86. @plain skee: It was well understood that the resistance of a metal drops as the temperature is lowered, for reasons more or less as you have cited. The problem was that in the case of superconductivity the resistance actually became zero even above absolute zero, where the vibrations are still present.

    @th: Latakos demonstrated that Popper's picture doesn't describe the way real science works.

    ReplyDelete
  87. Let us address back the original topic about the arXiv filtering. Since it only accept “small mutations” of nowadays official standpoints it ends up being a subtle way to reject any unwelcome departure from conventional standpoints in order to preserve the very many established interests.

    I agree with David Schroeder who said: “Perhaps arXiv should have a dedicated section labeled "Pseudoscience", in which the rejected papers are dumped. That way at least everyone gets a fair hearing. Should it be later determined that a paper placed in the Pseudoscience repository meets the criteria of good science, then it could be reassigned to a legitimate category”.

    I think the new section should not be labelled "Pseudoscience" but instead simply: “Rejected papers”. This way people could get their own opinion about what is rejected and why it is. It will be much fairer!

    ReplyDelete
  88. Brian,

    That's right, re Lakatos. That is why Popper framed his work by 'verisimilitude' -- truth likeness. The point is, how confident can one be in a theory developed from observation alone? If there is no demonstrable independence of theory from result, one risks proving what one assumed in the first place.

    That's acceptable for mathematics, toxic to science -- because there is no way from there to show that one describes an objective mind-independent phenomenon. Now perhaps you believe that ultimately there is no mind-independent reality; you would never be able to show that it is true.

    With Popper's philosophy, it is possible to show that reality is not mind-independent in a rationally objective way, a problem he attacked in the book, Realism and the Aim of Science. One may be compelling and avoid the charge of mysticism.

    ReplyDelete
  89. Wes, no offense taken.

    In fact, objectivity and inter-subjectivity are not operationally different. The problem with quantum mechanics that you cite -- so many interpretations -- is the central problem of induction.

    Mathematically complete theories (special relativity, e.g.) do not suffer from mistakes in interpretation acquired inductively.

    ReplyDelete
  90. Wes, I didn't want to wander off topic, by referencing the story you linked. I will, though, speak to the Emily Dickinson poem that Tammet quotes:

    "The brain is wider than the sky,
    For, put them side by side,
    The one the other will contain
    With ease, and you beside."

    This experience is familiar to mathematicians who recognize that digital processing is an artificial imposition on brain mechanics. Numbers are our own invention.

    ReplyDelete
  91. "As I pointed out earlier, to a historian of science it is much too simplistic to baldly claim that 'Einstein was by no stretch an outsider in 1905', especially without reference to any evidence to back the claim."

    My evidence are several biographies, in English and German, some by people who knew him personally. Sure, he was not the typical run-of-the-mill physicist, but certainly not an outsider in the sense that he came from nowhere and revolutionized physics unexpectedly, perhaps because he was an outsider. It depends on the definition, of course. In many respects Feynman was an outsider, or Dyson, but at the same time members of the establishment. I think that we can agree that he was not an outsider in the sense often claimed by crackpots.

    ReplyDelete
  92. The Trouble with Physics is a really good book; I appreciate Lee Smolin's honesty. More importantly, it helped me thoroughly comprehend the "point" Brian Josephson is making in the above quote.

    In his book Smolin states that the Multiverse is a relatively new concept but Hindu, Taoist, and Buddhist yogis/yoginis have been talking about the Multiverse for thousands of years - if not tens of thousands. And not one or two yogis/yoginis but many. The Multiverse entered the yogi/yogini discourse primarily because many of them have developed their mindstream to the point where they can remember their previous lifetimes, some of which have occurred in Universes distinct from this one! This, of course, would not even be considered relevant or valid data to most in the scientific community. But yet if the mathematics seems to imply the existence of a Multiverse some scientists accept its existence as fact, argument closed! And this is indeed a bit strange.

    Perhaps this is related to the question of objectivity in mathematics. I think many people consider mathematics objective because the structure determines the theory; if two distinct subjects, of approximately equal ability, both start with the exact same structure they can't help but derive the exact same theory. But the fact remains that we, as subjects, determine the structure! It's just that, with science, what one is being asked to accept is fully encapsulated in that structure and the conclusions, the theory, follows deductively. But as Smolin points out in the later part of his book, a great deal is taken on faith, faith in the integrity of the community. And the same is true of the mystic communities.

    The mystics have a structure in the form of a technology, a set of yoga's and meditations which have been honed but continue to evolve. Engaging in this technology is a rather difficult but very rewarding method of knowledge acquisition. In the very beginning one must have faith in the community in the sense that they represent living examples of the validity of the technology. But after only a few months of consistent practice one begins to see results and faith is no longer necessary (although the initial faith generally transforms into something closely approximating love).

    The point is the similarity in the situations. Regarding mathematics and science, one has two available rational options: you can subject yourself to the arduous process of learning the technology and making the deductions yourself or you can just trust the experts. Regarding the mystic path one also has available two rational options: you can subject yourself to the arduous process of learning the technology and making the deductions yourself or you can just trust the experts.

    Clearly, especially after reading Smolin's book, trusting the experts in one endeavor but not in the other doesn't seem very rational at all. And the reward for learning the mystical technology cannot be overly praised; it is the end of suffering, the conquering of time, the ability to "[r]est in a spacious modality, neither yearning for the past, anticipating the future, nor clinging to the present." Rest in a spacious modality - it's the yearning, anticipating, and the clinging that lead to suffering; why this is so is trivial.

    I would love to continue this discussion with Dr. Josephson and th ray but I must get back to work; I am in the midst of an arduous process . . .

    If I have offended you Dr. Hossenfelder I most sincerely apologize but on occasion "resistance is necessary for existence." To quote Lee Smolin, "We must have a basis for disagreeing with the majority without being labeled a crank."

    With regards,
    Wes Hansen

    ReplyDelete
  93. @Georges: Thanks for the supportive comment, on my mention of a pseudoscience category at arVix. But as Phillip Helbig pointed out "We already have that. It's called viXra. No serious scientist reads stuff there."

    I only browsed a few papers at viXra, and my impression is that it is a mixed bag of "The Good, the Bad, and the Ugly", to borrow the title from a 1967 Spaghetti Western. There's an excellent overview of the site's mission and purpose by its founder Phillip Gibbs. https://www.quora.com/Are-there-any-serious-papers-on-viXra

    I'm definitely thinking of posting several speculative papers there.

    ReplyDelete
  94. Fast machine-learning online optimization of ultra-cold-atom experiments

    http://www.nature.com/articles/srep25890

    ReplyDelete
  95. I was in the library the other day, looking for a specific book, and found this wonderful analysis: Where Mathematics Comes From: How the Embodied Mind Brings Mathematics into Being:

    http://www.cogsci.ucsd.edu/~nunez/web/FM.PDF

    I'm sure most formally educated individuals have read this book and if they haven't, they should, especially if they're an educator. Written by George Lakoff, a cognitive linguist somewhat notorious for his book, "Women, Fire, and Dangerous Things," title of which refers to an aboriginal language which places women, fire, and dangerous things all in the same linguistic category, and Rafael Nunez, a cognitive psychologist specializing in the nature and origin of mathematical ideas, the book represents the initiation of a methodology for the cognitive analysis of mathematical ideas. They start with scientific research indicating that babies only a few days old can subitize and recognize addition and subtraction involving arrays of objects up to roughly four in number. We share this innate ability with several of our relatives in the animal realm including pigeons, parrots, raccoon, rats, and chimpanzees. They then demonstrate how the entirety of, I would say, classical analysis emerges as an extension of this innate ability via a few standard cognitive schemas and metaphorical mappings, the metaphors being grounded in our experience interacting with the environment. The key element, to me anyway, is that the metaphorical mappings preserve inferential structure!

    They conclude by stating the Theory of Embodied Mathematics, said theory being developed in the earlier chapters, and then demonstrating that theory explicitly with a cognitive analysis of Euler's formula. Their analysis is simply amazing to me! The book is 450 pages but it's well worth reading the entire thing. The key conclusion is that mathematics is a function of how our brains work and how we interact with the environment and is driven by a desire for precision. "Mathematics is a systematic extension of the mechanisms of everyday cognition. Any fit between mathematics and the world is mediated by, and made possible by, human cognitive capacities. Any such fit occurs in the human mind, where we cognize both the world and mathematics." Of course this doesn't exclude the possibility that the entire Cosmos is an embodied mind utilizing similar or analogous cognitive capacities, schemas, and metaphors!

    This is what interests me about this whole mysterious business. Humans come into existence with an innate ability to recognize small groupings of distinct objects and the addition and subtraction of distinct objects to/from those groupings. This, of course, implies an innate ability to recognize certain symmetries! Based on that innate ability, we have, over the course of time, been able to construct the rich, complex, and ever-expanding Mathematical Universe. The key point here is the innate ability! Every toy model I am familiar with, whether Game of Life, Self-Generating Systems, Cellular Automata, or etc., begins with some innate ability, some simple rule or small set of rules, and from that innate ability emerges often quite complex pattern. And what is it that reductionist science is ultimately looking for? It is ultimately looking for the one simple rule, the innate ability, which gives rise to all of the complexity which is our Cosmos! Does it not seem reasonable to conjecture that that innate ability is grounded in something closely approximating cosmic intelligence? Does it not seem reasonable to conjecture that we humans exist in a type of mutually beneficial symbiosis with that intelligence, hence, our own innate abilities?

    ReplyDelete
  96. I just don't fathom how many seemingly reasonable people can consider the above conjectures absurd yet thoroughly embrace the idea that for every atomic decay event a continuum of Universes, all identical except for the time of decay, spontaneously splinter off from the pre-measurement Universe! Although after reading the above book, I can certainly imagine that there is a cognitive analytical explanation for it; probably something to do with a metaphorical mapping from General Relativity to Quantum Relativity, hence, Relative State, combined with an abhorrence for certain religious tyrannies. Of course that leads to something else I find unfathomable, the tendency of people, driven by an abhorrence for tyranny, to become themselves tyrannical . . .



    ReplyDelete
  97. I ponder the question for a bit (just a day or so) and it struck me that.....

    1) we have to implement priorities (or impose limits) on all levels of education (or training)

    I like standards, a generalized standard seems highly appropriate. The methods used in basic education enjoyed a lot of testing. At the top we could still have those with superior credentials pick the topics you should study (complexity) AND decide which ones to hide (efficiency)

    2) A (to me) quite surprising factor in setting the limits is the pool of available scientists.

    I've tried to explain the following a bunch of times but all to often people just don't get it. (One can see they don't get it in their work.) It seems something so stupidly simple it flies over peoples head. But here goes...

    The automations and services we provide and create make up chains. One can take the elevator down to the basement, get in the car, drive it to an airport, fly to some place else, rent a car there and travel an elevator to your hotel room.

    Would we remove the airport the other components continue to make up other chains. One might use a boat or drive the car to the destination.

    Odds are good however for the hotel and the car rental to depend far more on the airport than the elevator in your home/apartment complex.

    Would we remove the roads from the chain the purpose of owning a car is gone.

    A different example of such a chain would be purchasing a product on the internet.

    We need a service to be able to find a website offering the product we need, then the website has a number of carefully defined and carefully calibrated steps to convert interested visitors into customers.

    This chain is interesting as an example only because of the amount of effort invested in not wasting the visitors time, not creating unnecessary complexity or generally getting rid of everything that gets in the way of the conversion ratio.

    Now I dare suggest the chain of education [proverbially] to be full of missing airports and needlessly complex websites.

    For most people the science literature is that hotel you cant get to. The papers considered interesting enough for publication is the diversity of strangely styled rooms one can rent.

    There might be a theoretical market for papers equal to a room entirely decorated with a leopard skin motive. If those interested in that kind of thing cant get to the hotel... then it's academic.

    Every worthy publication deserves to have its readers. If we fail to produce those it wasn't worth writing.

    Look at all those people slaving away in utterly useless occupations? People are reading harry potter and they are playing pokemon go.

    Not that all this real world/science stuff couldn't be interesting to them. We just have a system of disjointed links that require a lot of determination to see a chain in.

    To become a scientist one first has to climb down on the outside of the building, then one has to walk 10 000 miles to the airport, pilot a barely functional airplane then sleep in a tent.

    The eventual science is less impressive than its accomplishment.

    If you feel that non of this makes any sense to you that would be exactly what I mean.

    ha-ha

    ReplyDelete
  98. "This life of yours which you are living is not merely a piece of the entire existence, but it is in a certain sense the whole; only this whole is not so constituted that it can be surveyed in one single glance.
    -- Erwin Schrödinger, My View of the World

    ReplyDelete
  99. FYI a very similar article has been published in Nautilus http://nautil.us/issue/41/selection/what-counts-as-science.

    ReplyDelete
  100. slh,

    Yes, I know. I also know that the author read my blogpost, but didn't bother to link to it. So excuse me for not being too excited about it.

    ReplyDelete
  101. I read this entire comment section in one sitting. I followed the argument as it ping ponged between commenters, understanding the points and counterpoints as they were presented. What sciences are represented in these remarks? I see math, physics, psychiatry?, Buddhism (it's not considered science by mainstream but is a practice that has potentially reproducible truths) theology, philosophy... I am going back to college again, and this debate represents what most stimulates me intellectually. Your help in understanding what fields to further educate myself in would be very helpful. My initial thought is quantum physics and philosophy? How does one validate by research subjective experiences that might be explained by quantum physics? I hope I don't sound too uneducated, but that's pretty much what I think I want to do.

    ReplyDelete

COMMENTS ON THIS BLOG ARE PERMANENTLY CLOSED. You can join the discussion on Patreon.

Note: Only a member of this blog may post a comment.