Monday, April 07, 2014

Will the social sciences ever become hard sciences?

The term “hard science” as opposed to “soft science” has no clear definition. But roughly speaking, the less the predictive power and the smaller the statistical significance, the softer the science. Physics, without doubt, is the hard core of the sciences, followed by the other natural sciences and the life sciences. The higher the complexity of the systems a research area is dealing with, the softer it tends to be. The social sciences are at the soft end of the spectrum.

To me the very purpose of research is making science increasingly harder. If you don’t want to improve on predictive power, what’s the point of science to begin with? The social sciences are soft mainly because data that quantifies the behavior of social, political, and economic systems is hard to come by: it’s huge amounts, difficult to obtain and even more difficult to handle. Historically, these research areas therefore worked with narratives relating plausible causal relations. Needless to say, as computing power skyrockets, increasingly larger data sets can be handled. So the social sciences are finally on the track to become useful. Or so you’d think if you’re a physicist.

But interestingly, there is a large opposition to this trend of hardening the social sciences, and this opposition is particularly pronounced towards physicists who take their knowledge to work on data about social systems. You can see this opposition in the comment section to every popular science article on the topic. “Social engineering!” they will yell accusingly.

It isn’t so surprising that social scientists themselves are unhappy because the boat of inadequate skills is sinking in the data sea and physics envy won’t keep it afloat. More interesting than the paddling social scientists is the public opposition to the idea that the behavior of social systems can be modeled, understood, and predicted. This opposition is an echo of the desperate belief in free will that ignores all evidence to the contrary. The desperation in both cases is based on unfounded fears, but unfortunately it results in a forward defense.

And so the world is full with people who argue that they must have free will because they believe they have free will, the ultimate confirmation bias. And when it comes to social systems they’ll snort at the physicists “People are not elementary particles”. That worries me, worries me more than their clinging to the belief in free will, because the only way we can solve the problems that mankind faces today – the global problems in highly connected and multi-layered political, social, economic and ecological networks – is to better understand and learn how to improve the systems that govern our lives.

That people are not elementary particles is not a particularly deep insight, but it collects several valid points of criticism:

  1. People are too difficult. You can’t predict them.

    Humans are made of a many elementary particles and even though you don’t have to know the exact motion of every single one of these particles, a person still has an awful lot of degrees of freedom and needs to be described by a lot of parameters. That’s a complicated way of saying people can do more things than electrons, and it isn’t always clear exactly why they do what they do.

    That is correct of course, but this objection fails to take into account that not all possible courses of action are always relevant. If it was true that people have too many possible ways to act to gather any useful knowledge about their behavior our world would be entirely dysfunctional. Our societies work only because people are to a large degree predictable.

    If you go shopping you expect certain behaviors of other people. You expect them to be dressed, you expect them to walk forwards, you expect them to read labels and put things into a cart. There, I’ve made a prediction about human behavior! Yawn, you say, I could have told you that. Sure you could, because making predictions about other people’s behavior is pretty much what we do all day. Modeling social systems is just a scientific version of this.

    This objection that people are just too complicated is also weak because, as a matter of fact, humans can and have been modeled with quite simple systems. This is particularly effective in situations when intuitive reaction trumps conscious deliberation. Existing examples are traffic flows or the density of crowds when they have to pass through narrow passages.

    So, yes, people are difficult and they can do strange things, more things than any model can presently capture. But modeling a system is always an oversimplification. The only way to find out whether that simplification works is to actually test it with data.

  2. People have free will. You cannot predict what they will do.

    To begin with it is highly questionable that people have free will. But leaving this aside for a moment, this objection confuses the predictability of individual behavior with the statistical trend of large numbers of people. Maybe you don’t feel like going to work tomorrow, but most people will go. Maybe you like to take walks in the pouring rain, but most people don’t. The existence of free will is in no conflict with discovering correlations between certain types of behavior or preferences in groups. It’s the same difference that doesn’t allow you to tell when your children will speak the first word or make the first step, but that almost certainly by the age of three they’ll have mastered it.

  3. People can understand the models and this knowledge makes predictions useless.

    This objection always stuns me. If that was true, why then isn’t obesity cured by telling people it will remain a problem? Why are the highways still clogged at 5pm if I predict they will be clogged? Why will people drink more beer if it’s free even though they know it’s free to make them drink more? Because the fact that a prediction exists in most cases doesn’t constitute any good reason to change behavior. I can predict that you will almost certainly still be alive when you finish reading this blogpost because I know this prediction is exceedingly unlikely to make you want to prove it wrong.

    Yes, there are cases when people’s knowledge of a prediction changes their behavior – self-fulfilling prophecies are the best-known examples of this. But this is the exception rather than the rule. In an earlier blogpost, I referred to this as societal fixed points. These are configurations in which the backreaction of the model into the system does not change the prediction. The simplest example is a model whose predictions few people know or care about.

  4. Effects don’t scale and don’t transfer.

    This objection is the most subtle one. It posits that the social sciences aren’t really sciences until you can do and reproduce the outcome of “experiments”, which may be designed or naturally occurring. The typical social experiment that lends itself to analysis will be in relatively small and well-controlled communities (say, testing the implementation of a new policy). But then you have to extrapolate from this how the results will be in larger and potentially very different communities. Increasing the size of the system might bring in entirely new effects that you didn’t even know of (doesn’t scale), and there are a lot of cultural variables that your experimental outcome might have depended on that you didn’t know of and thus cannot adjust for (doesn’t transfer). As a consequence, repeating the experiment elsewhere will not reproduce the outcome.

    Indeed, this is likely to happen and I think it is the major challenge in this type of research. For complex relations it will take a long time to identify the relevant environmental parameters and to learn how to account for their variation. The more parameters there are and the more relevant they are, the less the predictive value of a model will be. If there are too many parameters that have to be accounted for it basically means doing experiments is the only thing we can ever do. It seems plausible to me, even likely, that there are types of social behavior that fall into this category, and that will leave us with questions that we just cannot answer.

    However, whether or not a certain trend can or cannot be modeled we will only know by trying. We know that there are cases where it can be done. Geoffry West’s city theory I find a beautiful example where quite simple laws can be found in the midst of all these cultural and contextual differences.
In summary.

The social sciences will never be as “hard” as the natural sciences because there is much more variation among people than among particles and among cities than among molecules. But the social sciences have become harder already and there is no reason why this trend shouldn’t continue. I certainly hope it will continue because we need this knowledge to collectively solve the problems we have collectively created.


  1. Hi Bee,

    A nice piece regarding subject matter which you might say it be you that had me to become concerned about as to care about. My only comment is in respect to your feeling that social science might not ever become asvhard a science as physics I think is still to be decided about by what is true in physics itself, more specifically as to what is found to be tue in QM in respect to the limits of Quantum Computing. That is it is still possible that we could find no problem to be too complex for the power of quantum computers.



  2. The development rapidly making the social sciences harder has been not computing power but the realization that much behavior is programmed, especially by genetics. As the genetic roots of behavior become better understood, the ability to predict and control behavior will become more precise.

    So far, that knowledge has been used mostly for such purposes as getting us to buy more Big Macs and Cokes. Conceivably, it might even be harnessed to make people less violent and able to lead better lives.

  3. It may be that in some ultimate sense human behaviour is no more predictable in the spacious now than say QM as limits or any statistical system partly prediciable yet so in the average like experiments of cold fusion or ESP,the foundations an open question no matter how much we extend wisdom in complexity.
    The second revolution in psychology was behaviourism after psychodynamics of hidden mental things. (& in the political world what is happening, freedom as vague conspiracies aside what is covert is rarely what is in explanations in the news)
    The third expected revolution in psychology did not happen unless we believe we stumbled on it by the wide spread drug culture which brings us back to nerve nets or biological complexity hardly a hard science yet. What did we imagine a harder revolution to be if physics itself is not better resolved.?
    The good news that this question of social gravity is asked and clearly presented sensibly in scientific terms by a more mentally sound new generation of our scientists.
    That this blog does so is encouraging. It in a careful way predictable for a better human evolution that fulfills what remains irreducibly human in a sea of those in effect promoting lies.
    But a better prediction or observation as to our idealized punctuated hopeful monster outcomes past fits and starts is that in our future we can truly ask with so much civilization in the world why still so much inhumanity in war and personal violence.
    For when that revolution comes the vanishing of violence we may only look back on with respect for struggling humans with sorrow. For in such a world even if the power of one individual is that he can destroy the world it cannot happen, nor that society will act to crush him as it collapses on itself with failed decisions. That such social gravity becomes hard science is possible.

  4. Economics embraces hundreds of years of intensely dense, local and global real world data to accurately predict nothing: mercatilism, Marx, the Boys from Chicago, Obamunism; the operative value of anything (taxation, Quantitative Easing, ZIRP}. Psychology is a roiling disaster: pathology, education, social policy, teenagers, Korporate Kulture, prisons, elections, art, even its own rules and billing books (DSM), P greater than 0.99.

    The "soft sciences" are fiber-gorged floating excrement stinking of their own imposed importance. They are flashy mediocrities holding a vast audience diligently converting glitter into a tyranny of immersive falsehoods. A society inundated with rules from religious and secular political classes ignoring productive ends (e.g., Rapa Nui moai), will collapse.

    What economist ever got rich playing the stock market? The empirically most valid collection of psychologists ever to appear started WWII (and lost it).

  5. My guess is, the validity and scope of social science and physics will converge mutually. After all, the quantum gravity and landscapes of stringy theories is nothing, what can be considered a hard science even today, despite the pile of math behind all of it. At this place we can just ask the Bee, which physical, i.e. testable/measurable prediction she did during her whole scientific carrier? Just single one, please - and we'll see...

  6. Zephir,
    I can name one although it is a thought experiment never quite resolved to your satisfaction in this social setting.
    Show that a presumably intelligent scientific and social entity labled Zephir understands the higher level of what the Bee is saying.

  7. Well, Zephir, I predicted you'd still be alive when you've reached the comment box. And I was right...

  8. This was just a postdiction, not prediction (i.e. interpretation of my above post). And it has nothing to do with your scientific carrier (you were correct, but you're not payed for it).

  9. What are you seeking to predict in the social sciences? For the most part, we don't try to follow each molecule in a gas; the properties that interest us are the properties of the aggregate.

    People who employ other people are most interested in predicting the performance of the prospective employees. They've taken over crude measures like IQ and standardized tests; but have also gone to big data like Google:

    Google at least recognizes that performance is not an intrinsic property of the employee but a joint property of the employee and the environment that Google provides.

    Advertisers, politicians I think are more interested in predicting of aggregates of people, not of individuals.

    etc., etc.

    Then of course, there are questions like "What will Putin do?" and the problem is that even if you had a predictive science in place, you won't get enough initial value data; even with all the existing spying and snooping technologies.

  10. There is a problem with the idea that genetics will make human behavior predictable. For instance, the scientists found neural circuits that cause monkeys to identify snakes in their visual field faster than other neutral objects. Nice, clearly being able to see snakes faster carries an evolutionary advantage for monkeys, and this neural circuit is genetically determined, etc., etc.

    So what happens to these neural circuits in humans? Well, it turns out humans too can spot threatening objects in their visual fields faster than neutral objects. So the monkey neural circuit is there in humans as well. BUT! for humans, what a threatening object is, is not hardwired into the brain, it is learned! When little children learn guns are a threat, thereafter this monkey neural circuit kicks in, and they can identify guns faster than neutral objects - but only after they've learned guns are a threat.

  11. Physicists applying mathematical models to studying human behavoirs make perfect sense. They have been doing this since at least the 50's in applying game theory to economics and war games. Wall St. from what I've read and heard loves physicists and their abilities to understand and model complex systems. I can imagine physicists who work on pattern recognition algorithms for large detectors at the LHC involved in analyzing big data from various social sciences. An interesting project that physicists are working on in the dynamics of New York City is mentioned here:
    I would hope that social scientists would welcome a collaboration with physicists.

  12. Jerry,
    machines sometimes make rare human mistakes but if they do not get confused as if desparate game theory shows cooperation the best strategy - a lesson social systems seem slow to learn.
    Besides, there are better ways to set up pattern recognition in the LHC. Humans still better than computers with that. But that requires we understand the nature of particles we are searching for to program it.

    Arun, for awhile now I have enjoyed your posts - even ones from long ago.

  13. Sure stands to reason, if one posits that hard sciences have become softer by virtue of a new-found subjectivity - i.e. requiring an observer in any statement. Another reason is that social dynamics revolve around individuated individuals, but there is more and more evidence that what goes for one person also goes for part of a person, and goes for a group of people. Examples here include schizophreic behavior, left and right brained behavior, and group dynamics.

  14. Zephir:

    1) You just demonstrated (again) that you didn't read the post you were commenting on.

    2) You have told us several times before that you have not read any of my papers and have no interest in reading them, so why should I waste time in explaining you what I do?

    3) I have written several times about my work and a link to my publication list is in the sidebar. Not that I honestly believe you will actually look at it.

    4) I do not think and never said that the purpose of all scientific inquiry must be making predictions. Your misunderstand is probably due to

    1) You didn't read what I wrote...



  15. /* why should I waste time in explaining you what I do */

    Hello, Bee - You don't have to explain anything. I was only interested about some testable prediction, by which could distinguish your hard science from soft social science qualitatively. That's the whole point.

    Best Zeph.

  16. After correctly predicting what people said before they verbalized what they said, I was severely reprimanded and ostracized not to do this in this future.

    Now I am a closet whatever.
    I keep to myself about what is about to be said.

    That way no one feels insecure.

    Still, no one ever feels threaten when no amount of improvising you will ever perform musically is an unbreakable code of melody no one can recognize.

    That is a nice subtle soft! sciencey way of stating super-determinism where no one takes issue with it.

    If you know the script to any role,
    then the predictability will anger those convinced that no role will ever play any part of their life.

    Like the role of being a mother, father, child or physicist.

    All good. All soft. All the best.

  17. Stuart: Off-topic, please don't do that. Edgar: Sorry, your reply had to go too.

  18. The universe is now deterministic,
    arxiv:1404.1207 "Spontaneous Creation Of The Universe From Nothing"

    That the soft - and especially the social - sciences are empirical disasters offers some hope that tomorrow will recant.

  19. Uncle Al,
    As philosophy the question is still open. As to applying it to science and social science spontanity can be excluded if nothingness is construded as singularity. Symmetry breaking is a simple fact of arithmetic which needs a wider view in physical group dimensions and coherent looping. That DNA can be read in 4's and there are now tetraquarks observable tells us much about physical systems. We reach the paradox again of where the information may go.
    I did not read this arXiv. We cannot simply put the generational problem in terms of supersymmetric particles

  20. This blog post and the West interview brings to mind Wheeler's "law without law" ( and other "constructivist" ideas. I also read a book by the MIT engineer, Adrian Bejan, which postulates that scaling laws, what he calls the "Constructal Law," are a fundamental property of all Thermodynamic systems(to him, everything in "existence" is a thermodynamic system). As to recommending the book, I'm neutral, but to me it didn't really seem to explain anything, rather, it was just a collection of observations; however, the prevalence and usefulness of scaling laws is rather profound (

    As to the problems facing our world, perhaps there is nothing we can do, perhaps it's the manifestation of a scaling phenomenon. Back in 2005, the Berkeley physicist and his grad student,Richard Muller and Robert Rohde, respectively, discovered an emergent pattern in the fossil record indicating massive extinction events occur with regularity every ~62 million years, so, we're overdue! It's like we're stuck in some strange attractor where life evolves to a relative degree of complexity and then collapses. Of course the objective becomes finding a trajectory (a scaling law) that culminates in the velocity necessary to escape the current attractor so we can embody a deeper one representing a greater degree of relative complexity. So, you know, we can think of it as a race: too slow and we meet West's singularity; too fast and we burn up in exit; just right and we meet up with Kurzweil's singularity. So, in my world, which is dominated by the Red Queen and her court of impossibility, what we do is spend significantly less on politics and more on scientific research with emphasis on AI . . . or, we could all become Evangelical Christians and go play in the silly theme park they built overlooking Tel Maggido AKA Armeggedon. IT'S THE END OF THE WORLD AS WE KNOW IT AND I FEEL FIIIIIIIIIIIINNNNNEEEEE . . .

  21. Regarding free will, I would like to say again that you're making a categorical error. When you talk about existence, you should be careful whether you're talking about fundamental existence or existence at a macroscopic level.
    Human, like a baseball, does not exist at the fundamental level. But your argument that human does not have free will is about the fundamental existence if free will. Of course this is true, but it is also trivial- you're saying that a macroscopic existence does not possess some fundamental property. True but obvious.
    The free will that we should be talking about is, loosely speaking, the macroscopic free will. So the question becomes "whether some macroscopic possess some macroscopic property". Now this is the correct question and for this question Yes is a possible answer.

  22. Tytung,

    You're not listening to what I am saying. Macroscopic properties can always be derived from the microscopic properties. They become 'not fundamental' by deciding to neglect certain facts about them for the sake of simplicity. Of course you can do that and of course then you have 'freedom', but you have generated that freedom just by willfully ignoring information. I don't know what you think you gain by doing that. Best,


  23. Hi,
    Thanks for the reply.
    Yes I understand that we gain nothing by that way.
    But shouldn't it that when we say "A has property B", both A and B must be at the same level (of emergence)?
    Yes we need to throw away information to get "free will", but if we retain those all information, we cannot talk about "A" anymore.

  24. “To begin with it is highly questionable that people have free will.”

    When folks come to the door with their pamphlets inquiring if I know Jesus, I am civil and they in turn do not beat me over the head with their Bibles. Yet, here I am repeatedly bludgeoned by assertions that seem to be more a matter of faith than provable science. A reference beyond your thoroughly familiar blog post would be welcomed.

    Forgive me if this seems strident and perhaps even rude. Consider it theatrical excess. Yet still, the notion of all-encompassing clockwork determinism oppresses me. It is contrary to my sense of how the world works (a sentiment perhaps not bearing much weight for a quantum physicist).

    Free will is an exotic blossom far out at the tip of the branch, let’s leave it aside. What deeply concerns me is that the rationale that denies free will has consequences for the branch itself. That is, it removes entirely the determinative potential of any present moment and I don’t believe that the complexity of living systems can arise under those conditions.
    Consider a branch physical rather than metaphoric. Is it not a dynamical system that is adapting to complex and changing stresses through a complimentary series of small determinative present moments?

    “Macroscopic properties can always be derived from the microscopic properties.”

    Surely there is a compelling Lego logic here and conceivably this could be proven. But, perhaps there is another, meta-level organizing principle at work, something not conveyed in the microscopic description.

    Consider that a pitched baseball speeding through the strike zone has the kinetic energy equivalent to about 10^20 electron volts and that the net energy gain in a photon/chlorophyll antenna interaction is one electron. This means that 10^20 energy impulses, widely disparate in time and space, have somehow been reconciled into the confluence of a single vector. On the microscopic level this is a highly improbable event.

    Now, no physical laws have been violated, but can this be completely explained at the microscopic level? Do we expect that the barnacles attached to the hull should determine the navigation of the ship?

    I am not asking for much. Perhaps only one deterministically valid present moment in 10^50 would allow me my familiar world.


  25. This of course should read,
    "Consider that a pitched baseball speeding through the strike zone has the kinetic energy equivalent to about 10^20 electron volts and that the net energy gain in a photon/chlorophyll antenna interaction is one electron volt.


Comment moderation on this blog is turned on.
Submitted comments will only appear after manual approval, which can take up to 24 hours.