Saturday, October 03, 2020

What are Differential Equations and how do they work?

[This is a transcript of the video embedded below. Some parts of the text may not make sense without the graphics in the video.]

Today I want to talk about that piece of mathematics which describes, for all we currently know, everything: Differential Equations. Pandemic models? Differential equations. Expansion of the universe? Differential equations. Climate models? Differential equations. Financial markets? Differential equations. Quantum mechanics? Guess what, differential equations.

I find it hard to think of anything that’s more relevant for understanding how the world works than differential equations. Differential equations are the key to making predictions and to finding out what is predictable, from the motion of galaxies to the weather, to human behavior. In this video I will tell you what differential equations are and how they work, give you some simple examples, tell you where they are used in science today, and discuss what they mean for the question whether our future is determined already.

To get an idea for how differential equations work, let us look at a simple example: The spread of a disease through the population. Suppose you have a number of people, let’s call it N, which are infected with a disease. You want to know how N will change in time, so N is a function of t, where t is time. Each of the N people has a certain probability to spread the disease to other people during some period of time. We will quantify this infectiousness by a constant, k. This means that the change in the number of people per time equals that constant k times the number of people who already are infected.

Now, the change of a function per time is the derivative of the function with respect to time. So, this gives you an equation which says that the derivative of the function is proportional to the function itself. And this is a differential equation. A differential equation is more generally an equation for an unknown function which contains derivatives of the function. So, a differential equation must be solved not for a parameter, say x, but for a whole function.

The solution to the differential equation for disease spread is an exponential function, where the probability of infecting someone appears in the exponent, and there is a free constant in front of the exponential, which I called N0. This function will solve the equation for any value of this free constant. If you put in the time t equals zero, then you can see that this constant N0 is simply the number of infected people at the initial time.

So, this is why infectious diseases begin by spreading exponentially, because the increase in the number of infected people is proportional to the number of people who are already infected. You are probably wondering now how these constants relate to the basic reproduction number of the disease, the R naught we have all become familiar with. When a disease begins to spread, this constant k in the exponent is (R0-1)/ τ, where τ is the time an infected person remains infectious.

So, R naught can be interpreted as the average number of people someone infects. Of course in reality diseases do not continue spreading exponentially, because eventually everyone is either immune or dead and there’s no one left to infect. To get a more realistic model for disease spread, one would have to take into account that the number of susceptible people begins to decrease as the infection spreads. But this is not a video about pandemic models, so let us instead get back to differential equations. Another simple example for a differential equation is one you almost certainly know, Newton’s second law, F equals m times a. Let us just take the case where the force is a constant. This could describe, for example, the gravitational force near the surface of the earth, in a range so small you can neglect that the force is actually a function of the distance from the center of Earth. The equation is then just a equals F over m, which I will rename to small g, and this is a constant. a is the acceleration, so the second time-derivative of position. Physicists typically denote the position with x, and a derivative with respect to time with a dot, so that is double-dot x equals g. And that’s a differential equation for the function x of t.

For simplicity, let us take x to be just the vertical direction. The solution to this equation is then x(t)= gt2/2 + vt +x0, where v and x0 are constants. If you take the first derivative of this function, you get g times t plus v, and another derivative gives just g. And that’s regardless of what the two constants were.

These two new constants in this solution, v and x0, can easily be interpreted, by looking at the time t=0. x0 is the position of the particle at time t = 0, and, if we look at the derivative of the function, we see that v is the velocity of the particle at t=0. If you take an initial velocity that’s pointed up, the curve for the position as a function of time is a parabola, telling you the particle goes up and comes back down. You already knew that, of course. The relevant point for our purposes is that, again, you do not get one function as a solution to the equation, but a whole family of functions, one for each possible choice of the constants.

Physicists call these free constants which appear in the possible solutions to a differential equation “initial values”. You need such initial values to pick the solution of the differential equation which fits to the system you want to describe. The reason we have two initial values for Newton’s law is that the highest order of derivative in the differential equation is two. Roughly speaking, you need one initial value per order of derivative. In the first example of disease growth, if you remember, we had one derivative and correspondingly only one initial value.

Now, Newton’s second law is not exactly frontier research, but the thing is that all theories we use in the foundations of physics today are of this type. They are given by differential equations, which have a large number of possible solutions. Then we insert initial values to identify the solution that actually describes what we observe.

Physicists use differential equations for everything, for stars, for atoms, for gases and fluids, for electromagnetic radiation, for the size of the universe, and so on. And these differential equations always work the same. You solve the equation, insert your initial values, and then you know what happens at any other moment in time.

I should add here that the “initial values” do not necessarily have to be at an initial time from which you make predictions for later times. The terminology is somewhat confusing, but you can also choose initial values at a final time and make predictions for times before that. This is for example what we do in cosmology. We know how the universe looks today, that are our “initial” values, and then we run the equations backwards in time to find out what the universe must have looked like earlier.

These differential equations are what we call “deterministic”. If I tell you how many people are ill today, you can calculate how many will be ill next week. If I tell you where I throw a particle with what initial velocity, you can tell me where it comes down. If I tell you what the universe looks like today, and you have the right differential equation, you can calculate what happens at every other moment of time. This consequence is that, according to the natural laws that physicists have found so far, the future is entirely fixed already; indeed, it was fixed already when the universe began.

This was pointed out first by Pierre Simon Laplace in 1814 who wrote:
“We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.”

This “intellect” Laplace is referring to is now sometimes called “Laplace’s demon”. But physics didn’t end with Laplace. After Laplace wrote those words, Poincare realized that even deterministic systems can become unpredictable for all practical purposes because they are “chaotic”. I talked about this in my earlier video about the Butterfly effect. And then, in the 20th century, along came quantum mechanics. Quantum mechanics is a peculiar theory because it does not only use an differential equations. Quantum mechanics uses another equation in addition to the differential equation. The additional equation describes what happens in a measurement. This is the so-called measurement update and it is not deterministic.

What does this mean for the question whether we have free will? That’s what we will talk about next week, so stay tuned.


  1. Sabine wrote: "you want to know how N will change in time"

    In this case, N being an integer, it cannot be described by a differential equation. Of course you are aware of that. Some people frequently forget that differential equations are not an exact representation of reality, but only of averages (or probabilities).

    > "a piece of mathematics which describes, for all we currently know, everything"

    Differential equations reign supreme also in stochastic models. But ignoring the graininess and randomness inherent to the real world has led famous physicists to believe that quantum physics is somehow special and fundamentally unintelligible.

  2. I'm neither a physicist nor a statistician, but I've worked at times in the operations research field, where practitioners select models that best reflect, say, the rate at which to replace tires in a vehicle fleet before they fail. The impression I have is that they go to a fairly limited set of available differential equations that, when the variables are plugged in, give a "close enough" result. But this isn't really a strict probability. In fact, if we look at COVID models, "close enough" has turned out to be pretty far out of whack. I think you're imputing a level of precision to mathematical modeling of the real world that in fact doesn't exist.

  3. Understanding maxwell equations of electromagnetism was what motivated me to study calculus and physics and eventually go to college. I'm a third year mechatronics engineer student now.

    I just wanted to come by and say that your videos and Matt's (from PBS space time) really motivated me to try and dive a little deeper into QM, specially your video about bra-ket notation.
    I just finished Susskind 10 lectures on QM and the plan is to take a look at Feynman's 3 book now.
    Let's see how long what I already know of math and physics can take me on this rabbit hole.

    If physicists sometimes get a bit lost in math, sometimes we engineer students sometimes get a bit lost in physics =P
    Huge props for your work, professor. On top of all the rest, you are a great educator. Thanks for that!

  4. I learned about differential equation in high school. In particular it was the simple dx/dt = ax. With the calculus I understood this lead to dx/x = adt and the left is a log and the right is just at and you get the exponential. I thought this was pretty nifty. Then of course this gets generalized to many differential equations, and in particular partial differential equations.

    It is worth noting there is a structure beyond differential equations. We all know that functions produce sets of numbers if we insert numbers into them. Differential equations give functions when we impose initial or boundary conditions onto them. Differential equations emerge from systems of differential forms. These come from cohomology rings, and because topology gives a class of geometries, where any one is determined by imposing a coordinate system. We might think of a geometry as what happens when a coordinate condition is permitted, where this is a bit analogous to putting an initial or boundary condition to get a solution from a differential equation. The ADMH bundle construction of Atiyah, Donaldson et al is a nice example. In this sense we have this odd level beyond differential equations.

    I thought about writing about quantum complexity, entropy, and its relationship to measurement or decoherence. However, maybe I will wait on that due to time limits. There is a role for these stochastic situations, but there is a form of Poincare recurrence. This quantum recurrence may have something to do with the stability of the deSitter vacuum. The point I was interested in making is the differential equation for complexity with Rindler time is analogous to the Galileo equation for simple motion.

    With epidemiology it appears my brick-head president has gotten a direct lesson on that. It is a bit like I say there are two hard ways to come to understand electrodynamics. The long hard way that is best is to study physics. The quick way that gives an idea of it is to urinate into a wall socket. Trump chose a form of the latter.

    I got a cold a few weeks ago. The cough and symptoms were distinct from Covid and the cold receded rather quickly. The problem is in its wake for a couple of weeks the receding cold symptoms became Covid-like. The expectorating cough of the cold switched into the dry mild cough of Covid. This infernal disease is endodermal or infects tissue with ACE2 receptors and I am reading this has irreversible consequences. So if you have dx/dt = F(x) then a reversal of time is an overall minus sign on this.

  5. So next week it will be more about philosophy than about physics. I don’t mind.

  6. On the one hand one can marvel that a formal mathematical structure can model the behavior of "stuff" so well. On the other hand, we realize that these differential equations are the product of a brain, made out of "stuff". So if the "stuff" has certain properties, those properties are expressed by anything made out of that "stuff". So math isn't an idealization at all, but a reflection of "stuff" in a structure of "stuff". No surprise. And then, if the representation is deterministic, it is only because the "stuff" is deterministic. Could there be any other kind of "stuff"? I ought to tell a joke at this point, lest I be accused of being "stuffy". But I can't think of one. Such is my stuff.

  7. “If I tell you how many people are ill today, you can calculate how many will be ill next week.”
    It should work also in the other direction of time: If I tell you how many people are ill today; you can calculate how many were ill last week. Unfortunately, this does not work. The constant in the equation is not constant. It changes with time. Why? Because it is my free will to decide whether I stay at home or go to the party?

  8. The greatest danger in fore-casting or post-casting with differential equations is illustrated by that parabolic arc.
    Given three points on a parabolic arc we can compute the initial conditions and values forward or backward in time indefinitely; but try this with the arc of a projectile. Post cast far enough, and it claims the missile was at some time in the past miles beneath the earth. Forecast far enough, and it predicts the missile will again be many miles within the earth. Neither is true.

    The issues with the missile are obvious, something will stop it in either direction: The reason for its arc to begin is unknown, but that reason exists. The reason it will stop is it will encounter an obstacle.

    But post-casting and fore-casting in cosmology has similar perils. In post-casting we can encounter conditions we currently do not understand about condensed matter; like what actually caused expansion to begin (similar to what caused the missile to fly in the first place). And in the end, we may assume nothing can stop the expansion, because we can't think of anything that would. Even though we know our current description of reality is incomplete as long as mysteries remain unexplainable (wave-function collapse, dark matter, dark energy, a workable GR in the quantum realm).

    All such fore-casts and post-casts should carry the warning labels of old maps; unknown regions were labeled "Here There Be Dragons". It is in general a mistake to fore-cast or post-cast indefinitely and draw any conclusions from it.

  9. I am a little disappointed that the post did not mention finite-difference equations. I had a calculus course in high school and a couple in college, and only learned about finite-difference equations in a job-related Masters program. It had a significant effect on my world view. (From a world of infinities everywhere to one of vast but discrete systems.)

    They are similar in form and in solution-form to differential equations but the variables are not continuous (x = i*deltax, where i is an integer and deltax is a constant, the minimum increment). In reality, most differential equations are approximations and the actual cases are finite-difference equations. For example, fluid-flow, e.g. the Navier-Stokes differential equation. Calculus assumes continuity with no lower bound. Fluids are composed of molecules--they have a lower bound.

    Intuitively it makes sense that as deltax goes to zero (and i therefore goes to infinity), finite-difference equations and calculus converge to the same answer, so one can assume continuity and gain all the theorems and known solutions of calculus, but sometimes the existence of the lower bound can have a significant effect. I wonder if the a0 acceleration level in MOND might not be symptom of this.

    Both finite-difference and differential equations are deterministic, of course. I take it that determinism was the main point of the post.

  10. I am having a terrible mental block when it comes to differential equations. I can pick one out n a crowd, but I don't know what gives rise to them. That is, I don't know why so many systems are expressed as a differential equation. And I am very suprised that many solutions to differential equations involve euler's number e.

  11. Hi Sabine,

    Should I understand that the constants of nature (and all free parameters) may be symptoms of the reign of differential equations ? or of inability to describe/imagine/understand a lower level dynamics ?


  12. In first year physics, we hit two high points - the damped driven harmonic oscillator in mechanics, and the RLC circuit in electricity and magnetism. 'This is the same differential equation as we saw last term' was a moment of triumph.

  13. Determinism has implications for computational issues. Consider AI: our brains are running the deterministic "I" program. So why not AI? The answer hinges on whether the "I" program can be implemented on some other device. And what sort of algorithm? Is any implementation of the "I" program computationally intensive (no efficient shortcut) or not? For that matter, the deterministic algorithm or program governing the workings of the universe that we attempt to model in shorthand, in the form of differential equations, could be intensive It could remain deterministic but the differential laws are always and will be forever approximate. It's interesting to think of the halting problem in this sort of framework. It's clear that the "I" program can reproduce all of known mathematics, because it has CREATED all of known mathematics. Why could not an implementation of that program/algorithm in some substrate other than a human brain REPRODUCE all of known mathematics? I guess we can ask whether the "I" algorithm running in that part of the universe we call the human brain can be self-referential to the extent required to duplicate itself. Many fascinating questions.

  14. On free will:

    "My will is wet *) said Winnifred and pulled her husband into bed"(John O'Mill).

    *) law.

  15. A couple of mathematical points connected with the article above, which should be considered before drawing philosophical conclusions (e.g. about determinism, etc):

    (1) The standard Cauchy/Peano theorem on differential equations only guarantees existence of a solution; it does not guarantee uniqueness.

    (2) Differential equations exist which have solutions, but have no computable (ie calculable) solutions (e.g. example from Aberth 1970s).

    (3) Arguably using an example like "infection modelling" is more an example of a stochastic differential equation: exact numbers are not predicted for a later time, unlike in more Newtonian differential equation examples.

    The conclusion I would draw from these results is that a differential equation is more like a set of constraints on the future behaviour of the initial system, rather than a computer program-like determination of what exactly will evolve.

    1. While what you say is correct, this is not the case for the differential equations that we use in the foundations of physics, the Schroedinger equation (linear, first order), and Einstein's Field Equations (on whose initial value problem much ink has been spilled).



Comment moderation on this blog is turned on.
Submitted comments will only appear after manual approval, which can take up to 24 hours.
Comments posted as "Unknown" go straight to junk. You may have to click on the orange-white blogger icon next to your name to change to a different account.