Has Protein Folding Been Solved?

[This is a transcript of the video embedded below.]


Protein folding is one of the biggest, if not THE biggest problem, in biochemistry. It’s become the holy grail of drug development. Some of you may even have folded proteins yourself, at least virtually, with the crowd-science app ‘’Foldit”. But then late last year the headlines proclaimed that Protein Folding was “solved” by artificial intelligence. Was it really solved? And if it was solved, what does that mean? And, erm, what was the protein folding problem again? That’s what we will talk about today.

Proteins are one of the major building blocks of living tissue, for example muscles, which is why you may be familiar with “proteins” as one of the most important nutrients in meat.

But proteins come in a bewildering number of variants and functions. They are everywhere in biology, and are super-important: Proteins can be antibodies that fight against infections, proteins allow organs to communicate between each other, and proteins can repair damaged tissue. Some proteins can perform amazingly complex functions. For example, pumping molecules in and out of cells, or carrying substances along using motions that look much like walking.

But what’s a protein to begin with? Proteins are basically really big molecules. Somewhat more specifically, proteins are chains of smaller molecules called amino acids. But long and loose chains of amino acids are unstable, so proteins fold and curl until they reach a stable, three-dimensional, shape. What is a protein’s stable shape, or stable shapes, if there are several? This is the “protein folding problem”.

Understanding how proteins fold is important because the function of a protein depends on its shape. Some mutations can lead to a change in the amino acid sequence of a protein which causes the protein to fold the wrong way. It can then no longer fulfil its function and the result can be severe illness. There are many diseases which are caused by improperly folded proteins, for example, type two diabetes, Alzheimer’s, Parkinson’s, and also ALS, that’s the disease that Stephen Hawking had.

So, understanding how proteins fold is essential to figuring out how these diseases come about, and how to maybe cure them. But the benefit of understanding protein folding goes beyond that. If we knew how proteins fold, it would generally be much easier to synthetically produce proteins with a desired function.

But protein folding is a hideously difficult problem. What makes it so difficult is that there’s a huge number of ways proteins can fold. The amino acid chains are long and they can fold in many different directions, so the possibilities increase exponentially with the length of the chain.

Cyrus Levinthal estimated in the nineteen-sixties that a typical protein could fold in more than ten to the one-hundred-forty ways. Don’t take this number too seriously though. The number of possible foldings actually depends on the size of the protein. Small proteins may have as “few” as ten to the fifty, while some large ones can have and a mind-blowing ten to the three-hundred possible foldings. That’s almost as many vacua as there are in string theory!

So, just trying out all possible foldings is clearly not feasible. We’d never figure out which one is the most stable one.

The problem is so difficult, you may think it’s unsolvable. But not all is bad. Scientists found out in the nineteen-fifties that when proteins fold under controlled conditions, for example in a test tube, then the shape into which they fold is pretty much determined by the sequence of amino acids. And even in a natural environment, rather than a test tube, this is usually still the case.

Indeed, the Nobel Prize for Chemistry was awarded for this in 1972. Before that, one could have been worried that proteins have a large numbers of stable shapes, but that doesn’t seem to be the case. This is probably because natural selection preferentially made use of large molecules which reliably fold the same way.

There are some exceptions to this. For example prions, like the ones that are responsible for mad cow disease, have several stable shapes. And proteins can change shape if their environment changes, for instance when they encounter certain substances inside a cell. But mostly, the amino acid sequence determines the shape of the protein.

So, the protein folding problem comes down to the question: If you have the amino-acid sequence, can you tell me what’s the most stable shape?

How would one go about solving this problem? There are basically two ways. One is that you can try to come up with a model for why proteins fold one way and not another. You probably won’t be surprised to hear that I had quite a few physicist friends who tried their hands at this. In physics we call that a “top down” approach. The other thing you can do is what we call a “bottom up” approach. This means you observe how a large number of proteins fold and hope to extract regularities from this.

Either way, to get anywhere with protein folding you first of all need examples of how folded proteins look like. One of the most important methods for this is X-ray crystallography. For this, one fires beams of X-rays at crystallized proteins and measures how the rays scatter off. The resulting pattern depends on the position of the different atoms in the molecule, from which one can then infer the three-dimensional shape of the protein. Unfortunately, some proteins take months or even years to crystallize. But a new method has recently much improved the situation by using electron microscopy on deep-frozen proteins. This so-called Cryo-electron microscopy gives much better resolution.

In 1994, to keep track of progress in protein folding predictions, researchers founded an initiative called the Critical Assessment of Protein Structure Prediction, CASP for short. CASP is a competition among different research teams which try to predict how proteins fold. The teams are given a set of amino acid sequences and have to submit which shape they think the protein will fold into.

This competition takes place every two years. It uses protein structures that were just experimentally measured, but have not yet been published, so the competing teams don’t know the right answer. The predictions are then compared with the real shape of the protein, and get a score depending on how well they match. This method for comparing the predicted with the actual three-dimensional shape is called a Global Distance Test, and it’s a percentage. 0% is a total failure, 100% is the high score. In the end, each team gets a complete score that is the average over all their prediction scores.

For the first 20 years, progress in the CASP competition was slow. Then, researchers began putting artificial intelligence on the task. Indeed, in last year’s competition, about half of the teams used artificial intelligence or, more specifically, deep learning. Deep learning uses neural networks. It is software that is trained on large sets of data and learns recognize patterns which it then extrapolates from. I explained this in more detail in an earlier video.

Until some years ago, no one in the CASP competition scored more than 40%. But in the last two installments of the competition, one team has reached remarkable scores. This is DeepMind, a British Company that was acquired by Google in twenty-fourteen. It’s the same company which is also behind the computer program AlphaGo, that in twenty-fifteen was first to beat a professional Go player.

DeepMind’s program for protein folding is called AlphaFold. In twenty-eighteen, AlphaFold got a score of almost 60% in the CASP competition, and in 2020, the update AlphaFold2 reached almost 90%.

The news made big headlines some months ago. Indeed, many news outlets claimed that AlphaFold2 solved the protein folding problem. But did it?

Critics have pointed out that 90% is still a significant failure rate and that some of the most interesting cases are the ones for which AlphaFold2 did not do well, such as complexes of proteins, called oligomers, in which several amino acids are interacting. There is also the general problem with artificial intelligences, which is that they can only learn to extract patterns from data which they’ve been trained on. This means the data has to exist in the first place. If there are entirely new functions that don’t make an appearance in the data set, they may remain undiscovered.

But well. I sense a certain grumpiness here of people who are afraid they’ll be rendered obsolete by software. It’s certainly true that the AlphaFold’s 2020 success won’t be the end of the story. Much needs to be done, and of course one still needs data, meaning measurements, to train artificial intelligence on.

Still I think this is a remarkable achievement and amazing progress. It means that, in the future, protein folding predictions by artificially intelligent software may save scientists much time-consuming and expensive experiments. This could help researchers to develop proteins that have specific functions. Some that are on the wish-list, for example, are proteins to stimulate the immune system to fight cancer, a universal flu vaccine, or proteins that breaking down plastics.

Where do atoms come from?

[This is a transcript of the video embedded below.]

Matter is made of atoms. You all know that. But where do atoms come from? When were they made and how? And what’s the “island of stability”? That’s what we will talk about today.

At first sight, making an atom doesn’t seem all that difficult. All you need are some neutrons and protons for the nucleus, then you put electrons around them until the whole thing is electrically neutral, done. Sounds easy. But it isn’t.

The electrons are the simple part. Once you have a positively charged nucleus, it attracts electrons and they automatically form shells around the nucleus. For more about atomic electron shells, check my earlier video.

But making an atomic nucleus is not easy. The problem is that the protons are all positively charged and they repel each other. Now, if you get them really, really close to each other, then the strong nuclear force will kick in and keep them together – if there’s a suitable amount of neutrons in the mix. But to get the protons close enough together, you need very high temperatures, we’re talking about hundreds of millions of degrees.

Such high temperatures, indeed much higher temperatures, existed in the early universe, briefly after the big bang. However, at that time the density of matter was very high everywhere in the universe. It was a mostly structureless soup of subatomic particles called a plasma. There were no nuclei in this soup, just a mix of the constituents of nuclei.

It was only when this plasma expanded and cooled, that some of those particles managed to stick together. This created the first atomic nuclei which could then catch electrons to make atoms. From this you get Hydrogen and Helium and a few other chemical elements with their isotopes up to atomic number 4. The process of making atomic nuclei, by the way is called “nucleosynthesis”. And this part of nucleosynthesis that happened a few minutes after the big bang is called “big bang nucleosynthesis”.

But the expansion of plasma after the big bang happened so rapidly that only the lightest atomic nuclei could form in that process. Making the heavier ones takes more patience, indeed it takes a few hundred million years. During that time the universe continued to expand, but the light nuclei collected under the pull of gravity and formed the first stars. In these stars, the gravitational pressure increased the temperature again. Eventually, the temperature became large enough to push the small atomic nuclei into each other and fuse them to larger ones. This nuclear fusion creates energy and is the reason why stars are hot and shine.

Nuclear fusion in stars can go on up to atomic number twenty-six, which is iron, but then it stops. That’s because iron is the most stable of the chemical elements. Its binding energy is the largest. So, if you join small nuclei, you get energy out in the process until you hit iron, after which pushing more into the nucleus begins to take up energy.

So, with the nuclear fusion inside of stars, we now have elements up to iron. But where do the elements heavier than iron come from? They come from a process called “neutron capture”. Some fusion processes create free neutrons, and the neutrons, since they have no electric charge, have a much easier time entering an atomic nucleus than protons. And once they are in the nucleus, they can decay into a proton, an electron, and an electron-antineutrino. If they do that, they have created a heavier element. A lot of the so-created nuclei will be unstable isotopes, but they will spit out bits and pieces until they hit on a stable configuration.

Neutron capture can happen in stars just by chance every now and then. Over the course of time, therefore, old stars breed a few of the elements heavier than iron. But the stars eventually run out of nuclear fuel and die. Many of them collapse and subsequently explode. These supernovae distribute the nuclei inside galaxies or even blow them out of galaxies. Some of the lighter elements which are around today are actually created from splitting up these heavier elements by cosmic rays.

However, neutron capture in old stars is slow and stars only live for so long. This process just does not produce sufficient amounts of the heavy elements that we have here on Earth. Doing that requires a process that’s called “rapid neutron capture”. For this one needs an extreme environment of very high pressure with lots of neutrons that bombard the small atomic nuclei. Again, some of the neutrons enter the nucleus and then decay, leaving behind a proton, which creates heavier elements.

For a long time astrophysicists thought that rapid neutron capture happens in supernovae. But that turned out to not work very well. Their calculations indicated that supernovae would not produce a sufficient amount of neutrons quickly enough. The idea also did not fit well with observations. For example, if the heavy elements that astrophysicists observe in some small galaxies –called “dwarf galaxies” – had been produced by supernovae, that would have required so many supernovae that these small galaxies had been blown apart and we wouldn’t observe them in the first place.

Astrophysicists therefore now think that the heavy elements are most likely produced not in supernovae, but in neutron star mergers. Neutron stars are one of the remnants of supernovae. As the name says, they contain lots of neutrons. They do not actually contain nuclei, they’re just one big blob of super-dense nuclear plasma. But if they collide, the collision will spit out lots of nuclei, and create conditions that are right for rapid neutron-capture. This can create all of the heavy elements that we find on Earth. A recent analysis of light emitted during a neutron star merger supports this hypothesis because the light contains evidence for the presence of some of these heavy elements.

You may have noticed that we haven’t checked off the heaviest elements in the periodic table and that there are a few missing in between. That’s because they are unstable. They decay into smaller nuclei in times between a few thousand years and some micro-seconds. Those that were produced in stars are long gone. We only know their properties because they’ve been created in laboratories, by shooting smaller nuclei at each other with high energy.

Are there any other stable nuclei that we haven’t yet discovered? Maybe. It’s a long-standing hypothesis in nuclear physics that there are heavy nuclei with specific numbers of neutrons and protons that should have life-times up to some hundred thousand years, it’s just that we have not been able to create them so far. Nuclear physicists call it the “island of stability”, because it looks like an island if you put each nucleus on a graph where one axis is the number of protons, and the other axis is the number of neutrons.

Just exactly where the island of stability is, though, isn’t clear and predictions have moved around somewhat over the course of time. Currently, nuclear physicists believe reaching the island of stability would require pushing more neutrons inside the heaviest nuclei they previously produced.

But the maybe most astonishing thing about atoms is how so much complexity, look around you, is built up from merely three ingredients neutrons, protons, and electrons.

I hope you enjoyed this video. You can now support this channel on Patreon. So, if you want to be part of the story, go check this out. I especially want to thank my super-supporters on Patreon. Your help is greatly appreciated.

Was the universe made for us?

[This is a transcript of the video embedded below.]


Today I want to talk about the claim that our universe is especially made for humans, or fine-tuned for life. According to this idea it’s extremely unlikely our universe would just happen to be the way it is by chance, and the fact that we nevertheless exist requires explanation. This argument is popular among some religious people who use it to claim that our universe needs a creator, and the same argument is used by physicists to pass off unscientific ideas like the multiverse or naturalness as science. In this video, I will explain what’s wrong with this argument, and why the observation that the universe is this way and not some other way, is evidence neither for nor against god or the multiverse.

Ok, so here is how the argument goes in a nutshell. The currently known laws of nature contain constants. Some of these constants are for example, the fine-structure constant that sets the strength of the electromagnetic force, Planck’s constant, Newton’s constant, the cosmological constant, the mass of the Higgs boson, and so on.

Now you can ask, what would a universe look like, in which one or several of these constants were a tiny little bit different. Turns out that for some changes to these constants, processes that are essential for life as we know it could not happen, and we could not exist. For example, if the cosmological constant was too large, then galaxies would never form. If the electromagnetic force was too strong, nuclear fusion could not light up stars. And so on. There’s a long list of calculations of this type, but they’re not the relevant part of the argument, so I don’t want to go through the whole list.

The relevant part of the argument goes like this: It’s extremely unlikely that these constants would happen to have just exactly the values that allow for our existence. Therefore, the universe as we observe it requires an explanation. And then that explanation may be god or the multiverse or whatever is your pet idea. Particle physicists use the same type of argument when they ask for a next larger particle collider. In that case, they claim it requires explanation why the mass of the Higgs boson happens to be what it is. This is called an argument from “naturalness”. I explained this in an earlier video.

What’s wrong with the argument? What’s wrong is the claim that the values of the constants of nature that we observe are unlikely. There is no way to ever quantify this probability because we will never measure a constant of nature that has a value other than the one it does have. If you want to quantify a probability you have to collect a sample of data. You could do that, for example, if you were throwing dice.Throw them often enough, and you get an empirically supported probability distribution.

But we do not have an empirically supported probability distribution for the constants of nature. And why is that. It’s because… they are constant. Saying that the only value we have ever observed is “unlikely” is a scientifically meaningless statement. We have no data, and will never have data, which allow us to quantify the probability of something we cannot observe. There’s nothing quantifiably unlikely, therefore, there’s nothing in need of explanation.

If you look at the published literature on the supposed “fine-tuning” of the constants of nature, the mistake is always the same. They just postulate a particular probability distribution. It’s this postulate that leads to their conclusion. This is one of the best known logical fallacies, called “begging the question” or “circular reasoning.” You assume what you need to show. And instead of showing that a value is unlikely, they pick a specific probability distribution that makes it unlikely. They could as well pick a probability distribution that would make the observed values *likely, just that this doesn’t give the result they want to have.

And, by the way, even if you could measure a probability distribution for the constants of nature, which you can’t, then the idea that our particular combination of constants is necessary for life would *still be wrong. There are several examples in the scientific literature for laws of nature with constants nothing like our own that, for all we can tell, allow for chemistry complex enough for life. Please check the info below the video for references.

Let me be clear though that finetuning arguments are not always unscientific. The best-known example of a good finetuning argument is a pen balanced on its tip. If you saw that, you’d be surprised. Because this is very unlikely to happen just by chance. You’d look for an explanation, a hidden mechanism. That sounds very similar to the argument for finetuning the constants of nature, but the balanced pen is a very different situation. The claim that the balanced pen is unlikely is based on data. You are surprised because you don’t normally encounter pens balanced on their tip.You have experience, meaning you have statistics. But it’s completely different if you talk about changing constants that cannot be changed by any physical process. Not only do we not have experience with that, we can never get any experience.

I should add there are theories in which the constants of nature are replaced with parameters that can change with time or place, but that’s a different story entirely and has nothing to do with the fine-tuning arguments. It’s an interesting idea though. Maybe I should talk about this some other time? Let me know in the comments.

And for the experts, yes, I have so far specifically referred to what’s known as the frequentist interpretation of probability. You can alternatively interpret the term “unlikely” using the Bayesian interpretation of probability. In the Bayesian sense, saying that something you observe was “unlikely”, means you didn’t expect it to happen. But with the Bayesian interpretation, the whole argument that the universe was especially made for us doesn’t work. That’s because in that case it’s easy enough to find reasons for why your probability assessment was just wrong and nothing’s in need of explaining.

Example: Did you expect a year ago that we’d spent much of 2020 in lockdown? Probably not. You probably considered that unlikely. But no one would claim that you need god to explain why it seemed unlikely.

What does this mean for the existence of god or the multiverse? Both are assumptions that are unnecessary additions to our theories of nature. In the first case, you say “the constants of nature in our universe are what we have measured, and god made them”, in the second case you say “the constants of nature in our universe are what we have measured, and there are infinitely many other unobservable universes with other constants of nature.” Neither addition does anything whatsoever to improve our theories of nature. But this does not mean god or the multiverse do not exist. It just means that evidence cannot tell us whether they do or do not exist. It means, god and the multiverse are not scientific ideas.

If you want to know more about fine-tuning, I have explained all this in great detail in my book Lost in Math.

In summary: Was the universe made for us? We have no evidence whatsoever that this is the case.

---

You can join the chat on this video today (Saturday, Jan 16) at 6pm CET/Eastern Time here.

The Mathematics of Consciousness

[This is a transcript of the video embedded below.]


Physicists like to think they can explain everything, and that, of course, includes human consciousness. And so in the last few decades they’ve set out to demystify the brain by throwing math at the problem. Last year, I attended a workshop on the mathematics of consciousness in Oxford. Back then, when we still met other people in real life, remember that?

I find it to be a really interesting development that physicists take on consciousness, and so, today I want to talk a little about ideas for how consciousness can be described mathematically, how that’s going so far, and what we can hope to learn from it in the future.

The currently most popular mathematical approach to consciousness is integrated information theory, IIT for short. It was put forward by a neurologist, Giulio Tononi, in two thousand and four.

In IIT, each system is assigned a number, that’s big Phi, which is the “integrated information” and supposedly a measure of consciousness. The better a system is at distributing information while it’s processing the information, the larger Phi. A system that’s fragmented and has many parts that calculate in isolation may process lots of information, but this information is not “integrated”, so Phi is small.

For example, a digital camera has millions of light receptors. It processes large amounts of information. But the parts of the system don’t work much together, so Phi is small. The human brain on the other hand is very well connected and neural impulses constantly travel from one part to another. So Phi is large. At least that’s the idea. But IIT has its problems.

One problem with IIT is that computing Phi is ridiculously time consuming. The calculation requires that you divide up the system which you are evaluating in any possible way and then calculate the connections between the parts. This takes up an enormous amount of computing power. Estimates show that even for the brain of a worm, with only three hundred synapses, calculating Phi would take several billion years. This is why measurements of Phi that have actually been done in the human brain have used incredibly simplified definitions of integrated information.

Do these simplified definitions at least correlate with consciousness? Well, some studies have claimed they do. Then again others have claimed they don’t. The magazine New Scientist for example interviewed Daniel Bor from the University of Cambridge and reports:

“Phi should decrease when you go to sleep or are sedated via a general anesthetic, for instance, but work in Bor’s lab has shown that it doesn’t. “It either goes up or stays the same,” he says.”

I contacted Bor and his group, but they wouldn’t come forward with evidence to back up this claim. I do not actually doubt it’s correct, but I do find it somewhat peculiar they’d make such a statements to a journalist and then not provide evidence for it.

Yet another problem for IIT is, as the computer scientist Scott Aaronson pointed out, that one can think of rather trivial systems, that solve some mathematical problem, which distribute information during the calculation in such a way that Phi becomes very large. This demonstrates that Phi in general says nothing about consciousness, and in my opinion this just kills the idea.

Nevertheless, integrated information theory was much discussed at the Oxford workshop. Another topic that received a lot of attention is the idea by Roger Penrose and Stuart Hamaroff that consciousness arises from quantum effects in the human brain, not in synapses, but in microtubules. What the heck are microtubules? Microtubules are tiny tubes made of proteins that are present in most cells, including neurons. According to Penrose and Hameroff, in the brain these microtubules can enter coherent quantum states, which collapse every once in a while, and consciousness is created in that collapse.

Most physicists, me included, are not terribly excited about this idea because it’s generally hard to create coherent quantum states of fairly large molecules, and it doesn’t help if you put the molecules into a warm and wiggly environment like the human brain. For the Penrose and Hamaroff conjecture to work, the quantum states would have to survive at least a microsecond or so. But the physicist Max Tegmark has estimated that they would last more like a femtosecond, that’s only ten to the minus fifteen seconds.

Penrose and Hameroff are not the only ones who pursue the idea that quantum mechanics has something to do with consciousness. The climate physicist Tim Palmer also thinks there is something to it, though he is more concerned with the origins of creativity specifically than with consciousness in general.

According to Palmer, quantum fluctuations in the human brain create noise, and that noise is essential for human creativity, because it can help us when a deterministic, analytical approach gets stuck. He believes the sensitivity to quantum fluctuations developed in the human brain because that’s the most energy-efficient way of solving problems, but it only becomes possible once you have small and thin neurons, of the types you find in the human brain. Therefore, palmer has argued that low-energy transistors which operate probabilistically rather than deterministically, might help us develop artificial intelligent that’s actually intelligent.

Another talk that I thought was interesting at the Oxford workshop was that by Ramon Erra. One of the leading hypothesis for how cognitive processing works is that it uses the synchronization of neural activity in different regions of the brain to integrate information. But Erra points out that during an epileptic seizure, different parts of the brain are highly synchronized.

In this figure, for example, you see the correlations between the measured activity of hundred fifty or so brain sites. Red is correlated, blue is uncorrelated. On the left is the brain during a normal conscious phase, on the right is a seizure. So, clearly too much synchronization is not a good thing. Erra has therefore proposed that a measure of consciousness could be the entropy in the correlation matrix of the synchronization. Which is low both for highly uncorrelated and highly correlated states, but large in the middle, where you expect consciousness.

However, I worry that this theory has the same problem as integrated information theory, which is that there may be very simple systems that you do not expect to be conscious but that nevertheless score highly on this simple measure of synchronization.

One final talk that I would like to mention is that by Jonathan Mason. He asks us to imagine a stack of compact disks, and a disk player that doesn’t know which order to read out the bits on a compact disk. For the first disk, you then can always find a readout order that will result in a particular bit sequence, that could correspond, for example, to your favorite song.

But if you then use that same readout order for the next disk, you most likely just get noise, which means there is very little information in the signal. So if you have no idea how to read out information from the disks, what would you do? You’d look for a readout process that maximizes the information, or minimizes the entropy, for the readout result for all of the disks. Mason argues that the brain uses a similar principle of entropy minimization to make sense of information.

Personally, I think all of these approaches are way too simple to be correct. In the best case, they’re first steps on a long way. But as they say, every journey starts with a first step, and I certainly hope that in the next decades we will learn more about just what it takes to create consciousness. This might not only allow us to create artificial consciousness and help us tell when patients who can't communicate are conscious, it might also help us allow to make sense of the unconscious part of our thoughts so that we can become more conscious of them.

You can find recordings of all the talks at the workshop, right here on YouTube, please check the info below the video for references.

---

You can join the chat about this video today (Saturday, Jan 9) at noon Eastern Time or 6pm CET here.