How artificial intelligence reads minds

[This is a transcript of the video embedded below.]


Human communication works by turning thought into motion. Whether that’s body language, or speech, or writing – we use muscles in one way or another to get out the information that we want to share. But sometimes it would be really handy if we could communicate directly from our brain, to one another or with a computer. How far is this technology along? How does it work? And what’s next? That’s what we will talk about today.

Scientists currently have two ways to figure out what’s going on inside your brain. One way is to use functional Magnetic Resonance Imaging, the other is using electrodes.

Functional Magnetic Resonance Imaging or fMRI for short measures the flow of blood to different regions of the brain. The blood-flow is correlated with neural activity, so an fMRI tells you what parts of the brain are activated in a certain task. I previously made a video about Magnetic Resonance Imaging, so if you want to know how the physics works, check this out.

The problem with fMRIs is that they require people to lie in a big machine. It isn’t only that using this machine is expensive, it also takes some time to take an fMRI, which means that the temporal resolution isn’t great, typically a few seconds. So fMRI can’t tell you much about fast and temporary processes.

The other way to measure brain activity is electroencephalography, EEG for short, which measures tiny currents in electrodes that are placed on the skin on the head. The advantage of this method is that the temporal resolution is much better. The big disadvantage though is that it gives you only a rough idea about the region where the signal is coming from. A much better way is to put the electrodes directly on the surface of the brain, but this requires surgery.

Elon Musk has the idea that one day people might be willing to have electrodes implanted into their brain and he has put some money behind this with his “neuralink” project. But it’s difficult to get a research project approved if it requires drilling holes into other people’s heads, so most studies currently use fMRI – or people who already have holes in their head for one reason or another.

Before we talk about what recent studies have found, I want to briefly thank our tier four supporters on patreon. Your support is of great help to keep this channel going. And you too can be part of the story, go check out our page on patreon, the link is in the info below.

Let us then have a look at what scientists have found.

Researchers from Carnegie Mellon and other American universities have done a very interesting series of experiments using fMRI. In the first one, they put eleven trial participants in the MRI machine and showed them a word on a screen. The participants were asked to think of the concept related to a noun, for example an apple, a cat, a refrigerator, and so on. Then they gave the brain scans of 10 of these people to an artificially intelligent software, together with the word that the people were prompted with. The AI looked for patterns in the brain activity that correlated with the words, and then guessed what the 11th person was thinking of from the brain scan alone. The program guessed correctly about three quarters of the time.

That’s not particularly great, but it *is better than chance – it’s a proof of principle. And along the way the researchers made a very interesting finding. The study had participants whose first language was either English or Portuguese but their brain signature was independent of that. Indeed, the researchers found that in the brain, the concept encoded by a word doesn’t have much to do with the word itself. Instead, the brain encodes the concept by assigning different attributes to it. They have identified three of these attributes:

1) Eating related. This brain pattern activates for words like “apple”, “tomato” or “lettuce”
2) Shelter related. This pattern activates for example for “house”, “closet”, or “umbrella”, and
3) A body-object interaction. For example, if the concept is “pliers” the brain also activates the part representing your hand using the pliers.

This actually allows the computer to predict to some extent how the signal of a concept will look like even if the computer hasn’t seen data on that before. The researchers checked this by combining different concepts to sentences such as “The old man threw the stone into the lake”. Out of 240 possible sentences, the computer could pick the right one in eighty-three percent of cases. It is not that the computer can tell the whole sentence but it knows its basic components, it knows the semantic elements.

The basic finding of this experiment, that the brain identifies concepts by a combination of attributes, has been confirmed by other experiments. For example, another 2019 study, which also used fMRIs asked participants to think of different animals and found that the brain roughly classifies them by attributes like size, intelligence, and habitat.

In the last decade there have also been several attempts to find out what a person sees from their brain activity. For example, in 2017 a team from Kyoto University published a paper in which they used deep learning – so, artificial intelligence again – to find out what someone was seeing from their fMRI signal. They trained the software to recognize general aspects of the image, like shapes, contrast, faces, etc. You can judge the results for yourself. Here you see the actual images that the trial participants looked at, and here the reconstruction by the artificial intelligence – I find it really impressive.

What about speech or text? In April 2019, researchers from UCSF published a paper in Nature reporting they had successfully converted brain activity directly to speech. They worked with epilepsy patients that already had electrodes on their brain surface for treatment. What the researchers looked for were the motor signals that correspond to the sounds in speech, like the tongue, jaw, lips, and so on. Again, they let a computer figure out how to map the brain signal to speech. What you are about to hear is one of the participants reading a sentence and then what the software recreated just from the brain activity.

That’s pretty good, isn’t it? Unfortunately, it took weeks to decode the signals with that quality, so it’s rather useless in practice. But a new study that appeared just a few weeks ago has made a big leap forward for using brain-to text software by looking not at the movements related to producing sounds, but at the movements that come with handwriting.

The person who they worked with is paralyzed from the neck down and has electrodes implanted on his brain already. He was asked to imagine writing the letters of the alphabet, which was used to train the software, and later the AI could reproduce the text from brain activity when the subject imagined writing whole sentences. And, it could do that in real time. That allowed the paralyzed man to text at a speed of about 90 characters per minute, which is quite similar to what able-bodied people reach with text-messaging, about 135 characters. The AI was able to identify characters with over 94% accuracy, and with autocorrect that went up to 99%. So, as you can see, on the side of signal analysis, research has progressed quite rapidly in the past couple of years. But for technological applications the problem is that fMRIs are impractical, EEGs aren’t precise enough, and not everyone wants to have a USB port fused to their brain. Are there any other options?

Well, one thing that researchers have done is to genetically modify zebrafish larvae so that their neurons are fluorescent when active. That way you can measure brain activity non-invasively. And that’s nice, but even if you did that with humans, there’s still the skull in the way, so that doesn’t seem very promising.

More promising is an approach pursued by NASA which is to develop an infrared system to monitor brain activity. That still requires users to wear sensors around their head but it’s non-invasive. And several teams of scientists are trying to monitor brain activity by combining different non-invasive measurements: electrical and ultrasound and optical. The US military, for example, has put 104 million dollars into the Next-generation Nonsurgical Neurotechnology Program, or N cube for short which has the aim of controlling military drones.

We live in a momentous period in the history of human development. It’s the period when humans leave behind the idea that conscious thought is outside of science. So, all of a sudden, we can develop technologies to aid the conversion of thought into action. I find this incredibly interesting. I expect much to happen in this field in the coming years, and will update you from time to time, so don’t forget to subscribe.

Asteroid Mining – A fast way to get rich?

[This is a transcript of the video embedded below.]


Asteroids are the new gold mines. Fly to an asteroid, dig up its minerals, become a billionaire. I used to think this is crazy and will never make financial sense. But after a lot of reading I’m now thinking maybe it will work – by letting bacteria do the digging. How do you dig with bacteria? Is it even legal to mine asteroids? And will it happen in your lifetime? That’s what we’ll talk about today.

Space Agencies like NASA and ESA have found about 25000 asteroids. In 2020 alone, they discovered 3000 new near earth asteroids. About 900 of them have an extension of 1 kilometer or more.

What makes asteroids so interesting for mining is that their chemical composition is often similar to what you find in the core of our planet. Metals from the platinum group are very expensive because they are useful but rare in Earth’s crust. On an asteroid, they can be much easier to dig up. And that’s a straight-forward way to get rich – very, very rich.

The asteroid Psyche, for example, has a diameter of about two-hundred kilometers and astrophysicists estimate it’s about ninety percent metal, mostly iron and nickel. Lindy Elkins-Tanton, NASA's lead scientist of the Psyche mission, estimated the asteroid is worth about 10 quintillion US dollars. That’s a 1 followed by 19 zeros. Now imagine that thing was made of platinum...

NASA, by the way, is planning a mission to Psyche that’s to be launched in 2022. Not because of the quintillions but because they want to study its composition to learn more about how planetary systems form.

How does one find an asteroid that’s good for mining? Well, first of all it shouldn’t take forever to get there, so you want one that comes reasonably close to earth every once in a while. You also don’t want it to spin too much because that’d make it very hard to land or operate on it. And finally you want one that’s cheap to get to, so that means there’s a small amount of acceleration needed during the flight, a small “Delta V” as it’s called.

How many asteroids are there that fit these demands? The astrophysicist Martin Elvis from Harvard estimated it using an equation that’s now called the Elvis equation. It’s similar to the Drake equation which one uses to estimate the number of extraterrestrial civilizations by multiplying a lot of factors. And like the Drake equation, the Elvis Equation depends a lot on the assumptions that you make.

In any case, Elvis with his Elvis equation estimates that only about 10 of the known asteroids are worth mining. For the other ones, the cost-benefit ratio doesn’t work out, because they’re either too difficult to reach or don’t have enough worth mining or they’re too small. In principle one could also think of catching small asteroids, and bringing them back to earth, but in practice that’s difficult: The small ones are hard to find and track. At least right now this doesn’t work.

So the first two problems with asteroid mining are finding an asteroid and getting there. The next problem is digging. The gravitational pull on these asteroids is so small that one can’t just drill into the ground, that would simply kick the spacecraft off the asteroid.

The maybe most obvious way around this problem is to anchor the digging machine to the asteroid. Another solution that researchers from NASA are pursuing are shovels that dig in two opposite directions simultaneously, so there’s no net force to kick the machine off the asteroid. NASA is also looking into the option that instead of using one large machine, one could use instead a swarm of small robots that coordinate their tasks.

Another smart idea is optical digging. For this one use mirrors and lenses to concentrate sunlight to heat up the surface. This can burn off the surface layer by layer, and the material can then be caught in bags.

And then there is mining with bacteria. Using bacteria for mining is actually not a new idea. It’s called “biomining” and according to some historians the Romans have been doing it 2000 years ago already – though they almost certainly didn’t understand how it works since they didn’t know of bacteria to begin with. But we know today that some bacteria eat and decompose minerals. And during their digestion process, they separate off the metal that you want to extract. So basically, the idea is that you ship the bacteria to your asteroid, let them eat dust, and wait for them to digest.

On Earth, biomining is responsible for approximately twenty percent of global copper production and five percent of global gold production. But how can bacteria survive on asteroids? You can’t very well put them into space suits!

For one you wouldn’t directly dump the bacteria onto the asteroid, but put them into some kind of gel. Still there are pretty harsh conditions on an asteroid and you need to find the right bacteria for the task. It’s not hopeless. Microbiologists know that some species of bacteria have adapted to temperatures that would easily kill humans. Some bacteria can for example live at temperatures up to one-hundred thirteen degrees Celsius and some at temperatures down to minus twenty-eight Celsius. At low metabolic rates, they’ve been found to survive even at minus fourty degrees. And some species of bacteria survive vacuum as low as 10 to the minus five Pascal, which should allow them to survive in the vicinity of a spacecraft.

What about radiation? Again, bacteria are remarkably resistant. The bacterium Deinococcus radiodurans, for example, can cope with ionizing radiation up to twenty kiloGray. For comparison, in humans, acute radiation poisoning sets in at about zero point seven Gray. The bacteria easily tolerate twenty-thousand times as much!

And while the perfect bacterium for space mining hasn’t yet been found, there’s a lot of research going on in this area. It looks like a really promising idea to me.

But, you may wonder now, is it even legal to mine an asteroid? Probably yes. This kind of question is addressed by the nineteen sixty-seven Outer Space Treaty, which has been signed by one hundred eleven countries including the United States, Russia, and almost all of Europe.

According to that treaty, celestial bodies may not be subject to “national appropriation”. However, the treaty does not directly address the extraction of “space resources”, that is stuff you find on those celestial bodies. Some countries have interpreted this to mean that commercial mining does not amount to national appropriation and is permitted.

For example, since 2015 American citizens have the right to possess and sell space resources. Luxembourg has established a similar legal framework in 2017. And Russia too is about to pass such legislation.

This isn’t the only development in the area. You can now make a university degree in space resources, for example at the Colorado School of Mines, the University of Central Florida, and the University of Luxembourg. And at the same time several space agencies are planning to visit more asteroids. NASA wants to fly not only to Psyche, but also to Bennu, that is expected to get close to Earth in September twenty twenty-three.

The Chinese National Space Administration has proposed a similar mission to retrieve a sample from the asteroid Kamo’oalewa. And there are several other missions on the horizon.

And then there’s the industry interest. Starting about a decade ago, a number of start-ups appeared with the goal of mining asteroids, such as Planetary Resources and Deep Space Industries. These companies attracted some investors when they appeared but since then they have been struggling to attract more money, and they have basically disappeared – they’ve been bought by other companies which are more interested in their assets than in furthering the asteroid mining adventure.

The issue is that asteroid mining is real business, but it’s business in which there’s still tons of research to do: How to identify what asteroid is a good target, how to get to the asteroid, how to dig on it. And let’s not forget that once you’ve managed to do that, you also have to get the stuff back to earth. It’d take billions of up-front investments and decades of time to pay off even in the best case. So, while it’s promising, it looks unlikely to me that private investors will drive the technological development in this area. It will likely remain up to tax funded space agencies to finance this research for some more time.

2+2 doesn't always equal 4

[This is a transcript of the video embedded below.]



2 plus 2 makes 5 is the paradigmatic example of an obvious falsehood, a falsehood that everybody knows to be false. Because 2 plus 2 is equal to 1. Right? At the end of this video, you’ll know what I am talking about.

George Orwell famously used two plus two equals five in his novel nineteen eighty-four as an example for an obviously false statement that you can nevertheless make people believe in.

The same example was used already in seventeen eighty-nine by the French priest and writer Emmanuel Sieyès in his essay “what is the third estate”. At this time the third estate – the “bourgeoisie” – made up the big bulk of the population in France, but wasn’t allowed to vote. Sieyes wrote
“[If] it be claimed that under the French constitution two hundred thousand individuals out of twenty-six million citizens constitute two-thirds of the common will, only one comment is possible: it is a claim that two and two make five.” This was briefly before the French revolution.

So you can see there’s a heavy legacy to using two plus two is five as an example of an obvious untruth. And if you claim otherwise that can understandably upset some people. For example, the mathematician Kareem Carr recently got fire on twitter for pointing out that 2+2 isn’t always equal four.

He was accused of being “woke” because he supposedly excused obviously wrong math as okay. Even he was surprised at how upset some people got about it, because his point is of course entirely correct. 2+2 isn’t always equal to four. And I don’t just mean that you could change the symbol “4” with the symbol “5”. You can do that of course, but that’s not the point. The point is that two plus two is a symbolic representation for the properties of elements of a group. And the result depends on what the 2s refer to and how the mathematical operation “+” is defined.

Strictly speaking, without those definitions 2+2 can be pretty much anything. That’s where the joke comes from that you shouldn’t let mathematicians sort out the restaurant bill, because they haven’t yet agreed on how to define addition.

To see why it’s important to know what you are adding and how, let’s back up for a moment to see where the “normal” addition law comes from. If I have two apples and I add two apples, then that makes four apples. Right? Right.

Ok, but how about this. If I have a glass of water with a temperature of twenty degrees and I pour it together with another glass of water at 20 degrees, then together the water will have a temperature of 40 degrees. Erm. No, certainly not.

If both glasses contained the same amount of water, the final temperature will be one half the sum of the temperatures, so that’d still be 20 degrees, which makes much more sense. Temperatures don’t add by the rule two plus two equals four. And why is that?

It’s because temperature is a measure for the average energy of particles and averages don’t add the same way as apples. The average height of women in the United States is 5 ft 4 inches, and that of men 5 ft 9 inches, but that doesn’t mean that the average American has a height of 11 ft 1. You have to know what you’re adding to know how to add it.

Another example. Suppose you switch on a flashlight. The light moves at, well, the speed of light. And as you know the speed of light is the same for all observers. We learned that from Albert Einstein. Yes, that guy again. Now suppose I switch on the flashlight while you come running at me at, say, ten kilometers per hour. At what velocity is the light coming at *you. Well, that’s the speed of light plus ten kilometers per hour. Right? Erm, no. Because that’d be faster than the speed of light. What’s going on?

What’s going on is that velocities don’t add like apples either. They merely approximately do this if all the velocities involved are much smaller than the speed of light. But strictly speaking they have to be added using this formula.

Here u and v are the two velocities that you want to add and w is the result. C is the speed of light. You see immediately if one of the velocities, say u, is also the speed of light, then the resulting velocity stays the speed of light.

So, if you add something to the speed of light, the speed of light doesn’t change. If you come running at me, the light from my flashlight still comes at you with the speed of light.

Indeed, if you add the speed of light to the speed of light because maybe you want to know the velocity at which two light beams approach each other head on, you get c plus c equals c. So, in units of the speed of light, according to Einstein, 1+1 is 1.

That’s some examples from physics for quantities that just have different addition laws. Here is another one from mathematics. Suppose you want to add two numbers that are elements of a finite group, to keep things simple, say one with only three elements. We can give these elements the numbers zero, one, and two.

We can then define an addition rule on this group, which I’ll write as a plus with a circle around it, to make clear it’s not the usual addition. This new addition rule works like this. Take the usual sum of two number, then divide the result by three and take the rest.

So, for example 1+2 = 3, divide by three, the rest is 0. This addition law is defined so that it keeps us within the group. And with this addition law, you have 1 plus 2 equals 0. By the same rule 2 plus 2 equals one.

I know this looks odd, but it’s completely standard mathematics, and it’s not even very advanced mathematics, it just isn’t commonly taught in school. This remainder after division is usually called the modulus. So this addition law can be written as the plus with the circle equals the normal plus mod 3. A set of numbers with this addition law is called a cyclic group.

You can’t only do this with 4, but with any integer number. For example if you take the number 12, that just means if you add numbers to something larger than 12 you start over from zero again. That’s how clocks work, basically, 8+7=3, add another 12 and that gives 3 again. We’re fairly used to this.

Clocks are a nice visual example for how to add numbers in a cyclic group, but time-keeping itself is not an example for cyclic addition. That’s because the “real” physical time of course does not go in a circle. It’s just that on a simple clock we might not have an indicator for the time switching from am to pm or to the next day.

So in summary, if you add numbers you need to know what it is that you are adding and take the right addition law to describe what you are interested in. If you take two integers and use the standard addition law, then, yes, two plus two equals four. But there are many other things those numbers could stand for and many other addition laws, and depending on your definition, two plus two might be two or one or five or really anything at all. That’s not “woke” that’s math.

Why do we see things that aren't there?

[This is a transcript of the video embedded below.]

A few weeks ago, we talked about false discoveries, scientists who claimed they’d found evidence for alien life. But why is it that we get so easily fooled and jump to conclusions. How bad is it and what can we do about it? That’s what we will talk about today.


My younger daughter had spaghetti for the first time when she was about two years old. When I put the plate in front of her she said “hair”.

The remarkable thing about this is not so much that she said this, but that all of you immediately understood why she said it. Spaghetti are kind of like hair. And as we get older and learn more about the world we find other things that also look kind of like hair. Willows, for example. Or mops. Even my hair sometimes looks like hair.

Our brains are pattern detectors. If you’ve seen one thing, it’ll tell you if you come across similar things. Psychologists call this apophenia, we see connections between unrelated things. These connections are not wrong, but they’re not particularly meaningful. That we see these connections, therefore, tells us more about the brain than about the things that our brain connects.

The famous Rorschach inkblot test, for example, uses apophenia in the attempt to find out what connections the patient readily draws. Of course these tests are difficult to interpret because if you start thinking about it, you’ll come up with all kinds of things for all kinds of reasons. Seeing patterns in clouds is also an example of apophenia.

And there are some patterns which we are particularly good at spotting, the ones that are most important for our survival, ahead of all: Faces. We see faces everywhere and in anything. Psychologists call this pareidolia.

The most famous example may be Jesus on a toast. But there’s also a Jesus on the butt of that dog. There’s a face on Mars, a face in this box, a face in this pepper, and this washing machine has had enough.

The face on Mars is worth a closer look to see what’s going on. In 1976, the Viking mission sent back images from its orbit around Mars. When one of them looked like a face, a guy by name y Richard C. Hoagland went on TV to declare it was evidence of lost Martian civilization. But higher resolution images of the same spot from later missions don’t look like faces to us anymore. What’s going on?

What’s going on is that, when we lack information, our brain fills in details with whatever it thinks is the best matching pattern. That’s also what happened, if you remember my earlier video, with the canals on Mars. There never were any canals on Mars. They were imaging artifacts, supported by vivid imagination.

Michael Shermer, the American science writer and founder of The Skeptics Society, explains this phenomenon in his book “The believing brain”. He writes: “It is in this intersection of non-existing theory and nebulous data that the power of belief is at its zenith and the mind fills in the blanks”.

He uses as example what happened when Galileo first observed Saturn, in 1620. Galileo’s telescope at the time had a poor resolution, so Galileo couldn’t actually see the rings. But he could see there was something strange about Saturn, it didn’t seem to be round. Here is a photo that Jordi Busque took a few months ago with a resolution similar to what Galileo must have seen. What does it look like to you? Galileo claimed that Saturn was a triple planet.

Again, what’s happening is that the human brain isn’t just a passive data analysis machine. The brain doesn’t just look at an image and says: I don’t have enough data, maybe it’s noise or maybe it isn’t. No, it’ll come up with something that matches the noise, whether or not it has enough data to actually draw that conclusion reliably.

This makes sense from an evolutionary perspective. It’s better to see a mountain lion when there isn’t one than to not see a mountain lion when there is one. Can you spot the mountain lion? Pause the video before I spoil your fun. It’s here.

A remarkable experiment to show how we find patterns in noise was done in 2003 by researchers from Quebec and Scotland. They showed images of random white noise to their study participants. But the participants were told that half of those images contained the letter “S” covered under noise. And sure enough, people saw letters where there weren’t any.

Here’s the fun part. The researchers then took the images which the participants had identified as containing the letter “S” and overlaid them. And this overlay clearly showed an “S”.

What is going on? Well, if you randomly scatter points on a screen, then every once in a while they will coincidentally look somewhat like an “S”. If you then selectively pick random distributions that look a particular way, and discard the others, you indeed find what you were looking for. This experiment shows that the brain is really good at finding patterns. But it’s extremely bad at calculating the probability that this pattern could have come about coincidentally.

A final cognitive bias that I want to mention which is built into our brain is anthropomorphism, that means we assign agency to inanimate objects. That’s why we, for example, get angry at our phones or cars though that makes absolutely no sense.

Anthropomorphism was first studied in 1944 by Fritz Heider and Marianne Simmel. They showed people a video in which squares and triangles were moving around. And they found the participants described the video as if the squares and triangles had intentions. We naturally make up such stories. This is also why we have absolutely no problem with animation movies whose “main characters” are cars, sponges, or potatoes.

What does this mean? It means that our brains have a built-in tendency to jump to conclusions and to see meaningful connections when there aren’t any. That’s why we have astrophysicists who yell “aliens” each time they have unexplained data, and why we have particle physicists who get excited about each little “anomaly” even though they should full well know that they are almost certainly wasting their time. And it’s why, if I hear Beatles songs playing on two different radio stations at the same time, I’m afraid Paul McCartney died.

Kidding aside, it’s also why so many people fall for conspiracy theories. If someone they know gets ill, they can’t put it down as an unfortunate coincidence. They will look for an explanation, and if they look, they will find one. Maybe that’s some kind of radiation, or chemicals, or the evil government. Doesn’t really matter, the brain wants an explanation.

So, this is something to keep in mind: Our brains come up with a lot of false positives. We see patterns that aren’t there, we see intention where there isn’t any, and sometimes we see Jesus on the butt of a dog.