Lisa Falco on the Gender Data Gap, Health, and AI Bias

A data scientist for more than 15 years, Lisa Falco has always been fascinated by medicine, but didn’t see herself becoming a doctor. Still, she managed to find her way to the field she loved, working where technology meets healthcare.

In 2015, she started exploring health with a female-centric perspective after joining Ava Women. The founders of this Zurich-based startup wanted Lisa to help them develop a cycle and fertility tracking bracelet.

For four years and a half, the Swiss scientist worked tirelessly on this bracelet, constantly astonished by all that she was learning about the workings of the female body. She and her team even discovered things that had never been observed before.

This experience made her want to share this knowledge that she believed essential, especially to women. “If you’re a lawyer and you want to prove that you’re right, you have to come up with the right evidence, right? Knowing the scientific background of what you’re experiencing kind of provides you with the evidence.” So she wrote a book full of comprehensive information presented in an accessible manner that she hopes women find useful: Go Figure! The Astonishing Science of the Female Body. 

We sat down with Lisa Falco to talk about the use of data.

The following interview has been edited for more clarity with Lisa Falco’s permission.

A “basic” question, but sometimes the simplest things aren’t necessarily common knowledge. What is a data scientist exactly? What does he/she do on a daily basis?

Lisa Falco — So a data scientist looks at data. Data can be in any kind of form. Like images. Images are data. For instance, as a data scientist, you can look at medical images and try to extract information from them. Or in the case of wearable technology, you can look at a continuous flow of data — from the signals that the wearable measures —, and you make sense of it, filtering and extracting the most important features.

Then down the line, very often, you would use machine learning methods to really extract some kind of knowledge from that. It’s very closely linked to statistics. You find things that are not random in your data, things that are already there. But you need to sort out the “random noise” that is around it to extract knowledge.

So it’s not just about studying numbers. (laughs)

(laughs) No, no. You want to understand the fundamental process behind something, what is real. For instance, the heart rate signal. Your heart rate goes up and down, up and down… And it changes all the time — if you walk, if you sit, if you run. As a data scientist, you can extract those different parts of the signal, and then find, identify some things that are linked to a phenomenon that can’t be explained by walking or running. You really need a lot of data because otherwise you can’t just say “oh, this is not coincidental.” If you have enough data, you can prove that something is statistically significant. Otherwise it could be due to some other random effects. Which is why you should never trust clinical studies that have only 50 participants because that is not really enough to come up with a conclusion.

We often hear that we’re living in a data-driven world, in that we base so many decisions on data nowadays. Why is data so important?

It’s the way you provide proof that something is not random. It allows you to correlate things. For example, if I’m talking to you and you say you feel a certain way today and then you feel another way tomorrow, that can be specific to you. Is this a global phenomenon or is it just your perception of things? Then if we look at data from a thousand women and we see that, in this particular situation, they are having a similar feeling or similar symptom then that data can show you that it’s a real thing. It might not explain why just yet. But then we can use the data to correlate it to other knowledge that we might have, and see if this is actually something that is common. And it’s kind of the basis for all clinical science because we don’t understand all the small processes that are happening in our bodies. We can just make correlations. And the more data you have, the more solid you can prove that the correlations are real, that they are so-called “statistically significant.” That’s why data is very important to bring medicine forward, but also other things. You can use it also just to boost the business of your company in all different sectors. But especially in medicine, it’s incredible what you can achieve if you have a lot of data.

There is — not just in medicine, but across all fields — what is referred to as the gender data gap: there is a lack of data about women’s and girls’ experiences in the world. Why do you think that’s the case?

The gap is a traditional gap that has accumulated over a very long time. For many years, women were actually excluded from clinical trials, which is typically where you would collect the data. It was not to be discriminatory against women, that was not the purpose. Men and women were just not considered to be fundamentally different. Women were seen as men with the “complication” of hormones. And that complication made it more difficult to extract results from the studies. So it was easier to just say “let’s just study men, they’re more constant, they don’t have this complication.” And to another extent, it was also to protect women because they could be pregnant.

It’s only since 2016 that women have to be included in clinical trials [in the United States, at least]. So we are starting to fill the gap. But there’s a lot of work to be done because we have decades and decades of data that are missing. So even though it didn’t come from a bad intention, it has had quite bad consequences that we’re seeing today.

I also think that hormones were not really taken seriously because most of the doctors and scientists used to be men, and as a scientist, you tend to research what lies close to your heart. I’m not saying that men cannot relate to women, but now that we have many more women who are doctors and scientists, it’s different. It has contributed to pushing things forward. We’re in a very positive momentum. We still have a long way to go, but a lot of things are happening and change is coming.

What “bad consequences,” as you said, are you thinking of?

A lot of medicines were traditionally only tested for men, and recommendations were done then for men. Aspirin, for example, is supposed to be good for the heart and the blood flow in men, but they have not really shown the same thing for women.

Since men and women are actually fundamentally different, hormones do make a difference. Hormones change how our bodies are composed. Women have a much higher percentage of body fat than men. Women also have different enzymes and digest drugs differently.

With the Covid-19 vaccines, we saw that women were reacting much more strongly. Some studies have shown that, although women were a part of the test groups, the evidence of the efficacy of the vaccines wasn’t presented separately [by sex]. So it’s not only that you need to collect the data, you also need to analyze it separately. And then you could see that women had more secondary effects, which is really important knowledge to have. You might make different decisions when you do. You might also want to adapt the vaccine. Should we really have the same vaccine dose? These are things to consider, always.

Earlier you mentioned how, historically, women were viewed as men with a “complication,” which led to excluding women from trials. And yet at the same time, the biologically female body, in its differences with the male body, was used as an argument to discriminate against women.

Yes, exactly. I also always like to point out that we’re different on average. Women are not the average of women, and men are not the average of men. We’re all distributions, and there’s a lot of overlap in the distributions of how men are and how women are. Which is why the future of medicine is more about personalized medicine, to understand which factors about your body make you react a certain way to certain medicines. It can be linked to your genome, your proteome, your body composition…

What do you mean when you say “we’re all distributions”?

Distribution is a statistical term. Most things in nature are normally distributed. It looks like a bell curve. For instance intelligence. You have the average IQ in the middle, and then there’s a distribution around it of different IQs. The difference in IQ between men and women is quite an interesting example because if you look at the distributions, if you superpose the graphs, you’ll see that they are more or less identical. But actually, there is something that slightly pushes up the average for men: the most extreme geniuses tend to be men. And that is at least what we know. Maybe female geniuses have been hidden away somewhere. But if you pick a man and a woman randomly off the street, you can’t tell how likely it is that one will be more intelligent than the other. That’s why the distributions matter.

Generally, people like very simplistic explanations of things. They like to say “men are like this,” “women are like that,” and just look at the averages and then compare. But that makes absolutely no sense. It’s not because you’re a man that you’re automatically going to be a good driver for example. The spectrum is large. You could just as well be a terrible driver because the distribution is huge. You cannot infer anything by meeting a person because you never know where they are on the distribution.

As pointed out by Caroline Criado Perez in her 2019 book Invisible Women, this gap can even prove to be life-threatening. Among the most striking examples she gives is how heart attacks can go undiagnosed in women. What has been your experience with the lack of data and the bias in data collection while working on women’s health?

It’s a lack of data but also a lack of perception, I would say. We do associate heart attacks more with men, but cardiovascular disease is also the number one killer of women so it’s important to be aware of that. Also Alzheimer’s, for instance, is actually more prevalent in women, which is something many people don’t know.

Since I began working in a female-centric health environment, this gap has rather been an opportunity for me than a problem. With the teams I have worked with, we’ve had a chance to publish really cool findings and help advance women’s health.

But there are many things around how women react to certain medications in different parts of their cycle, things that I’ve been discovering about myself that I did not manage to find any kind of literature about… And just the fact that when we launched the Ava fertility bracelet, we were the first ones to see that the heart rate goes up a few days before ovulation. No one had actually measured the heart rate over the full menstrual cycle before. All research prior was just about one measurement point in the first half of the cycle then another measurement point at the second half of the cycle. But when we started looking at all these parameters that were changing, we could see how the heart rate went up before ovulation, which no one had shown. Another thing that’s really interesting is that when you’re pregnant, the heart rate starts going down a few weeks before giving birth. And no one had seen that before. That just shows how much is still possible to find out.

And while doing research for your own book, which is all about the female body, was there something in particular that struck you?

The data gap was never the focus of my book, to be honest. I wanted to highlight how exceptional the female body is. I read… 700 articles (laughs) to decipher and really understand the fundamentals, and I was continuously surprised by all the facts that were there. “This can’t be true. How did I not know this?!” So it’s also about knowing what is already known but hidden. The… treasures of all those clinical studies.

I think one part of the problem is about the data gap, but what’s also important is to make the knowledge that is out there available to people. 

Do you think that’s where journalists can enter into play? 

Absolutely. It’s really important to highlight everything that is coming up there.

But it can be hard when you’re not a specialist. One might read a study and not grasp what is important in it. How can journalists make that work?

It is difficult if you’re not an expert, and we see a lot of journalists… not always doing a great job when reporting on health and science. There are a few major papers, like The New York Times, that have amazingly well-researched articles. They have the right people to analyze it because you do need some basic knowledge.

Even for me sometimes, it can be hard to understand everything directly from the scientific article itself. Then I have a few sites I like to go to. Like the Mayo Clinic or Harvard Medical School, they provide quite easily comprehensible articles about a lot of phenomena. Also Healthline, they have doctors double-checking all the information. 

Or you can go directly to the source. I’m sure that if there’s an article that seems interesting, the researcher will be very happy to talk to you, and talk you through it.

Earlier you said not to trust a study with only 50 participants. What are some red flags to look out for, to know to trust or not to trust a study?

I mean, definitely the number of participants. (laughs) And if the study has been done over a long time, if they have done a good analysis of confounding factors. Although it will be hard for you to just come in and say “oh, did they do it the right way or not?”

The papers that come from peer-reviewed journals are more reliable because experts have assessed the quality of the content already. If it has been published in Nature, you can be quite confident in it. You also have The American Journal of Medicine and BMJ. If you have a high-impact journal, you also have better reviewers.

It’s never 100 percent solid, of course. There are some amazing examples — like the fake article about the benefits of chocolate in 2015. It managed to get published in a [non-peer-reviewed] journal, but… it was all made up. Still it was distributed a lot in the media because of course, everybody wants to believe something like that! I still can’t accept that it was not a real article (laughs).

You work on health-related issues, but you’re a woman in tech, an industry known for its gender gaps, from its workforce to the data it works with. In recent years, we’ve heard a few times about how artificial intelligence perpetuates bias. Is that something you’ve been confronted with in your work?

It’s been a huge problem in the field. If you apply AI blindly, it will model the data as it is. The data is modeling the world as it is, and the world has a lot of prejudice in it. Right now I’m working in a consulting company that is implementing AI solutions not only within health but also banking, insurance and so on. What I’ve been working on with my colleagues this last year is coming up with what we call a “responsible AI” framework. We’ve been working on how you can make concrete suggestions, in your tech development, so that you make sure that you don’t build in this bias. Because nobody wants to build in the bias, it doesn’t come from malice, it’s not with a bad intent. It’s done simply by not paying enough attention to the right things at the right moment. And we’re not the only ones that are working on this.

There’s a movement for a more ethical implementation of AI. People are made more aware of this so I’m very hopeful for the future there as well. 

Is there something that you want to add, to leave the journalists reading this interview with?

I would like to highlight the positive momentum we’re in, and the opportunities that come with focusing a bit more on women. It actually helps destigmatize things around hormones and women’s health. You know, a lot of women are often not taken seriously. I don’t think journalists would fall into that, but it’s important to ask “why do women have this or that perception and what is lying behind it?” Always dig a little bit deeper than the simplified explanations and never be dismissive of the experiences that women are having because there is often a scientific explanation behind it.