It is no secret that FP4 is my favorite part of a MotoGP weekend. Every Saturday afternoon I watch the live timing carefully for signs of which MotoGP rider has the best race pace, usually pinging comments back and forth with Neil Morrison over WhatsApp.
Once the results PDF is published, I pore over the Analysis timesheets(link is external), showing times and sector times for each lap, as well as which tires were used, and how fresh or used they were.
Based on that information, plus the outcome of qualifying, listening to what riders have to say and discussing the day with others, I try to make as informed a guess as possible of what might happen in the race.
I try to estimate who looks to have the best race pace, based on lap times set in longer runs on very used tires. And if a rider hasn’t used older tires – switching between two different rear tires, for example – I try to estimate whether their pace on used tires drops off more than the times in FP4 show.
Is all this effort worth it, or am I wasting my time? I felt it was time to put my hypothesis that FP4 is the most important and instructive session to the test. Is the outcome of the race closely correlated to the results of FP4? Or is there another session which is more useful to that extent.
To test my hypothesis, I devised a number of virtual championships. Using the same points system as the MotoGP championship – 25 points for 1st, 20 for 2nd, 16 for 3rd, 13 for 4th, 11 for 5th and then 10 down to 1 points for 6th through 15th position – I scored every rider based on where they finished in FP4 at every MotoGP event during the season.
To test the usefulness of FP4 as a session, I chose two other sessions to compare it against: warm-up, because it is the only other session that doesn’t count towards qualifying; and qualifying, as that is the most important of the time sessions. I then compared those three sessions against the actual outcome of the 2022 MotoGP championship.
The reason for choosing qualifying is fairly self-evident: with passing getting harder, grid position is increasingly important.
The reason for comparing with the warm-up session is less obvious. It is the one session where I genuinely have no idea of the usefulness of the outcome. It does not seem intuitively to correlate to position in the race.
It is generally used either to make final tweaks to setup, or to try to understand how a rear tire will hold up in the race. But at only 20 minutes, there is not enough time to do too much.
What I found was that it was good to test my assumptions against data. Because while FP4 may be a very useful session to examine, the actual result – the riders ranked by fastest lap times set during the session – proved not to be anywhere near as useful as I thought.
Before we proceed, a warning: there are going to be a lot of tables in this piece, which will necessitate a lot of scrolling. And because of the way the tables are formatted, they may not fit neatly onto screens, especially mobile ones.
First, a table comparing the scores of each rider in each session, compared to their final scores in the MotoGP championship:
|20||49||Fabio Di Giannantonio||Ducati||10||32||64||24|
The above table is useful to get a general idea of how each rider did, but there are few conclusions we can draw from it beyond it being clear that the points scored in the championship correlate very broadly with their results in other sessions. It shows the roughly blatantly obvious: fast riders are fast.
There is one rider I would draw your attention to, however, as he is something of a recurring theme: Brad Binder’s sixth place in the championship, with 188 points, is more than double his total in any of the other sessions.
They used to call Valentino Rossi a Sunday man, but Binder’s race results are radically better than in any other session. But more of that later.
So first to test my hypothesis that FP4 is significant. Bearing in mind that we lost two FP4 sessions in 2022 – Argentina to travel delays, and Motegi because of the tight travel schedule from Aragon to Japan – the results sorted by finishing position show a very mixed message:
|20||49||Fabio Di Giannantonio||Ducati||10||24||-14||20|
The top two in the championship take the top two spots in FP4 as well. The fact that their positions are reversed – Fabio Quartararo performs best in FP4, leading Pecco Bagnaia by 31 points – would seem to confirm the idea that Quartararo has been the fastest rider in race trim throughout the year, though he could not convert that into a successful championship defense.
Behind the two leaders, however, the picture is much more muddied. Marco Bezzecchi showed his real speed in FP4, often finishing close to the top and ending third in the FP4 standings.
But he couldn’t replicate that in the race: Bezzecchi scored 95 fewer points in the race than in FP4, and his championship position of 14th is 11 places worse than the third he managed in FP4.
That is perhaps to be expected for a rookie. He scored 3 DNFs and finished outside the points on two other occasions.
There was plenty of variance among the veteran riders as well. Johann Zarco was fourth best in FP4, but couldn’t replicate that in the race, a common theme across warm-up and qualifying as well.
KTM’s Miguel Oliveira was stronger in FP4 than in the race, while Enea Bastianini, Jack Miller, and Aleix Espargaro all performed vastly better during the race than during FP4.
To get a better idea of who did better in the race than in FP4, below is the table sorted by points difference (the difference between the points in the virtual FP4 championship and the actual riders’ championship):
|10||49||Fabio Di Giannantonio||Ducati||10||24||-14||20|
Here, we get a taste of just how much better Brad Binder is on Sunday afternoon than on Saturday. The South African improved on his virtual FP4 performance by a total of 113 points over the season.
That is nearly double the improvement of Aleix Espargaro, the next most improved rider from FP4 to the race. Enea Bastianini, Alex Rins, and Luca Marini are notable here.
At the other end of this table, beyond Marco Bezzecchi, we see Franco Morbidelli. Given Morbidelli’s struggles in 2022, his pace in FP4 not translating to the race underlines his problem, which is qualifying, as we will see later.
How well does FP4 correlate to the result of the race? Using standard deviation as an approximation – put in very rough terms, the average number of positions the result in the virtual FP4 championship differs from the actual championship – riders finish FP4 within 4.34 places of their championship position. (This is very crudely put.)
The problem with standard deviation inside a small sample – 22 riders finished inside the top 15 during the 20 races this year – is that a couple of outliers can strongly skew the results.
And there are a few prominent outliers: Marco Bezzecchi is 11 positions in the virtual FP4 standings ahead of his final championship position; Franco Morbidelli is 8 positions worse; and Brad Binder does 8 positions better.
But to interpret data correctly, you have to use it all, rather than just cherry pick the bits that fit your narrative. You can make a case for excluding outliers as being not representative, but there are outliers in all three comparison sets we are using.
One way of trying to work around the outliers is looking at the mode. The mode is the most common value in a given set of data.
Put in terms we are using, it is the number of position changes the largest number of riders achieve. For FP4, the mode is -2, meaning that riders tend to finish 2 positions better in the championship than in FP4.
How does FP4 compare to warm-up? Let’s look at the virtual warm-up championship standings first:
|20||49||Fabio Di Giannantonio||Ducati||32||24||8||20|
This was the session which I genuinely had little idea of, and so for me, it produces the biggest surprises. I certainly hadn’t expected to see Fabio Quartararo leading this championship, yet he is the clear leader here, ending top with a 23 point advantage over Aprilia’s Maverick Viñales.
Viñales is less of a surprise, not so much in the fact that it was Viñales, but in that I expected an unexpected name to be close to the top of the results in warm-up. It confirms the sense of randomness I get from the Sunday morning session.
That sense of randomness is far from being justified, however, as we shall see later.
At first glance, there seem to be a fair few outliers in warm-up. Besides Maverick Viñales, Takaaki Nakagami does 8 positions better in the warm-up championship than in the actual riders championship.
Brad Binder once again features prominently, improving 11 positions between warm-up and riders standings, and Alex Rins does 6 places better in the riders championship than in warm-up.
Yet that impression deceives. Calculate the standard deviation in warm-up, and it comes to 4.11, which is lower than FP4.
That implies that the spread of differences is smaller, meaning that the result in warm-up is slightly more closely correlated to the actual race results than FP4. That was something I was not expecting.
|17||49||Fabio Di Giannantonio||Ducati||32||24||8||20|
Compare the difference in points between the virtual warm-up championship and the final riders standings and it is once again Brad Binder who is head and shoulders above the rest. The South African outscored his warm-up standings by 133 points in the race.
Second was Pecco Bagnaia, perhaps a sign that the Italian was focused more on setup than anything else on Sunday morning, and used the session effectively.
The fact that his Ducati Lenovo teammate Jack Miller was also 58 points better in the race than in warm-up underlines this impression, that Ducati were very effective at finding improvements during warm-up.
At the other end of the table, the two outliers mentioned earlier, Maverick Viñales and Takaaki Nakagami lost the most in the race compared to warm-up.
For Nakagami, that is a given, repeating a pattern from previous years. Viñales’ performance is perhaps down to the process of adaptation to the Aprilia, though that is a hypothesis we can test again at the end of 2023.
Also notable are the Pramac Ducatis of Johann Zarco and Jorge Martin. Both Pramac riders were scoring highly in morning warm-up, only to come up short in the races.
What about qualifying? Does grid position determine the final outcome to a much greater extent than any other session? The answer to that is not quite as clear cut as you might hope:
|16||49||Fabio Di Giannantonio||Ducati||64||24||40||20|
It should be no surprise that the rider with the best qualifying performance is also the rider who won the MotoGP championship in 2022.
In our virtual qualifying championship, Pecco Bagnaia ended up with 293 points, 28 more than in the actual riders championship. He also ended up with a BMW M3 Touring, the car which BMW present to the best qualifier of the year.
Behind Bagnaia, however, a slightly different picture emerges. Jorge Martin proves his speed over a single lap by finishing second in the virtual QP championship, just 11 points behind Bagnaia.
But, this also demonstrates how he has struggled to turn great grid position into results: Martin ended up ninth in the riders championship, scoring a whopping 130 fewer points compared to qualifying.
Fabio Quartararo is third in the virtual QP standings, and losing just 2 points between QP and riders championship, confirms his consistency.
The 2021 MotoGP champ did as well in qualifying as he did in the race. That may also explain the second half of his season, where he struggled to get onto the front row of the grid, and paid the price for that.
Jack Miller and Johann Zarco out-qualified their race performances, underperforming by 51 and a massive 73 points respectively. Aleix Espargaro slipped a little, dropping 14 points, or roughly three quarters of a point per race.
Enea Bastianini, on the other hand, did vastly better in the race than in qualifying, ending third in the riders championship but only seventh in the virtual QP championship.
What this particular table makes clear is just how overwhelmingly good the Ducati is over a single lap.
There are seven Ducatis in the top ten of the qualifying standings, with rookie Fabio Di Giannantonio the only exception. Even then, the Italian did 40 points and 4 positions better in qualifying than in the race.
Four of the top five riders in the virtual QP championship were on Ducatis, compared to three of five in the riders standings, and five of the top ten. On average Ducatis qualified 2.25 positions better than they finished in the race.
As you might expect, the correlation between qualifying position and race result is stronger than for both FP4 and warm-up. The standard deviation is 3.89, a bit better than warm-up and almost half a position better than FP4.
And if you only take the top half of the standings, the first 12 riders, the standard deviation is even stronger, 3.68. You might cautiously suggest that there is a solid link between qualifying on the first four rows of the grid and the race result. But again, a caveat that this is a ridiculously small sample set.
It is also, perhaps, a flawed way of doing it. A better way might be to collect the data for each race individually, and work out the standard deviation for that.
But this has already been a time-consuming exercise, and sorting the data for each race would take much, much longer.
|19||49||Fabio Di Giannantonio||Ducati||64||24||40||20|
If we sort the results by differences between qualifying championship, we get a much more interesting picture. And one which underlines just how much better Brad Binder is on Sunday afternoon.
The South African outscores his virtual qualifying standings by 104 points, and finishes eight places higher in the rider standings than in qualifying.
What this table shows, however, is that the bike also plays a role. Second in the improvement stakes is Binder’s Red Bull KTM teammate Miguel Oliveira, with a 93-point improvement over qualifying to Binder’s 104 points.
That is pretty strong evidence for both the strength and the weakness of the KTM RC16. The KTM riders have consistently complained of a lack of turning and especially a lack of grip.
A lack of grip makes it really hard to squeeze the extra few tenths out of a new tire, which makes the difference between qualifying at the front and having to battle your way through.
By contrast, if you don’t have much mechanical grip, then when grip starts to drop as the tires go off, you tend to suffer less.
The bike is already set up to handle low grip, and so less changes than for bikes which are set up to exploit the grip available at the start of a race.
That may explain why the KTMs are capable of passing other bikes, when conventional wisdom has it that this is difficult.
Add that to the fact that the bike seems to brake quite well, and the KTM riders have weapons with which to fight. Unfortunately, this data also emphasizes the need for them to do just that.
It is also worth singling out Brad Binder once again. Across all three of the virtual championships, Binder outperforms his position in practice during the race. However, that is both a virtue and a vice.
The fact that Binder struggles in qualifying is borne out by the fact that teammate Miguel Oliveira is in a similar position. But Binder also does poorly in FP4 and warm-up, where Oliveira is much stronger. In FP4, Oliveira is stronger than in the race, where Binder is much worse.
We may cautiously draw the conclusion that this is Binder’s Achilles heel. There is no doubt that the South African turns up on Sunday. But his results in the remainder of the sessions raise legitimate questions as to what he is doing on Friday and Saturday, and if he wouldn’t benefit from focusing on improving there.
Looking beyond the KTMs, the table shows strong improvement between race and qualifying for both Alex Rins and Enea Bastianini, improving by 60 and 50 points respectively.
Alex Marquez also deserves a mention here, scoring 35 more points on race day than he did in qualifying, the only Honda rider to manage that feat.
At the other end of the table we see the double-edged sword of Ducati’s outstanding qualifying performance.
The bottom five – that is, the five riders who lost the most between the qualifying and the race – are all Ducati riders, with Jorge Martin suffering the worst, with a difference of 130 points, ahead of Johann Zarco, Jack Miller, Fabio Di Giannantonio, and Marco Bezzecchi.
Martin’s gap is notable because of its size. But that is also an artifact of just how outstanding the Pramac Ducati rider is over a single lap. He loses a lot of points because he qualifies so far forward.
Martin scored five pole positions, and started from the front row on four other occasions. The points difference between first and third is 9 points, between first and fifth is 14 points, so qualifying so well makes it easy to lose points in the race by losing just a couple of positions.
But Martin also managed five DNFs, a 22nd, a 13th, a 10th and a 9th. The Pramac rider’s speed is not in question. But on the basis of this analysis, it is pretty clear that Ducati’s decision to elevate Enea Bastianini to the factory squad instead of Martin is probably the right one.
The comparison between qualifying and racing is undoubtedly useful, but it also requires a note of caution. Ducati’s strong performance in qualifying makes it vulnerable in the race.
It is easier to lose positions if you start on the front row than gain positions. If you qualify badly, there are more riders ahead of you to pass.
To take an extreme example, look at Marc Marquez’ 2019 season. It is widely regarded as the nearest thing to a perfect year since Giacomo Agostini was winning everything on the MV Agusta against paltry competition. In 2019, Marquez did not finish worse than second, with 12 wins and a single DNF.
Yet on average, he finished only a single position better than he qualified, and scored 36 points more during the races than he did in qualifying. Solid, you might say, but not spectacular.
That category error is also present in a fun and interesting graphic I saw on Twitter, made by user @mgp1official. That chart shows the number of positions gained or lost by each rider throughout the season.
It is similar to the table above, with a few crucial differences: Brad Binder leads, but Alex Marquez is ahead of Miguel Oliveira in terms of places gained, while Alex Rins and Enea Bastianini are well down the order.
The fundamental flaw with the graphic is apparent from the names in fourth, fifth, and sixth. Raul Fernandez, Darryn Binder, and Remy Gardner all made up a lot of places during the season by the simple expedient of starting at the back of the grid.
If you start last and finish last, but five riders crash in front of you, you gain five places even though you may be a second a lap slower than the riders who crashed out ahead of you.
This is also the reason why Fernandez, Darryn Binder, Gardner, and Andrea Dovizioso don’t appear in most of these tables. They very rarely cracked into the top 15, and so were not awarded points under the system I used.
The system I chose does not show passes, or positions gained, but I feel it does show a more direct correlation between performance in practice and qualifying, and the races.
Finally, what conclusions can we draw from all of this? It’s complicated. The clearest correlation is between qualifying and race results, but that hardly qualifies as a novel insight.
But the dominance of Ducati in qualifying should be cause for concern given the introduction of sprint races in 2023. In a format where tire degradation is not a factor, and where fuel consumption is also less significant, Ducati should be able to convert strong grid positions into race results more easily. This will only increase the importance of qualifying.
My own hypothesis, that FP4 correlates highly to race results, and looking for clues for race performance during warm-up is a largely pointless exercise, has been shown to be wrong.
At least, in terms of outright times: the lap time set in warm-up is more tightly correlated to performance in the riders championship than the time set in FP4.
However, it is worth returning to how I use the data in FP4. Outright lap time is far from the most important factor in trying to figure out who will do well and who will struggle during the race.
Far more important is pace, that is the lap time which riders can maintain on very worn tires. That, I would contend, is a far more useful measure.
So, was this entire exercise a waste of time? Not at all. There are still plenty of lessons which are clear from the data. Ducati dominate qualifying, with Pecco Bagnaia able to translate that into a championship. Jorge Martin is fast over a single lap, but can’t turn that into championship points.
The KTM struggled in qualifying, but was a much more potent weapon in the race. Brad Binder is truly a Sunday rider, and would do well to work on the rest of the weekend as well.
It is always worth diving into data. But you must also be cautious with the lessons that data teaches.
Finally, my apologies for an inadequate understanding of statistics. Others will have a better grasp of the subtleties of the data I am trying to wrangle here. But something is better than nothing.