The Rob Young Investigation: Key findings

Yesterday, the investigation that I had been commissioned to do with Roger Pielke Jr on the allegations of Rob Young cheating during his TransAmerica run was published by Skins.  The report concludes, very strongly, that Young had to have traveled large parts of the run in the vehicle, until he was discovered by Asher Delmott and subsequently observed by a group of volunteers.

You can download the full report at this site, but below I have pasted the section of the report that deals with the data confirming his cheating, which is what we identified as the smoking gun.  It’s pasted from the full report.

I’d like to thank Skins for taking the initiative to do this – when I was contacted by Jaimie Fuller in June this year, he said that he wanted independent evaluation of the allegations and data, and thereafter, he and his team made every attempt to provide all possible data, and were extremely accommodating of the process that Roger and I felt needed to be followed.  It was an exercise in integrity on their part, and that deserves significant recognition – it certainly makes a change from sponsors protecting cheating.

Second, this investigation was the result of online investigators who responded to Asher Delmott’s initial Letsrun post (this is all in the fully report), and then took it upon themselves to scour his data records and social media posts to build a picture of suspicion.  Our investigation confirmed many of the theories they had developed.  And while there was the usual amount of vitriol and personal attack in a good deal of the discussion, there was also some very high quality analysis, which we were able to use to a) start drawing the picture, and b) complete the loop once we had the important cadence data described in this post.

As for Young, I note that he has continued to protest his innocence, despite the very strong evidence.  I have no idea where the story goes from here – everyone deserves redemption and second chances, but continued denial is not a good place to start the search for it.  I would advise the truth, and then perhaps look to rebuild an ultra-marathon career if possible.  That process can only follow the truth, and this exercise, along with the background to it, have raised significant doubts over what is true and what is not.

Ross

During the record attempt, Rob Young used two TomTom watches, described below. The data from the TomTom watches was uploaded to Young’s TomTom account throughout the run.  It is known that this was happening because screen shots of the uploaded records were being posted to the Rob Young Facebook page during the attempt, though these records, and the associated Facebook posts, were deleted from Facebook once allegations of cheating began to surface (See the sections below on the Kansas incident).

The TomTom record was however the final data source provided to investigators, after Strava and MapMyRun files had been initially provided via log-in details to these sites where Young had uploaded the original TomTom data. The Strava and MapMyRun data were however found to be incomplete and otherwise unsatisfactory.  Those records, discussed below, were compared to the TomTom record for validation purposes, but either did not have cadence data, or produced a calculated cadence number based on a formula, and were thus deemed inappropriate for reliable analysis.

Further, we established that TomTom data, uploaded directly, could not be altered, unlike MapMyRun and Strava entries uploaded from files, which can be edited.  Therefore, a priority for the investigators was to obtain both the watches (to have a record of the data in their collected, “raw” form) and the original TomTom data files, which would have been backed up to a local computer (two laptops that accompanied the attempt).

TomTom log-in information was eventually obtained on July 11, 2016 (the investigation began July 1).  We requested, but were not provided access to the backup TomTom files from the two laptops that accompanied the attempt. These would have included all TomTom files uploaded from the watches. The watches were provided to us but had been completely cleared of all data. We know that the TomTom files that we were provided were a subset of all runs recorded during the attempt. An unknown number of files were not provided to us.

The TomTom data that was provided to us (the “record” of the attempt) formed the basis for the primary analysis of the record attempt, with other records serving validation purposes only. None of the other evidence that we looked at, including interviews, provided any basis for an alternative explanation for what was displayed in the TomTom data.

A total of 322 running sessions, spanning from May 13 to June 16, were found in this record.[1] This compares to 299 sessions recorded in the hand-written log book. Futhermore, we have evidence that more than 322 sessions were actually collected by the TomTom watches, based on screenshots for certain runs and segments of runs not present in the data files that we were provided. Certain sessions were duplicated, and overlapping segments were also found. Taking these issues into account, the exact number of total running sessions is unknown.

With these issues understood, we proceeded to use the TomTom data provided to us as the Data-of-Record for the analysis of the attempt.

The TomTom record of 322 sessions covered 2113 miles between May 13 and June 16, 2016.

Session length ranged from 0 miles to 33.99 miles, with 230 of the 322 sessions covering 3 miles or more.  64 of the sessions were run at a pace faster than 9 min/mile.  In order to account for some of the “noise” in the data, we focused specifically on these subsets of the total record with longer runs (more than 3 miles, 230 sessions) and faster runs (faster than 9 min/mile, 64 sessions).[2]

Appendix A contains a tabulated record of these sessions, which are also summarized in an Excel file that accompanies this report. We have also made available to accompany this report all of the TomTom records that were provided to us by Young.

The primary focus of the analysis is cadence data, rather than speed or the performance during the runs.  We chose this approach because analyzing the running performances invites allegations and rebuttals that are by nature subjective and unprovable – any individual can claim to be capable of running at a certain speed for a certain period provided that speed is reasonable (that is, not obviously unrealistic).  In the absence of direct observation and bench-marking against that athlete’s known capabilities, any claim, even if exceptional, is impossible to confirm or to refute.  Since we have no direct evidence of what Rob Young is capable of, and since the data of record is the only performance data relevant to this analysis, this approach would be fraught with subjectivity and suppositions, even if suggestive.

Cadence data, on the other hand, might reveal cheating, irrespective of performance, because:

1. It is more constrained than actual running speed – even elite athletes running at vastly greater speeds do so at cadences that are similar to those achieved by recreational runners
2. It is a direct means of assessing the primary investigative focus in this case, namely whether Young gained unauthorized assistance in the attempt.

Consequently, cadence data are key to our focus.

The TomTom record allowed us to examine the cadence data, which had previously been absent from any analysis of the attempt.  The MapMyRun record does provide a step number for each session, but we analysed this data and discovered that it is not a true step count, but rather a number calculated by the software based on an estimated relationship between the number of steps taken per mile and running pace (min per mile).

1. Infeasible and impossible cadence values

Figure 1 on the following page shows the cadence in steps per minute (counted as each foot-strike) as recorded by the TomTom watches for the 230 sessions longer than 3 miles. These are displayed on the graph in chronological order from left to right.  The typical ranges for slow walking, normal walking and a typical minimum for jogging are shown.  Also, the incident involving Asher Delmott is shown, as is the timing of Delmott’s first post on the LetsRun website which led to subsequent analyses and accusations against Young.

Figure 1: Cadence during sessions longer than 3 miles

 

It is clear from this figure that many sessions have extremely low cadences.  Of the 230 logged sessions longer than 3 miles:

• 44 have a cadence lower than 20 steps/min
• 26 have a cadence between 20 steps/min and 40 steps/min
• 16 have a cadence between 40 steps/min and 60 steps/min

What is most notable is that after the Asher Delmott post (June 7), the number of sessions with a cadence below 60 steps/min (corresponding to a slow walk) virtually disappears, as shown by the figure below:

Figure 2: Change in cadence after Asher Delmott LetsRun post

 

That is:

• Two sessions out of 54 runs after the Delmott post have a cadence below 60 (4%), compared to;
• 84 out of 176 sessions (48%) prior to the Delmott post on LetsRun
• Instead, most sessions after the Delmott post fall within the range that is expected for a mixture of normal walking and typical running (42 out of 52 sessions, 78%), and which is consistent with the paces logged by Young during these sessions

Cadence data can only be interpreted when the other details of the running session are known, however, because a person who is traveling very slowly on foot would be expected to have a very low cadence.  They may stop frequently, causing both speed and cadence to drop, and this could account for those sessions observed to have extremely low cadence values.

Therefore, we next looked at the relationship between cadence and running pace for sessions that were completed at a faster running pace, specifically by examining only sessions that were run at a pace faster than 9 min per mile.  The graph below shows the cadence values of those faster paced sessions only.

Figure 3: Stride rate during sessions faster than 9 min/mile pace

 

It is clear that when Rob Young was running faster than 9 min/mile (64 sessions in total), there were a number of sessions with an implausibly low cadence.  These sessions ranged in distance from 1.02 miles to 31.98 miles, and in pace from 4:56/mile to 8:51/mile.  Of these 64 sessions:

• 39 had a cadence lower than 20 steps/min
• 18 had a cadence between 20 steps/min and 40 steps/min
• Only three had a cadence greater than 90 steps/min

It is unequivocally impossible for a runner to maintain a pace of 9 min/mile or faster with cadence values this low. The data strongly suggest that the TomTom watches cannot have been worn by a runner during these sessions – they must have covered the distance without the taking of steps, which implies inside a vehicle for all or part of the logged session.[3]

One potential counter-argument that could be made is that the watches were malfunctioning or otherwise not working properly. We note, firstly, that no one has made this argument to us.  Secondly, and specific to the data, referring to Figures 1 and 2, if both watches were not working properly, then the pattern of very low cadence would continue all the way to the end of the record attempt.[4]  This did not happen – immediately after the Delmott LetsRun post, and during the period of observation, the cadence values (on both watches used in the attempt) returned to what would be described as ‘typical’ or normal cadence values.

Furthermore, Figure 3 reveals that after the LetsRun post, not a single run faster than 9 min/mile was achieved, at any cadence.  In combination, Figures 1, 2 and 3 show that Young continued to run after the Delmott post, but at slower speeds and with typical cadence values.  This refutes any suggestion that the atypical cadence data can be attributed to simultaneously malfunctioning watches.
Another explanation of the data is that Young was running while the watches were repeatedly left (accidentally) in a vehicle.  But when this question was posed to both him and Michael Speicher, both emphatically denied that any runs had been performed without the watches on Young as he ran.

Specifically, here is what Rob Young told us when asked:

[7:27:08] Ross Tucker: Next question, straight forward, Did you, at any stage of the run, benefit from driving in a vehicle, as indicated by the watch data?

[7:29:15] hania: No!!!!!  but i did jump on the truck several time less than 5 times as a dog chased me and only for 100meters maximum. We did run that distand (sic) to make it up

We asked Rob if he had ever traveled in the vehicle:

[7:30:44] Ross Tucker: Sorry, just to come back to that, we want to be clear about the data that we’ve got and have analysed. that the watch data indicates that fairly long distances, much longer than can be explained by signal loss or a short run away from dog, were travelled at speeds suggesting a vehicle.

[7:31:03] Ross Tucker: And we wanted to give you a chance to respond to that data

[7:31:16] Ross Tucker: The data is very clear on this, so any further comment?

[7:37:10] I was never in the vehicle at any point unless we had stopped and went for food or to a safe resting point – oh i did at points when i came in the vehicle  for a break leave my watch and live tracker and headed back out to run again but fairly quickly realised and put it back on me – i state again on everything, at no point did i use the vehicle for self gain in the run

We put similar questions to Michael Speicher:

[17-Aug-16 9:27:25] Roger Pielke Jr.: The cadence data indicates that the watch-of-record was in the vehicle at times. When we spoke to Rob earlier today, he indicated that there were instances when he did leave the watch-of-record in the vehicle by mistake and would quickly realize it and put it back on. Do you recall any such incidents?

[17-Aug-16 9:29:50] Michael Speicher: I have no knowledge of him coming back into to vehicle

[17-Aug-16 9:30:19] Roger Pielke Jr.: Ok, thanks. 

[17-Aug-16 9:31:11] Roger Pielke Jr.: Before leaving the cadence data, we want to be very clear here that we are following what the data says. The cadence data is unequivocal in what it shows. It will be released with our report to be openly examined. The data shows, without a doubt, that the watched traveled in a vehicle. We want you to have every opportunity to respond to this now, as it will become public. Any further comment?

[17-Aug-16 9:34:35] Michael Speicher: Rob, at all times, was in charge of the watches and in charge of the data. He was in charge of changing the watches. I was not his minder.

Absent any explanation offered by the team to counter the implications of the data, we therefore conclude that the only potential explanation for these infeasible low cadence values, present despite relatively high running speeds, is that the watches had to have been in a vehicle for part or all of the logged sessions.

1. Impossible step lengths

The next part of our analysis involved calculating the average step lengths that would be required for Rob Young to cover the known distance at the known pace.  Because there is a known and established relationship between running pace and step length, this method allows all the performance factors – cadence, speed, distance and pace to be factored into a single outcome that may be deemed feasible or infeasible.

By way of introducing this method, in order to cover a given distance at a given pace, a certain combination of cadence (steps/min) and step length is required.  If the distance, pace and stride rate for a session is known, then it is possible to calculate the required step length of that session.

Consider the following illustrative example:

• Cadence, as reported by TomTom = 170 steps/min
• Distance covered during session = 5 miles
• Time taken for session = 47:00

The step length can be calculated as follows:

Step length (m)          =          Distance in meters/(cadence x time in minutes)

For this illustrative example:

Step length (m)          =          (5 miles x 1609m)/(170 x 47)

=          1.00 m per step

We present the step length findings below for the Rob Young attempt, again based on the data-of-record:

Figure 4 shows the calculated average step length during runs longer than 3 miles.

Figure 4: Calculated average step length for every session longer than 3 miles

 

Of the 230 sessions longer than 3 miles:

• 82 had a step longer than 2m, which we deem to be a conservative cut-off for implausible step length given the average pace Young was running for these sessions
• 18 had a step length longer than 40m
• 14 had a step length between 20m and 40m
• 14 had a step length between 10m and 20m
• 19 had a step length between 5m and 10m
• 148 sessions had step lengths of <2m

The 82 runs with step lengths >2m are all clearly implausible and impossible for running, and are the result of the exceptionally low cadence combined with relatively fast running speeds that we described previously (Figures 1-3).

For reference, a runner who is running at 3-hour marathon pace would be expected to have an average step length of between 1.30m and 1.50m, given typical cadence values at this pace.

Similarly, a runner who is taking 2m long steps, and who is running with a typical cadence (150 to 180 steps/min) would be running at a pace between 4:30 and 5:20 per mile.  These are the stride parameters that would be observed in world class half marathon and marathon runners.  We would deem any step length longer than 2m to be clearly infeasible and unrealistic for Rob Young during a Trans-USA Record Attempt.

It is clear from the data that Rob Young’s TomTom record has a number of sessions with step lengths much, much higher than this, despite never reaching these running paces. Some of the step lengths are clearly not humanly possible by anyone.

This is further confirmed by Figure 5, which shows the calculated step lengths for sessions faster than 9 min/mile:

Figure 5: Calculated step length for sessions faster than 9 min/mile pace

Confirming the previous findings, for faster paced runs:

• 62 out of 64 sessions have a calculated step longer than 2m
• 57 out of 64 sessions have a calculated step length greater than 5m
• 29 of 64 sessions have a calculated step length over 20m
• 16 of 64 sessions have a calculated step length longer than 40m

Notably, not a single instance of impossible or infeasible step lengths was observed after the LetsRun post on July 7 (Figure 4 and Figure 5), and no sessions faster than 9 min/mile pace were logged after this point either (Figure 5).

The change in the proportion of sessions with different calculated step lengths for sessions longer than 3 miles after the Asher Delmott LetsRun post is summarized in the figure below.

Figure 6: Change in step length after Asher Delmott LetsRun post

 

Finally, we analyzed the relationship between running pace and step length.  This method allows us to identify sessions that are infeasible or impossible taking into account the cadence, calculated step length and the running pace.  Figure 7 below shows the findings for sessions longer than 3 miles (Fig 7A) and sessions faster than 9 min/mile (Fig 7B).

Figure 7: Pace per mile as a function of step length for longer sessions (A) and faster sessions (B)

 

For ease of viewing, we have removed the most extreme values from both graphs.  These were sessions which had an average step length greater than 150m (three sessions).  We also highlight the sessions deemed infeasible, having step lengths longer than 2m, with red symbols.

It is clear that in the majority of the longer runs, and in almost all the faster paced runs, the pace achieved is done with step lengths that are impossible (longer than 2m).  Indeed, only 2 faster paced runs out of 64 were achieved with a step length less than 2m (Figure 7B).

One of these two runs was the first run of the attempt. The data we were provided includes a duplicate file (i.e., identical) of the very first run in the attempt These two files have different dates, May 13^th and 14^th.  We hypothesize that these two files reflect (a) a time zone error on one of the watches (during its first use); and (b) one of the watches traveling by bicycle while Young ran alongside.[5]  One of the duplicate files has step lengths above 2m (consistent with a bike) and one below (consistent with a run). In addition, the Reinvestigation website has documented several other instances of duplicate files being uploaded from the two watches indicative of a bike being used while a run was taking place.[6]  The duplicate files do not bear on our bottom line conclusions.

We conclude that the sessions with impossible step lengths could only have been achieved with the watch traveling for part or all of the route by vehicle, because we can rule out faulty equipment and the possibility that Rob Young ran without the watch by accident.

1. Impact of time of day on performance, cadence and average step length

Figure 8 below shows the pace and calculated step lengths for sessions longer than 3 miles logged during the day, and for sessions longer than 3 miles logged at night.  We have again cut the x-axis off at 150 to provide a clearer picture of the finding (because several sessions had step lengths of >150m).

Figure 8: Pace during day and night sessions as a function of average step length

 

This figure reveals that the vast majority of the infeasible and impossible step lengths occur at night.  The breakdown is as follows:

Table 1: Comparison of sessions logged during day-time and night-time hours

 

The average pace of night sessions is faster, with a greater average step length.  60% of night sessions have a step length greater than 2m (which we deem infeasible).  Indeed, the average step length at night is 2.86m, which is considerably higher than both the day average (1.15m) and any step length that would considered possible given the pace of the runs.

Of the 82 infeasible sessions with a step length longer than 2m, 63 or 77% come during the night-time hours.

We conclude that during at least more than half of the night time runs, the watch covered all or part of the logged sessions in or on the vehicle.  That this did not occur in the same high proportions during the day:

1. Provides us with further evidence that the watches are not faulty, since their measurement of cadence and related parameters clearly differs from day to night and is not consistently or uniformly wrong, and;
2. Is suggestive that the cover of darkness, and the reduced likelihood of being discovered, was likely a factor in deliberate attempts to cover the route with vehicle assistance.

The possibility that observation and risk of discovery changes the behavior is explored next.

1. Impact of observation by the Geezers on step length and implications

After the LetsRun post, we observe the almost complete disappearance of infeasible and unrealistic cadence and step length data.  This period coincided with Rob Young’s observation by a group who called themselves the “Geezers.”  This group joined Rob on June 11, and accompanied him uninterrupted for the next five days before the Record Attempt was ended on June 16.[7]

The period of observation by the Geezers makes for a useful comparison against the period prior to observation, both for sessions logged at night, and during the day. The two periods offer a quasi-experimental design for this part of the investigation.

The day vs night comparisons are also important, because we have already shown that the majority of the impossible cadences and step lengths were found to occur at night (Figure 8 and Table 1).

Figure 9 shows Rob Young’s pace as a function of calculated step length during the period before and during the observation by the Geezers in order to examine how the presence of observers influences the stride parameters and performances.

Figure 9: Pace vs step length before (9A) and during (9B) observation by the Geezers

 

The breakdown and summary of the two periods is shown in Table 2:

Table 2: Comparison of sessions longer than 3 miles logged before and during observation by the Geezers

 

These graphs and Table represent the “smoking gun” in the analysis of the TomTom data.

They show:

1. Before observation, a high number of sessions had impossible step length implications.  This has been shown in various ways previously (Figures 4-7 and Table 1).
2. Before observation, there was a significant difference between day-time and night-time sessions, with the majority of impossible stride parameters coming from night-time sessions (see also Table 1 for details)
3. Once the Geezers joined Rob Young, two critical changes can be observed:

1. First, the impossible stride parameters disappeared completely (Figure 9 and Table 2).  During the period of Geezer observation, not a single session longer than 3 miles had a step length above 2m, and the longest step length calculated was 1.06m.  Prior to observation, 82 out of 195 sessions had step lengths longer than 2m.  Most of these came during night-time hours.
2. Second, the difference between day-time and night-time sessions disappears.  Rob Young continued to run at a slightly faster pace at night than during the day, but the relationship between pace and step length (Figure 7) is consistent with published literature and is similar between the day and night-time sessions
3. Of interest is that the greatest ‘outlier’ of all these performances are those that occurred before observation, during the nighttime hours.  Here, Young had infeasible step lengths (3.57m average), cadence (45 steps/min) and pace (10:02 min/mile) compared to all other categories (daytime before observation and both day- and nighttime during observation).

Finally, as confirmation of our analysis described above, we report a key finding of an analysis that was conducted by a group of investigators who began mining Young’s data, uploaded to Strava, to analyse his performances after the allegations of cheating emerged.  One of their key findings, described on a website established to report various lines of evidence (http://ryinvestigation.blogspot.co.uk/2016/07/strava-data-analysis.html), was the pair of histograms shown below.

They show the time spent (in hours) at various running paces during one-hour time bands.  The top panel shows the period prior to observation by the Geezers, and it is clear that a significantly larger portion of Young’s runs are spent running faster than 8 min/mile (orange and red shading) between 6pm and 8am than during the daylight hours.

In contrast, during observation, there is a marked change, with much less time spent running at faster paces during the night-time hours.

Figure 10: Histogram plots of time spent at various running paces as a function of time of day

(Source: ryinvestigation blogspot)

We interpret this analysis to strongly support our conclusion, based on the cadence data rather than running paces, that Young gained assistance using a vehicle, predominantly during the night-time hours, to achieve all his faster pace runs (see Figure 7B) prior to observation.  The presence of independent observers eliminated this practice, and profoundly changed the running paces he was capable of, in accordance with changes to the measured stride parameters achieved (Figures 4 – 7).

1. Bottom Line

We conclude that this data is strongly suggestive that before being observed by the Geezers, Rob Young was receiving assistance in or on a vehicle for all or part of his runs.  This was particularly the case during night-time hours.  Once the observation from the Geezers begins, with no opportunity to cover distance without running, performance (pace per mile) and the stride parameters (both cadence and step length) immediately return to typical values, and are more consistent with what was observed during daylight hours before observation.

This analysis also confirms that the watches cannot have been faulty, because time-of-observation would have no impact on the measured parameters if this were the case.  Instead, the data indicate normally functioning TomTom watches, which had to have travelled by vehicle for some or part of the logged sessions during the period prior to observation.

When asked about these anomalies and infeasible parameters, neither Rob Young nor Michael Speicher was able to offer any plausible explanation.

Our conclusion is expressed in terms stronger than the “balance of probabilities” threshold stated in the terms of reference. The data tell a very compelling story.

Ross

Footnotes

[1] The attempt actually began May 14. We believe that a duplicate run on May 13^th may have been the result of a time zone mis-setting on one of the watches. The hand-written log records the first run as May 13^th which was subsequently corrected to May 14^th.

[2] Some sessions fall into both categories.

[3] It is, of course, possible that the watches were either inside or outside (e.g., on top of) the vehicle, the available data does not differentiate.

[4] An online search of known problems with TomTom watch cadence data revealed a few examples of cadences that are too fast, but none in the other direction.

[5] The GPS track takes the watches on bike paths and across areas where a vehicle could not travel.

[6] See: http://ryinvestigation.blogspot.com/2016/06/was-bike-used-to-cheat.html and http://ryinvestigation.blogspot.com/2016/06/yet-another-bike-ride.html

[7] Some details on the “Geezers” and their accompaniment of Young can be found here: https://www.theguardian.com/lifeandstyle/2016/jun/22/robert-young-marathon-record-attempt-cheating-rumors

8 Note that the embargoed version of the report (29 September) contained an error in Table 2 – the average step length for day was indicated as 1.14m and for “Day” and 1.03m for “Night”. The values have been corrected and accurately reported in Table 2 above