January 4, 2022 update – I’ve thought about this a couple of times and I’m not convinced there is any thing presented below justifies adjusting draft order of players. I still stand behind the conclusions on the number of players at each position though.
I’ll start by saying that I’m not expert in IDP best ball, or best ball at all for that matter. I’ve participated in one best ball eliminator with IDP (s/o @seahawksdan8 on twitter). But with sleeper adding best ball scoring this year, I’ve seen more and more best ball leagues with IDP. This is a look at player variability at each position, and how that might change roster construction in a best ball draft versus a format where you set a roster each week.
I started by looking at the The IDP Guys 2020 Invitational results at each position, and calculated the average, standard deviation and coefficient of variance of the weekly scores for the top scorers at each of the 5 IDP positions they used (Defensive Tackle, Defensive End, Linebacker, Cornerback and Safety). I talked about the IDP Invitational previously here. It isn’t a best ball league, but I’m using it and its scoring as an example to show the weekly variance at each position. These are the scoring settings they used.

Worth noting that there is a scoring premium given to Cornerbacks. That said, this is a discussion of variance, so the actual scoring is somewhat irrelevant.
All other things equal, positions with a lower weekly variance should be drafted a little earlier and in smaller numbers as there’s more confidence in their averages, and positions with a higher weekly variance should get drafted later and in larger numbers. Of course, there are other factors, including the scoring and starting requirements in each league. Here’s what the variance looks like at each position.

The tables hows that Linebackers are the least volatile on a weekly basis, followed by Cornerbacks & Safeties, leaving Defensive Ends and Defensive Tackles as the most volatile. The conclusion of this is that Linebackers should be drafted a little bit earlier and in fewer numbers than you otherwise would because you can be more confident in their weekly scores. Defensive Ends and Tackles can be drafted a little bit later and in larger numbers than you otherwise would, because they are more variable, and you want a couple more options to take advantage the variance at that position. Cornerback and Safety fall in the middle of the other positions.
If you had asked me beforehand, I would have told you that Cornerback would be more variable on a weekly basis than Safety, because I would expect Cornerbacks to have fewer tackles, but a higher chance to get pass deflections or interceptions, but the 2020 scoring shows that this isn’t the case.
The IDP Invitational was hosted on My Fantasy League and used their default positions, so several players pass rushers get classified as LB based on their team’s scheme (TJ Watt, Khalil Mack). You would expect those players to have the same weekly variability as Defensive Ends, as they all share the same primary roles on the field.
I also looked at the coefficient of variance of the top 48 players are each position. I found that the Coefficient of Variance increased for the top 48 set by 5 to 10% for Linebackers, Defensive Ends and Defensive Tackles, but no increase for Cornerbacks and Safeties. It is also worth noting that while I’m showing the average
Find me on twitter @djkelltown. When I’m not talking about fantasy football, I’m probably talking about my NHL team (Winnipeg Jets) or CFL team (Winnipeg Blue Bombers)