A Data-Driven Approach To Precision Rifles, Optics & Gear
Home / Ammo & Handloading / Precision & Group Size – Statistics for Shooters Part 3
Precision & Group Size

Precision & Group Size – Statistics for Shooters Part 3

This article is going to dive into the most effective ways to quantify the precision of a rifle and ammo particularly when it comes to group sizes and dispersion (i.e. how the bullet holes on target are distributed or spread out). We’re going to build on Part 1 that laid the foundation, and Part 2 that looked at quantifying muzzle velocity variance – and in this final article of my Statistics for Shooters series we’re going to apply those concepts in two dimensions to analyze groups.

In Modern Advancements for Long-Range Shooting Vol. 2, researcher Bryan Litz provides a great introduction to this with a central question that all of us shooters are wondering:

“One common question shooters ask about measuring groups is: should I shoot 3-shot groups or 5? Is it necessary to shoot 10 shot groups?  There’s no easy answer to this question, but we can say that the more shots you fire, the more decisive and accurate your understanding of precision will be. If you’re just trying to see what your rifle’s 3-shot average group size is, then you just shoot a few 3-shot groups and you know. But if you’re trying to decide if there’s a precision difference between ammo loaded with primer A vs. primer B, then a couple of 3-shot groups probably won’t answer that question. So it very much depends on what question you’re trying to answer when you consider how many shots are necessary.”

“So it very much depends” is right! This article will dive down that rabbit hole, and equip you to answer the following questions for your specific application:

  • How many shots per group do I need?
  • Is extreme spread the best way to measure my groups?
  • Should I exclude fliers?
  • What is the most effective and accurate way to compare groups between two loads? (That doesn’t require a math degree!)
  • Are there any practical tips or shortcuts that can help with this?

My goal is to help you get the most out of the shots you fire at the range doing load development or even comparing factory ammo, and make more informed decisions that will help us get more rounds on target. As with all the other articles in my Statistics for Shooters series, I spent an absurd amount of time arduously crafting this article and creating visuals so it was approachable by shooters who aren’t math nerds because I firmly believe these concepts can help a TON of people in the long-range community.

Precision vs. Accuracy

Before we dive into the details, we need to clarify two central terms related to this topic. Many shooters use the terms “precision” and “accuracy” interchangeably, but there is an important difference – in both shooting and statistics.

Precision vs. Accuracy for Shooting

“Note that here we use the specific statistical meaning of these terms, even though in casual usage they are often sloppily interchanged. Accuracy refers to how well shot groups are centered on a target, and is essentially a problem of sighting-in a shooting system. If the shot group doesn’t center on the desired target the assumption is that the weapon merely needs to have its point of aim adjusted. It is implicitly assumed that the aim adjustments can be made finely enough so that any difference between the average POA and POI would be statistically insignificant. Precision describes the spread of individual shots about the center point of a shot group. A precise shooting system will, over many shots to and from the same points, produce a tight shot group with smaller distances between individual shots than a less precise system. However, in contrast to accuracy, precision can’t be “adjusted” into a shooting system by dialing some knob on the weapon.” – Ballistipedia.com

Both accuracy and precision are important to hitting targets, but when we’re shooting groups we’re testing precision. Often that’s also referred to as dispersion, which can be thought of as how the bullet holes on target are distributed or spread out. That is what this article will be focused on.

The Problem with Extreme Spread and Other “Range Statistics”

The most common way to try to quantify the precision or dispersion of a rifle/ammo system is to measure the extreme spread (ES). ES is typically expressed as the center-to-center distance between the two shots that are furthest apart. That is easy to measure by hand and it is a useful stat. However, like any descriptive statistic, we are simplifying multiple data points into a single number – so by definition, we are losing some level of detail, but it does make it much easier to do direct comparisons.

The Problem with Extreme Spread For Rifle Shooting

“Measuring the extreme spread of shot groups is quick and easy, but it’s not actually a very good measure of dispersion. What do I mean by a good measure? A good measure should give you useful information, which is information you can use to make good decisions. When you look at the extreme spread of a 5-shot group, that measurement is determined by only 2 out of the 5 shots. In other words, only 40% of the shots are considered in the measurement. Even worse, for a 10-shot group, a center-to-center measurement is only using information from 20% of the total shots. Since the extreme spread, center-to-center measurement, is determined by only a small portion of the total shots available, it’s just sort of an indicator of precision.” – Bryan Litz

Bryan certainly isn’t alone on this view. Ballistipedia groups ES into what it refers to as “Range Statistics,” which also includes measurements like the smallest enclosing circle and others. They say these types of measurements “are more commonly used because they are easier to calculate. But they are statistically far weaker because they virtually ignore inner data points.” They point out that ES and other range statistics will always increase in size as we increase the number of shots. “They are the least efficient statistics, but are also the most commonly used because they are so easy to measure in the field and so familiar to shooters.

That isn’t to say that ES is completely irrelevant. A stat that is easy to measure at the range, and that everyone is familiar with can be helpful! However, other statistics can help us make better decisions on how a rifle/ammo system will perform in the future, and maximize the benefit from every shot fired.

A Better Measurement: Mean Radius (a.k.a. Average Distance To Center)

“There are many alternative measures of precision,” explains Litz. “You could measure the location of each shot and calculate the vertical and horizontal standard deviation (SD), radial standard deviation (RSD), circular error probable (CEP), etc.  You could get really carried away with statistical methods of characterizing precision.” We could really nerd out and talk about Rayleigh distributions and computing the corrected sum of squares – but Bryan is right, it’s easy to get carried away! (Read about these other measurements.)

There is a much simpler approach that allows us to get most of the practical benefits of those more sophisticated methods. Here is the stat Bryan Litz recommends that can provide that goldilocks balance of statistical rigor and insight, without getting carried away (excerpt from Modern Advancements In Long Range Shooting, Vol. 2):

Taking a step back and considering options for measuring dispersion; we want something more descriptive than extreme spread, but don’t want to go crazy with statistics. It’s my opinion that the mean radius of a shot group is a well suited measurement for this task.  Mean radius, also known as average to center is self-explanatory; it’s the average distance from each shot to the center of the group. Here are the reasons I think mean radius is a good measure of precision:

  1. Mean radius uses information from every shot in a group, not just the two most distant points.  Because of this, mean radius can provide a higher confidence measure of precision than extreme spread.
  2. The more complicated measures involving standard deviation assume a normal, or Gaussian distribution; as in the bell curve.  It’s arguable weather or not shot groups follow a Gaussian distribution or not [REF 3], and we don’t need to stir up arguments with statisticians.
  3. There is a free software program called On Target [REF 8] which automatically calculates mean radius (On Target calls it average to center or ATC).  So you don’t have to do any of the tedious math.
  4. When compared with the more sophisticated methods of measuring precision, mean radius is effectively just as good at resolving relative precision.

There are some criticisms of the mean radius measure; in particular:

  1. Extreme spread can be measured simply with a ruler or calipers; no computer required.  Depending on the projects, people, and equipment involved, it may be better overall to use the simpler measure (extreme spread) to avoid confusion.
  2. When groups of shots are so tight (bug-holes), you can’t always locate each shot, so a mean radius measurement wouldn’t be possible.

Although mean radius is a little harder to measure than extreme spread, it provides us with a better measurement of actual precision.  I’m not claiming that mean radius is technically superior in general, only that it’s a good tool for the job, considering practical application.

Just a few more comments on statistics and then we’ll get into looking at dispersion for various shooting applications.

Circular Error Probable (CEP) is a common metric used to quantify dispersion.  CEP is defined as a circle in which 50% of shots will land inside.  CEP and mean radius are very similar, within 5% or 6%, but are technically not the same thing.

If you want to know how big your group is likely to be based on the mean radius, you can multiply by 2.1 to get the 95% radius (R95).  For example, suppose you shoot a 10-shot group at 1000 yards with a mean radius of 3 inches.  2.1 times 3 inches gives you the radius of the circle in which 95% of your shots can be expected to land, so that’s 6.3 inches.  Remember that’s a radius, so the total group size would be around 12.6 inches.

We discussed how your center-to-center group measurements can be expected to grow with more shots.  Mean radius will also grow with increasing number of shots, but not as much as center-to-center.  In other words, you can expect a 24% difference in group size when going from a 5-shot to a 10-shot group measuring center-to-center.  But the mean radius will be closer to the same value for 5, 10, or 20-shots.  Basically, this means you converge on the final answer in fewer shots when measuring mean radius.  Another way to look at it is, using mean radius, you can resolve smaller differences in precision with fewer shots.  So if you’re testing to see if ammo loaded with primer A is more or less precise than ammo loaded with primer B, mean radius or average-to-center is more likely to discover the true answer.

Great explanation, Bryan! But, let’s not just take his word for it. What do other industry experts say?

“The US Military uses the measurement of the mean radius of all the shots in each group as the definition of the angular accuracy of a given firearm, not this ‘extreme shot distance’ measurement,” explains Chris Long, a respected researcher, and writer on precision rifles. Ballistipedia corroborates that saying, “Armed forces also often explicitly uses the more statistically powerful Mean Radius and Circular Error Probable measures.”

Mean radius, also referred to as Average Distance To Center (ATC), is easy enough to understand. By that, I mean that it isn’t some abstract value that is hard to see how it relates to the physical group printed on the target. And it gives us more reliable insight that can lead to better decisions.

While many recreational shooters may not be familiar with mean radius, that isn’t because it is new. It has been used in serious research for years. Here is an excerpt from an old issue of American Rifleman on the subject:

“Mean radius is the mean [or average] distance of bullet impacts from the center of the test group. It is used in government ammunition acceptance because it takes account of every shot and comes close to maximizing the test information. While there is no exact relationship between this measure and the simpler and more convenient group diameter, the 10-shot group diameter averages slightly over 3 times the mean radius.” … “These examples illustrate the sensitiveness of the extreme spread to the number of shots in the group. Indeed, as the table indicates, the measures made to only the outside shots of the group, e.g. the extreme spread, are very sensitive to the number of shots, while the measures made to all the shots, e.g. the mean radius, are far less so. It may be added that the latter measures are also less variable in their representation of the group; they are more efficient. This explains why the target testing of U.S. military rifle ammunition is by mean radius.

How To Measure/Calculate Mean Radius

Okay, let’s say we just fired a group – how do we calculate the mean radius? While it’s possible to measure and calculate by hand, I wouldn’t suggest it. Virtually nobody does it that way. The easiest way is to snap a photo of the target and use software to analyze it. The software will generate X and Y coordinates for the center of each hole, and then do the measurements and math for us.

Desktop apps like OnTarget will do this (costs $12), and some phone apps allow you to snap a photo of a target and analyze the group, like TargetScan ($18). A few months ago, I collaborated (on a volunteer basis) with the maker of a phone app called Ballistic X ($8) to add more advanced statistics to its analysis, including Mean Radius and some of the other advanced statistics used to quantify dispersion. The photo below is from a prototype of the changes I helped test for them, but these new features haven’t been released in Ballistic X at the time this was written, but I’m told they will be very soon (Update 12/16/2020: The owner of Ballistic X told me, “We are hoping to have a Beta for testing before Christmas. Then a full released after!”) . (Ballistic X for Apple | Ballistic X for Android)

Ballistic X Mean Radius Average To Center Extreme Spread CEP Target Analysis

Let me say this as clearly as I can: If you are comparing two loads or two rifles and trying to decide which is superior in terms of precision, comparing the mean radius of the groups fired will lead to more reliable conclusions than comparing extreme spread. If you got lost on the rest of the “math talk,” that should be your key takeaway. You have to use an app to calculate it, but doing that will help you get more benefit from those shots you fired at the range.

Should We Exclude Flyers?

There is no lack of strong opinions on this topic, but I will try to take a practical, yet science-based, approach to thinking through this question. This turns out to be critical when trying to quantify, model, and predict dispersion.

In general, excluding an observation just because it is an outlier is not good science, so most serious researchers don’t believe it’s justifiable to filter out data points. (Good article on this topic.)

To exclude a shot from a group just because it appears to be a ‘flyer’ is bad measurement technique and would lead one to underestimate long-range dispersion.” – A User’s Guide to Dispersion Analysis by Bruce Winker

“In target shooting a ‘flier’ is a shot that flies wide of the target, or appears to be an outlier. Every shooter experiences fliers. Sometimes the cause of a flier is known or can be found. But in statistics we should be careful to discriminate between ‘outliers’ and ‘fliers,’” Ballistipedia explains. “Not every outlier is a flier.”

The answer to our question often comes down to the difference between an “outlier” and a “called flyer.” (Note: Both flyer and flier are accepted spellings. 😉)

The dispersion of a group is primarily driven by two factors:

  1. Mechanical precision of the weapon system (i.e. barrel, optics, ammunition, and other parts, combined with how all those things attach and interact under recoil)
  2. Aiming error of the shooter (i.e. the trigger was pulled when the sights weren’t perfectly centered on target or perhaps your body position or rest wasn’t consistent shot-to-shot causing the weapon to move differently under recoil in a way that skews the bullet’s trajectory)

While our goal is to quantify #1 in complete isolation – with shoulder-fired rifles aimed by a human, #2 is unavoidable to some degree. Often we simply try to minimize it as much as possible. However, if as the shooter breaks a shot they notice their sights were not perfectly centered on the target and they “call” that (i.e. admits it, often verbally) before they looked to see where that bullet fell in their group – that is a called flyer, and there is a valid reason to exclude that shot from the group.

Here’s the catch: If you look at the target and notice that the last shot didn’t fall within the rest of the group, you can’t then say “You know, I must have pulled that one. I’ll exclude it.” Often this comes down to ego vs. good science. Be honest with yourself! If you want to make the best decisions then you’d only exclude called flyers from your groups – and leave any other outliers in the results.

Have you ever thought about why we are so quick to want to exclude an outlier in our groups? Molon suggests that is also related to groups with just a few shots in them: “When we’re firing three or five-shot groups with a flier, it is only natural to assume that it was caused by a flinch or ‘pulling’ the shot. Therefore, since the flier was our own fault, the tendency is to eliminate it from any reporting of group size.” But, if instead of stopping after 3 or 5-shots we continued to fire bullets until we had a 10-shot group, it’s likely the “obvious outlier” might become an obvious part of the group that eventually formed. MacDonald presents this helpful example:

Rifle Flier or Outlier In Group Overlay From Adam

Let’s focus on Group 2 above. If we had just fired Group 2 on a target, it would be natural to assume the highlighted shot that fell out of the group must have been our fault. However, when we look at the overlay of all the groups, we get a clearer picture of the rifle’s true dispersion and we see that shots from other groups also landed in a similar location on the right. While it would be completely natural to blame ourselves for a outlying shot like that and exclude it as a “flier” – it would also be wrong and cause us to underestimate the rifle’s true dispersion.

With practice and practical steps like double-checking your scope’s parallax, using a firm rear bag and solid front support, ensuring your body position and trigger pull is consistent shot-to-shot, etc., most shooters can get to the point where their influence on the rifle can be minimized so that it isn’t one of the largest variables when it comes to dispersion. There will always be some impact from the shooter for any shoulder-fired weapon where a human is pulling the trigger, but if you’re reading this – I’d suspect you are likely one of the highly detailed, educated shooters who should be slower to blame yourself for fliers.

Rail Gun Rifle Setup Unlimited Class

Finally, Ballestipedia.com explains the more technical reason that we shouldn’t exclude outliers: “We have accepted unbounded distributions as models of the shooting process, and so we have to also accept that outliers are part of both the model and the real world and that our model can correctly account for them if they are part of the modeled process. Granted, if I had a rail gun on an indoor range and had triple-checked every component of every round I sent downrange I may not accept an unbounded normal distribution as a model of my shot dispersion. But once we allow for outdoor conditions and normal ammunition, not to mention a shooter operating the gun, then in the normal course of events we will get outliers, and they are representative of the underlying normally-distributed process.” (Note: If you aren’t sure what “normally-distributed” means, read Part 1.)

Said simply, if you exclude anything but shots that were truly “called fliers” resulting from clear shooter error – you will underestimate dispersion. It might make you feel better about your group size, but it’s not going to help you hit targets at long range.

What Is A Good Sample Size?

What is a good sample size? The short answer comes down to how minor of an improvement you are trying to detect and how much confidence you want to have that the results will be a good predictor of future performance – but I’ll try to give you a more direct answer than that, without getting too technical. 😉

Recently I listened to a few of the sessions from the 2020 Modern Day Rifleman Summit, including two lectures by Hornady’s Ballistician, Jayden Quinlan. I enjoyed both of them, but here is an especially relevant excerpt I pulled from the 2nd lecture:

Jayden Quinlan, Hornady Ballistician

Jayden Quinlan, ballistician at Hornady, says that the very first thing he does in his process of setting up a new rifle is “getting a really good handle on what the system is capable of. I’m glad we talked about this part because I believe it often gets overlooked. One part of that is cost because there is a cost associated every time we pull the trigger, both in barrel life and cost of components. The second part is ego: We don’t want to know how bad our system really is. I have a 1/2 MOA rifle every time I do my part – that cracks me up every time I hear it. So here is my typical response to someone who says “I have a 1/4 MOA or 1/2 MOA rifle.” My first question is, “Over how many shots? 2 or 5 or 10 or 30?” Because that’s important to know. If we’re talking about some kind of dispersion we need to know the sample to quantify if it is good or not. Because a 1/4 MOA group that is 2 shots doesn’t really tell me a whole lot, but a 1/4 MOA group that is 10 shots tells me a whole lot. So how many shots? That’s the first question, and generally, it is a super small sample size and it was done once, and the next group was 1 inch.

The argument is if you truly have a 1/2 MOA rifle at 1000 yards, and it’s a shoulder-fired, hunting-type, lightweight field rifle and it shoots 1/2 MOA at 1000. If it does, then take it to the heavy Benchrest Nationals because you’re probably going to win. You know?! Look at the average group size of those rifles. I’m sorry, I don’t believe you.

Putting ego aside, what is it really capable of? So I shoot no less than 10 shot groups when I’m testing my dispersion. Dispersion is pretty much linear, and it goes in non-linear when you add in velocity and drag and stuff like that. So aiming error and dispersion are kind of linear. So I shoot them at 100 yards, and no less than 10 shots. And I don’t have a big giant ego fit if it is 1 MOA, and I don’t have a big giant ego if it is 3/4 MOA.

We’ve done enough large sample size testing in 50 shot groups to understand that you’re chasing your tail sometimes when you’re trying to squeeze that last little bit out of it, and when you think you have it your sample size isn’t large enough for that to actually be valid. Because when I go to a match, how many shots am I shooting? 200 rounds? 250? So how representative is my sampling of 10 to my capability for 200 or 250? Not as good as you think.

So I shoot no less than 10 for my baseline. Let’s say it is 1 MOA. Okay, then 1 MOA is my baseline. Then I move on to testing my muzzle velocity …”

Well said, Jayden! Lots of wisdom there.

I’ve yet to see any respected expert who is familiar with statistics claim that a 3-shot group is adequate to quantify a rifle’s precision. In fact, after rigorous statistical analysis, Ballistipedia.com says, “If you’re shooting 3-shot groups … then you are wasting bullets.” While many keyboard snipers love to cite how tiny their 3-shot groups are, they often fail to mention that their groups impacted at different spots on the target. The exact center of each bughole shifts around a bit relative to the point of aim. Again, let’s refer to MacDonald’s illustration below and we can see that the 5-shot groups on the left are each smaller than the composite group in the overlay, but we can also notice that the center of each of the 5-shot groups’ shifts around slightly.

This is a big part of why 10-shot groups are a much better indicator of the true dispersion of a rifle/ammunition combination – and a rifle’s true zero. (Great article with more on this: Statistics, Shooting, and the Myth of the 3-Shot Group)

Let’s step away from stats and science for a minute, and simply reflect on our own experience: We’ve all been at the range and had a couple of shots go through the same hole. Why are we so hesitant to send another bullet at that same target? If we’re honest, we’ve all felt that! It’s because our intuition tells us that one more shot could mess up that sweet bughole we have going – and our intuition is right! It’s exponentially easier to get a tiny group with just a couple of shots – but that largely has to do with luck. Fire enough 3-shot groups with virtually any rifle and you’ll eventually get a lucky one that measures under 0.25 MOA – and that looks cool on Instagram or a forum. But, the more shots we fire the more accurate read we get on a weapon system’s true dispersion.

“Some shooters may have two or three 3-shot groups to prove the load is really accurate. It really takes more shooting than that to make a judgment on a load’s accuracy potential. Three shots forming a tight cluster is nice to look at, but it is little more than an accident. Shooting three-shot groups to see how everything is working is essentially a waste of time and components. … Ten shots are a more reliable indicator when it comes to predicting what a load is likely to do in the future.” – Rick Jamison in an article published in Precision Reloading

When it comes to sampling size, most experts recommend 5, 7, or 10 shot groups for personal/hobby use. Those tasked with the most rigorous levels of research likely want much larger sample sizes (potentially 30, 50, or more). For example, the US Army Marksmanship Unit at Fort Benning, Georgia uses a minimum of 3 consecutive 10-shot groups fired with the rifle in a machine rest when testing service rifles.

At the 1992 Barcelona Olympics, USA Shooting Team members Launi Meili and Robert Foth won the gold and silver medals in the three-position rifle events. The Olympians used a type of Federal Gold Medal ammunition that was new at the time to aid them in their victories. That was the first time in over 30 years that an American won an Olympic medal in one of the small-bore shooting events while using American-made ammunition. When Federal’s Director of Product Engineering, Dave Longren, was asked about the accuracy/precision development and multifaceted testing of the Federal ammunition that helped the US Olympians win gold and silver medals, he had this to say: “The standard test string was three 10-shot groups, with the most attention paid to the 30-shot composite. When you’re working at this level, the traditional five 5-shot group test simply doesn’t give you statistically valid results.”

In general, statisticians consider any study with less than 30 data points to be a “small sample size.” So, it’s interesting to note the US Army Marksmanship Unit and Olympic-level competitors rely on three consecutive 10-shot groups – “with the most attention paid to the 30-shot composite.”

There was a good article posted by Molon on AR-15.com that captured an important point:

Institutions and organizations that buy enormous amounts of ammunition and weapons are far more interested in facts than sales hype and propaganda and most of them demand that accuracy/precision testing be conducted using 10-shot groups. This includes the Federal Bureau of Investigation, Crane Naval Surface Warfare Center, the US Army Marksmanship Unit, and the US military’s acceptance testing of both 5.56mm ammunition and weapons. On the other hand, businesses that use 3-shot groups for making their accuracy claims are usually trying to sell something.

When I first read that, I immediately recognized that statement had the ring of truth to it, but I also thought another scenario where it’s popular to cite 3-shot group sizes is when someone is trying to impress other people online. 😉 Molon went on to hit that head-on: “Ten shots are a more reliable indicator when it comes to predicting what a load is likely to do in the future. The problem with 10-shot groups is that when you report them, everyone thinks you aren’t shooting very well or that the ammunition is not good because the group sizes are so much larger than three or five-shot groups.” A large part of why so many continue with 3-shot groups likely gets back to Jayden’s comment about ego and the fact that “we don’t want to know how bad our system really is.”

The NRA’s 5×5 Method

The NRA’s standard for testing precision is to shoot five consecutive 5-shot groups and report the average extreme spread, which some serious statisticians say “is actually rather efficient.” As I mentioned in the first article in this series, when we’re firing groups to determine the capability of the system, often we’re doing that because we want to be able to predict how it will perform in the future. When it comes to predicting the future, we can’t do that with absolute certainty, so instead, it’s more appropriate to talk about ranges and probabilities with confidence intervals. So let’s do that in this case, and it’ll help us understand how reliable of a predictor the average extreme spread of 5 groups with 5 shots each really is. Let’s say that we measured the average ES of five consecutive 5-shot groups to be exactly 1.0 MOA. Ballistipedia.com tells us, the 90% confidence interval would be 0.81 MOA – 1.20 MOA. “This means, for example: if we measure an average 5×5 extreme spread of 1.0 MOA then, nine times out of ten, we would expect that same shooter, gun, and ammunition to produce five 5-shot groups measuring between 0.8 MOA and 1.2 MOA.”

However, while the NRA’s 25-shot, 5×5 method is good, “Following the best estimation methodology you can measure precision as effectively as the NRA protocol does but use only 3/4 as many bullets. Following the NRA protocol you’re spending 32% more ammo than necessary to get the same precision estimates.” (Ballistipedia). Granted, it might be easier to measure the average ES of 5 groups than to perform all the steps necessary for “the best estimation methodology,” but mean radius doesn’t involve a lot more complexity and can provide more reliable insight from less than 25 shots.

The Hard (But Important) Truth

Here are a few conclusions that I realize may challenge “conventional wisdom,” but they’re incredibly important:

“Targets are a bit more complex than muzzle velocities because a group is two-dimensional, not one-dimensional,” Denton Bramwell explains. “However, similar to the muzzle velocity case, group size is variation, and is a slippery devil to estimate using small samples.  If you are using five-shot groups to evaluate group size, you should expect that groups will naturally vary plus or minus about 50% with no change whatever in the load, rifle, or shooter’s performance. … If the rifle is truly a 1” machine, then the shooter should expect that 95% of his five-shot groups will fall between ½” and 1 ½”, with absolutely no change in real performance. You cannot reliably estimate the long-term characteristic of the rifle with just one or two five-shot groups. Groups within plus or minus 50% of the true long term average do not indicate any real change.

Remember what Jayden Quinlan, the Hornady ballistician, said: “We’ve done enough large sample size testing in 50 shot groups to understand that you’re chasing your tail sometimes when you’re trying to squeeze that last little bit out of it, and when you think you have it your sample size isn’t large enough for that to actually be valid.”

Engleman agrees, and takes the conclusion a step further: “I believe that you will find that by making statistically sound judgments, many loads produce statistically similar results and loads, in general, are not as finicky as current conventional wisdom would lead us to believe. … In reality, your shooting performance will probably improve since you can spend more time practicing with your set load and less time playing around with load combinations that have little or no statistical significance in terms of shooting performance.

So What Now?

Obviously, we can’t be expected to shoot three consecutive 10-shot groups of each combination of powder load, bullet jump, primers, and other variables we’re trying to dial in for our rifle. I like the way Adam MacDonald said it in his article Thinking Statistically: “The rifle talks to us by generating samples at $1 a pop. If we want to know how it truly works, we need to play its game. With enough samples, we can try to measure the population, but it can be expensive.”

So what is a guy to do? What is the right balance between statistical accuracy and not burning out our barrel before we work up a good load? Here are a few tips and tricks that can help us find our way to a good load in fewer shots:

Tip #1: Start With What The Winners Are Using

Engleman suggests a good shortcut: “The more data you include in a calculation, the more statistically accurate it will be. So with respect to component choices, start with the same components the winners are shooting. There is far more data in all the rounds they fire collectively with good results than you can possibly collect on your chronograph. If it works for them, it will work for you.”

I realize that may sound like a lazy answer, but it is a more valid solution than the small sample size load development most shooters do. If nothing else, it is a logical and effective shortcut to help us narrow the search for a load that performs well in your rifle – with fewer shots.

Since 2012, I’ve been publishing extensive information on the components that the top-ranked long-range shooters in the country are using, and you can access those articles here: What The Pros Use, and look for articles about load data, bullets, powders, primers, brass, etc. for a variety of popular precision rifle calibers and cartridges.

Tip #2: Eliminate Bad Loads Early – Even Based On Small Sample Sizes

Here is another shortcut: Identify what doesn’t work quickly. “If you fire 5 shots and the group is big, you can pretty confidently rule out this load,” explains MacDonald. “The ES, SD, and your intuition would all agree – better steer clear of this one. Why? It’s only 5 shots! The reasoning is that the relation between a sample and its population is asymmetric. It’s more likely that a bad load will produce a small group than it is for a good load to produce a large group. We exploit this and it’s why iteration works.”

MacDonald goes on to explain one of the most important keys: “When you think you’ve found something good, fire a couple more of the same load and see if your fortune repeats itself. Then proceed with verification. Once you happen across a load that appears promising, don’t stop there.

Tip #3: Include More Data For Free By Using Mean Radius

In statistics, the more data points you have, the more confident you can be in your results. So let’s think back on a quote from Bryan Litz mentioned at the top of this article: “When you look at the extreme spread of a 5-shot group, that measurement is determined by only 2 out of the 5 shots. In other words, only 40% of the shots are considered in the measurement. Even worse, for a 10-shot group, a center-to-center measurement is only using information from 20% of the total shots. Since the extreme spread, center-to-center measurement, is determined by only a small portion of the total shots available, it’s just sort of an indicator of precision.”

Extreme spread ignores a large portion of the shots we fired. We pay $1 or more for every round fired, so let’s make sure we get the full benefit from every one of them! Mean radius is the average distance to the center of the group for every bullet hole. That means if you fired 10 shots, the result is based on 10 measurements (i.e. the distance of each shot to the center of the group) – instead of just the measurement between the furthest two shots for ES. That is a big deal because it helps us more confidently characterize and quantify the precision of the weapon system.

Tip #4: Avoid Bugholes When Using Mean Radius

So we want groups with larger sample sizes, but that presents a problem when it comes to mean radius. Litz explains, “When groups of shots are so tight (bug-holes), you can’t always locate each shot, so a mean radius measurement wouldn’t be possible.” The 10-shot group below illustrates the problem. We can’t identify where the center of all 10 shots fell, so we actually can’t measure the distance from each shot to the center of the group. In the scenario below, we can only measure the ES of that paper target, unless we were using some type of electronic target system.

Precision Rifle 10 Shot Group

In this article, I referenced a lot of experts and serious researchers who recommended that 10-shot groups were more indicative of the rifle’s capability, but most of the time they were referring to ES group size measurements, not necessarily mean radius measurements. Because the mean radius is a more efficient and reliable statistic, it can more accurately quantify the rifle’s precision in fewer shots. However, that doesn’t mean we can fire a single 5-shot group, measure the mean radius, and arrive at the truth of our rifle’s performance.

To avoid the “bughole” problem Litz mentioned, I typically limit my groups to 5-shots per target so that I can more accurately identify each hole and calculate the mean radius, but I will fire multiple 5-shot groups. This approach is similar to the NRA 5×5 method, except I average the mean radius over all the groups, rather than ES. Sometimes I even overlay the targets on top of one another, so I can look at the composite picture as if I fired all of the groups at the same target. (Note: A killer feature of a target analysis app would be to allow a shooter to pick multiple target photos and have them automatically overlaid on top of each other. Not only could you do stats on the composite group, but you could also see any slight shifts in the center of the groups. Update 12/16: Some of my readers mentioned that OnTarget TDS has features similar to this.)

Tip #5: Resist Chasing That Last Little Bit Of Precision

I’ve been at the range doing load development before and thought I discovered something special. After trying countless permutations of components and specs, I’d finally found a load that was uber-optimized for my rifle. Thinking back on those moments when I was trying to fine-tune a load and squeeze out that last bit of precision from my rifle, I now realize I was likely just chasing my tail – just as Jayden said. I thought I found something, but I was making decisions on truly insufficient data. I didn’t have a large enough sample size to support my conclusions. It’s very likely that if I would have continued shooting more groups, my uber-optimized load would have ultimately performed very similar to other loads where the powder or components varied slightly. I was convinced that I saw a pattern in my group sizes, but in reality, it could have as easily been the natural variation that should be expected within small sample sizes – and it likely was.

As humans, we tend to see patterns everywhere. That’s important when making decisions and judgments and acquiring knowledge; we tend to be uneasy with chaos and chance (T. Gilovich, How We Know What Isn’t So, 1991). Unfortunately, that tendency can often lead to us finding meaningful patterns in meaningless noise. (Dr. Michael Shermer in Patternicity, Scientific American, Dec. 2008).

Statistics is a tool that can help us differentiate between true patterns and meaningless noise.

Our innate desire to see patterns in chaos can lead to some funny things when it comes to analyzing groups and load development. I’ve heard intelligent, accomplished shooters explain that if a 3-shot group forms an upside-down triangle then you know you found a very special load. Unfortunately, statistics seem to tell us that might be closer to palm reading than real science. Things like the ladder load development method, with only one shot at each powder charge, might be similar. I realize that may be tough for some veteran shooters to accept, but it’s the hard truth.

I’ll restate Engleman’s conclusion one more time because it’s such a key point:

I believe that you will find that by making statistically sound judgments, many loads produce statistically similar results and loads, in general, are not as finicky as current conventional wisdom would lead us to believe. … In reality, your shooting performance will probably improve since you can spend more time practicing with your set load and less time playing around with load combinations that have little or no statistical significance in terms of shooting performance.

Having said all that, it is likely wise to ignore any performance difference that is 10% or less – unless you are willing to fire 50-100+ shots to verify if there is a legitimate difference or that was simply the natural variation that we should expect to occur with so many random/independent factors at play.

One of the coolest benefits of PrecisionRifleBlog.com becoming so popular is that over the past several years I’ve had the opportunity to meet some of the most respected long-range shooters in the world and have some really interesting conversations with them. I’ve been struck by how little time many of them spend doing load development. That isn’t to say that none of them do extensive load development, but I bet most of them spend exponentially less time fine-tuning a load than you might think. Guys like Scott Satterlee and others have shared their load development methods (like on this podcast), and it doesn’t involve spending multiple days at the range tinkering.

That is why today, I no longer spend days at the range doing load development. I’ve followed the lead of several pros, and when I find a load that gives me good performance, I load up a bunch of it and go practice. The only time I might spend more than a day tinkering with a load is when I’m shooting ELR and my bullet’s time of flight will be extended to 3 seconds or more. In that niche scenario, super-consistent velocities can be the difference in a first-round hit or miss, so it may be worth the extra effort to fine-tune a load – but then again, if I choose to go down that path I know I’ll need to fire a lot of rounds to verify any slight improvements I find are real and not noise from natural variation.

Honestly, a lot of shooters enjoy tinkering with load development and even find it therapeutic. If that is fun for you – continue on. I often say, “Don’t fix happy!” However, it’s important to be aware that if you aren’t making decisions on significant sample sizes, then you are likely seeing patterns in noise. I hesitate to use this analogy because it’s crude – but I can’t think of a better one that captures the essence of this problem: “It’s like masturbation. It might make you feel good, but it sure isn’t getting you anywhere.” 😉

“It’s not just about firing more shots. It’s about understanding what to expect, and planning your tests so that you can walk away with confidence. Even 20 shot groups can be useless depending on the scenario, and the best of us make this mistake all the time.” – Adam McDonald

How Much Does Group Size Matter?

A couple of years ago, I took a very objective look at how much performance improvement shrinking group sizes would have in terms of hit probability at long range. I knew as a long-range shooter, I (like most of us) can obsess over every little detail. We think everything is important! After all, we’re trying to hit relatively small targets that are so far you may not even be able to see them with the naked eye. While you can get away with a lot of minor mistakes and still ring steel at short and medium ranges, as you extend the range small mistakes or tiny inconsistencies are magnified. So, most things are important … but to differing degrees. So I wanted to have some hard numbers to help me understand when I reached that point of diminishing returns in terms of group size and when squeezing out that last bit of precision might not have a measurable impact in terms of hits out in the field. I will say that I was more than a little surprised by the results, and I’d encourage you to go read that article: How Much Does Group Size Matter?

Summary & Key Points

Whew! We made it. We covered a lot of ground! Here’s a quick recap of the key takeaways from this article:

  • Accuracy and precision are not the same things. Accuracy refers to how well a group of shots is centered on a target. Precision describes the spread of individual shots about the center point of a group of shots. Unlike accuracy, precision can’t be adjusted by dialing a knob on the weapon.
  • Extreme Spread (ES) is not a very good measure of dispersion. Range statistics like ES are statistically far weaker because they virtually ignore inner data points. They are the least efficient statistics but are also the most commonly used because they are so easy to measure in the field and so familiar to shooters.
  • Mean radius, also known as average to center, is the average distance from each shot to the center of the group.
  • Mean radius uses information from every shot in a group, not just the two most extreme points. Because of this, mean radius can provide a higher confidence measure of precision than ES. However, mean radius is a little harder to measure than ES.
  • Mean radius allows us to resolve smaller differences in precision with fewer shots. If you are comparing two loads or two rifles and trying to decide which is superior in terms of precision, comparing the mean radius of the groups fired will lead to more reliable conclusions than comparing ES.
  • Don’t exclude outliers in your groups unless one was undeniably a result of human error.
  • 10-shot groups are a more reliable indicator when it comes to predicting what a load is likely to do in the future.
  • By making statistically sound judgments, you may find that many loads produce statistically similar results and loads, in general, are not as finicky as conventional wisdom would lead us to believe.
  • As humans, we naturally tend to see patterns everywhere, but that can often lead to us finding meaningful patterns in meaningless noise. Statistics is a tool that can help us differentiate between true patterns and meaningless noise.
  • It’s not just about firing more shots. Plan your tests and analyze your targets in a way that you’ll be able to walk away with confidence in your decisions.

I’ll leave you with one last quote from Statistics, Shooting & The Myth of the 3 Shot Group:

Final conclusion: What counts is hitting the target. Realistic target engagement > punching paper at 100 yards 10 shots at a time > punching paper at 100 yards 3 shots at a time > sitting in my office writing about statistics.”

Isn’t that the truth?! Let’s get out and shoot! 😉

Other Articles In The “Statistics For Shooters” Series

Here is a recap of all the articles in this “Statistics for Shooters” Series:

  1. How To Predict The Future: Fundamentals of statistics for shooters
  2. Quantifying Muzzle Velocity Consistency: Gaining insight to minimize our shot-to-shot variation in velocity
  3. Quantifying Group Dispersion: Making better decisions when it comes to precision and how small our groups are (this article)
  4. Executive Summary: This article recaps the key points from all 3 articles in a brief bullet-point list

You can also view my full list of works cited if you’re interested in diving deeper into this topic.

About Cal

Cal Zant is the shooter/author behind PrecisionRifleBlog.com. Cal is a life-long learner, and loves to help others get into this sport he's so passionate about. Cal has an engineering background, unique data-driven approach, and the ability to present technical information in an unbiased and straight-forward fashion. For more info, check out PrecisionRifleBlog.com/About.

Check Also

6.5 Creedmoor Ammo Muzzle Velocity

6.5 Creedmoor Ammo Test Part 4: Live-Fire Muzzle Velocity Details

This is the second article in this series covering the muzzle velocity data I collected. ...

36 comments

  1. When you are doing these 10 shot groups how long do you wait between shots?

    • Good question, Eric. I typically wait until the barrel is cool to touch. Occasionally I’ll even use a repeat timer app on my phone to create a consistent gap even between shots of maybe 1 to 2 minutes, instead of firing straight through it. But between strings, I usually just wait until the barrel cools back down close to ambient temperature.

      Thanks,
      Cal

      • Cal, I’m one of those chasers of the perfect group (have never attained it). Your articles have convinced me I need to find an acceptable group size for each rifle based on its purpose (hunting, PRS, long range, etc.) and go with it. I have on a few occasions reached the point of no return in load development by shooting so many rounds searching for the “perfect” load that the barrel was no longer capable of the original accuracy.
        One quick question. I have read your articles concerning the ELR matches and have watched a number of them on YouTube. Do any of these matches use the new ELR camera systems in order to allow shooters the benefit of spotting rounds that don’t contact the target. In a number of the videos I’ve watched, the shooter, wind caller and spotter have not been able to spot impacts at extreme range. It would seem beneficial to all the shooters to have access to the ELR cameras in order to spot and adjust misses. I don’t see where this would be an unfair advantage if all shooters had access to the camera system. What are your thoughts on this?
        Keep up the great articles.

      • Ha! I’m with you, Wade. I appreciate the honesty. There are a lot of us who have gone down that rabbit trail!

        For the ELR question, no I’m not aware of any competitions that allow the shooter or spotter to watch a down-range camera feed while they are shooting. I guess they’d say that takes some of the sport out of it. I honestly don’t have a strong opinion one way or the other. I’d probably be a proponent of allowing guys to see the camera feed. Ultimately, we’re trying to do something really difficult with or without the down-range cameras. While there are some of us that can hit without that (at least some of the time), that might open this up to more people enjoying ELR. Nobody enjoys sending expensive rounds down range blindly, and I think more ringing steel is usually good for whatever sport you’re in. People have more fun, and that in turn grows the sport.

        I do agree that it can be REALLY tough at times to spot shots at Extreme Long Range. Honestly, it sometimes becomes as much about spotting shots as it is shooting – literally! That’s why you see some guys bring out some truly awesome optical devices to try to spot impacts, so they can make corrections and get impacts on subsequent shots. Honestly, that is one of the biggest drivers behind a lot of guys going from a 375 caliber to a 416 caliber. Many people think a 416 Barrett must have a lot better ballistics than something like a 375 CheyTac, but a huge part of why some of the top competitors went over to the 416 was that it’s much easier to spot those impacts from 2 miles away. The 416 has a lot more recoil, which none of us love, but it also carries a lot more energy downrange so instead of being a tiny little speck of dust flying up, it is more like a grenade going off. That might be a little dramatic, but it is much, much easier to spot. 😉

        Thanks for sharing your thoughts!
        Cal

  2. Hey Cal, this series keeps getting more interesting all the time! A thought., why not have multiple aim points or small dots on a single target to establish a single radial delta for each shot and no bug hole issues? cheers Paul

    • Hey, Paul. Glad you enjoyed this series. I wasn’t sure how people would respond to it but thought it would sure be helpful if I could present it in a way that didn’t lose people.

      I have tried something similar to what you’re suggesting, with a single bullet at each target. It was similar to this photo Aaron Hipp posted on Instagram where he was doing a bullet jump test. (Read more about that here.)
      Aaron Hipp Bullet Jump Test

      That works well, except you have to overlay those in software to be able to determine the “center of the group”. Unfortunately, that is still a pretty manual process, until someone comes out with an app that can merge multiple photos or aim points automatically. I bet that happens at some point. It would be a killer feature, and honestly, wouldn’t be that difficult to code.

      Doing 5 or sometimes even 7 shots per group, I can still keep track of individual bullet holes and you are able to establish the “center of group” fairly easily with a sample size that big. If you think about each bullet hole as X and Y coordinates, the “center of the group” is simply the average of all the X coordinates and the average of all the Y coordinates. As I mentioned in the last post, it is relatively easy to get fairly close to an average for a population, even with 5-10 samples. But, you just wouldn’t be able to determine that “center of group” very well with just 1-3 bullet holes.

      Ultimately, I bet there is an app that comes out as a result of this series of articles. I know there are several app developers that are reading this, so I hope I gave them some ideas for things they could automate to open new doors for us as shooters. There seem to be some obvious ones related to target analysis to me that I bet quite a few of us would be willing to pay for. Based on my experience from the past, I’d bet within 6 months you see something pop on the market to fill some of the needs I mentioned in the articles. (If you’re an app developer reading this, you should get started! It seems like the first to market may be the one to capitalize on the opportunity!)

      Thanks for the comments, Paul.

      Thanks,
      Cal

      • Hi Cal,

        I believe On Target TDS will automatically overlay a hole from individual bulls just as you describe. I just got this software (from one of your references in this series). I tried the auto overlay by simulating a bullet hole by punching the target bulls with a pen. It picked up most of them, the holes were pretty small compared to an actual LR bullet hole. It should pickup the larger holes more consistently.

        Best,

        Lee

      • That’s awesome. Honestly, I have the paid version of On-Target, but haven’t ever used the TDS feature. I can’t wait to try it out. It seems like a pretty ideal software package … except it’s only on the desktop. I wish there was something like that on my phone. I seem to always have that on me at the range, but if it only works on your desktop it’s a little more hassle. But, it’s good to know that works. At least there is some option out there!

        Thanks,
        Cal

      • Cal,

        I should have mentioned in my previous that the On Target TDS software will automatically overlay holes when using a target it recognizes. You can view and/or print the targets from within the program.

        Lee

  3. Great article Cal! Most important take away for me is “Start with what the winners are using.” Always loved the “What the Pros are Using” articles as they give me an insight to what is working for the top shooters and has always helped me focus my attention on what works. After moving to 6 Dasher, having the data about what the top shooters are seeing with their rifles (https://precisionrifleblog.com/2019/09/02/6mm-dasher-load-data/) really saved a lot of time and money. I know what to expect from the 6 Dasher load and was able to just go straight to the ‘best’ components without all the extra trial and error. I totally agree with utilizing the data from the thousands of rounds fired by them in addition to my own. I now know the window of performance to expect and can avoid chasing performance that may not exist. Again, great article on the Statistics for Shooters. It really provided insight on what’s important, what’s useless, and what you’re getting with each shot.

    • That’s awesome, Steve. Glad to know that was helpful. I knew collecting the exact load details from those guys would interest a lot of people. I wasn’t thinking about this at the time, but my intuition did say that it could be a shortcut to finding a load for your rifle. It might not be identical to those other guys, because barrels and rifles differ – but it at least gets you in the ballpark and helps you know what components seem to work well together. I know I always start with that data!

      I appreciate you taking the time to mention that being helpful, and this series, too.

      Thanks,
      Cal

  4. Cal Zant, congratulations on another great article.
    What equipment you used and the caliber
    Greetings from Brazil

    • Thanks, Humberto.

      I’m not sure I understand your question about the equipment and caliber. Were you asking what I used in one of the groups in the examples?

      Thanks,
      Cal

  5. The paid version of OnTarget allows you to overlay multiple targets on top of each other. They call it TDS virtual groups. I’ve found that to be very useful.

  6. Fantastic article, answered a whole lot of questions I was pondering, another outstanding job well done!
    One simple minded idea to consider would be to use a common ringed target that has an x, 10, 9, 8, etc and use the score generated to evaluate that load compared to others. That would use the data from a tight cluster as well as the one or two outliers.

    • Thanks, Marc. That could work. The only downside compared to mean radius is that approach would be testing accuracy and not just precision. But, if you have a good zero for the load that could work.

      Thanks,
      Cal

  7. Hi Cal. I’ve always had an interest in statistical analysis and found your articles to be superbly laid out and easy to follow. I can only imagine the amount of time and concentration it took to lay the information out so well, so kudos!

    Quick question that may be much harder to answer: With what you know now about statistical analysis, would you call any of the conclusions/insights you’ve made in past articles statistically inconclusive?

    I know you are careful about how you assert findings, and I appreciate that they are qualified, but I’m particularly curious about how the recent bullet jump research holds up.

    Thanks – JW

    • Thanks, JW. I appreciate you sharing how much these articles helped. Your “quick question” is a tough one! I’m certain there are things I’ve published before or conclusions I’ve drawn from a sample size that was too small to fully support the conclusion. Often times though, I am trying to simply present the results and not necessarily make claims about some future performance. I try hard to present the data I collect, along with the full details about sample size and even how the tests were performed in detail, because I don’t want to be a source of bad information. I’ve said before that I would rather my website get shut down completely than become a source of bad information – there is already too much of that out there!

      Now the bullet jump data I presented was actually reviewed by a professional statistician that did explicitly say that “statistically significant conclusions are appropriate” for the data that Mark had collected. Honestly, if he wouldn’t have had a statistician review that data, I likely wouldn’t have been comfortable enough to publish those results. I was fully aware that those results flew in the face of “conventional wisdom”, and I really don’t want to mislead anyone or perpetuate bad information. Here is an excerpt specifically related to this from the first post I published related to Mark’s bullet jump data:

      Are These Results Statistically Significant?
      As Mark began to share his preliminary findings with friends in the shooting industry, a couple of people had concerns about how repeatable these tests were or how much confidence they could have in the results. Of course, from Mark’s perspective, it wasn’t like he was just trying to promote a theory. He clearly doesn’t have anything to gain by sharing this info. So he was just as interested in learning whether these patterns were random or if there might really be something here.

      So, a couple of months ago Mark asked a professional statistician to analyze this data he’d collected for the 105gr Hybrid. After doing calculations and looking at the data a few different ways, the statistician’s professional opinion was that “statistically significant conclusions are appropriate.” He calculated p-values less than 0.05 with two different approaches, meaning there is strong evidence that the patterns aren’t random. The statistician was careful to add, “Extrapolations to other tests, bullet type and environmental conditions must be made with caution by the subject matter expert.” That means we shouldn’t assume these results for the 105 Hybrid represent how other bullets might behave, which is an important point.

      So in that particular instance, it seems that a professional statistician did the necessary p-test and found the data to support the hypothesis. But, there was a subsequent post about Mark’s 18-Shot Challenge, and I’m not sure that sample size is enough to draw meaningful conclusions – but again, that doesn’t mean there isn’t something there or it isn’t helpful. Some of this is finding the “right” balance between purist and pragmatist – for you. I doubt any of us are going to fire 30-shot groups of every different load specification we test, although that is what real statisticians would suggest. But, where that line is can really only come down to the individual to decide. Of course, having said all that … it’s also why I invested another $1,000 in ammo for the 6.5 Creedmoor ammo test that I did. I knew I needed a larger sample size to be able to trust the results, so I personally invested another $1,000 in ammo for that test. I think the whole thing ended up costing me over $4,000, which was all out-of-pocket … but I had this stuff in the back of my mind! Obviously, I believe in it!

      Great question!

      Thanks,
      Cal

  8. Mahmoud Shmaitelly

    Cal,

    This Target Analysis online tool is free and still available. I like what it does for free!

    http://taran.ptosis.ch/taran.html

    Thanks,

    M.A.Sh

  9. Cal, I appreciate the time you put into this. I think you’ve convinced me to start using an app to measure ATC of shots in addition to extreme spread. Even though I have engineering background I learned some things from your interesting presentation. I often use Es instead of Sd for velocity and had heard that Sd of under 10 shots wasn’t that reliable. Reason I tended to use Es is because I often worked up loads shooting 5 shot groups round robin a la Dan Newberry OCW method, so I’m not capturing velocity of a given load on consecutive shots. It was easier for me to subtract max/min to calc Es for a load than plug all the velocities into a spreadsheet for Sd calc. I could rule out a load with a large Es pretty quickly.
    A high master highpower shooter once told me his method for precision load workup: 3 shot groups, then 5 shots of most promising, then 10 shot of best one or 2, finally 20 shots for record in a match. Pretty definitive results but quite a few rounds down range. Except for hunting rifles, I rarely will shoot less than 5 shots to gauge precision. From my experience, it’s too easy to get a misleadingly small 3 shot group that sends you down a path of a using a suboptimal load.
    regarding the killer app to overlay targets, seems like it shouldn’t be that hard. existing shot measurement app could be “taught” to identify the common center of each target, then plot points on X Y coordinates (after rotating each image to have exactly the same X Y orientation). Then create a composite of all the X-Y coordinates and run the math on that. Beyond my skills but should be doable.
    Good luck moving that forward!

    • Thanks, Tim! I’m with you. Honestly, as I was reading all those other sources and trying to craft this content, it changed the way I thought about this stuff, too. So you’re not alone!

      I think the high master highpower shooter was on to something. I do something kind of similar, where I just continue to add shots but eliminate loads as I go until I have a winner. That seems like a good, pragmatic approach.

      And I agree that the app wouldn’t be hard to write. I actually have a degree in Computer Science and was a professional software developer earlier in my career, and have the whole thing built in my head … but don’t have the time to actually invest in it. It’s not too complex. I bet someone does it. I think the power would be in being able to search/filter for groups fired based on attributes. For example, it’d be cool to overlay all groups you’d ever fired of a certain cartridge, or out of a particular rifle, or out of a particular rifle with a particular bullet, or out of a particular rifle with a particular bullet and powder type and charge, etc. That would be a way to let you assemble a larger sample size from shots you already fired, but be able to see the composite group that might span multiple months or even years … but had common elements. I think there is statistical power in that! I pitched that idea to the Ballistic X guy, but maybe someone else will run with it. Ultimately, I don’t care who does it – I just want to be able to buy an app that does it for me! 😉

      Thanks for comments,
      Cal

  10. Carl E. Osborn, Ph.D., J.D.

    Hey Cal:
    I’m probably missing the obvious here – graduate school and my training in statistics was 40 years ago – but why are we talking about mean radius rather than SD radius? In spite of its more inclusive nature, calculating a mean radius is still calculating a mean and we already know about its weaknesses vis a vis outliers when used as a measure of central tendency. I’ve been sitting here with my morning coffee trying to decide whether I’ve missed an obvious reason why this won’t work and feeling like an idiot after someone with better statistical skills points it out but it looks like I’m gonna press the send button here anyway.

    • Thanks, Carl. Great question. You haven’t missed anything obvious. There are a couple reasons I agree with Bryan Litz on the general recommendation to use mean radius, but honestly … any of these measurements would be a significant improvement over ES. Here are a few thoughts related to your question:

      1. With muzzle velocity, we like SD because it describes how much variance there is from the average. But with groups, this gets a little funny, and I’ll try to describe what I mean … although it’s kind of complex. Let’s say you fired a 10-shot group that formed a perfect circle around the point of aim. Every single bullet hole was precisely 1.0″ from the point of aim. The extreme spread would be 2.0″ (I know that’s “big” … but lets keep this simple). That means the radius of all 7 shots was 1.0″, which would also be an average of 1.0″. But what would the standard deviation of the radii be? 0.0″, right? There is no variance from the average. But, is that telling us what we want to know? Is 0.0″ SD on a 2.0″ group a good thing?
      2. This is kind of a technical thing, but you mentioned “SD radius” – but “Radial Standard Deviation (RSD)” is not what it sounds like. Honestly, that measurement is biased and misnamed because it does not express the standard deviation of shot radii. (Read more about that here.) I don’t know who’s bright idea that was to start calling it that. The fact that there is another established measurement with that same name causes some confusion.
      3. Probably the most important part is related to you wisely noticing that we’re using the mean/average, which is more prone to outliers – like I covered in detail in Part 1. I am actually with you there. Personally, I think median radius might be a better measurement, because of all the reasons I covered in that first post … namely “When we’re talking about what is most likely to happen in the future, the median can often be a better choice than the average.” I almost went into this in this post, but thought I was probably already too long-winded … so I stuck with just explaining mean radius. But, CEP is one of the other measurement methods mentioned in several quotes I mentioned in this article. The military uses that to prove ammo/weapons, too. The excerpt from Bryan explained “Circular Error Probable (CEP) is a common metric used to quantify dispersion. CEP is defined as a circle in which 50% of shots will land inside. CEP and mean radius are very similar, within 5% or 6%, but are technically not the same thing.” It’s actually the radius of the circle, which that quote doesn’t make entirely clear. So if the CEP is the number that half of the shots feel inside of and half fell outside of, that is basically the same thing as the median. So I’m pretty sure you could just call it median radius. Funny how we have different names for everything! It certainly doesn’t help make this easy! I think I could argue about the merit of median radius over mean radius, but like Bryan said, “CEP and mean radius are very similar, within 5% or 6%” … and not worth arguing about. Either are a significant improvement over what 99% of shooters are using today in the shooting community, which is ES. So I didn’t want to make an academic argument.
      4. Finally, I think average/mean is more easily understood by most shooters than the median. It’s not some abstract idea that is hard for the common shooter to understand. I’m not arguing about technical merit here, but ultimately if people don’t understand what it physically represents on the target then you’ve lost some value.

      All that to say that you aren’t wrong in your thought process! If nothing else, I appreciate knowing that this got you thinking about it. 😉 And more than that, I appreciate you asking a question and putting yourself out there. I am that same way! I’m never afraid to ask a question if I don’t understand. Sometimes there is a simple answer that I was just missing … but in this case, it’s not a simple answer. Sure hope it makes sense!

      Thanks,
      Cal

  11. Cal

    What do you consider a good mean radius size group for PRS type shooting? The “old” way of thinking was a 3/4 to 1 MOA ES shooting gun and ammunition was acceptable for PRS type plate banging. Of course smaller was better, but this acceptable. Mean radius of even a 1MOA ES style group is quite a bit smaller.

    Thanks

    • Great question! Why didn’t I cover that in this post? You know, I spent days thinking about this and that never crossed my mind – but it sure would be helpful for context.

      I included a target image in the post of a 7-shot group from my 300 Norma Mag. The ES of that measured 0.767 MOA (labeled as “Group”), and the mean radius was 0.221 MOA.
      300 Norma Group

      Now that doesn’t mean there is some kind of equation to convert between those. But, I fired a BUTT-LOAD of groups with 6.5 Creedmoor match ammo from two different rifles (the test that I’m about to start publishing the results for), and those groups averaged from the “low two’s” (like 0.208 MOA for the mean radius up to the “high three’s” (like 0.399 or even one at 0.460 MOA). Those were the average over 8 groups fired from two different rifles. I had a couple of 5-shot groups where the mean radius was as low as 0.09-0.14, but that was just a single group and as you average that over multiple 5-shot groups you land at the numbers I just mentioned. That is all with match-grade factory ammo, too. I’d say anything that is 0.300 MOA or higher for mean radius is easily below my standards, and the related ES of those were all 0.8 MOA or larger. In the 6.5 Creedmoor ammo test, the ones that were in the “low two’s” had an ES from 0.5-0.6 MOA. Remember, those were all 5-shot groups … and 3-shot groups with the same rifles and ammo would be significantly smaller, and they were averaged over several groups.

      The US Military uses mean radius for some of their weapon ammo tests, and for the legendary M24 sniper rifle their requirement is defined in MIL-R-71126(AR), 3.15.7 Targeting and Accuracy:

      M24 Sniper Rifle Accuracy Standard

      So my quick math for 1.3 inches at 200 yards and 1.9 inches at 300 yards is about 0.6 MOA. That might sound impressive, but remember that is mean radius and not extreme spread. That means for that target at 200 yards, it is 1.3 inches from the center of the group to the bullet impact on average. The extreme spread of a group like that is likely closer to 4 inches at 200 yards. Now that military standard was in 1992 and rifles and ammo have come a long way since then! That is also the bare minimum to be accepted for service. I certainly wouldn’t make a mean radius of 0.6 MOA my personal goal. Honestly, I’d be pretty pissed off if that’s what my rifle shot!

      My personal goal would probably be to find something with a mean radius under 0.250 MOA for a group with 5+ shots in it. That shouldn’t be too hard with good ammo and a good rifle. Like always, the lower the better! I would suspect lots of guys will aim for 0.200 MOA. That might be hard to actually pull off with large sample sizes.

      Hope that provides some context. I’m sure others might be able to chime in with what their experience has been or what their goals are for a good load.

      Thanks,
      Cal

      • Thanks, for replying. I like small groups when I hand load, but but like to shoot factory ammunition at times too. That is usually nowhere near as good as the hand loads in group size and not even close on ES/SD numbers for speed, but still hits targets pretty well. The mean radius method seems like a better way to test, especially with the factory stuff.

  12. Thanks for all the hard work putting these together, really good info here. One thing I always look for is if an outlier occurs at a specific place, ie, last round in mag always seems to go a bit left. Took me awhile to figure out that the mag was feeding that round in hard enough to hit the chamber wall and reset the bullet seat.

    • That’s an interesting point, Brian. There sure could be something causing an outlier. I’ve had very serious Benchrest guys tell me they even found patterns with where the bullet would go in the group based on how many shots were through the rifle since it was cleaned. They cleaned after every string, but I think they told me they’d fire a fouler (or maybe two) and then fire shots for record and shot #1 always seemed to go high, then shot #2 would be right, shot #3 might be left, etc. Now that is a little different than your example where it’s a mechanical reason, but it’s an interesting point. There has to be something that is the root cause of the dispersion (or a series of things all interplaying together). If you can figure out what that is, that’s great!

      Thanks,
      Cal

  13. Once again, well done. When using ATC with a 10 shot group what would be an acceptable ATC to determine a minimum overall sub moa rifle. Ignoring the actual group size. In other words, how do we use ATC and what ATC moa range is desirable.

  14. You’re pretty much regurgitating Bryan Litz’s books with this particular presentation. I don’t know if you’re running out of interesting topics or not because you’ve put out some excellent excellent studies and reviews in the past but not so much lately. I’m looking forward to your next post.

    • Ha! Well, Heath, I couldn’t disagree more. I actually think this 3-part series might be one of the most helpful and useful things I’ve ever written – but you’re entitled to your opinion!

      I do quote stuff Bryan has done pretty regularly, but that’s because he’s one of the only serious researchers that has done objective studies on many of these topics and published them in a way that is accessible by the general public. I also quote some of the Hornady ballisticians, like Dave Emary and Jayden Quinlan, because they also put out stuff to the general public. But outside of those guys (and me), I’m not sure there are many other people regularly putting out material on the more technical topics of shooting or publishing original, objective research. So that’s why I quote them so often. Actually, in this 3-part series, I quoted other people more than I did Bryan, so it’s funny that you feel like this is a “regurgitating Bryan Litz’s books”.

      I’m certainly not running out of interesting topics. I thought this one was interesting, and have got a ton of emails and comments from others that they also found this interesting and very helpful. It just seems like you didn’t value this one. Maybe you’ll like the next one. If not, I’ll give you a full refund! 😉

      Thanks,
      Cal