PrecisionRifleBlog.com A DATA-DRIVEN Approach to Precision Rifles, Optics & Gear
Related Articles
I wrapped up the testing portion of the scope field test last night!!! Although I significantly underestimated how much time this would require, I committed to do it … and now the hard part is over. I did get some great data that may offer new insight, and help fellow shooters see what they’re paying for.
I’ve already started crunching the data, and plan to start publishing results within the next 2 weeks. Instead of analyzing all of the data and writing all the content up front, I plan to break it down into sections and publish each one as I go. So this will essentially be a series of posts, and one might cover the results of the double-blind optics tests, while another one talks about the findings related to how precise the click values were. There may be 3-4 posts total to cover the results, and the final post will wrap it all up with some overall scores and a summary.
Now, I need some input from you. I’ve been debating on how to weight the overall scores. Essentially, I tested optical performance, mechanical performance, and I’m also trying to quantify things like ergonomics and other specialized features. How should each of those be weighted in the overall score? Is optical clarity or mechanical precision more important? Please take a minute to let me know what you think by answering the one question poll below. The poll will only be open for 1 week (closes 6/19).
Update 6/24/14: Thanks to everyone for providing input on this. 709 people completed the poll, and it really helped me understand how the overall scores should be weighted. Here are the results:
You can see mechanical and optical performance were both obviously rated as very important, but mechanical was rated as slightly higher priority on average. Most shooters view precise mechanical adjustments as a cost to even be a serious candidate at this price point. Then on the less important end of the scale was ergonomics and what I grouped as “advanced features” (I didn’t know what else to call it). It looks like they are also very similar in ranking, although a few more people thought things like reticle options, locking turrets, zero stop, etc. were slightly important, and some even said that stuff as VERY important. Thanks again for the feedback. And, as I’ve mentioned to a few people … don’t worry, I won’t dumb all the results down to top-level scores. I plan to publish exhaustive details of each and every test. So if you don’t agree with the overall scoring weight, you can look at the results for each test and come up with your own conclusions. I just want to provide some type of summary score for people who prefer the “cliff notes” version, and I wanted the weights I used to lineup with what the shooting community in general thought was important … not just based on what I thought was important. Thanks!
Here are a few more photos from the field tests:
A quick suggestion regarding your survey: I would rate all 4 elements as ‘very important’, and suspect most other shooters will as well. However, if I had to RANK them relatively in order of importance, it would be as follows:
1. optical performance (most important)
2. ergonomics
3. mechanical performance
4. advanced features (least important, but still pretty darn important)
My point is that it may be more useful to ask people to rank their choices against each other rather than to ask if each one is important. ‘Cuz they’re all important.
Thanks for all of your hard work, and I look forward to reading your posts over the next couple of weeks.
Great tip. I originally designed it as a ranking question, but that actually doesn’t give you the same information. For example, I personally think optics & mechanics are both really important, but maybe mechanical precision slightly higher priority if I had to pick between the two. While ergonomics, reticles, and other features aren’t trivial, in my opinion they aren’t even in the same ballpark as those other two. I can specify all this stuff with a Likert scale question like the one I created. Ranking just gives order, not intensity. So you’d just know that I think Mechanics, Optics, Ergonomics, Advanced Features. There would be no differentiation of intensity between each one … only relative order.
I did add the tip that tells people to not just rate everything as important in response to your tip. Maybe that will help. I do plan to throw out any results where someone did that. Luckily so far, out of 100+ people, virtually everyone has put thought into it and not rated them all as VERY Important. Thanks again for the tip.
I’m not certain that Cal’s survey format isn’t spot on…considering I’m a bit shocked that someone would rate optical performance and ergonomics above mechanical performance. With all due respect, what good is a scope that you can see your target crisply, with a great deal of comfort and style, but cannot reliably depend on for tracking???? Our vastly different viewpoints would not accurately be depicted in a “rank order”. Thanks Cal for the time and effort you have put in on this project, I’m really anxious to read your findings.
Hi Clete. Thanks for your perspective. I freely admit to not being an experienced shooter, and so may have different priorities than you. I guess I’m assuming that any scope in this price range will have ‘acceptable’ performance in all categories. The question then becomes, which element is most important to you? If I flip your (very valid) question around, I come up with, “What good is great mechanical performance if you can’t see the target clearly?” With my 54 year old eyes and tri-focal lenses, I can’t physiologically compensate for less-than-excellent optics. Younger eyes may not have this issue.
Three rules for guys over 50:
1. Never bypass a urinal.
2. Never waste an erection.
3. Never trust a fart.
I’m sure there are a few different views on this, and that’s okay. Like I mentioned to R Byrd, I plan to publish details and scores from each test. The only debate is on the one “summary score” I’m hoping to publish for people who are doing a fly-by comparison, and don’t want to hear about all the different things I tested and don’t care how different aspects are weighted. There really are more people like that out there than you might think!
So if someone doesn’t agree with the weights I land on for the overall score (which will be determined based on the input you guys are giving me on this survey), I’ll provide all the details necessary for someone to make their own score based on different priorities. I just want to make sure that the one I publish isn’t just based on my view of what is important, but instead represents what the long-range shooting community in general views as important (or at least what my readers consider important). I am really sensitive to not represent my own opinion too boldly on this website. My goal is to try to present information in the most unbiased way possible.
And although I typically edit out off-topic comments … Robert, the 3 rules for guys over 50 was funny. At least made me laugh! So I’ll leave it up there. Thanks again for the comments.
I respectfully suggest you not weight the categories, but do a detailed job of rating each one as possible.
I see reviews that seem to dumb down their audience by telling them what the most important aspect of a thing is. I find that people care about things in different degrees based on there specific needs. Some could care less about size and weight where others think it is paramount. By weighting the results you are telling the audience “you’re not smart enough to decide what’s most important to you.” Your great work will allow us to see which scopes excelled in which areas
Absolutely. I will provide exhaustive detail of the individual tests, including specific results and how they were conducted. I’m a detailed person myself, so I appreciate when people allow me to dig into the details. I definitely won’t gloss over the tests with some kind of high-level summary, which is why these are going to take a lot of time to write.
At the same time, not everyone wants to be forced to dig into the details to come to a conclusion. Some people just prefer to get a top-level executive summary.
I plan to cater to both, just like I did in my Rangefinder Field Test. There was a post that gave a top-level summary, and other posts that got into the lower-level details. Some people said I went into WAY too much detail on the rangefinder posts, but I know there are still people like you that prefer to know the underlying data … so I’ll do both.
Thanks for the input.
Robert, your “over 50 rules” are both noted and appreciated…I will need those sooner than later. ha ha ha…in regards to the mechanism vs. optical debate…consider this, when evaluating quality scopes: better to know that your turret adjustments consistently place the cross hairs precisely were they should be on a fuzzy target (you don’t need to see it sweat, you just need to identify it accurately), than to be able to clearly see how far off you just missed through a pristine piece of glass with inaccurate tracking. As a note…I prefer both to be exceptional, and also appreciate a good value when presented (Thanks Cal!) . 🙂 Good luck, as you continue forward. Cheers!
ty for your hard work, it is appreciated and very valued !!!
My name is Cal … but I’ve been called worse things than Ty! I appreciate the encouragement nonetheless. Thanks!
Ty means thank you.
Ha! You guys and your textspeak!