We have long argued that the Standard Field Sobriety Tests (SFST) used in DWI arrests is junk science. The so called "tests" have never been peer reviewed and are actually not science at all.
Dr. Greg Kane decided to examine the field sobriety validation studies.
Dr. Kane has examined the raw data that cops, and the public for that
matter, never see. His conclusions, the numbers just don't add up.
Why should you always refuse field sobriety tests? Read on to find out.
1. Name, Background, Resume
Greg Kane. live in metro Denver. I have degrees in math and in physics from Rice, 1980. I went to med school at UT Houston; I practiced internal medicine from '91 till '98. Since '98 I've been a consultant to attorneys working on medical malpractice claims. See www.medmalEXPERTS.com
Greg Kane. live in metro Denver. I have degrees in math and in physics from Rice, 1980. I went to med school at UT Houston; I practiced internal medicine from '91 till '98. Since '98 I've been a consultant to attorneys working on medical malpractice claims. See www.medmalEXPERTS.com
2. How did you get involved with SFST?
We medical doctors spend a lot of time interpreting imprecise physical tests. A spot on an x-ray may mean cancer, or it may mean nothing. Do you cut the lady open, or do you send her home? Medicine has developed a sophisticated mathematics for answering questions like that. In med school they teach you that math. As a math - physics guy, I though it was cool. My post-graduate research dealt with how that mathematics worked out for one specific type of x-ray. So I come to the question of SFST accuracy with an interest and some expertise in the mathematics of "accuracy."
As an expert broker I get calls from defense attorneys looking for a doctor to counter FST evidence. Too often these cases involve drivers who failed an FST, had a BAC of zero, and are now charged with driving while intoxicated by, say, a therapeutic level of some benign medicine prescribed by their own doctor. The government's reasoning: they failed an FST, they must have been impaired.
We medical doctors spend a lot of time interpreting imprecise physical tests. A spot on an x-ray may mean cancer, or it may mean nothing. Do you cut the lady open, or do you send her home? Medicine has developed a sophisticated mathematics for answering questions like that. In med school they teach you that math. As a math - physics guy, I though it was cool. My post-graduate research dealt with how that mathematics worked out for one specific type of x-ray. So I come to the question of SFST accuracy with an interest and some expertise in the mathematics of "accuracy."
As an expert broker I get calls from defense attorneys looking for a doctor to counter FST evidence. Too often these cases involve drivers who failed an FST, had a BAC of zero, and are now charged with driving while intoxicated by, say, a therapeutic level of some benign medicine prescribed by their own doctor. The government's reasoning: they failed an FST, they must have been impaired.
Because I happen to know how science says FST results are correctly interpreted,
I know the government's theory is wrong. The most difficult thing
in life is to know how to do a thing right and to watch somebody else
do it wrong, without comment. Also, it's un-American to convict people
with secret evidence, and with "scientific" tests that don't work.
3. What did you learn about SFST validation studies?.
First, all the usual stuff defense attorneys complain about. They were never peer reviewed. They're chock-a-block with procedural and logical flaws.
Second, I discovered things that don't get talked about. For example
l The "accuracy" statistic the NHTSA uses to validate the SFST is a technical mathematical statistic that does NOT reflect the likelihood that a DUI defendant who failed the test was impaired.
3. What did you learn about SFST validation studies?.
First, all the usual stuff defense attorneys complain about. They were never peer reviewed. They're chock-a-block with procedural and logical flaws.
Second, I discovered things that don't get talked about. For example
l The "accuracy" statistic the NHTSA uses to validate the SFST is a technical mathematical statistic that does NOT reflect the likelihood that a DUI defendant who failed the test was impaired.
The NHTSA's "accuracy" statistic is open to manipulation. Simply by manipulating the group of drivers you choose to "study," you can set up your validation study beforehand so it is certain to "discover" whatever arrest accuracy you've been paid to validate. Not only is it possible to manipulate study groups this way, that's actually how it's done in real life. NHTSA contractors do manipulate their study groups in a way that uses this statistical trick. Every NHTSA FST validation study that "discovers" a high FST accuracy uses this trick. Every validation study that fails to use this statistical trick also fails to "discover" a high FST accuracy.
SFST studies do not study SFSTs. They study officers instincts. The "accuracies" they report are not the accuracies of the standardized FST, they are the "accuracies" of officers' unstandardized gut instincts about whether each driver is impaired or not. Validation study officers are as accurate as they are only because they repeatedly ignore the SFST. If they did actually rely on the SFST, their arrest accuracies would be substantially worse. That's right, worse. SFSTs are less accurate than officers' gut instincts, and validation study reports prove it.
All FST validation studies keep the accuracy of the SFST itself secret.
When I looked at the raw data for the 1998 San Diego study, I was shocked.
To a first approximation the SFST works like this: Everybody fails. Everybody
fails, and officers release people their gut instinct tells them are not
impaired.
At the 0.04% BAC level:
296 drivers took the SFST
292 failed-99%.
4 passed- 1 %
On innocent people the accuracy is 7%! If juries rely on the SFST to decide
the guilt of drivers charged with DWAI at the current 0.05% level, they
will wrongly convict 93% of the innocent drivers who go to trial.
4. Tell us about your mathematical analysis. How is it done?
Basically it's as if the government were pushing Youth Cream. The claim is, rub YC on your face and you wake up looking, acting and feeling young. So the government pays for Youth Cream validation studies done at an elementary school, and guess what, 95% of the people who used the cream in the scientific study did look, act, and feel young. Then the government claims the contractors' research proves YC is highly accurate at making people young. But the secret isn't in the cream, it's in how contractors picked the group they "studied."
4. Tell us about your mathematical analysis. How is it done?
Basically it's as if the government were pushing Youth Cream. The claim is, rub YC on your face and you wake up looking, acting and feeling young. So the government pays for Youth Cream validation studies done at an elementary school, and guess what, 95% of the people who used the cream in the scientific study did look, act, and feel young. Then the government claims the contractors' research proves YC is highly accurate at making people young. But the secret isn't in the cream, it's in how contractors picked the group they "studied."
It's the same for SFSTs. Validation contractors "discover"
high accuracies because they load their study groups with drunks.
The home page of my web site fieldsobrietytest.info links several of my published articles on how sensitivity-specificity-predictive-value science applies to SFSTs.
5. Officers are taught these tests are highly accurate. Is that true?
Yes and No.
Yes, there is a technical mathematical statistic called "accuracy," and Yes the "accuracy" of SFSTs is high in the manipulated study groups in the government studies.
The home page of my web site fieldsobrietytest.info links several of my published articles on how sensitivity-specificity-predictive-value science applies to SFSTs.
5. Officers are taught these tests are highly accurate. Is that true?
Yes and No.
Yes, there is a technical mathematical statistic called "accuracy," and Yes the "accuracy" of SFSTs is high in the manipulated study groups in the government studies.
No, SFSTs are not accurate in the everyday sense that the answers they
give are usually correct. The never before published raw data I managed
to uncover proves that within the margin of error a failed SFST carries
no implication of impairment. None. You can't tell whether the person
who failed the test is actually impaired, or whether they're just
one of the 93% of innocent people who also fail the test.
6. How do sober drivers fare on SFSTs?
At the 0.04% BAC, on innocent people the accuracy is 7%! That's not a typo. Seven percent. If juries rely on the SFST to decide the guilt of drivers charged with DWAI at the current 0.05% level, they will wrongly convict 93% of the innocent drivers who go to trial.
6. How do sober drivers fare on SFSTs?
At the 0.04% BAC, on innocent people the accuracy is 7%! That's not a typo. Seven percent. If juries rely on the SFST to decide the guilt of drivers charged with DWAI at the current 0.05% level, they will wrongly convict 93% of the innocent drivers who go to trial.
At 0.08% BAC, on innocent people the SFST is only 29% accurate! That's
worse than a coin toss. If juries rely on the standardized field sobriety
test to decide the guilt of drivers charged with DUI at the 0.08% level,
they will falsely convict 71% of the innocent drivers who go to trial.
7. What should defense lawyers know about these validation studies?
SFST studies do not study SFSTs. They study officers' instincts. All FST validation studies keep the accuracy of the SFST itself secret. The raw data proves the accuracy of the SFST itself is terrible.
Far from proving SFSTs are highly accurate, the most recent, most up to date NHTSA validation study (1998 San Diego) actually proves that study officers routinely ignored SFST interpretation guidelines. The claimed SFST "accuracy" comes from studies whose test interpretation is not standardized.
Because SFST interpretation criteria are not standardized (or rather, are standardized, but study officers simply ignored the standardized criteria) it is impossible for the officer in your next DUI case to reproduce the validation studies' accuracies. The math proves that your client's failed FST may mean she was 100% likely to be impaired, or it may mean her chance was 0%. Or anything in between. That's the best FST science can do. FST results do not predict alcohol impairment.
SFST studies do not study SFSTs. They study officers' instincts. All FST validation studies keep the accuracy of the SFST itself secret. The raw data proves the accuracy of the SFST itself is terrible.
Far from proving SFSTs are highly accurate, the most recent, most up to date NHTSA validation study (1998 San Diego) actually proves that study officers routinely ignored SFST interpretation guidelines. The claimed SFST "accuracy" comes from studies whose test interpretation is not standardized.
Because SFST interpretation criteria are not standardized (or rather, are standardized, but study officers simply ignored the standardized criteria) it is impossible for the officer in your next DUI case to reproduce the validation studies' accuracies. The math proves that your client's failed FST may mean she was 100% likely to be impaired, or it may mean her chance was 0%. Or anything in between. That's the best FST science can do. FST results do not predict alcohol impairment.
8. You've stated you analysis will also work for drug dogs and drug
detection experts. Please explain.
Scientific tests simply do not have an accuracy, a simple single number that reflects how often the test gives the correct answer. So if anyone tells you they do, if anyone interprets a test result on the basis of that sort of one-number accuracy statistic, they are mistaken.
Scientific tests simply do not have an accuracy, a simple single number that reflects how often the test gives the correct answer. So if anyone tells you they do, if anyone interprets a test result on the basis of that sort of one-number accuracy statistic, they are mistaken.
If there was a study in which the drug dog got the right answer 80% of
the time, and later that same dog alerted on your client's bag, that
does not mean the probability your client's bag contained drugs was
80%. The true number, the scientific number, may be as low as 0%.
Ditto DREs. Ditto partial fingerprints, bite marks, hair analysis, etc. For the math, see the web site, or google "sensitivity specificity predictive value"
Ditto DREs. Ditto partial fingerprints, bite marks, hair analysis, etc. For the math, see the web site, or google "sensitivity specificity predictive value"
9. Anything else.
This stuff can make a real difference in real criminal cases. An attorney
here in metro Denver has had FST accuracy evidence excluded at trial by
insisting the prosecution provide expert evidence that the officer's
interpretation of the FST's accuracy statistic as indicating the probability
of impairment is scientific (they can't, it isn't), and by providing
my sworn statement that standard scientific methods of test interpretation
differ from the prosecutions'.
Dr. Greg Kane's Website