Printer Friendly Page
VETS' COMMISSION SURVEY HITS SOUR NOTES --
Reports of
veteran harassment and questions on methodology
call results of VDBC survey into question.
Story below:
---------------
The Veterans' Disability Benefits Commission
(VDBC) is studying exactly what their name implies.
Many say their goal is to cut benefits.
Their report is due out this Fall.
Background on VDBC here...
http://www.vawatchdog.org/
milcom/vdbcstackeddeck.htm
And here...
http://www.vawatchdog.org/milcom/
vdbcsmisleadingquestions.htm
The VDBC is interviewing over 22,000
service-connected disabled vets as part of their study.
I have recommended that veterans not take this
survey. The VDBC has no "need to know" your personal information.
And, the general feeling is that this information will be put to bad
use.
Background on the survey, with backlinks,
here...
http://www.vawatchdog.org/
nfDEC06/nf120406-4.htm
Now, we get some disturbing news. The
VDBC notified the 22,000 vets by letter that they would be part of the
survey. They were told they would be called by the company hired
to do the survey.
A number of veterans have told me they refused
to take the survey when contacted by the survey company.
Now comes word that the survey company is
calling back and making statements felt to be harassing and even
threatening. Vets have told me that on a second phone call from
the survey company (this after saying they wouldn't take the survey)
they are being told in a harsh tone that the survey is "from VA
Secretary Nicholson" and that it's "important" and that their
participation is "expected."
This is downright harassment. The vets
have already said no...and, invoking the name of the VA Secretary has
threatening tones.
The VDBC should be ashamed of these tactics.
Many of the veterans "chosen" to take this survey are suffering from
PTSD. This type of harassment can only cause unneeded stress and
anxiety.
This is clearly not in the best interest of
veterans.
Now...another note about the survey.
I received the following email from a veteran.
I have removed any identifying information.
The information included in this email leads me
to believe that the VDBC survey will provide no meaningful results.
-----
Dear Larry:
Just discovered your site. Thanks so much, it's great.
Hadn't seen your site before I took the Veterans Disability Benefits
Survey so I took it. Had I known I wouldn't have done so. However, there
may be a silver lining here. I designed and conducted surveys
professionally for a long time. It is being done so poorly, I was
appalled when I took it. So, I sent the following letter to the VDB
Commission, as well as my congressmen and women, and the chairs of the
House and Senate VA Committees.
The letter should be self explanatory. Hope it does some good. Let me
know if you have questions, or I can help with this.
From: Veteran
March 6, 2007
Veterans Disability Benefits Commission
1101 Pennsylvania Ave. NW - 5th Floor
Washington D.C. 20004
Subject: Veteran's Disability Benefits Survey
I am a Vietnam veteran receiving VA disability benefits. As such I was
selected to participate in the Veteran's Disability Benefits Survey
being conducted for the Veteran's Disability Benefits Commission. After
responding to the telephone survey recently, I am disturbed by what I
heard. A simple analysis of my responses will give a misleading view of
my situation. Let me explain.
When I was working, I was trained in the design and conduct of surveys.
This included not only study design and sample selection, but also
proper question wording and questionnaire design. I conducted many
surveys for my agency, gaining valuable experience. I am troubled by the
design and implementation of this survey. Please note the following
concerns.
First, many questions were difficult to understand. Some I never did
understand and I am a college educated professional. Some of the
questions were so long, the interviewer had great difficulty reading
them to me. What kind of responses can we expect from any respondent who
doesn't understand the question? Typically, when a respondent doesn't
understand, they will provide a response anyway, especially when urged
to do so by the interviewer as I was. If they don't understand the
question, they will guess at its meaning, and their guesses are often
wrong. Thus they are answering a question different from the intended
one. The same condition is true if the respondent does not understand
the meaning of any of the allowed responses. How can we trust the
responses from these questions?
Second, the allowed range of responses was inadequate. Often they
weren't relevant to the question. Most questions were scaled multiple
choice, the respondent picks one of five possible responses provided by
the interviewer. The five point scales were typically in the form of
from "much better" to "much worse", or from "much less" to "much more".
In many cases, "none of the above" was the answer I would have provided
if I had been allowed to provide my own response. As an example, one
such question asked about the status of my condition. On a scale of one
to five, "was it much worse?" at one end of the scale, to "was it much
better?" at the other. My condition has changed. Some parts are better,
some parts are worse. My condition is so much different now that it is
difficult to describe it as better or worse. I chose the middle point of
the scale, "no change" because it isn't better or worse. However as we
have already seen, "no change" is also inaccurate because the condition
has changed. The study designer failed to account for this possibility.
What do you know about my condition from the response I provided? How
can we trust the responses to these questions?
Third, the interviewer was inadequately trained. She struggled to read
questions to me, often not understanding questions herself. When I
didn't understand, all she could do was reread the question, which did
not help. Often she had to ask a supervisor for instruction on how to
proceed after I asked for clarification. Several times the supervisor
had to tell her how to continue with the questionnaire. How can we trust
these results?
In my professional opinion, the study designers display a totally
inadequate understanding of the study subject, resulting in poor study
design. The questionnaire was not pretested adequately leading to
amateurish questionnaire design and question wording. The study attempts
a simplistic analysis of a complex situation. The survey covers such a
wide variety of disabilities that one set of questions can not do the
job. Too many questions weren't applicable to some types of
disabilities. Also, the survey is likely to have a large "non-response"
bias which will lead to skewed results if not fully corrected. This
study is almost guaranteed to provide an inaccurate analysis.
I am concerned that a simplistic analysis of my responses will give a
misleading view of my situation. I am so concerned that I want to
withdraw my responses. Results of this survey are likely to give an
inaccurate analysis of the situation leading to inappropriate policy
recommendations.
Signed, Veteran
---------------
Larry Scott --
