Without even getting into the horrors of pointless research design or poorly thought out data collection methodology... lets just focus on some simple demographics questions...
Students Design
1. Age2. Date of Birth
3. Today's Date
4. Postcode
5. Gender
6. Ethnicity
7. Religion
8. Relationship/Marital status
- Single,never married
- Single,previously married
- Defacto relationship
- Married
....
You get the idea. There is a certain rational for very very blunt questions... but its crap. I have lost count of the number of surveys I have prepared (got to be in the range of 60+ by now) and I also usually have the pleasure of processing the data that comes off the back end. My point is I have seen the reaction to blunt and poorly worded questions and its BAD.
These questions are AMBIGUOUS. The data that is collected will be similarly AMBIGUOUS, making the research CRAP.
These questions impose ASSUMPTIONS on the participant that they may neither feel comfortable answering truthfully or make the research (and the researcher) look foolish. Preventing the participant engaging with the research truthfully is simply BIASING the research and turning the collected data into a stew of half truths and garbage.
These questions display IGNORANCE and BIAS on the part of the researcher.
Firsly, ask the student why they are collecting the data. (Usual answer is ... my supervisor said it was a good idea....WRONG!... go back to the start... try again.) The right answer is, "I need to UNDERSTAND something about my sample population". or "I want to be able to describe my sample population clearly" or "I think this is an important variable and need to accuratly measure it so I can identify outliers and remove participants who are unsuitable from my study" or "I need to split my sample population on this variable" or "I have ethical clearance to study a particular population and need to clearly identify suitable participants"... etc. There needs to be a very clear rationale for asking ANY question. Following on from that rationale is then a clear understanding of exactly what data that question needs to collect. Is it specific CATEGORICAL data? Is it a gently worded open ended question?
If you're splitting the sample population on a clearly defined categorical boundary then the wording of your question needs to deal quite specifically with that boundary and clarify for the participant which group they fall into. It also needs to ensure that it does not BIAS the participants to try to put themselves in the WRONG group. There are various reasons why someone may choose to lie about some information. The point is to make it easy for them to answer correctly/honestly/comfortably.
1. Age
Asking for the participants age is fairly pointless. Help the participant provide you with clean, useful data by asking them for their age in years, or years and months or whatever degree of precision you are interested in. Let them know if you would prefer it rounded up or down, if you want really unambiguous data. (
N.B peole usually round their age down....unless they are young and want to seem older.. so if your mention a boundary of 18 years... you will find some young people will try to ... ease themselves over the line...
2. Date of Birth
DO NOT ASK FOR PARTICIPANTS DATE OF BIRTH!
Just don't. This is sensitive information. There is a great deal of conditioning going on to stop people giving their date of birth to any online form. So you will get lies and ommisions from literate participants and accurate information from technically naive participants. But you have no way to tell the difference. So its pointless and unreliable data.
In the event that your survey contains any other identifying information, this turns your data set from anonymous to containing identifying data that you are required to store securely for 7 years. This also makes your research project higher risk and should be reviewed by a full ethics review board.
3. Todays Date
WTF?
If you are doing the research on paper... then have the courtesy to put the date in yourself. If you are using an electronic system... it should time stamp the results automatically.
Firstly, if you actually care about the date the research was conducted, why are you trusting the participant to get it right? Help them out. Only ask the participant for information you do not already have.
4. Postcode
Fair enough.... But which one? What are you trying to understand by asking that question? Remember a postcode is about location, but as people can and do move, its also implicitly a time dependant question.
Do you mean
"What is the postcode of the location you are filling in this research form now?"
"What is the postcode of your current residence? "
"What is the postcode of your home if you live away from home to study?"
"What is the postcode you have lived in for the majority of the previous year?"
"What is the postcode that you work in at your current job?" (What if they travel for work?)
"Postcode" depends on time and place. What place do you mean and what time do you mean?
5. Gender
Oh my.... where do I start?
Having done surveys for Gay, Lesbian, Bisexual, Transgender and "Other" groups, I have some experience of just how complex this question actually is for some sections of the population to answer and then for the resercher to do something with the data.
There are many possible dimensions to the question..if we stick to asking about the simple expression of an old fashion gender stereotype, not counting sexuality or sexual preferences, reproduction capacity, nurture capacity, social role etc... then the question can be expanded to...
What gender cues do you choose to publicly express?
What gender do you choose to privatly express?
What gender does your body physically express?
What gender does your choromsomes express?
How strongly do you identify as or express the gender choice publicly?
Do you always express the same gender publicly?
Do you express the same gender cues to everyone?
The spectrum of answers include, but is not limited to:
None
Male
Female
Both
Hermaphorodite/Intersex (Physically expressing aspects of both)
Androgenous (Physcially expressing neither)
Transgender (Neither or both but not either specifically, ambiguous or electing to live as one or the other. Various meanings... basically ambiguous)
Transexed (Assigned a gender at birth but choosing another later in life. May include social, hormonal and surgical alterations)
I have also seen scenarios where the participant answered honestly with something like this
"Male with my family, female in public and privatly with my partners I tend to play a dominant woman... until they get to know the real me".
So go and code that into a neat box... I dare you.
Here are some links to add even more detail
http://en.wikipedia.org/wiki/Gender#Gender_taxonomy
http://web.uvic.ca/~ahdevor/HowMany/HowMany.html
So the question gets back to what the student actually wants to know..... how precise the student wants the answer and how comfortable they want their participant to be answering the question.
6. Ethnicity
Seriously.... what fucking rock have you been living under for the other bit of your life? Or series of rocks...
I have built surveys to run in a number of different countries and the answers to this question and the sensitivities surrounding it are "varied" to say the least.
Is the student asking about the participants genetic heritage? And if so, why?
Is the student trying to understand the cultural context that the participant brings to the discussion? If so then ask about that....
Is the student trying to collect nationanlities?
Are they asking about where the participant origionated from or their current legal status?
Is the student just an uniformed idiot? .... pretty obviously
In a country that is awash with migrants, immigrants, their children, other peoples children, stolen generations, the decendants of slaves, visitors, temorary residents, illegal immigrants, various long established cultural groups... you will get a very very very wide range of answers to this question.
In a country where there are very strong ethnic/cultural/national groups ( such as Malasia) asking such a question will probably only get you one of three answers (Malay, Chinese, Indian) there are a few others but this is the big three. The problem is that these are social group "Identities" which tells you only as much as any stereotype can.... which is sweet fuck all. Essentially, your doing "Bad Research"(tm)
7. Religion
.... ok... there is a clear patten of dumb ignorance here.... Without beating a dead horse, the best phrasing for this question I have seen is:
What religious or spirital group if any do you identify with?
However this question does not ask about "Practising" or "strength of beleif"... simply because measuring string is not what is going on... this is simply a bullshit question for trying to categories the participant group and make a nice table or graph in some really crap research report. Even asking the question marks the researcher out as an idiot. But I digress....
8. Relationship status
Talk about double barrel answers to a single barrel question...
Ok. The researcher is trying to figure out what the participants relationship is currently... and what it might have been before. (But without clarifying time frames or any other contextual information)
The big problem I see is just how many descriptors are missing from the list.
My list of favorites goes something like this....
How would you describe your relationship status?
- Single
- Divorced
- Widowed
- Sole Parent
- Shared Custody Parent
- Abandoned
- Grown Up
- Independant
- In a new relationship
- In a Defacto relationship
- In a Civil Union
- In a Religious Marriage
- In a Partnership
- In a Committed Relationship
- In an open Relationship
- Getting Divorced
- Seperated
- OTHER....
9. Highest level of education completed
As we have all sorts of people showing up at University I have gotten into the habit of expecting a quite varied range of answers to this question. I still meet mature age students who report only doing their sixth form or not really going to school at all. I meet migrants and refugees who have had fragmentary schooling in one or more countries that just does not map to a simple list.
My current favorite list is:
- Minimal formal education
- Yr 10 (Or Equivalent)
- Yr 12 (Or Equivalent)
- Certificate/Diploma (less than 1 year of study)
- Apprenticeship/Trade Qualification/TAFE Diploma (More than 1 year of study)
- Undergraduate Degree
- Post-graduate Qualification (Honors/Diploma/Associate Diploma)
- Masters/Doctoral Course/PhD
- Other
Again, all these answers are dependant on what the researcher is trying to do with the data they are trying to collect. There are levels of precision that are useless for some purposes but never the less allow the participants to feel comfortable and happy to provide accurate answers, even when the researcher is going to homogenise the answers into some simplistic scale....
People factors..... gotta love 'em.
No comments:
Post a Comment