Data-Based Decision Making
Value the use of data as the starting point for professional work.
REFLECTION
Adults returning to college have to cope with the stress schoolwork places on their lives. The stress is compounded if they have full-time jobs and families competing for their time. For our Ed 690, Methods of Inquiry class, our team wanted to study how these returning students deal with this stress. Through this exercise I learned several things that will be useful in my future instructional design work.
I learned the importance of completing a literature review before embarking on a case study. Through reading about past studies we were able to both avoid reinventing the wheel and get a clearer idea of the direction our study should take. In this case we were able to find out the motivations for adult learners to take on the burden of higher education and some of the things that helped alleviate their stress. This helped us formulate our questions.
The assignment helped me understand better the practical application of methods use to evaluate data. We used the five-point Likert Scale to measure the degree respondents agreed with a statement. Additionally, both the Mann-Whitney U test and Chi Square test were applied to the data. I discovered the Mann-Whitney U test is best used when the data is a relatively small sample and it’s ordered in the kind of 1 through 5 rating the Likert Scale produces. Chi Square tests evaluates the significant differences between proportions for two or more groups in a data set.
One of the things that fascinated me in the 690 class was how relatively small numbers of people can be used to interpolate for much larger groups. However, the catch is how you select those people. Random selection from a broad number of sources seems to work best. Because of our time constraints, we invited students from the classes we teach along with fellow EDTEC students to take the survey. Ideally we would have chosen a broader sample.
Another thing I learned is that it’s not easy to word good survey questions. Slight differences in wording can be confusing or lead to incorrect interpretations of the question. For instance we wanted to ask about the support respondents receive from their work place. These were three versions of the question:
- Is the support your work place gives employees who pursue an education satisfactory or not satisfactory?
- Do you feel the support your work place gives employees who pursue an education satisfactory or not satisfactory?
- Are you personally satisfied the support your work place gives employees who pursue an education satisfactory or not satisfactory?
These are three different ways of asking the same question, but they will probably produce different responses. The first is more detached, the second asks about personal feelings and the third personal satisfaction. Which kind of response did we want? We grappled with these shadings of meaning.
Another lesson learned is sometimes the conclusion is inconclusive. In this case we were hoping to reveal a difference between genders and generations in how they deal with the stress of school. However, we found there is no significant difference, except in smoking (Males tended to use smoking more than females as a stress-reliever). It’s disappointing when you find no real differences between groups, but this is valuable knowledge, too.
One thing I discovered about myself through this work is that this is my least favorite aspect of Instructional Design. I do not enjoy doing these studies since the work is tedious. I prefer to plunge right into the design. However, I also learned the crucial necessity of doing these statistical studies. The end result of our study was not what we thought it would be. If I had created training based on my personal assumptions or anecdotal evidence, I would have created something divorced from reality.