Okay, I haven’t been posting my journals lately, which I’m sure you’ve all been missing, but here’s a new one to keep you happy. 🙂
For this one, I need to search the Internet using the phrase “data collection techniques” and find websites that discussed different types of information gathering: views from people, direct observation of behavior or testing, and physical records.
So here are the sources I found:
First up, MBA-Lectures.com. According to them, the observation method “gives the opportunity to record..behavior directly.” They do point out that observation might be more time consuming and costly than other methods, plus it is often subjective because the observer can include their own biases in the results.
Next, OCHA Disaster Response Preparedness Toolkit. Strangely appropriate since it’s Zombie Awareness Month. They also discuss observation, but for them, it’s observation of a situation. They do, however, have a lot of information about interviewing to gather information. Because they are looking to gather information from a number of people about their needs, this is actually helpful because I would like to be able to interview faculty to get their views. In something I take as being a good way to get qualitative data, they suggest beginning an interview with general conversation about “life in the area, and about their experiences during the disaster.” For me, this would be replaced by general conversation about teaching, grading, and assessing students, and then moving onto the actual assignment they gave their students and how they graded and assessed it.
They do also look at physical records. For their purposes, they look at calendars. These can help them know what was going on during a given time period; helpful for them since they are looking to get life back to normal, and knowing what was “normal” for that date can help them have a goal to work towards. Obviously, this isn’t directly related to what I’ll be looking at, but I wanted to mention it because to me it tells me that I should consider what kinds of records I want to look at. Should I find a way to look at past records of grades? Perhaps I should include a question in the qualitative section of the study that looks at whether or not faculty members who follow a rubric feel that it is fair and if they graded their students based on what they felt was right as opposed to what the rubric said was? (I say this because there have been times that I felt a student got too high or low a grade based on a rubric…)
Next up: University of Florida Cooperative Extension Service. This is a bit old – 1992! – what I found from them is actually quite helpful. It’s about “selecting a data collection technique.” It’s really about survey instruments, and it discusses reliability and validity. I suppose I didn’t have to bring it up here at all, but I wanted to keep it for future reference.
Northern Arizona University also had a great website up. For “Methods of Data Collection,” they include a list of methods and examples, including documents, observations, surveys, experimental, other field methods (nominal groups, technique, and Delphi), and multimethods approaches. It let me know that I’m definitely interested in doing a mixed-methods study: quantitative, gathering information from surveys about a grading/assessment experiment, and qualitative, doing interviews of faculty members to understand the whys as well as the whats.
Finally, another surprising source: the Center for Rural Studies. Their website is all about gathering information about an area and a group of people. Again, lots of discussion about surveying, observing, and gathering data from collected sources (such as census data, vital statistics, etc.)
So, found me some sources! What I’m supposed to do now is:
Critically analyze the methods you are considering for your draft proposal and develop the specific details of these methods.
Well, right now, I am going to ask faculty members to use a particular assignment in order to measure course outcomes (that are already in place). I will create a rubric to assess the assignment based on those course outcomes using Bloom’s Taxonomy. I will give the faculty members training on how to apply the rubric. The faculty members will then grade the assignment as they normally would, using letter grades. Once they finished with the traditional grading, they would then assess the assignment according to the rubric I had given them.
Then I will gather data, finding out what letters grades and what assessment numbers were assigned. This data can be compared, looking to see if the letter grades reflect the course outcomes achieved (or that were failed to be achieved).
After analyzing the data, I can meet with the faculty members and interview them, asking for information about how they grade and what they think about the grades they assigned versus the outcomes they assessed.
At this point, I’m not sure what I can do to make this better. I’m open to suggestions, but I like to think I’m on track with it.