I just completed day 1 of an NREMT item writing workshop at NREMT headquarters in Columbus, Ohio. So far it has been a great experience and I have learned a lot about the process of constructing test items.
Item Writer Selection
The NREMT regularly invites EMS educators to apply to be item writers. I submitted a resume and cover letter asking to participate. You can get more information about being an item writer by contacting the registry or speaking with them at a conference. When I was selected I was asked which level I would like to participate. I requested EMT Basic and a preferred date.
Workshop Preparation
About 6 weeks before the workshop I received my item writing assignment. I, along with the other item writers in my cohort, was asked to writer 25 items. We were asked to construct items in specific categories, such as cardiac, and in specific areas of the task analysis. For example, I was asked to construct a cardiac item that was related to pathophysiology of an acute coronary syndrome.
The preparation assignment also included reading a large packet of material about the registry test and constructing test items. It was very useful reading.
I was a bit anxious about writing the items because I knew they would be displayed to a group of my peers and together we would tear down and rebuild the items.
Workshop Day One
This morning after about an hour of logistics and introduction we got right into the business of reviewing and rebuilding the questions each group member submitted. Our questions were displayed, one at a time, on the conference room screen. Together we worked to refine and improve the question. Many times the final question looked nothing like the initial question. Since we all wrote questions for the same areas of the EMT-Basic task analysis there were some very similar questions for a few topics, such as cardiac rhythm interpretation (there is a pretty small slice about this topic an EMT-Basic is expected to know).
Lessons Learned … so far
These are some of my lessons learned so far that I think are valuable for students and educators ….
1. Students, it is well worth your time to actually read your textbook.
2. Knowing and being able to apply the basic life support algorithm is the essence of being an EMT.
3. Group review can make a bad item better. Group review can make a bad item worse. And group review can make a good item worse.
4. Items are constructed from a diverse group of field professionals and educators from across the country.
5. Very little, if any, extraneous information in an item survives the group editing process.
6. It is impossible to know if any of the items we are constructing will ever make it as actual scored registry questions. Every item will be pilot-tested by hundreds of actual registry test takers. After pilot testing the item will be analyzed and it will only be added to the live pool if it meets specific criteria.
//
