READING ASSIGNMENT
Title of the Article : Evaluation of Biomedical Text-mining Systems : Lessons Learned from Information Retrieval
Author : Hersh, William MD
Publisher : Henry Stewart Publications
URL : http://search.ebscohost.com. accessed 02 June 2008.
MEDLINE DATABASE
ABSTRACT
The paper reviewed the state of the science in the evaluation of text-mining systems, with a particular focus on the lessons learned from similar work done in the more mature area of information retrieval (IR). The rest of the paper then provided an overview of the measures used in IR; a review of the largest IR evaluation initiative, the text Retrieval Conference (TREC); and a critique of current evaluation methods in biomedical text-mining.
THREE THINGS I LEARNED FROM MY READING ASSIGNMENT
· Information retrieval system evaluation is necessary prior to application because this determines if techniques found effective by system-oriented evaluation will translate into a user’s ability to better perform information tasks.
· The most frequently employed performance measures of IR are recall and precision. Along these criteria, my interest was ticked by the concept of the “gold standard” which focuses on the relevance of retrieved documents. Sad to note however that there is evidence suggesting that differences in recall and precision do not affect performance of real-world tasks among users because of varying knowledge, underlying skills and cognitive abilities.
· I have encountered the Text Retrieval Conference (TREC) which is designed to serve as a means to increase communication among academic, industrial and governmental IR researchers. This initiative highlights the value of collaboration in the information field for a more successful IR.
IMPLICATIONS OF THE NEW THINGS I LEARNED TO MY WORK
It is undeniable that as an information professional, I cater to different users with differing needs,skills, and expectations. This entails careful evaluation before implementing or introducing a new IRS to the library, be it developed in-house (e.g. OPAC, database for literature holdings etc.) or from an outside source (commercialized LIS). User-friendliness, appropriateness and efficiency should be the foremost considerations. The evaluation process should also not just be focused on the technical soundness of the system but much more on its end-user compatibility and benefits. External input and insights from colleagues will also be a plus and a strategic move to better apply technology in information retrieval.
Read. SSE