A comparative evaluation of searching by controlled language and natural language in experimental N.A.S.A. data base

An evaluation test was made of an experimental data-base prepared by the Space Documentation Service of the European Space Agency, consisting of some 44,000 items from NASA STAR for 1973 and 1974. With this data-base it was possible to search on natural language terms in the titles and abstracts, in addition to the normal searches on controlled language index terms. The on-line searches were carried out at four centres, each centre being responsible for ten questions, with two searches in the alternative search modes being made by different people for each question. Up to twenty-five documents retrieved in the two searches for each question were sent to the originator of the question for relevance assessment. The results are presented in a number of different ways, but in every case the natural language searches showed a significantly higher recall ratio than the controlled language, with little difference in the precision ratios. It is suggested that the main reason for the superiority of natural language searching is the greater exhaustivity of the abstracts as compared to the indexing.