本文關(guān)鍵詞： 異或算法 Hash函數(shù) 中文詞庫(kù) 自適應(yīng)字長(zhǎng)　出處：《計(jì)算機(jī)研究與發(fā)展》2011年S1期 　論文類(lèi)型：期刊論文

【摘要】：中文搜索引擎中的詞庫(kù)是提高文本信息存儲(chǔ)與查找效率的關(guān)鍵.以異或Hash算法為基礎(chǔ),根據(jù)對(duì)不同字長(zhǎng)詞出現(xiàn)概率的統(tǒng)計(jì)結(jié)果,利用詞條的機(jī)內(nèi)編碼和漢字筆畫(huà)數(shù),把不同字長(zhǎng)的詞散列到不同的Hash值區(qū)間.實(shí)驗(yàn)結(jié)果表明該算法可以將Hash值的沖突率降低到0.327‰,該方法可用于計(jì)算機(jī)語(yǔ)料庫(kù)建設(shè)和中文輸入法等自然語(yǔ)言處理過(guò)程.
[Abstract]:The lexicon in Chinese search engine is the key to improve the efficiency of text information storage and search. Based on the XOR or Hash algorithm, according to the statistical results of the occurrence probability of different word length words. By using the in-machine coding of entries and the number of strokes in Chinese characters, different word lengths are hashed to different intervals of Hash values. The experimental results show that the collision rate of Hash values can be reduced to 0.327 鈥，

本文編號(hào)：1491533