基于爬蟲的小企業(yè)搜索系統(tǒng)的設計與實現(xiàn)
發(fā)布時間：2019-01-10 11:14
【摘要】：隨著互聯(lián)網的不斷發(fā)展,網絡已逐步成為眾人獲取信息的一種主要手段,而且目前一些門戶網站的信息內容正以驚人的速度增加著。面對如此大量的信息,如何更全面、更準確地獲取最新、最有效的信息已經成為我們把握機遇迎接挑戰(zhàn)的首要條件。針對一些門戶網站,目前傳統(tǒng)的WEB搜索引擎具有更新慢,準確度低等幾個缺點,為了提高針對一些網站的信息檢索能力,本課題研究設計了一套基于爬蟲技術的站內搜索引擎系統(tǒng)。 本論文首先簡要介紹和分析基于爬蟲技術的搜索引擎系統(tǒng)的研究背景、國內外現(xiàn)有的搜索技術。接著,進行需求分析,提出系統(tǒng)應具備實時性和高準確性等特點,對此提出系統(tǒng)的總體設計框架、模塊劃分,以及模塊相關介紹。 本系統(tǒng)采用Maven進行項目管理,選用Velocity模板技術實現(xiàn)網絡機器人,基于Compass和中文分詞技術,采用Service模式設計搜索框架以及采用Webwork、 Spring等J2EE技術,采用MVC模式、Command模式等以及多種RPC技術等實現(xiàn)多種搜索接口。該系統(tǒng)可為企業(yè)提供通用的垂直搜索服務,具備實時性、通用性等特點,并可通過多種搜索接口可以很方便的與企業(yè)應用集成。
[Abstract]:With the continuous development of the Internet, the Internet has gradually become a major means for people to obtain information, and at present, the information content of some web portals is increasing at an alarming rate. In the face of so much information, how to obtain the latest and most effective information more comprehensively and accurately has become the most important condition for us to seize the opportunity to meet the challenge. The traditional WEB search engine has several shortcomings such as slow update, low accuracy and so on. In order to improve the ability of information retrieval for some websites, In this paper, a search engine system based on crawler technology is designed. Firstly, this paper briefly introduces and analyzes the research background of search engine system based on crawler technology, and the existing search technology at home and abroad. Then, the requirement analysis is carried out, and the characteristics of real-time and high accuracy are put forward. The overall design framework, module partition and module related introduction of the system are put forward. In this system, Maven is used for project management, Velocity template technology is used to realize network robot, based on Compass and Chinese word segmentation technology, Service pattern is used to design search framework, J2EE technology such as Webwork, Spring is adopted, and MVC mode is adopted. Command mode and various RPC technologies to achieve a variety of search interfaces. The system can provide a general vertical search service for enterprises, with the characteristics of real-time, versatility, and can be easily integrated with enterprise applications through a variety of search interfaces.
【學位授予單位】：大連理工大學
【學位級別】：碩士
【學位授予年份】：2012
【分類號】：TP391.3
本文編號：2406239
[Abstract]:With the continuous development of the Internet, the Internet has gradually become a major means for people to obtain information, and at present, the information content of some web portals is increasing at an alarming rate. In the face of so much information, how to obtain the latest and most effective information more comprehensively and accurately has become the most important condition for us to seize the opportunity to meet the challenge. The traditional WEB search engine has several shortcomings such as slow update, low accuracy and so on. In order to improve the ability of information retrieval for some websites, In this paper, a search engine system based on crawler technology is designed. Firstly, this paper briefly introduces and analyzes the research background of search engine system based on crawler technology, and the existing search technology at home and abroad. Then, the requirement analysis is carried out, and the characteristics of real-time and high accuracy are put forward. The overall design framework, module partition and module related introduction of the system are put forward. In this system, Maven is used for project management, Velocity template technology is used to realize network robot, based on Compass and Chinese word segmentation technology, Service pattern is used to design search framework, J2EE technology such as Webwork, Spring is adopted, and MVC mode is adopted. Command mode and various RPC technologies to achieve a variety of search interfaces. The system can provide a general vertical search service for enterprises, with the characteristics of real-time, versatility, and can be easily integrated with enterprise applications through a variety of search interfaces.
【學位授予單位】：大連理工大學
【學位級別】：碩士
【學位授予年份】：2012
【分類號】：TP391.3
【參考文獻】
相關期刊論文 前10條
1 楊樹林;胡潔萍;;JSON數(shù)據交換格式及其在數(shù)據驗證中的應用[J];北京印刷學院學報;2008年04期
2 張盼;聶剛;;基于Lucene的全文檢索系統(tǒng)的設計與實現(xiàn)[J];電腦知識與技術;2010年01期
3 徐立新;雷相波;;應用Maven管理項目[J];電腦知識與技術;2010年10期
4 徐宏發(fā);王衛(wèi)平;;基于SOA的數(shù)字圖書館互操作開放框架[J];計算機工程與應用;2006年34期
5 陸榮幸,郁洲,阮永良,王志強;J2EE平臺上MVC設計模式的研究與實現(xiàn)[J];計算機應用研究;2003年03期
6 俞華鋒;;Memcached在大型網站中的應用[J];科技信息(科學教研);2008年01期
7 李蕾,王楠,鐘義信,郭祥昊,韓鵬,賈自燕,高清霞;基于語義網絡的概念檢索研究與實現(xiàn)[J];情報學報;2000年05期
8 姜強;;SOA的規(guī)劃與設計[J];軟件導刊;2010年11期
9 趙國棟;;SOA觀點 SOA，，重在實踐[J];信息系統(tǒng)工程;2006年08期
10 劉純波,李琦,承繼成;基于XML-RPC的分布式地理信息系統(tǒng)計算模型[J];中國圖象圖形學報;2003年06期
本文編號：2406239
本文鏈接：http://www.sikaile.net/kejilunwen/sousuoyinqinglunwen/2406239.html
