An Approach for Web Page Change Detection using Segmentation

Abstract

On the basis of changes in web pages, web crawler tends to find, index and re-index the information. Change detection is an activity that aims to identify a particular set of changes according to user specification. The foremost challenge for the web crawler is to detect changes with minimal usage of resources. Continual query and diff algorithm tends to improve the methods for change detection in web crawler. Mild modification in the architecture of web crawler in conjunction with segmentation can enhance its functionality. The main objective of using segmentation in proposed tool is that it offers the customization in change detection technique by selecting particular region of the web page. In this thesis, a tool named “CFinder” has been proposed, which use diff algorithm in combination with segmentation to identify navigational, structural, and content changes. The proposed tool merged the functionality of generalized with personalized change detection.