Skip to main navigation Skip to search Skip to main content

Block-level links based content extraction

  • Shixing Shen*
  • , Hui Zhang
  • *Corresponding author for this work
  • Beihang University

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

We present block-level links based content extraction (BLCE) - a method to extract content from the web pages by using the link attributes of blocks, which contains the number of links and the length of link text (anchor text). We describe how to divide one webpage into blocks and how to merge the similar blocks into one, then compute the number of links and the total length of anchor text. We find that extracting content only with the number of links and length of anchor text is not effective because the number of links and length of link text are proportional to the length of page. Density of links is a good method to solve this. So we use the content links ratios and the content anchor text ratios to describe the link attribute of the blocks. BLCE performs better than other methods especially in the new web pages with DIV and CSS where traditional algorithm can't work well.

Original languageEnglish
Title of host publicationProceedings - 2011 4th International Symposium on Parallel Architectures, Algorithms and Programming, PAAP 2011
Pages330-333
Number of pages4
DOIs
StatePublished - 2011
Event2011 4th International Symposium on Parallel Architectures, Algorithms and Programming, PAAP 2011 - Tianjin, China
Duration: 9 Dec 201111 Dec 2011

Publication series

NameProceedings - 2011 4th International Symposium on Parallel Architectures, Algorithms and Programming, PAAP 2011

Conference

Conference2011 4th International Symposium on Parallel Architectures, Algorithms and Programming, PAAP 2011
Country/TerritoryChina
CityTianjin
Period9/12/1111/12/11

Keywords

  • Block-level links
  • Content extraction
  • Merge block

Fingerprint

Dive into the research topics of 'Block-level links based content extraction'. Together they form a unique fingerprint.

Cite this