Automatic topic detection with an incremental clustering algorithm

Xiaoming Zhang; Zhoujun Li

doi:10.1007/978-3-642-16515-3_43

Automatic topic detection with an incremental clustering algorithm

Xiaoming Zhang^*
, Zhoujun Li

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

At present, most of the topic detection approaches are not accurate and efficient enough. In this paper, we proposed a new topic detection method (TPIC) based on an incremental clustering algorithm. It employs a self-refinement process of discriminative feature identification and a term reweighting algorithm to accurately cluster the given documents which discuss the same topic. To be efficient, the "aging" nature of topics is used to precluster stories. To automatically detect the true number of topics, Bayesian Information Criterion (BIC) is used to estimate the true number of topics. Experimental results on Linguistic Data Consortium (LDC) datasets TDT4 show that the proposed method can improve both the efficiency and accuracy, compared to other methods.

Original language	English
Title of host publication	Web Information Systems and Mining - International Conference, WISM 2010, Proceedings
Pages	344-351
Number of pages	8
Edition	M4D
DOIs	https://doi.org/10.1007/978-3-642-16515-3_43
State	Published - 2010
Event	2010 International Conference on Web Information Systems and Mining, WISM 2010 - Sanya, China Duration: 23 Oct 2010 → 24 Oct 2010

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number	M4D
Volume	6318 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	2010 International Conference on Web Information Systems and Mining, WISM 2010
Country/Territory	China
City	Sanya
Period	23/10/10 → 24/10/10

Keywords

Incremental clustering
TDT
Term reweighting
Topic Detection

Access to Document

10.1007/978-3-642-16515-3_43

Cite this

Zhang, X., & Li, Z. (2010). Automatic topic detection with an incremental clustering algorithm. In Web Information Systems and Mining - International Conference, WISM 2010, Proceedings (M4D ed., pp. 344-351). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6318 LNCS, No. M4D). https://doi.org/10.1007/978-3-642-16515-3_43

@inproceedings{f967b8d9fcd943179d856e2a7bd32e72,

title = "Automatic topic detection with an incremental clustering algorithm",

abstract = "At present, most of the topic detection approaches are not accurate and efficient enough. In this paper, we proposed a new topic detection method (TPIC) based on an incremental clustering algorithm. It employs a self-refinement process of discriminative feature identification and a term reweighting algorithm to accurately cluster the given documents which discuss the same topic. To be efficient, the {"}aging{"} nature of topics is used to precluster stories. To automatically detect the true number of topics, Bayesian Information Criterion (BIC) is used to estimate the true number of topics. Experimental results on Linguistic Data Consortium (LDC) datasets TDT4 show that the proposed method can improve both the efficiency and accuracy, compared to other methods.",

keywords = "Incremental clustering, TDT, Term reweighting, Topic Detection",

author = "Xiaoming Zhang and Zhoujun Li",

year = "2010",

doi = "10.1007/978-3-642-16515-3\_43",

language = "英语",

isbn = "3642165141",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

number = "M4D",

pages = "344--351",

booktitle = "Web Information Systems and Mining - International Conference, WISM 2010, Proceedings",

edition = "M4D",

note = "2010 International Conference on Web Information Systems and Mining, WISM 2010 ; Conference date: 23-10-2010 Through 24-10-2010",

}

Zhang, X & Li, Z 2010, Automatic topic detection with an incremental clustering algorithm. in Web Information Systems and Mining - International Conference, WISM 2010, Proceedings. M4D edn, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. M4D, vol. 6318 LNCS, pp. 344-351, 2010 International Conference on Web Information Systems and Mining, WISM 2010, Sanya, China, 23/10/10. https://doi.org/10.1007/978-3-642-16515-3_43

Automatic topic detection with an incremental clustering algorithm. / Zhang, Xiaoming ; Li, Zhoujun.
Web Information Systems and Mining - International Conference, WISM 2010, Proceedings. M4D. ed. 2010. p. 344-351 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6318 LNCS, No. M4D).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Automatic topic detection with an incremental clustering algorithm

AU - Zhang, Xiaoming

AU - Li, Zhoujun

PY - 2010

Y1 - 2010

N2 - At present, most of the topic detection approaches are not accurate and efficient enough. In this paper, we proposed a new topic detection method (TPIC) based on an incremental clustering algorithm. It employs a self-refinement process of discriminative feature identification and a term reweighting algorithm to accurately cluster the given documents which discuss the same topic. To be efficient, the "aging" nature of topics is used to precluster stories. To automatically detect the true number of topics, Bayesian Information Criterion (BIC) is used to estimate the true number of topics. Experimental results on Linguistic Data Consortium (LDC) datasets TDT4 show that the proposed method can improve both the efficiency and accuracy, compared to other methods.

AB - At present, most of the topic detection approaches are not accurate and efficient enough. In this paper, we proposed a new topic detection method (TPIC) based on an incremental clustering algorithm. It employs a self-refinement process of discriminative feature identification and a term reweighting algorithm to accurately cluster the given documents which discuss the same topic. To be efficient, the "aging" nature of topics is used to precluster stories. To automatically detect the true number of topics, Bayesian Information Criterion (BIC) is used to estimate the true number of topics. Experimental results on Linguistic Data Consortium (LDC) datasets TDT4 show that the proposed method can improve both the efficiency and accuracy, compared to other methods.

KW - Incremental clustering

KW - TDT

KW - Term reweighting

KW - Topic Detection

UR - https://www.scopus.com/pages/publications/78649526450

U2 - 10.1007/978-3-642-16515-3_43

DO - 10.1007/978-3-642-16515-3_43

M3 - 会议稿件

AN - SCOPUS:78649526450

SN - 3642165141

SN - 9783642165146

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 344

EP - 351

BT - Web Information Systems and Mining - International Conference, WISM 2010, Proceedings

T2 - 2010 International Conference on Web Information Systems and Mining, WISM 2010

Y2 - 23 October 2010 through 24 October 2010

ER -

Zhang X , Li Z. Automatic topic detection with an incremental clustering algorithm. In Web Information Systems and Mining - International Conference, WISM 2010, Proceedings. M4D ed. 2010. p. 344-351. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); M4D). doi: 10.1007/978-3-642-16515-3_43

Automatic topic detection with an incremental clustering algorithm

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this