Á¤º¸°úÇÐȸ³í¹®Áö (Journal of KIISE)
ÇѱÛÁ¦¸ñ(Korean Title) |
BERT ÀÓº£µù°ú ¼±ÅÃÀû OOV º¹»ç ¹æ¹ýÀ» »ç¿ëÇÑ ¹®¼¿ä¾à |
¿µ¹®Á¦¸ñ(English Title) |
Automatic Text Summarization Based on Selective OOV Copy Mechanism with BERT Embedding |
ÀúÀÚ(Author) |
ÀÌż®
°½Â½Ä
Tae-Seok Lee
Seung-Shik Kang
|
¿ø¹®¼ö·Ïó(Citation) |
VOL 47 NO. 01 PP. 0036 ~ 0044 (2020. 01) |
Çѱ۳»¿ë (Korean Abstract) |
¹®¼ ÀÚµ¿ ¿ä¾àÀº ÁÖ¾îÁø ¹®¼·ÎºÎÅÍ ÁÖ¿ä ³»¿ëÀ» ÃßÃâÇϰųª »ý¼ºÇÏ´Â ¹æ½ÄÀ¸·Î ª°Ô ÁÙÀÌ´Â ÀÛ¾÷ÀÌ´Ù. »ý¼º ¿ä¾àÀº ¹Ì¸® »ý¼ºµÈ ¿öµå ÀÓº£µù Á¤º¸¸¦ »ç¿ëÇÑ´Ù. ÇÏÁö¸¸, Àü¹® ¿ë¾î¿Í °°ÀÌ Àúºóµµ ÇÙ½É ¾îÈÖ´Â ÀÓº£µù »çÀü¿¡¼ ´©¶ôµÇ´Â ¹®Á¦°¡ ¹ß»ýÇÑ´Ù. ¹®¼ ÀÚµ¿ ¿ä¾à¿¡¼ ¹Ìµî·Ï ¾îÈÖÀÇ ÃâÇöÀº ¿ä¾à ¼º´ÉÀ» ÀúÇϽÃŲ´Ù. º» ³í¹®Àº Selectively Pointing OOV(Out of Vocabulary) ¸ðµ¨¿¡ BERT(Bidirectional Encoder Representations from Transformers) ÇüÅÂ¼Ò ÀÓº£µù, Masked OOV, ÇüżÒ-to-¹®Àå º¯È¯±â¸¦ Àû¿ëÇÏ¿© ¹Ìµî·Ï ¾îÈÖ¿¡ ´ëÇÑ ¼±ÅÃÀû º¹»ç ¹× ¿ä¾à ¼º´ÉÀ» ³ô¿´´Ù. ±âÁ¸ ¿¬±¸¿Í ´Þ¸® Á¤È®ÇÑ Æ÷ÀÎÆà Á¤º¸¿Í ¼±ÅÃÀû º¹»ç Áö½Ã Á¤º¸¸¦ ¸í½ÃÀûÀ¸·Î Á¦°øÇÏ´Â ¼±ÅÃÀû OOV Æ÷ÀÎÆà º¹»ç ¹æ¹ý°ú ÇÔ²² BERT ÀÓº£µù°ú OOV ·£´ý ¸¶½ºÅ·, ÇüżÒ-¹®Àå º¯È¯±â¸¦ Ãß°¡ÇÏ¿´´Ù. Á¦¾ÈÇÑ OOV ¸ðµ¨À» ÅëÇؼ ÀÚµ¿ »ý¼º ¿ä¾àÀ» ¼öÇàÇÑ °á°ú ´Ü¾î ÀçÇö ±â¹ÝÀÇ ROUGE-1ÀÌ 54.97 ³ªÅ¸³µÀ¸¸ç, ¶ÇÇÑ ¾î¼ø ±â¹ÝÀÇ ROUGE-LÀÌ 39.23À¸·Î Çâ»óµÇ¾ú´Ù.
|
¿µ¹®³»¿ë (English Abstract) |
Automatic text summarization is a process of shortening a text document via extraction or abstraction. Abstractive text summarization involves using pre-generated word embedding information. Low-frequency but salient words such as terminologies are seldom included in dictionaries, that are so called, out-of-vocabulary (OOV) problems. OOV deteriorates the performance of the encoder-decoder model in the neural network. To address OOV words in abstractive text summarization, we propose a copy mechanism to facilitate copying new words in the target document and generating summary sentences. Different from previous studies, the proposed approach combines accurately pointing information, selective copy mechanism, embedded by BERT, randomly masking OOV, and converting sentences from morpheme. Additionally, the neural network gate model to estimate the generation probability and the loss function to optimize the entire abstraction model was applied. Experimental results demonstrate that ROUGE-1 (based on word recall) and ROUGE-L (longest used common subsequence) of the proposed encoding-decoding model have been improved at 54.97 and 39.23, respectively.
|
Å°¿öµå(Keyword) |
BERT
OOV ·£´ý ¸¶½ºÅ·
ÇüżÒ-¹®Àå º¯È¯±â
¹®¼¿ä¾à
¹Ìµî·Ï ´Ü¾î ÀνÄ
µö·¯´×
»ý¼º¿ä¾à
random masked OOV
text summarization
morpheme-to-sentence converter
recognition of unknown word
deep-learning
generative summarization
|
ÆÄÀÏ÷ºÎ |
PDF ´Ù¿î·Îµå
|