{"id":3835,"date":"2020-12-28T21:34:12","date_gmt":"2020-12-29T02:34:12","guid":{"rendered":"http:\/\/skimai.com\/?p=3835"},"modified":"2024-05-20T07:38:31","modified_gmt":"2024-05-20T12:38:31","slug":"tutoriel-sur-la-facon-de-regler-finement-bert-pour-le-resume","status":"publish","type":"post","link":"https:\/\/skimai.com\/fr\/tutorial-how-to-fine-tune-bert-for-summarization\/","title":{"rendered":"Tutoriel : Comment affiner le BERT pour la synth\u00e8se extractive"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_1 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/skimai.com\/fr\/tutorial-how-to-fine-tune-bert-for-summarization\/#Tutorial_How_to_Fine-Tune_BERT_for_Extractive_Summarization\" >Tutorial: How to Fine-Tune BERT for Extractive Summarization<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/skimai.com\/fr\/tutorial-how-to-fine-tune-bert-for-summarization\/#1_Introduction\" >1. Introduction<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/skimai.com\/fr\/tutorial-how-to-fine-tune-bert-for-summarization\/#2_Extractive_Summarization\" >2. Extractive Summarization<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/skimai.com\/fr\/tutorial-how-to-fine-tune-bert-for-summarization\/#BERT_Encoder\" >BERT Encoder<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/skimai.com\/fr\/tutorial-how-to-fine-tune-bert-for-summarization\/#Summarization_Classifier\" >Summarization Classifier<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/skimai.com\/fr\/tutorial-how-to-fine-tune-bert-for-summarization\/#3_Make_Summarization_Even_Faster\" >3. Make Summarization Even Faster<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/skimai.com\/fr\/tutorial-how-to-fine-tune-bert-for-summarization\/#4_Lets_Summarize\" >4. Let&#8217;s Summarize<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/skimai.com\/fr\/tutorial-how-to-fine-tune-bert-for-summarization\/#5_Conclusion\" >5. Conclusion<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/skimai.com\/fr\/tutorial-how-to-fine-tune-bert-for-summarization\/#References\" >References<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h1><span class=\"ez-toc-section\" id=\"Tutorial_How_to_Fine-Tune_BERT_for_Extractive_Summarization\"><\/span>Tutorial: How to Fine-Tune BERT for Extractive Summarization<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<pre><code>    Originally published by Skim AI's Machine Learning Researcher, Chris Tran<\/code><\/pre>\n<p><a href=\"https:\/\/github.com\/chriskhanhtran\/bert-extractive-summarization\"><img decoding=\"async\" src=\"https:\/\/img.shields.io\/badge\/GitHub-View_on_GitHub-blue?logo=GitHub\" alt=\"\" \/><\/a><\/p>\n<h2><span class=\"ez-toc-section\" id=\"1_Introduction\"><\/span>1. Introduction<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Summarization has long been a challenge in Natural Language Processing. To generate a short version of a document while retaining its most important information, we need a model capable of accurately extracting the key points while avoiding repetitive information. Fortunately, recent works in NLP such as Transformer models and language model pretraining have advanced the state-of-the-art in summarization.<\/p>\n<p>In this article, we will explore BERTSUM, a simple variant of BERT, for extractive summarization from <a href=\"https:\/\/arxiv.org\/abs\/1908.08345\">Text Summarization with Pretrained Encoders<\/a> (Liu et al., 2019). Then, in an effort to make extractive summarization even faster and smaller for low-resource devices, we will fine-tune DistilBERT (<a href=\"https:\/\/arxiv.org\/abs\/1910.01108\">Sanh et al., 2019<\/a>) and MobileBERT (<a href=\"https:\/\/arxiv.org\/abs\/2004.02984\">Sun et al., 2019<\/a>), two recent lite versions of BERT, and discuss our findings.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"2_Extractive_Summarization\"><\/span>2. Extractive Summarization<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>There are two types of summarization: <em>abstractive<\/em> and <em>extractive summarization<\/em>. Abstractive summarization basically means rewriting key points while extractive summarization generates summary by copying directly the most important spans\/sentences from a document.<\/p>\n<p>Abstractive summarization is more challenging for humans, and also more computationally expensive for machines. However, which summaration is better depends on the purpose of the end user. If you were writing an essay, abstractive summaration might be a better choice. On the other hand, if you were doing some research and needed to get a quick summary of what you were reading, extractive summarization would be more helpful for the task.<\/p>\n<p>In this section we will explore the architecture of our extractive summarization model. The BERT summarizer has 2 parts: a BERT encoder and a summarization classifier.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"BERT_Encoder\"><\/span>BERT Encoder<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><img decoding=\"async\" src=\"https:\/\/github.com\/chriskhanhtran\/minimal-portfolio\/blob\/master\/images\/bertsum.jpeg?raw=true\" alt=\"\" \/><\/p>\n<p><em>The overview architecture of BERTSUM<\/em><\/p>\n<p>Our BERT encoder is the pretrained BERT-base encoder from the masked language modeling task (<a href=\"https:\/\/github.com\/google-research\/bert\">Devlin et at., 2018<\/a>). The task of extractive summarization is a binary classification problem at the sentence level. We want to assign each sentence a label $y_i in {0, 1}$ indicating whether the sentence should be included in the final summary. Therefore, we need to add a token <code>[CLS]<\/code> before each sentence. After we run a forward pass through the encoder, the last hidden layer of these <code>[CLS]<\/code> tokens will be used as the representions for our sentences.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Summarization_Classifier\"><\/span>Summarization Classifier<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>After getting the vector representation of each sentence, we can use a simple feed forward layer as our classifier to return a score for each sentence. In the paper, the author experimented with a simple linear classifier, a Recurrent Neural Network and a small Transformer model with 3 layers. The Transformer classifier yields the best results, showing that inter-sentence interactions through self-attention mechanism is important in selecting the most important sentences.<\/p>\n<p>So in the encoder, we learn the interactions among tokens in our document while in the summarization classifier, we learn the interactions among sentences.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"3_Make_Summarization_Even_Faster\"><\/span>3. Make Summarization Even Faster<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Transformer models achieve state-of-the-art performance in most NLP bechmarks; however, training and making predictions from them are computationally expensive. In an effort to make summarization lighter and faster to be deployed on low-resource devices, I have modified the <a href=\"https:\/\/github.com\/nlpyang\/PreSumm\">source codes<\/a> provided by the authors of BERTSUM to replace the BERT encoder with DistilBERT and MobileBERT. The summary layers are kept unchaged.<\/p>\n<p>Here are training losses of these 3 variants: <a href=\"https:\/\/tensorboard.dev\/experiment\/Ly7CRURRSOuPBlZADaqBlQ\/#scalars\">TensorBoard<\/a><\/p>\n<p><img decoding=\"async\" src=\"https:\/\/github.com\/chriskhanhtran\/bert-extractive-summarization\/raw\/master\/tensorboard.JPG\" alt=\"\" \/><\/p>\n<p>Despite being 40% smaller than BERT-base, DistilBERT has the same training losses as BERT-base while MobileBERT performs slightly worse. The table below shows their performance on CNN\/DailyMail dataset, size and running time of a forward pass:<\/p>\n<table>\n<thead>\n<tr>\n<th align=\"left\">Models<\/th>\n<th align=\"center\">ROUGE-1<\/th>\n<th align=\"center\">ROUGE-2<\/th>\n<th align=\"center\">ROUGE-L<\/th>\n<th align=\"center\">Inference Time*<\/th>\n<th align=\"center\">Size<\/th>\n<th align=\"center\">Params<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td align=\"left\">bert-base<\/td>\n<td align=\"center\">43.23<\/td>\n<td align=\"center\">20.24<\/td>\n<td align=\"center\">39.63<\/td>\n<td align=\"center\">1.65 s<\/td>\n<td align=\"center\">475 MB<\/td>\n<td align=\"center\">120.5 M<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">distilbert<\/td>\n<td align=\"center\">42.84<\/td>\n<td align=\"center\">20.04<\/td>\n<td align=\"center\">39.31<\/td>\n<td align=\"center\">925 ms<\/td>\n<td align=\"center\">310 MB<\/td>\n<td align=\"center\">77.4 M<\/td>\n<\/tr>\n<tr>\n<td align=\"left\">mobilebert<\/td>\n<td align=\"center\">40.59<\/td>\n<td align=\"center\">17.98<\/td>\n<td align=\"center\">36.99<\/td>\n<td align=\"center\">609 ms<\/td>\n<td align=\"center\">128 MB<\/td>\n<td align=\"center\">30.8 M<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>*<em>Average running time of a forward pass on a single GPU on a standard Google Colab notebook<\/em><\/p>\n<p>Being 45% faster, DistilBERT have almost the same performance as BERT-base. MobileBERT retains 94% performance of BERT-base, while being 4x smaller than BERT-base and 2.5x smaller than DistilBERT. In the MobileBERT paper, it&#8217;s shown that MobileBERT significantly outperforms DistilBERT on SQuAD v1.1. However, it&#8217;s not the case for extractive summarization. But this is still an impressive result for MobileBERT with a disk size of only 128 MB.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"4_Lets_Summarize\"><\/span>4. Let&#8217;s Summarize<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>All pretrained checkpoints, training details and setup instruction can be found in <a href=\"https:\/\/github.com\/chriskhanhtran\/bert-extractive-summarization\/\">this GitHub repository<\/a>. In addition, I have deployed a demo of BERTSUM with the MobileBERT encoder.<\/p>\n<p><strong>Web app:<\/strong> <a href=\"https:\/\/extractive-summarization.herokuapp.com\/\"><a href=\"https:\/\/extractive-summarization.herokuapp.com\/\"><a href=\"https:\/\/extractive-summarization.herokuapp.com\/\"><a href=\"https:\/\/extractive-summarization.herokuapp.com\/\">https:\/\/extractive-summarization.herokuapp.com\/<\/a><\/a><\/a><\/a><\/p>\n<p><a href=\"https:\/\/extractive-summarization.herokuapp.com\/\"><img decoding=\"async\" src=\"https:\/\/img.shields.io\/badge\/Heroku-Open_Web_App-blue?logo=Heroku\" alt=\"\" \/><\/a><\/p>\n<p><img decoding=\"async\" src=\"https:\/\/github.com\/chriskhanhtran\/minimal-portfolio\/blob\/master\/images\/bertsum.gif?raw=true\" alt=\"\" \/><\/p>\n<p><strong>Code:<\/strong><\/p>\n<p><a href=\"https:\/\/colab.research.google.com\/drive\/1hwpYC-AU6C_nwuM_N5ynOShXIRGv-U51#scrollTo=KizhzOxVOjaN\"><img decoding=\"async\" src=\"https:\/\/img.shields.io\/badge\/Colab-Run_in_Google_Colab-blue?logo=Google&#038;logoColor=FDBA18\" alt=\"\" \/><\/a><\/p>\n<pre><code>import torch\nfrom models.model_builder import ExtSummarizer\nfrom ext_sum import summarize\n# Load model\nmodel_type = 'mobilebert' #@param ['bertbase', 'distilbert', 'mobilebert']\ncheckpoint = torch.load(f'checkpoints\/{model_type}_ext.pt', map_location='cpu')\nmodel = ExtSummarizer(checkpoint=checkpoint, bert_type=model_type, device='cpu')\n# Run summarization\ninput_fp = 'raw_data\/input.txt'\nresult_fp = 'results\/summary.txt'\nsummary = summarize(input_fp, result_fp, model, max_length=3)\nprint(summary)\n<\/code><\/pre>\n<p><strong>Summary sample<\/strong><\/p>\n<p>Original: <a href=\"https:\/\/www.cnn.com\/2020\/05\/22\/business\/hertz-bankruptcy\/index.html\"><a href=\"https:\/\/www.cnn.com\/2020\/05\/22\/business\/hertz-bankruptcy\/index.html\"><a href=\"https:\/\/www.cnn.com\/2020\/05\/22\/business\/hertz-bankruptcy\/index.html\"><a href=\"https:\/\/www.cnn.com\/2020\/05\/22\/business\/hertz-bankruptcy\/index.html\">https:\/\/www.cnn.com\/2020\/05\/22\/business\/hertz-bankruptcy\/index.html<\/a><\/a><\/a><\/a><\/p>\n<pre><code>By declaring bankruptcy, Hertz says it intends to stay in business while restructuring its debts and emerging a\nfinancially healthier company. The company has been renting cars since 1918, when it set up shop with a dozen\nFord Model Ts, and has survived the Great Depression, the virtual halt of US auto production during World War II\nand numerous oil price shocks. \"The impact of Covid-19 on travel demand was sudden and dramatic, causing an\nabrupt decline in the company's revenue and future bookings,\" said the company's statement.\n<\/code><\/pre>\n<h2><span class=\"ez-toc-section\" id=\"5_Conclusion\"><\/span>5. Conclusion<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>In this article, we have explored BERTSUM, a simple variant of BERT, for extractive summarization from the paper <strong>Text Summarization with Pretrained Encoders<\/strong> (Liu et al., 2019). Then, in an effort to make extractive summarization even faster and smaller for low-resource devices, we fine-tuned DistilBERT (Sanh et al., 2019) and MobileBERT (Sun et al., 2019) on CNN\/DailyMail datasets.<\/p>\n<p>DistilBERT retains BERT-base&#8217;s performance in extractive summarization while being 45% smaller. MobileBERT is 4x smaller and 2.7x faster than BERT-base yet retains 94% of its performance.<\/p>\n<p>Finally, we deployed a web app demo of MobileBERT for extractive summarization at <a href=\"https:\/\/extractive-summarization.herokuapp.com\/\"><a href=\"https:\/\/extractive-summarization.herokuapp.com\/\"><a href=\"https:\/\/extractive-summarization.herokuapp.com\/\"><a href=\"https:\/\/extractive-summarization.herokuapp.com\/\">https:\/\/extractive-summarization.herokuapp.com\/<\/a><\/a><\/a><\/a>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"References\"><\/span>References<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<ul>\n<li>[1] <a href=\"https:\/\/github.com\/nlpyang\/PreSumm\">PreSumm:  Text Summarization with Pretrained Encoders<\/a><\/li>\n<li>[2] <a href=\"https:\/\/huggingface.co\/transformers\/model_doc\/distilbert.html\">DistilBERT: Smaller, faster, cheaper, lighter version of BERT<\/a><\/li>\n<li>[3] <a href=\"https:\/\/github.com\/google-research\/google-research\/tree\/master\/mobilebert\">MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices<\/a><\/li>\n<li>[4] <a href=\"https:\/\/github.com\/lonePatient\/MobileBert_PyTorch\">MobileBert_PyTorch<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Tutorial: How to Fine-Tune BERT for Extractive Summarization Originally published by Skim AI&#8217;s Machine Learning Researcher, Chris Tran 1. Introduction Summarization has long been a challenge in Natural Language Processing. To generate a short version of a document while retaining its most important information, we need a model capable of accurately extracting the key points [&hellip;]<\/p>\n","protected":false},"author":1003,"featured_media":3857,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"single-custom-post-template.php","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[125,64,67],"tags":[74,81,79,75,73],"class_list":["post-3835","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-enterprise-ai-blog","category-how-to","category-ml-nlp","tag-bert","tag-fine-tune","tag-how-to","tag-nlp","tag-summarization"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Tutorial: How to Fine-Tune BERT for Extractive Summarization - by Skim AI<\/title>\n<meta name=\"description\" content=\"Tutorial on &quot;how to&quot; Fine-Tune BERT for Extractive Summarization. Part of a series of articles on using BERT for multiple use cases in NLP\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/skimai.com\/fr\/tutoriel-sur-la-facon-de-regler-finement-bert-pour-le-resume\/\" \/>\n<meta property=\"og:locale\" content=\"fr_FR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Tutorial: How to Fine-Tune BERT for Extractive Summarization - by Skim AI\" \/>\n<meta property=\"og:description\" content=\"Tutorial on &quot;how to&quot; Fine-Tune BERT for Extractive Summarization. Part of a series of articles on using BERT for multiple use cases in NLP\" \/>\n<meta property=\"og:url\" content=\"https:\/\/skimai.com\/fr\/tutoriel-sur-la-facon-de-regler-finement-bert-pour-le-resume\/\" \/>\n<meta property=\"og:site_name\" content=\"Skim AI\" \/>\n<meta property=\"article:published_time\" content=\"2020-12-29T02:34:12+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-05-20T12:38:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/skimai.com\/wp-content\/uploads\/2020\/12\/photo-1504711434969-e33886168f5c-scaled.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\n\t<meta property=\"og:image:height\" content=\"1707\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Greggory Elias\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u00c9crit par\" \/>\n\t<meta name=\"twitter:data1\" content=\"Greggory Elias\" \/>\n\t<meta name=\"twitter:label2\" content=\"Dur\u00e9e de lecture estim\u00e9e\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/\"},\"author\":{\"name\":\"Greggory Elias\",\"@id\":\"https:\/\/skimai.com\/uk\/#\/schema\/person\/7a883b4a2d2ea22040f42a7975eb86c6\"},\"headline\":\"Tutorial: How to Fine-Tune BERT for Extractive Summarization\",\"datePublished\":\"2020-12-29T02:34:12+00:00\",\"dateModified\":\"2024-05-20T12:38:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/\"},\"wordCount\":843,\"publisher\":{\"@id\":\"https:\/\/skimai.com\/uk\/#organization\"},\"image\":{\"@id\":\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/skimai.com\/wp-content\/uploads\/2020\/12\/photo-1504711434969-e33886168f5c-scaled.jpg\",\"keywords\":[\"bert\",\"fine tune\",\"how to\",\"nlp\",\"summarization\"],\"articleSection\":[\"Enterprise AI\",\"How to\",\"LLMs \/ NLP\"],\"inLanguage\":\"fr-FR\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/\",\"url\":\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/\",\"name\":\"Tutorial: How to Fine-Tune BERT for Extractive Summarization - by Skim AI\",\"isPartOf\":{\"@id\":\"https:\/\/skimai.com\/uk\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/skimai.com\/wp-content\/uploads\/2020\/12\/photo-1504711434969-e33886168f5c-scaled.jpg\",\"datePublished\":\"2020-12-29T02:34:12+00:00\",\"dateModified\":\"2024-05-20T12:38:31+00:00\",\"description\":\"Tutorial on \\\"how to\\\" Fine-Tune BERT for Extractive Summarization. Part of a series of articles on using BERT for multiple use cases in NLP\",\"breadcrumb\":{\"@id\":\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#breadcrumb\"},\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#primaryimage\",\"url\":\"https:\/\/skimai.com\/wp-content\/uploads\/2020\/12\/photo-1504711434969-e33886168f5c-scaled.jpg\",\"contentUrl\":\"https:\/\/skimai.com\/wp-content\/uploads\/2020\/12\/photo-1504711434969-e33886168f5c-scaled.jpg\",\"width\":2560,\"height\":1707,\"caption\":\"photo 1504711434969 e33886168f5c scaled\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/skimai.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Tutorial: How to Fine-Tune BERT for Extractive Summarization\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/skimai.com\/uk\/#website\",\"url\":\"https:\/\/skimai.com\/uk\/\",\"name\":\"Skim AI\",\"description\":\"The AI Agent Workforce Platform\",\"publisher\":{\"@id\":\"https:\/\/skimai.com\/uk\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/skimai.com\/uk\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"fr-FR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/skimai.com\/uk\/#organization\",\"name\":\"Skim AI\",\"url\":\"https:\/\/skimai.com\/uk\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/skimai.com\/uk\/#\/schema\/logo\/image\/\",\"url\":\"http:\/\/skimai.com\/wp-content\/uploads\/2020\/07\/SKIM-AI-Header-Logo.png\",\"contentUrl\":\"http:\/\/skimai.com\/wp-content\/uploads\/2020\/07\/SKIM-AI-Header-Logo.png\",\"width\":194,\"height\":58,\"caption\":\"Skim AI\"},\"image\":{\"@id\":\"https:\/\/skimai.com\/uk\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.linkedin.com\/company\/skim-ai\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/skimai.com\/uk\/#\/schema\/person\/7a883b4a2d2ea22040f42a7975eb86c6\",\"name\":\"Greggory Elias\",\"url\":\"https:\/\/skimai.com\/fr\/author\/gregg\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Tutoriel : Comment affiner le BERT pour la synth\u00e8se extractive - par Skim AI","description":"Tutoriel sur la mani\u00e8re d'affiner l'ORET pour la synth\u00e8se extractive. Fait partie d'une s\u00e9rie d'articles sur l'utilisation de l'ORET pour de multiples cas d'utilisation en TAL.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/skimai.com\/fr\/tutoriel-sur-la-facon-de-regler-finement-bert-pour-le-resume\/","og_locale":"fr_FR","og_type":"article","og_title":"Tutorial: How to Fine-Tune BERT for Extractive Summarization - by Skim AI","og_description":"Tutorial on \"how to\" Fine-Tune BERT for Extractive Summarization. Part of a series of articles on using BERT for multiple use cases in NLP","og_url":"https:\/\/skimai.com\/fr\/tutoriel-sur-la-facon-de-regler-finement-bert-pour-le-resume\/","og_site_name":"Skim AI","article_published_time":"2020-12-29T02:34:12+00:00","article_modified_time":"2024-05-20T12:38:31+00:00","og_image":[{"width":2560,"height":1707,"url":"https:\/\/skimai.com\/wp-content\/uploads\/2020\/12\/photo-1504711434969-e33886168f5c-scaled.jpg","type":"image\/jpeg"}],"author":"Greggory Elias","twitter_card":"summary_large_image","twitter_misc":{"\u00c9crit par":"Greggory Elias","Dur\u00e9e de lecture estim\u00e9e":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#article","isPartOf":{"@id":"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/"},"author":{"name":"Greggory Elias","@id":"https:\/\/skimai.com\/uk\/#\/schema\/person\/7a883b4a2d2ea22040f42a7975eb86c6"},"headline":"Tutorial: How to Fine-Tune BERT for Extractive Summarization","datePublished":"2020-12-29T02:34:12+00:00","dateModified":"2024-05-20T12:38:31+00:00","mainEntityOfPage":{"@id":"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/"},"wordCount":843,"publisher":{"@id":"https:\/\/skimai.com\/uk\/#organization"},"image":{"@id":"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#primaryimage"},"thumbnailUrl":"https:\/\/skimai.com\/wp-content\/uploads\/2020\/12\/photo-1504711434969-e33886168f5c-scaled.jpg","keywords":["bert","fine tune","how to","nlp","summarization"],"articleSection":["Enterprise AI","How to","LLMs \/ NLP"],"inLanguage":"fr-FR"},{"@type":"WebPage","@id":"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/","url":"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/","name":"Tutoriel : Comment affiner le BERT pour la synth\u00e8se extractive - par Skim AI","isPartOf":{"@id":"https:\/\/skimai.com\/uk\/#website"},"primaryImageOfPage":{"@id":"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#primaryimage"},"image":{"@id":"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#primaryimage"},"thumbnailUrl":"https:\/\/skimai.com\/wp-content\/uploads\/2020\/12\/photo-1504711434969-e33886168f5c-scaled.jpg","datePublished":"2020-12-29T02:34:12+00:00","dateModified":"2024-05-20T12:38:31+00:00","description":"Tutoriel sur la mani\u00e8re d'affiner l'ORET pour la synth\u00e8se extractive. Fait partie d'une s\u00e9rie d'articles sur l'utilisation de l'ORET pour de multiples cas d'utilisation en TAL.","breadcrumb":{"@id":"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#breadcrumb"},"inLanguage":"fr-FR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/"]}]},{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#primaryimage","url":"https:\/\/skimai.com\/wp-content\/uploads\/2020\/12\/photo-1504711434969-e33886168f5c-scaled.jpg","contentUrl":"https:\/\/skimai.com\/wp-content\/uploads\/2020\/12\/photo-1504711434969-e33886168f5c-scaled.jpg","width":2560,"height":1707,"caption":"photo 1504711434969 e33886168f5c scaled"},{"@type":"BreadcrumbList","@id":"https:\/\/skimai.com\/tutorial-how-to-fine-tune-bert-for-summarization\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/skimai.com\/"},{"@type":"ListItem","position":2,"name":"Tutorial: How to Fine-Tune BERT for Extractive Summarization"}]},{"@type":"WebSite","@id":"https:\/\/skimai.com\/uk\/#website","url":"https:\/\/skimai.com\/uk\/","name":"Skim AI","description":"La plateforme de travail des agents de l'IA","publisher":{"@id":"https:\/\/skimai.com\/uk\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/skimai.com\/uk\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"fr-FR"},{"@type":"Organization","@id":"https:\/\/skimai.com\/uk\/#organization","name":"Skim AI","url":"https:\/\/skimai.com\/uk\/","logo":{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/skimai.com\/uk\/#\/schema\/logo\/image\/","url":"http:\/\/skimai.com\/wp-content\/uploads\/2020\/07\/SKIM-AI-Header-Logo.png","contentUrl":"http:\/\/skimai.com\/wp-content\/uploads\/2020\/07\/SKIM-AI-Header-Logo.png","width":194,"height":58,"caption":"Skim AI"},"image":{"@id":"https:\/\/skimai.com\/uk\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.linkedin.com\/company\/skim-ai"]},{"@type":"Person","@id":"https:\/\/skimai.com\/uk\/#\/schema\/person\/7a883b4a2d2ea22040f42a7975eb86c6","name":"Greggory Elias","url":"https:\/\/skimai.com\/fr\/author\/gregg\/"}]}},"_links":{"self":[{"href":"https:\/\/skimai.com\/fr\/wp-json\/wp\/v2\/posts\/3835","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/skimai.com\/fr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/skimai.com\/fr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/skimai.com\/fr\/wp-json\/wp\/v2\/users\/1003"}],"replies":[{"embeddable":true,"href":"https:\/\/skimai.com\/fr\/wp-json\/wp\/v2\/comments?post=3835"}],"version-history":[{"count":0,"href":"https:\/\/skimai.com\/fr\/wp-json\/wp\/v2\/posts\/3835\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/skimai.com\/fr\/wp-json\/wp\/v2\/media\/3857"}],"wp:attachment":[{"href":"https:\/\/skimai.com\/fr\/wp-json\/wp\/v2\/media?parent=3835"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/skimai.com\/fr\/wp-json\/wp\/v2\/categories?post=3835"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/skimai.com\/fr\/wp-json\/wp\/v2\/tags?post=3835"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}