{"id":413,"date":"2021-01-14T06:00:29","date_gmt":"2021-01-13T21:00:29","guid":{"rendered":"https:\/\/staka.jp\/wordpress\/?p=413"},"modified":"2021-01-14T06:00:29","modified_gmt":"2021-01-13T21:00:29","slug":"%e3%83%95%e3%83%aa%e3%83%bc%e3%81%ae%e3%83%8b%e3%83%a5%e3%83%bc%e3%83%a9%e3%83%ab%e6%a9%9f%e6%a2%b0%e7%bf%bb%e8%a8%b3%e3%83%a2%e3%83%87%e3%83%abfugumt","status":"publish","type":"post","link":"https:\/\/staka.jp\/wordpress\/?p=413","title":{"rendered":"\u30d5\u30ea\u30fc\u306e\u30cb\u30e5\u30fc\u30e9\u30eb\u6a5f\u68b0\u7ffb\u8a33\u30e2\u30c7\u30ebFuguMT"},"content":{"rendered":"\n<p>\u82f1\u6587\u3092\u65e5\u672c\u8a9e\u8a33\u3059\u308b\u30cb\u30e5\u30fc\u30e9\u30eb\u6a5f\u68b0\u7ffb\u8a33\u30e2\u30c7\u30eb\u3092CC BY-SA 4.0\u3067\u516c\u958b\u3057\u305f\u3002<a href=\"https:\/\/staka.jp\/wordpress\/?p=276\">\u4ee5\u524d\u306e\u8a18\u4e8b<\/a>\u3067\u7d39\u4ecb\u3057\u305f\u624b\u6cd5\u3092\u7528\u3044\u6628\u5e7411\u6708\u306b\u69cb\u7bc9\u3057\u305f\u30e2\u30c7\u30eb\u3067\u3042\u308b<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><a href=\"https:\/\/fugumt.com\/FuguMT_ver.202011.1.zip\">FuguMT ver.2020.11.1 \uff08\u7d04440MB\uff09\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9<\/a><\/li><li>shasum: 0cf8a1fc540b4c7b4388b75b71858c0eb32e392a<\/li><li>\u30e9\u30a4\u30bb\u30f3\u30b9:  <a href=\"https:\/\/creativecommons.org\/licenses\/by-sa\/4.0\/deed.ja\">CC BY-SA 4.0<\/a>\u00a0 \uff08readme\u306b\u3082\u66f8\u3044\u305f\u901a\u308a\u3001\u4f5c\u8005\uff08Satoshi Takahashi\uff09\u306f\u672c\u30e2\u30c7\u30eb\u3092\u4f7f\u7528\u3057\u3066\u767a\u751f\u3057\u305f\u3042\u3089\u3086\u308b\u7d50\u679c\u306b\u3064\u3044\u3066\u4e00\u5207\u306e\u8cac\u4efb\u3092\u8ca0\u3044\u307e\u305b\u3093 \u3002\u5f15\u7528\u7b49\u3092\u884c\u3046\u5834\u5408\u306f\u3053\u306eBlog\u306eURL\u3092\u8a18\u8f09\u3059\u308b\u304b\u3001\u30ea\u30f3\u30af\u3092\u8cbc\u3063\u3066\u304f\u3060\u3055\u3044\u3002\uff09<\/li><\/ul>\n\n\n\n<p>\u6027\u80fd\u306f\u305d\u3053\u305d\u3053\uff08\u5f8c\u8ff0\uff09\u3002\u69cb\u7bc9\u624b\u6cd5\u306f\u672c\u683c\u7684\uff08Marian-NMT[1]\u3092\u7528\u3044\u305f Transformer  + Sentence Piece[2]\uff09\u3067\u3042\u308b\u3002<\/p>\n\n\n\n<p>\u7814\u7a76\u7528\u9014\u3067\u306e\u8a66\u7528\u3092\u524d\u63d0\u3068\u3057\u3066\u304a\u308a\u3001<strong>\u5f53\u7136\u306a\u304c\u3089<span style=\"text-decoration: underline;\">\u52d5\u4f5c\u3084\u51fa\u529b\u306b\u95a2\u3057\u3066\u4e00\u5207\u306e\u8cac\u4efb\u3092\u8ca0\u3048\u306a\u3044<\/span>\uff08\u91cd\u8981\u306a\u306e\u30672\u5ea6\u76ee\uff09<\/strong> [3] \u304cCreative Commons\u306b\u5f93\u3063\u3066\u5229\u7528\u53ef\u80fd\u306a\u30e2\u30c7\u30eb\u306f\u73cd\u3057\u3044\u3068\u601d\u3046\u3002<\/p>\n\n\n\n<p>\u5468\u8fba\u306e\u30b3\u30fc\u30c9\u3092\u542b\u3081\u3066github\u306bupload\u3057\u305f\u306e\u3067\u3001\u5229\u7528\u3059\u308b\u5834\u5408\u306f<a href=\"https:\/\/github.com\/s-taka\/fugumt\">github\u306efugumt\u30ea\u30dd\u30b8\u30c8\u30ea<\/a>\u3092\u53c2\u7167\u3044\u305f\u3060\u304d\u305f\u3044\u3002github\u306e\u30b3\u30fc\u30c9\u306f\u57fa\u672c\u7684\u306b<a href=\"https:\/\/devneko.jp\/demo\/\">\u30c6\u30b9\u30c8\u30b5\u30a4\u30c8<\/a>\u3068\u540c\u69d8\u3060\u304c\u30b7\u30f3\u30d7\u30eb\u306b\u4f7f\u7528\u53ef\u80fd\u306a\u69cb\u6210\u306b\u3057\u3066\u3044\u308b[3]\u3002<\/p>\n\n\n\n<p><strong><span style=\"text-decoration: underline;\">\u7ffb\u8a33\u3092\u884c\u3046\u5834\u5408\u306f\u8457\u4f5c\u6a29\u306b\u6ce8\u610f\u3002\u6700\u8fd1\u306fCC Zero\u306a\u3069\u81ea\u7531\u306a\u30e9\u30a4\u30bb\u30f3\u30b9\u3067\u516c\u958b\u3055\u308c\u308b\u8ad6\u6587\u7b49\u3082\u5897\u3048\u3066\u304d\u3066\u3044\u308b\u3068\u306f\u3044\u3048\u3001\u901a\u5e38\u3001\u8457\u4f5c\u7269\u3092\u81ea\u7531\u306b\u7ffb\u8a33\u3059\u308b\u3053\u3068\u306f\u3067\u304d\u306a\u3044\u3002 <\/span><\/strong><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u30e2\u30c7\u30eb\u306e\u8a73\u7d30<\/h3>\n\n\n\n<p>github\u306b\u8a18\u8f09\u306e\u901a\u308a FuguMT ver.2020.11.1 \u306f\u69d8\u3005\u306a\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3092\u7d44\u307f\u5408\u308f\u305b\u3066\u4f5c\u6210\u3057\u3066\u3044\u308b\u3002\u4f7f\u7528\u3057\u305f\u30c7\u30fc\u30bf\u91cf\u306f\u7d04660\u4e07\u5bfe\u8a33\u30da\u30a2\uff08\u65e5\u672c\u8a9e:690MB \u82f1\u8a9e:610MB\u3001\u7d041\u5104words\uff09\u3067\u3042\u308b\u3002 AWS p3.2xlarge \u4e0a\u3067 Marian-NMT + SentencePiece\u3092\u7528\u3044\u7d0430\u6642\u9593\u306e\u5b66\u7fd2\u3092\u884c\u3063\u305f\u3002<\/p>\n\n\n\n<p>\u516c\u958b\u3059\u308b\u30e2\u30c7\u30eb\u306f\u300cmodel.npz\u300d\u3068\u300cmodel_uncased.npz\u300d\u306e2\u3064\u3067\u3001\u524d\u8005\u306f\u82f1\u6587\u3092\u305d\u306e\u307e\u307e\u65e5\u672c\u8a9e\u8a33\u3059\u308b\u30e2\u30c7\u30eb\u3001\u5f8c\u8005\u306f\u82f1\u6587\u3092\u5c0f\u6587\u5b57\u306b\u7d71\u4e00\uff08sentence piece\u306e&#8211;normalization_rule_name = nmt_nfkc_cf\uff09\u3057\u65e5\u672c\u8a9e\u8a33\u3059\u308b\u30e2\u30c7\u30eb\u3067\u3042\u308b\u3002 fine tuning\u3082\u53ef\u80fd\u3002\u8a73\u7d30\u306a\u8a2d\u5b9a\u306fzip\u30d5\u30a1\u30a4\u30eb\u4e2d\u306e\u300cmodel.npz.yml\u300d\u300cmodel.npz.progress.yml\u300d\u3092\u78ba\u8a8d\u3057\u3066\u307b\u3057\u3044\u3002fine tuning\u6642\u306b\u306freadme\u3092\u8aad\u307f\u30e9\u30a4\u30bb\u30f3\u30b9\u306b\u5f93\u3063\u305f\u53d6\u6271\u3044\u3092\u304a\u9858\u3044\u3057\u305f\u3044\u3002<\/p>\n\n\n\n<p>\u3053\u306e\u30e2\u30c7\u30eb\u306e\u6027\u80fd\u306fKFTT \u30c6\u30b9\u30c8\u30c7\u30fc\u30bf\u306eBLEU\u306723\u7a0b\u5ea6\u3067\u3042\u308b\u3002\u6280\u8853\u6587\u66f8\u3001\u8ad6\u6587\u306b\u5bfe\u3057\u3066\u306f\u3082\u3046\u5c11\u3057\u6027\u80fd\u304c\u9ad8\u304f\u3001BLEU\u306730\u7a0b\u5ea6\u3068\u30aa\u30f3\u30e9\u30a4\u30f3\u3067\u516c\u958b\u3055\u308c\u3066\u3044\u308b\u7ffb\u8a33\u30a8\u30f3\u30b8\u30f3\u3068\u540c\u30ec\u30d9\u30eb\u306e\u6027\u80fd\u3092\u51fa\u3059\u3053\u3068\u304c\u3067\u304d\u308b\u3002BLEU\u306f\u826f\u3044\u6307\u6a19\u3068\u306f\u3044\u3044\u304c\u305f\u3044\u305f\u3081\u3001 <a href=\"http:\/\/plant-check.jp:8080\/\">\u30c6\u30b9\u30c8\u30b5\u30a4\u30c8<\/a> \u3067\u8a66\u3057\u3066\u307f\u308b\u3068\u5927\u4f53\u306e\u6027\u80fd\u304c\u5206\u304b\u308b\u3068\u601d\u3046\u3002\u3053\u306eBlog\u3092\u66f8\u3044\u3066\u3044\u308b\u6642\u70b9\u3067\u306f\u30c6\u30b9\u30c8\u30b5\u30a4\u30c8\u306e\u7ffb\u8a33\u30a8\u30f3\u30b8\u30f31\u306fFuguMT ver.2020.11.1\u306emodel.npz\u3092\u4f7f\u7528\u3057\u3066\u3044\u308b[4]\u3002<\/p>\n\n\n\n<p>\u591a\u304f\u306e\u5834\u5408 \u300cmodel.npz\u300d\u306e\u65b9\u304c\u826f\u3044\u7d50\u679c\u3092\u51fa\u529b\u3059\u308b\u304c\u3001\u7ffb\u8a33\u304c\u96e3\u3057\u3044\u6587\u306e\u5834\u5408\u300cmodel_uncased.npz\u300d\u306e\u65b9\u304c\u5b89\u5b9a\u3057\u305f\u7d50\u679c\u3092\u51fa\u529b\u3059\u308b\u3002github\u306efugumt\u30e9\u30a4\u30d6\u30e9\u30ea\u3067\u306f\u8907\u6570\u306e\u7ffb\u8a33\u7d50\u679c\u5019\u88dc\u304b\u3089\u826f\u3044\u3068\u601d\u308f\u308c\u308b\u51fa\u529b\u3092\u9078\u3076\u6a5f\u80fd\u3092\u5b9f\u88c5\u3057\u3066\u3044\u308b\u3002 <\/p>\n\n\n\n<h4 class=\"wp-block-heading\"> FuguMT ver.2020.11.1 \u3067\u5229\u7528\u3057\u305f\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8<\/h4>\n\n\n\n<p> FuguMT ver.2020.11.1  \u3067\u4f7f\u7528\u3057\u305fCreative Commons\u30e9\u30a4\u30bb\u30f3\u30b9\u306e\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306f\u4e0b\u8a18\u306e\u901a\u308a\u3002CC\u3067\u5229\u7528\u53ef\u80fd\u306a\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306b\u306f\u975e\u5e38\u306b\u52a9\u3051\u3089\u308c\u305f\u3002\u30d5\u30ea\u30fc\u306a\u30e9\u30a4\u30bb\u30f3\u30b9\u3067\u516c\u958b\u3055\u308c\u305f\u65b9\u3005\u306b\u611f\u8b1d\u3057\u305f\u3044\u3002FuguMT\u306e\u69cb\u7bc9\u306b\u306f\u4e0b\u8a18\u4ee5\u5916\u306b\u72ec\u81ea\u306b\u53ce\u96c6\u3057\u305f\u30c7\u30fc\u30bf\u3082\u5229\u7528\u3057\u3066\u3044\u308b[5]\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Japanese-English Subtitle Corpus (CC BY-SA 4.0):&nbsp;<a href=\"https:\/\/nlp.stanford.edu\/projects\/jesc\/\">https:\/\/nlp.stanford.edu\/projects\/jesc\/<\/a><ul><li>Pryzant, R. and Chung, Y. and Jurafsky, D. and Britz, D., JESC: Japanese-English Subtitle Corpus, Language Resources and Evaluation Conference (LREC), 2018<\/li><\/ul><\/li><li>\u4eac\u90fd\u30d5\u30ea\u30fc\u7ffb\u8a33\u30bf\u30b9\u30af (KFTT) (CC BY-SA 3.0):&nbsp;<a href=\"http:\/\/www.phontron.com\/kftt\/index-ja.html\">http:\/\/www.phontron.com\/kftt\/index-ja.html<\/a><ul><li>Graham Neubig, &#8220;The Kyoto Free Translation Task,&#8221;&nbsp;<a href=\"http:\/\/www.phontron.com\/kftt\">http:\/\/www.phontron.com\/kftt<\/a>, 2011.<\/li><\/ul><\/li><li>Tanaka Corpus (CC BY 2.0 FR):<a href=\"http:\/\/www.edrdg.org\/wiki\/index.php\/Tanaka_Corpus\">http:\/\/www.edrdg.org\/wiki\/index.php\/Tanaka_Corpus<\/a><ul><li><blockquote>Professor Tanaka originally placed the Corpus in the Public Domain, and that status was maintained for the versions used by WWWJDIC. In late 2009 the Tatoeba Project decided to move it to a Creative Commons CC-BY licence (that project is in France, where the concept of public domain is not part of the legal framework.) It can be freely downloaded and used provided the source is attributed.<\/blockquote><\/li><\/ul><\/li><li>JSNLI (CC BY-SA 4.0):<a href=\"http:\/\/nlp.ist.i.kyoto-u.ac.jp\/index.php?%E6%97%A5%E6%9C%AC%E8%AA%9ESNLI%28JSNLI%29%E3%83%87%E3%83%BC%E3%82%BF%E3%82%BB%E3%83%83%E3%83%88\">http:\/\/nlp.ist.i.kyoto-u.ac.jp\/index.php?%E6%97%A5%E6%9C%AC%E8%AA%9ESNLI%28JSNLI%29%E3%83%87%E3%83%BC%E3%82%BF%E3%82%BB%E3%83%83%E3%83%88<\/a><ul><li>\u5409\u8d8a \u5353\u898b, \u6cb3\u539f \u5927\u8f14, \u9ed2\u6a4b \u798e\u592b: \u6a5f\u68b0\u7ffb\u8a33\u3092\u7528\u3044\u305f\u81ea\u7136\u8a00\u8a9e\u63a8\u8ad6\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u591a\u8a00\u8a9e\u5316, \u7b2c244\u56de\u81ea\u7136\u8a00\u8a9e\u51e6\u7406\u7814\u7a76\u4f1a, (2020.7.3).<\/li><\/ul><\/li><li>WikiMatrix (Creative Commons Attribution-ShareAlike license):<a href=\"https:\/\/github.com\/facebookresearch\/LASER\/tree\/master\/tasks\/WikiMatrix\">https:\/\/github.com\/facebookresearch\/LASER\/tree\/master\/tasks\/WikiMatrix<\/a><ul><li>Holger Schwenk, Vishrav Chaudhary, Shuo Sun, Hongyu Gong and Paco Guzman, WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia, arXiv, July 11 2019.<\/li><\/ul><\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">\u4f7f\u3044\u65b9<\/h3>\n\n\n\n<p><a href=\"https:\/\/github.com\/s-taka\/fugumt\">fugumt<\/a>\u306b\u8a18\u8f09\u306e\u901a\u308a\u3002github\u306eDockerfile\u3092build\u3057\u3001\u30e2\u30c7\u30eb\u3092\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u3001marian-decoder\u7d4c\u7531\u3067\u4f7f\u3046<a href=\"https:\/\/github.com\/s-taka\/fugumt#marian-decoder%E3%81%AE%E5%AE%9F%E8%A1%8C\">\u624b\u9806<\/a>\u304c\u30c6\u30b9\u30c8\u3092\u884c\u3044\u3084\u3059\u3044\u3002CPU 1\u30b3\u30a2\u3067\u5b9f\u884c\u3057\u305f\u5834\u54081\u6587\u306e\u7ffb\u8a33\u306b2-3\u79d2\u306f\u5fc5\u8981\u3067\u3042\u308b\u3002<\/p>\n\n\n\n<p>fugumt\u30e9\u30a4\u30d6\u30e9\u30ea\u3067\u306f\u6587\u7ae0\u306e\u6587\u5206\u5272\u3001<a href=\"https:\/\/staka.jp\/wordpress\/?p=357\">\u8a33\u629c\u3051\u9632\u6b62\u30e2\u30fc\u30c9<\/a>\u306e\u5b9f\u884c\u3001\u7ffb\u8a33\u306e\u9069\u5207\u3055\u306e\u30b9\u30b3\u30a2\u30ea\u30f3\u30b0\u3001\u8907\u6570\u306e\u7ffb\u8a33\u5019\u88dc\u304b\u3089\u826f\u3044\u7ffb\u8a33\u6587\u3092\u9078\u3076\u6a5f\u80fd\u306a\u3069\u3092\u5b9f\u88c5\u3057\u3066\u3044\u308b\u3002<\/p>\n\n\n\n<p>pdf_server.py\u3092\u4f7f\u3046\u3068<a href=\"https:\/\/github.com\/euske\/pdfminer\">pdfminer<\/a>\u3092\u7528\u3044\u3066PDF\u6587\u66f8\u3092\u305d\u306e\u307e\u307e\u7ffb\u8a33\u3067\u304d\u308b\u3002pdf_server.py\u306fpickle\u3092\u51fa\u529b\u3059\u308b\u3002server.py\u306e\u6a5f\u80fd\u3067PDF\u3068\u8a33\u6587\u3092\u5bfe\u6bd4\u3057\u306a\u304c\u3089\u5185\u5bb9\u3092\u78ba\u8a8d\u3067\u304d\u308b\u3002 \u8a08\u7b97\u6642\u9593\u306f1\u30da\u30fc\u30b8\u3042\u305f\u308a1\u5206\u7a0b\u5ea6\u3067\u3042\u308b\u3002 <\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u4eca\u5f8c\u306e\u4e88\u5b9a<\/h3>\n\n\n\n<p>github\u306b\u3082\u66f8\u3044\u3066\u3044\u308b\u901a\u308a\u3001Tatoeba\u3068CCAligned\u3092\u7528\u3044\u305f\u30e2\u30c7\u30eb\u306f\u69cb\u7bc9\u6e08\u307f\u3067\u6027\u80fd\u8a55\u4fa1\u4e2d\u3067\u3042\u308b\u3002\u5bfe\u8a33\u30da\u30a2\u6570\u306f1400\u4e07\u306b\u9054\u3057\u3066\u3044\u308b\u3082\u306e\u306e\u3001CCAligned\u306e\u30c7\u30fc\u30bf\u306b\u6a5f\u68b0\u7ffb\u8a33\u3055\u308c\u305f\u3082\u306e\u304c\u591a\u6570\u6df7\u5728\u3057\u3066\u304a\u308a\u7dcf\u5408\u7684\u306a\u6027\u80fd\u304c\u5411\u4e0a\u3059\u308b\u304b\u306f\u691c\u8a3c\u3067\u304d\u3066\u3044\u306a\u3044\u3002\u3053\u308c\u3092\u542b\u3081\u3001\u4eca\u5f8c\u4e0b\u8a18\u306e\u3088\u3046\u306a\u3053\u3068\u3092\u3084\u308a\u305f\u3044\u3068\u601d\u3063\u3066\u3044\u308b[7]\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>\u5bfe\u8a33\u30da\u30a2\u30921400\u4e07\u30da\u30a2\u306b\u62e1\u5f35\u3057\u305f\u30e2\u30c7\u30eb\u306e\u6027\u80fd\u8a55\u4fa1\u30fb\u516c\u958b<\/li><li>FastAPI\u7b49\u3092\u5229\u7528\u3057\u305fAPI\u5316<ul><li>bottle\u306e\u5b9f\u88c5\u3092\u6539\u5584\u3057\u305f\u3044<\/li><\/ul><\/li><li>\u82f1\u8a9e\u2192\u65e5\u672c\u8a9e\u3060\u3051\u3067\u306f\u306a\u304f\u65e5\u672c\u8a9e\u2192\u82f1\u8a9e\u306e\u30a8\u30f3\u30b8\u30f3\u4f5c\u6210<ul><li>\u57fa\u672c\u7684\u306b\u306f\u30c7\u30fc\u30bf\u3092\u53cd\u8ee2\u3055\u305b\u308c\u3070\u4f5c\u308c\u308b<\/li><\/ul><\/li><li>Back Translation\u3001BART\u3084mT5\u306a\u3069\u4e8b\u524d\u5b66\u7fd2\u30e2\u30c7\u30eb\u306e\u6d3b\u7528\u3001\u6587\u8108\u3092\u4f7f\u3063\u305f\u7ffb\u8a33\u306a\u3069\u30e2\u30c7\u30eb\u306e\u9ad8\u5ea6\u5316<ul><li>Back Translation\u306e\u6d3b\u7528\u306f\u30de\u30b7\u30f3\u30d1\u30ef\u30fc\u304c\u3042\u308c\u3070\u53ef\u80fd<\/li><li>BART\u306f\u8a66\u884c\u3057\u305f\u304c\u6027\u80fd\u304c\u5411\u4e0a\u3057\u306a\u304b\u3063\u305f\u3002\u3068\u308a\u3042\u3048\u305a\u4e2d\u65ad\u3057\u305f\u691c\u8a3c\u3092\u518d\u958b\u3057\u305f\u3044\u3002<\/li><li>1400\u4e07\u5bfe\u8a33\u30da\u30a2\u306e\u3046\u30617\u5272\u7a0b\u5ea6\u306f\u6587\u8108\u3092\u8003\u616e\u53ef\u80fd[8]\u3067\u6587\u8108\u3092\u8003\u616e\u3059\u308b\u30e2\u30c7\u30eb\u306f\u69cb\u7bc9\u3057\u305f\u3044\uff08\u69cb\u7bc9\u3067\u304d\u308b\uff09\u3002<\/li><\/ul><\/li><li>\u82f1\u8a9e\u30fb\u65e5\u672c\u8a9e\u4ee5\u5916\u3078\u306e\u5bfe\u5fdc<ul><li>\u30d5\u30e9\u30f3\u30b9\u8a9e\u3001\u30c9\u30a4\u30c4\u8a9e\u3001\u30a4\u30bf\u30ea\u30a2\u8a9e\u3001\u30b9\u30da\u30a4\u30f3\u8a9e\u3001\u30ed\u30b7\u30a2\u8a9e\u3001\u30dd\u30eb\u30c8\u30ac\u30eb\u8a9e\u306f\u4e00\u5b9a\u7a0b\u5ea6\u306e\u5bfe\u8a33\u3092\u30b3\u30ec\u30af\u30b7\u30e7\u30f3\u3057\u3066\u3044\u308b\u3002<\/li><\/ul><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">\u305d\u306e\u4ed6<\/h4>\n\n\n\n<p>\u3068\u308a\u3042\u3048\u305a\u590f\u4f11\u307f\u306e\u5bbf\u984c\u3067\u4f5c\u3063\u305f\u3082\u306e\u3092\u5f62\u306b\u3057\u3066\u51ac\u4f11\u307f\uff08\uff0b1\u9031\u9593\uff09\u306b\u516c\u958b\u3067\u304d\u305f\u306e\u306f\u826f\u304b\u3063\u305f\u3002  FuguMT\u306f\u6c17\u8efd\u306b\u4f7f\u3048\u308b\u304c\u30e2\u30c7\u30eb\u6027\u80fd\u306f\u305d\u3053\u307e\u3067\u9ad8\u304f\u306a\u3044\u3002\u4f55\u3089\u304b\u306e\u30b5\u30fc\u30d3\u30b9\u3067\u5229\u7528\u3059\u308b\u5834\u5408\u306f\u5927\u624b\u7ffb\u8a33\u30b5\u30a4\u30c8\u306eAPI\u3068\u6bd4\u8f03\u3057\u3066\u307f\u3066\u307b\u3057\u3044\u3002 <\/p>\n\n\n\n<p>\u5143\u3005\u306e\u30e2\u30c1\u30d9\u30fc\u30b7\u30e7\u30f3\u306fSuperGLUE\u306e\u3088\u3046\u306a\u6a19\u6e96\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u65e5\u672c\u8a9e\u7248\u3092\u4f5c\u308a\u305f\u3044\u3068\u3044\u3046\u3082\u306e\u3060\u3063\u305f\u3002SuperGLUE\u306f\u6628\u5e74\u672b\u306b\u300c\u89e3\u6c7a\u300d\u3055\u308c\u3066\u3057\u307e\u3044\u3001xtreme\u3068\u3044\u3063\u305f\u30de\u30eb\u30c1\u30ea\u30f3\u30ac\u30eb\u306a\u30d9\u30f3\u30c1\u30de\u30fc\u30af\u3082\u6d41\u884c\u3063\u3066\u3044\u308b\u306e\u3067\u3084\u3084\u6642\u4ee3\u9045\u308c\u611f\u3082\u3042\u308b\u3002\u3002\u3002\u304c\u3001\u52c9\u5f37\u306b\u306f\u306a\u3063\u305f\u3002<\/p>\n\n\n\n<p>\u6b21\u306b\u4f55\u3092\u3084\u308b\u304b\u306f\u8003\u3048\u4e2d\u3002\u4eca\u5f8c\u306e\u4e88\u5b9a\u306b\u3042\u308b\u3053\u3068\u3092\u5730\u9053\u306b\u3084\u3063\u3066\u3044\u304f\u304b\u3001\u5168\u7136\u9055\u3046\u3053\u3068\u3092\u3084\u308b\u304b\u3082\u6b63\u76f4\u6c7a\u3081\u3066\u3044\u306a\u3044\u3002<\/p>\n\n\n\n<p>\u5bfe\u8a33\u30da\u30a2\u306e\u30c7\u30fc\u30bf\u306e\u516c\u958b\u306f\u4eca\u306e\u3068\u3053\u308d\u8003\u3048\u3066\u3044\u306a\u3044\u3002\u5546\u696d\u7684\u4fa1\u5024\u306e\u3042\u308b\u30c7\u30fc\u30bf\u3060\u3068\u601d\u3044\u3064\u3064\u3001\u30ea\u30fc\u30ac\u30eb\u30c1\u30a7\u30c3\u30af\u3084\u7a0e\u91d1\u306e\u554f\u984c\u306a\u3069\u69d8\u3005\u306a\u8ab2\u984c\u304c\u3042\u308b\u306e\u3067\u76f8\u5f53\u306e\u91d1\u984d\u3067\u306a\u3044\u3068\u6709\u511f\u63d0\u4f9b\u3082\u3057\u306a\u3044\u3068\u601d\u3046\u3002<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">\u811a\u6ce8<\/h4>\n\n\n\n<p>[1] Marian-NMT:&nbsp;<a href=\"https:\/\/github.com\/marian-nmt\/marian\">https:\/\/github.com\/marian-nmt\/marian<\/a><br \/>[2] SentencePiece:&nbsp;<a href=\"https:\/\/github.com\/google\/sentencepiece\">https:\/\/github.com\/google\/sentencepiece<\/a><br \/>[3] \u7279\u306b\u5dee\u5225\u7684\u306a\u7ffb\u8a33\u3092\u884c\u3046\u53ef\u80fd\u6027\u306a\u3069\u5341\u5206\u306a\u691c\u8a3c\u306f\u3067\u304d\u3066\u3044\u306a\u3044\u3002<br \/>[4] \u5b89\u5b9a\u6027\u306f\u82e5\u5e72\u52a3\u308b\u3002\u30c6\u30b9\u30c8\u30b5\u30a4\u30c8\u306fuwsgi+nginx\u3067\u52d5\u4f5c\u3055\u305b\u308b\u69cb\u6210\u3068\u3057\u3066\u3044\u308b\u3002\u3082\u3063\u3068\u3082\u3001Marian-NMT\u81ea\u4f53\u304c\u975e\u5e38\u306b\u826f\u304f\u3067\u304d\u305f\u30bd\u30d5\u30c8\u30a6\u30a7\u30a2\u306a\u306e\u3067\u3001npz\u30d5\u30a1\u30a4\u30eb\u3055\u3048\u3042\u308c\u3070\u4ed6\u306f\u3044\u3089\u306a\u3044\u304b\u3082\u3057\u308c\u306a\u3044\u304c\u30fb\u30fb\u30fb\u3002<br \/>[5]  \u3053\u308c\u3089\u30c7\u30fc\u30bf\u3092\u6a5f\u68b0\u7ffb\u8a33\u30e2\u30c7\u30eb\u69cb\u7bc9\u306b\u7528\u3044\u305f\u5834\u5408\u3001\u6a5f\u68b0\u7ffb\u8a33\u30e2\u30c7\u30eb\uff08npz\u30d5\u30a1\u30a4\u30eb\uff09\u306e\u30e9\u30a4\u30bb\u30f3\u30b9\u304c\u3069\u3046\u306a\u308b\u304b\u306f\u591a\u304f\u306e\u8b70\u8ad6\u304c\u3042\u308b\u3002\u3053\u3053\u3067\u306f\u51fa\u6240\u30fb\u30e9\u30a4\u30bb\u30f3\u30b9\u3092\u660e\u78ba\u306b\u3057\u3001\u5f15\u7528\u6761\u4ef6\u306f\u914d\u5e03\u30b5\u30a4\u30c8\u306e\u8868\u8a18\u306b\u5f93\u3063\u3066\u3044\u308b\u3002<br \/>[6] \u30c7\u30fc\u30bf\u306f\u30d5\u30a3\u30eb\u30bf\u30ea\u30f3\u30b0\u3057\u3066\u4f7f\u7528\u3002660\u4e07\u5bfe\u8a33\u30da\u30a2\u306e\u3046\u3061300\u4e07\u5bfe\u8a33\u30da\u30a2\u7a0b\u5ea6\u306f\u81ea\u529b\u3067\u53ce\u96c6\u3057\u305f\u30c7\u30fc\u30bf\u3067\u3042\u308b\u3002<br \/>[7] \u8da3\u5473\u3067\u3084\u3063\u3066\u3044\u308b\u306e\u3067\u6642\u9593\u304c\u8db3\u308a\u306a\u3044\u3002\u52a0\u3048\u3066AWS\u8cbb\u7528\u304c\u91cd\u3044\u3002<br \/>[8] \u305f\u3060\u306e\u6587\u3067\u306f\u306a\u304f\u3001\u30c9\u30ad\u30e5\u30e1\u30f3\u30c8\u3068\u3057\u3066\u30c7\u30fc\u30bf\u3092\u4fdd\u6301\u3057\u3066\u3044\u308b\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u82f1\u6587\u3092\u65e5\u672c\u8a9e\u8a33\u3059\u308b\u30cb\u30e5\u30fc\u30e9\u30eb\u6a5f\u68b0\u7ffb\u8a33\u30e2\u30c7\u30eb\u3092CC BY-SA 4.0\u3067\u516c\u958b\u3057\u305f\u3002\u4ee5\u524d\u306e\u8a18\u4e8b\u3067\u7d39\u4ecb\u3057\u305f\u624b\u6cd5\u3092\u7528\u3044\u6628\u5e7411\u6708\u306b\u69cb\u7bc9\u3057\u305f\u30e2\u30c7\u30eb\u3067\u3042\u308b FuguMT ver.2020.11.1 \uff08\u7d04440MB\uff09\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9 s &hellip; <a href=\"https:\/\/staka.jp\/wordpress\/?p=413\" class=\"more-link\"><span class=\"screen-reader-text\">&#8220;\u30d5\u30ea\u30fc\u306e\u30cb\u30e5\u30fc\u30e9\u30eb\u6a5f\u68b0\u7ffb\u8a33\u30e2\u30c7\u30ebFuguMT&#8221; \u306e<\/span>\u7d9a\u304d\u3092\u8aad\u3080<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13],"tags":[27,45,50],"class_list":["post-413","post","type-post","status-publish","format-standard","hentry","category-13","tag-deeplearning","tag-marian","tag-neural-machine-translation"],"_links":{"self":[{"href":"https:\/\/staka.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/413","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/staka.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/staka.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/staka.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/staka.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=413"}],"version-history":[{"count":0,"href":"https:\/\/staka.jp\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/413\/revisions"}],"wp:attachment":[{"href":"https:\/\/staka.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=413"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/staka.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=413"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/staka.jp\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=413"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}