自然语言处理NLP星空智能对话机器人系列:深入理解Transformer自然语言处理 SRL

Gavin大咖语录

在Gavin大咖看来,正常的AI技术人员很难破解Transformer的的玄妙主要是因为其无法体悟贝叶斯公式中P(B)的内涵,从而失去了领悟获得Transfomer真正力量的机会。

QQ图片20211106194820.jpg

在Gavin大咖看来,Transformer是人工智能、贝叶斯理论、认知模型演进的统一架构,Transformer是学术界(无意间)基于Bayesian理论通过神经网络实现的(计算机)认知模型持续的Evolving的理想架构体系。
QQ图片20211106195533.jpg
在Gavin大咖看来,理论上将Transformer能够更好的处理一切以“set of units”存在的数据,而计算机视觉、语音、自然语言处理等属于这种类型的数据,所以理论上讲Transformer会在接下来数十年对这些领域形成主导性的统治力。

QQ图片20211106194815.jpg
Gavin:Transformer是人工智能领域的新一代的引擎,本质是研究结构关系、工业界实践的核心是基于Transformer实现万物皆流。

QQ图片20211106195528.jpg

欢迎加Gavin大咖微信,加入星空智能对话交流群

Gavin二维码.png

本文是SRL系列博客的第6个示例,示例6选取了一个我们通常认为只是名词的单词。然而,比我们想象的更多的词可以是名词和动词。“toice”是曲棍球中的一个动词,用来将一个“puck" 穿过冰场并越过对手的球门线。“puck”是曲棍球中使用的圆盘。
曲棍球教练可以告诉球队训练冰球来开始新的一天。当教练喊叫时,我们就可以得到祈使句

"Now, ice pucks guys!"

请注意,“guys“ 可以指“人”,而不管他们的性别。

让我们运行代码,看看会发生什么

!echo '{"sentence": "Now, ice pucks guys!"}' | \
allennlp predict https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz -

运行结果如下

2020-12-20 09:09:27,286 - INFO - transformers.file_utils - PyTorch version 1.5.1 available.
2020-12-20 09:09:27.438284: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2020-12-20 09:09:29,226 - INFO - transformers.file_utils - TensorFlow version 2.4.0 available.
2020-12-20 09:09:30,428 - INFO - allennlp.common.file_utils - checking cache for https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz at /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724
2020-12-20 09:09:30,428 - INFO - allennlp.common.file_utils - waiting to acquire lock on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724
2020-12-20 09:09:30,429 - INFO - filelock - Lock 139618002246904 acquired on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724.lock
2020-12-20 09:09:30,429 - INFO - allennlp.common.file_utils - cache of https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz is up-to-date
2020-12-20 09:09:30,429 - INFO - filelock - Lock 139618002246904 released on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724.lock
2020-12-20 09:09:30,429 - INFO - allennlp.models.archival - loading archive file https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz from cache at /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724
2020-12-20 09:09:30,430 - INFO - allennlp.models.archival - extracting archive file /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724 to temp dir /tmp/tmpewur_o27
2020-12-20 09:09:34,712 - INFO - allennlp.common.params - type = from_instances
2020-12-20 09:09:34,712 - INFO - allennlp.data.vocabulary - Loading token dictionary from /tmp/tmpewur_o27/vocabulary.
2020-12-20 09:09:34,713 - INFO - filelock - Lock 139618002367880 acquired on /tmp/tmpewur_o27/vocabulary/.lock
2020-12-20 09:09:34,741 - INFO - filelock - Lock 139618002367880 released on /tmp/tmpewur_o27/vocabulary/.lock
2020-12-20 09:09:34,742 - INFO - allennlp.common.params - model.type = srl_bert
2020-12-20 09:09:34,742 - INFO - allennlp.common.params - model.regularizer = None
2020-12-20 09:09:34,742 - INFO - allennlp.common.params - model.bert_model = bert-base-uncased
2020-12-20 09:09:34,742 - INFO - allennlp.common.params - model.embedding_dropout = 0.1
2020-12-20 09:09:34,743 - INFO - allennlp.common.params - model.initializer = <allennlp.nn.initializers.InitializerApplicator object at 0x7efb596c07b8>
2020-12-20 09:09:34,743 - INFO - allennlp.common.params - model.label_smoothing = None
2020-12-20 09:09:34,743 - INFO - allennlp.common.params - model.ignore_span_metric = False
2020-12-20 09:09:34,743 - INFO - allennlp.common.params - model.srl_eval_path = /usr/local/lib/python3.6/dist-packages/allennlp_models/structured_prediction/tools/srl-eval.pl
2020-12-20 09:09:35,046 - INFO - transformers.configuration_utils - loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-config.json from cache at /root/.cache/torch/transformers/4dad0251492946e18ac39290fcfe91b89d370fee250efe9521476438fe8ca185.7156163d5fdc189c3016baca0775ffce230789d7fa2a42ef516483e4ca884517
2020-12-20 09:09:35,047 - INFO - transformers.configuration_utils - Model config BertConfig {
  "architectures": [
    "BertForMaskedLM"
  ],
  "attention_probs_dropout_prob": 0.1,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "type_vocab_size": 2,
  "vocab_size": 30522
}

2020-12-20 09:09:35,254 - INFO - transformers.modeling_utils - loading weights file https://cdn.huggingface.co/bert-base-uncased-pytorch_model.bin from cache at /root/.cache/torch/transformers/f2ee78bdd635b758cc0a12352586868bef80e47401abe4c4fcc3832421e7338b.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157
2020-12-20 09:09:37,949 - INFO - allennlp.nn.initializers - Initializing parameters
2020-12-20 09:09:37,949 - INFO - allennlp.nn.initializers - Done initializing parameters; the following parameters are using their default initialization from their code
2020-12-20 09:09:37,949 - INFO - allennlp.nn.initializers -    bert_model.embeddings.LayerNorm.bias
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.embeddings.LayerNorm.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.embeddings.position_embeddings.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.embeddings.token_type_embeddings.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.embeddings.word_embeddings.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.output.LayerNorm.bias
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.output.LayerNorm.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.output.dense.bias
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.output.dense.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.key.bias
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.key.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.query.bias
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.query.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.value.bias
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.value.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.intermediate.dense.bias
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.intermediate.dense.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.output.LayerNorm.bias
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.output.LayerNorm.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.output.dense.bias
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.output.dense.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.output.LayerNorm.bias
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.output.LayerNorm.weight
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.output.dense.bias
2020-12-20 09:09:37,950 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.output.dense.weight
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.key.bias
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.key.weight
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.query.bias
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.query.weight
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.value.bias
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.value.weight
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.intermediate.dense.bias
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.intermediate.dense.weight
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.output.LayerNorm.bias
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.output.LayerNorm.weight
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.output.dense.bias
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.output.dense.weight
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.output.LayerNorm.bias
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.output.LayerNorm.weight
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.output.dense.bias
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.output.dense.weight
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.key.bias
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.key.weight
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.query.bias
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.query.weight
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.value.bias
2020-12-20 09:09:37,951 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.value.weight
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.intermediate.dense.bias
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.intermediate.dense.weight
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.output.LayerNorm.bias
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.output.LayerNorm.weight
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.output.dense.bias
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.output.dense.weight
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.output.LayerNorm.bias
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.output.LayerNorm.weight
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.output.dense.bias
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.output.dense.weight
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.key.bias
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.key.weight
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.query.bias
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.query.weight
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.value.bias
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.value.weight
2020-12-20 09:09:37,952 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.intermediate.dense.bias
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.intermediate.dense.weight
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.output.LayerNorm.bias
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.output.LayerNorm.weight
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.output.dense.bias
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.output.dense.weight
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.output.LayerNorm.bias
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.output.LayerNorm.weight
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.output.dense.bias
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.output.dense.weight
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.key.bias
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.key.weight
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.query.bias
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.query.weight
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.value.bias
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.value.weight
2020-12-20 09:09:37,953 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.intermediate.dense.bias
2020-12-20 09:09:37,954 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.intermediate.dense.weight
2020-12-20 09:09:37,954 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.output.LayerNorm.bias
2020-12-20 09:09:37,954 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.output.LayerNorm.weight
2020-12-20 09:09:37,954 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.output.dense.bias
2020-12-20 09:09:37,954 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.output.dense.weight
2020-12-20 09:09:37,954 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.output.LayerNorm.bias
2020-12-20 09:09:37,954 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.output.LayerNorm.weight
2020-12-20 09:09:37,954 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.output.dense.bias
2020-12-20 09:09:37,954 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.output.dense.weight
2020-12-20 09:09:37,954 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.key.bias
2020-12-20 09:09:37,954 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.key.weight
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.query.bias
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.query.weight
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.value.bias
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.value.weight
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.intermediate.dense.bias
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.intermediate.dense.weight
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.output.LayerNorm.bias
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.output.LayerNorm.weight
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.output.dense.bias
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.output.dense.weight
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.output.LayerNorm.bias
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.output.LayerNorm.weight
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.output.dense.bias
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.output.dense.weight
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.key.bias
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.key.weight
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.query.bias
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.query.weight
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.value.bias
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.value.weight
2020-12-20 09:09:37,955 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.intermediate.dense.bias
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.intermediate.dense.weight
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.output.LayerNorm.bias
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.output.LayerNorm.weight
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.output.dense.bias
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.output.dense.weight
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.output.LayerNorm.bias
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.output.LayerNorm.weight
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.output.dense.bias
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.output.dense.weight
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.key.bias
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.key.weight
2020-12-20 09:09:37,956 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.query.bias
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.query.weight
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.value.bias
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.value.weight
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.intermediate.dense.bias
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.intermediate.dense.weight
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.output.LayerNorm.bias
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.output.LayerNorm.weight
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.output.dense.bias
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.output.dense.weight
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.output.LayerNorm.bias
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.output.LayerNorm.weight
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.output.dense.bias
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.output.dense.weight
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.key.bias
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.key.weight
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.query.bias
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.query.weight
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.value.bias
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.value.weight
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.intermediate.dense.bias
2020-12-20 09:09:37,957 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.intermediate.dense.weight
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.output.LayerNorm.bias
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.output.LayerNorm.weight
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.output.dense.bias
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.output.dense.weight
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.output.LayerNorm.bias
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.output.LayerNorm.weight
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.output.dense.bias
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.output.dense.weight
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.key.bias
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.key.weight
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.query.bias
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.query.weight
2020-12-20 09:09:37,958 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.value.bias
2020-12-20 09:09:37,968 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.value.weight
2020-12-20 09:09:37,968 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.intermediate.dense.bias
2020-12-20 09:09:37,968 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.intermediate.dense.weight
2020-12-20 09:09:37,968 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.output.LayerNorm.bias
2020-12-20 09:09:37,968 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.output.LayerNorm.weight
2020-12-20 09:09:37,968 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.output.dense.bias
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.output.dense.weight
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.output.LayerNorm.bias
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.output.LayerNorm.weight
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.output.dense.bias
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.output.dense.weight
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.key.bias
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.key.weight
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.query.bias
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.query.weight
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.value.bias
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.value.weight
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.intermediate.dense.bias
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.intermediate.dense.weight
2020-12-20 09:09:37,969 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.output.LayerNorm.bias
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.output.LayerNorm.weight
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.output.dense.bias
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.output.dense.weight
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.output.LayerNorm.bias
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.output.LayerNorm.weight
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.output.dense.bias
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.output.dense.weight
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.key.bias
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.key.weight
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.query.bias
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.query.weight
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.value.bias
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.value.weight
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.intermediate.dense.bias
2020-12-20 09:09:37,970 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.intermediate.dense.weight
2020-12-20 09:09:37,971 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.output.LayerNorm.bias
2020-12-20 09:09:37,971 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.output.LayerNorm.weight
2020-12-20 09:09:37,971 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.output.dense.bias
2020-12-20 09:09:37,971 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.output.dense.weight
2020-12-20 09:09:37,971 - INFO - allennlp.nn.initializers -    bert_model.pooler.dense.bias
2020-12-20 09:09:37,971 - INFO - allennlp.nn.initializers -    bert_model.pooler.dense.weight
2020-12-20 09:09:37,971 - INFO - allennlp.nn.initializers -    tag_projection_layer.bias
2020-12-20 09:09:37,971 - INFO - allennlp.nn.initializers -    tag_projection_layer.weight
2020-12-20 09:09:38,432 - INFO - allennlp.common.params - dataset_reader.type = srl
2020-12-20 09:09:38,433 - INFO - allennlp.common.params - dataset_reader.lazy = False
2020-12-20 09:09:38,433 - INFO - allennlp.common.params - dataset_reader.cache_directory = None
2020-12-20 09:09:38,433 - INFO - allennlp.common.params - dataset_reader.max_instances = None
2020-12-20 09:09:38,433 - INFO - allennlp.common.params - dataset_reader.manual_distributed_sharding = False
2020-12-20 09:09:38,433 - INFO - allennlp.common.params - dataset_reader.manual_multi_process_sharding = False
2020-12-20 09:09:38,433 - INFO - allennlp.common.params - dataset_reader.token_indexers = None
2020-12-20 09:09:38,433 - INFO - allennlp.common.params - dataset_reader.domain_identifier = None
2020-12-20 09:09:38,433 - INFO - allennlp.common.params - dataset_reader.bert_model_name = bert-base-uncased
2020-12-20 09:09:38,744 - INFO - transformers.tokenization_utils - loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt from cache at /root/.cache/torch/transformers/26bc1ad6c0ac742e9b52263248f6d0f00068293b33709fae12320c0e35ccfbbb.542ce4285a40d23a559526243235df47c5f75c197f04f37d1a0c124c32c9a084
input 0:  {"sentence": "Now, ice pucks guys!"}
prediction:  {"verbs": [], "words": ["Now", ",", "ice", "pucks", "guys", "!"]}

2020-12-20 09:09:39,466 - INFO - allennlp.models.archival - removing temporary unarchived model dir at /tmp/tmpewur_o27

transformer 无法找到动词:

prediction:  {"verbs": [], "words": ["Now", ",", "ice", "pucks", "guys", "!"]}

游戏结束!我们可以看到transformers 已经取得了巨大的进步,但是开发者仍然有很大的空间来改进模型。人类还在游戏中!尝试一些您自己的示例或示例,看看SRL可以做什么以及该方法的局限性。

星空智能对话机器人系列博客

评论

暂无评论