高性能小模型排行榜*记最佳得分(模型名称需带有nlpcc标识,模型参数低于1/9的bert-base参数量)
排行 | 模型 | 研究机构 | 测评时间 | Score | 认证 | CLUEWSC 2020 | CSL | CLUENER | CMRC2018 |
---|
1 | nlpcc-tiny-NEZHA-8L | Huawei Cloud & Noah's Ark lab | 20-08-25 | 78.897 | 待认证 | 80.345 | 82.933 | 78.058 | 74.250 |
2 | nlpcc-tiny-NEZHA-6L | Huawei Cloud & Noah's Ark lab | 20-09-08 | 77.587 | 待认证 | 80.690 | 81.200 | 76.459 | 72.000 |
3 | nlpcc-tiny-bert-6L | Xiaomi AI Lab | 20-06-28 | 76.860 | 待认证 | 80.690 | 78.567 | 79.082 | 69.100 |
4 | nlpcc-bert-base-test | mi-ailab-nlp-app | 20-03-31 | 76.047 | 待认证 | 74.828 | 81.267 | 77.441 | 70.650 |
5 | nlpcc-bert-tiny | TencentPretrain & TI-ONE | 20-05-20 | 73.507 | 待认证 | 73.448 | 74.167 | 75.613 | 70.800 |
6 | nlpcc-tiny-bert | Xiaomi AI Lab | 20-05-27 | 73.114 | 待认证 | 67.931 | 78.567 | 76.858 | 69.100 |
7 | nlpcc-tiny-NEZHA-4L | Huawei Cloud & Noah's Ark lab | 20-05-09 | 72.514 | 待认证 | 66.207 | 79.767 | 75.583 | 68.500 |
8 | nlpcc-xiaomi-tiny-6L | mi-ailab-nlp-app | 20-05-03 | 72.045 | 待认证 | 67.931 | 79.033 | 75.166 | 66.050 |
9 | nlpcc-bert-small | Tencent TEG | 20-05-01 | 69.704 | 待认证 | 63.448 | 76.767 | 74.851 | 63.750 |
10 | nlpcc-RoBERTa-tiny-clue | NUDT | 20-06-29 | 69.298 | 待认证 | 64.828 | 78.700 | 67.414 | 66.250 |
11 | nlpcc-tiny-bert-6L | mi-ailab-nlp-app | 20-04-25 | 68.577 | 待认证 | 62.414 | 76.733 | 73.262 | 61.900 |
12 | nlpcc-BERT-4T | Huawei-ICS-NLP | 20-05-20 | 67.885 | 待认证 | 63.793 | 75.500 | 72.996 | 59.250 |
13 | nlpcc-roberta_tiny | mi-ailab-nlp-app | 20-04-07 | 67.421 | 待认证 | 62.414 | 72.733 | 72.287 | 62.250 |
14 | nlpcc-tiny | mi-ailab-nlp-app | 20-04-02 | 67.237 | 待认证 | 62.759 | 72.433 | 71.507 | 62.250 |
15 | nlpcc-roberta-tiny-clue-baseline1 | test-baseline1 | 20-03-31 | 66.580 | 待认证 | 60.690 | 72.033 | 71.745 | 61.850 |
16 | nlpcc-RoBERTa-tiny-clue | dlnu-5.20 | 20-05-20 | 63.423 | 待认证 | 62.414 | 66.000 | 66.429 | 58.850 |
17 | nlpcc-RoBERTa-tiny-clue | dlnu-5.19 | 20-05-19 | 63.337 | 待认证 | 62.069 | 66.000 | 66.429 | 58.850 |
18 | nlpcc-RoBERTa-tiny-clue | dlnu-xiaomi | 20-05-15 | 62.043 | 待认证 | 62.414 | 66.000 | 60.908 | 58.850 |
19 | nlpcc-RoBERTa-tiny-clue | 民大F4 | 20-05-19 | 62.043 | 待认证 | 62.414 | 66.000 | 60.908 | 58.850 |
20 | nlpcc-roberta-tiny-clue | 中国平安财产险科技中心 | 20-05-11 | 61.191 | 待认证 | 60.690 | 68.567 | 60.805 | 54.700 |
https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TinyBERT
NEZHA-base模型做老师,采用tiny-bert蒸馏压缩技术
11M
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TinyBERT
NEZHA-base模型做老师,采用tiny-bert蒸馏压缩技术
11M
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
none
6层预训练
11M
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
none
中文bert-chinese
12层bert-base
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
https://github.com/dbiir/UER-py
-
-
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
none
6L
11m
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TinyBERT
NEZHA-base模型做老师,利用tiny-bert蒸馏压缩技术蒸馏4层小模型
四层Transformer结构,采用NEZHA模型的相对位置编码
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
none
6层预训练
11.9M
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
https://github.com/dbiir/UER-py
-
-
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
https://www.cluebenchmarks.com/
-
-
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
none
6层预训练
11m
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
huawei.com
BERT-4T
< 1/9 of bert base
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
none
roberta_tiny_clue
L4-H312-A12
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
none
bert-tiny (4layer)
L4-H312-A12
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
none
roberta_tiny_clue1
roberta_tiny_clue base-line1
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
https://github.com/wenbomi/csl-
无
无
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
https://github.com/pxk8001/NLPCC
-
-
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
https://github.com/wenbomi/csl-
无
无
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
https://github.com/wenbomi/csl-
无
无
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
https://github.com/CLUEbenchmark/CLUE
roberta-tiny
roberta-tiny
| C | E | N |
---|
C | 0.0 | 0.0 | 0.0 |
E | 0.0 | 0.0 | 0.0 |
N | 0.0 | 0.0 | 0.0 |
ALBERT(Ensemble)
GitHub/模型网址:
提交日期:9月17日
分数:9月17日
更多详情:
参数说明
单任务微调。我们从MNLI为RTE、STS和MRPC优化的模型开始
诊断信息
诊断主混淆矩阵
|
C |
N |
E |
C |
182 |
36 |
40 |
N |
81 |
189 |
116 |
E |
17 |
69 |
374 |
C = 对立
N = 不包含
E = 包含