HaHackathon Evaluation Results

SemEval 2021 Task 7

Task 1a: predict if the text would be considered humorous (for an average user)

Rank User F-Score Accuracy
       
1 PALI 0.982 0.9854
2 stce 0.975 0.9797
3 DeepBlueAI 0.96 0.9676
4 jm-team 0.96 0.9675
4 dalya 0.96 0.9675
5 mengyuan_jiayi 0.959 0.9667
6 stevenhuahua 0.958 0.9666
7 zain 0.958 0.9663
8 ThisIstheEnd 0.957 0.9655
9 MagicPai 0.957 0.9653
9 Meizizi 0.957 0.9653
10 mmmm 0.956 0.9647
11 fdabek 0.956 0.9647
12 Isra 0.955 0.964
13 DLJUST 0.954 0.9633
14 tathagataraha 0.953 0.9616
15 megatron 0.952 0.9612
16 CS-UM6P 0.951 0.9606
17 Amherst685 0.951 0.9604
18 MLXG 0.949 0.959
19 abcbpc 0.948 0.9587
20 StoneOpen 0.948 0.9583
21 Humor@IITK 0.948 0.9581
21 Ferryman 0.948 0.9581
22 reynier 0.948 0.9576
23 calamitylink 0.948 0.9572
24 ryanachi 0.946 0.9571
25 razvan.smadu 0.947 0.9566
26 emran 0.946 0.9564
27 DeathwingS 0.946 0.9563
28 zeus_yao 0.945 0.9557
29 apostaremczak 0.944 0.9544
30 LeoJ 0.943 0.9543
31 CHAOYUDENG 0.941 0.9538
32 gerarld 0.942 0.9532
33 kabil_ESSEFAR 0.938 0.9506
34 csecudsg 0.938 0.9496
35 mayukh 0.933 0.9468
36 Sakrah 0.926 0.9399
37 pakawat.nk 0.924 0.9386
38 Grenzlinie 0.925 0.9386
39 bousselham 0.92 0.9368
40 LucasHub 0.921 0.9364
41 zhaoyingjia123 0.921 0.9348
42 xjh 0.918 0.9345
43 Maoqin 0.919 0.9341
44 chenshi 0.916 0.9328
45 JAGD 0.916 0.9325
46 Han_Jiawei 0.912 0.9286
47 Zehao_Liu 0.906 0.9241
48 Anik 0.903 0.9233
49 GuanZhengyi 0.896 0.9205
50 chilai1996 0.897 0.9177
51 ayushnanda14 0.884 0.9081
52 alexkara23 0.872 0.8942
53 jam 0.857 0.884
54 LOLASING 0.849 0.8704
55 CHaines 0.817 0.8504
56 AlviIshmam 0.816 0.8489
57 milad.sayadamooz 0.527 0.629
58 MihaiSamson 0.078 0.063

 

Task 1b: if the text is classed as humorous, predict how humorous it is (for an average user), from 0 to 5.

Rank User RMSE
     
1 abcbpc 0.4959
2 mmmm 0.4977
3 Humor@IITK 0.521
4 mayukh 0.5257
5 tathagataraha 0.5263
6 fdabek 0.5271
7 Amherst685 0.5339
8 gerarld 0.5393
9 CS-UM6P 0.5401
10 jm-team 0.5446
10 dalya 0.5446
11 StoneOpen 0.547
12 alexkara23 0.5507
13 calamitylink 0.551
14 DLJUST 0.5555
15 DeathwingS 0.5561
16 MagicPai 0.5572
17 Han_Jiawei 0.5577
18 ryanachi 0.558
19 MihaiSamson 0.5598
20 DeepBlueAI 0.5607
21 mengyuan_jiayi 0.5621
22 Ferryman 0.5651
23 Anik 0.5694
24 pakawat.nk 0.57
25 Paima 0.5701
26 emran 0.5709
27 zain 0.5748
28 CHaines 0.5762
29 stevenhuahua 0.5831
30 reynier 0.5905
31 Meizizi 0.6136
32 razvan.smadu 0.62
33 LucasHub 0.6288
34 chenshi 0.6303
35 megatron 0.6307
36 Grenzlinie 0.6312
37 kabil_ESSEFAR 0.636
38 xjh 0.6385
39 Sakrah 0.6461
40 ThisIstheEnd 0.6539
41 csecudsg 0.6803
42 GuanZhengyi 0.701
43 zhaoyingjia123 0.7214
44 Maoqin 0.7405
45 apostaremczak 0.8497
46 jam 0.8609
47 JAGD 0.8847
48 abhideepmitra 1.0343
49 MLXG 2.1883
49 ayushnanda14 2.1883
49 LeoJ 2.1883
49 chilai1996 2.1883
50 milad.sayadamooz 2.5497

 

Task 1c: if the text is classed as humorous, predict if the humor rating would be considered controversial
i.e. the variance of the rating between annotators is higher than the median. This is a binary task.

Rank User F-Score Accuracy
       
1 PALI 0.4943 0.6302
2 mmmm 0.4699 0.6279
3 dalya 0.4699 0.627
3 jm-team 0.4699 0.627
4 ThisIstheEnd 0.4602 0.6261
5 DeepBlueAI 0.465 0.6257
6 CS-UM6P 0.4537 0.6242
6 kabil_ESSEFAR 0.4537 0.6242
6 CHaines 0.4537 0.6242
6 Ferryman 0.4537 0.6242
6 tathagataraha 0.4537 0.6242
6 abcbpc 0.4537 0.6242
7 fdabek 0.4537 0.6233
8 mayukh 0.478 0.621
9 Humor@IITK 0.452 0.6209
10 reynier 0.4732 0.6197
11 calamitylink 0.4764 0.6111
12 alexkara23 0.4732 0.599
13 mengyuan_jiayi 0.5106 0.5814
14 JAGD 0.465 0.5722
15 Anik 0.5301 0.5628
16 LucasHub 0.5333 0.5591
17 chenshi 0.5301 0.5547
18 Maoqin 0.5561 0.5488
19 Grenzlinie 0.5203 0.5455
20 StoneOpen 0.5561 0.5427
21 xjh 0.5447 0.5205
22 stevenhuahua 0.5626 0.4991
23 gerarld 0.5659 0.4972
24 Han_Jiawei 0.5268 0.4904
25 emran 0.5545 0.4888
26 ryanachi 0.5024 0.4883
27 Amherst685 0.522 0.4842
28 DLJUST 0.548 0.4813
29 MihaiSamson 0.5008 0.4752
30 pakawat.nk 0.5496 0.4683
31 jam 0.4374 0.4624
32 abhideepmitra 0.5366 0.4612
33 zhaoyingjia123 0.4407 0.4603
34 csecudsg 0.5366 0.4423
35 GuanZhengyi 0.5593 0.4271
36 milad.sayadamooz 0.5463 0
36 MLXG 0.5463 0
36 ayushnanda14 0.5463 0
36 LeoJ 0.5463 0
36 chilai1996 0.5463 0
36 razvan.smadu 0.5463 0
36 apostaremczak 0.4341 0

 

Task 2a: predict how generally offensive a text is for users.
This score was calculated regardless of whether the text is classed as humorous or offensive overall.

 

Rank User RMSE
     
1 DeepBlueAI 0.412
2 mmmm 0.419
3 calamitylink 0.423
4 abcbpc 0.4275
5 fdabek 0.4406
6 stevenhuahua 0.4454
7 megatron 0.4456
8 MagicPai 0.446
9 emran 0.4467
10 dalya 0.4469
11 StoneOpen 0.4489
11 gerarld 0.4489
12 mayukh 0.45
13 Amherst685 0.453
14 reynier 0.4532
15 jm-team 0.456
16 Humor@IITK 0.4607
17 zeus_yao 0.4621
18 Paima 0.4655
19 ThisIstheEnd 0.4691
20 CS-UM6P 0.4696
21 kabil_ESSEFAR 0.4759
22 Grenzlinie 0.4761
23 tathagataraha 0.4772
24 MihaiSamson 0.4788
25 Ferryman 0.4813
26 DLJUST 0.4822
27 LucasHub 0.5027
28 Sakrah 0.5059
29 xjh 0.5151
30 Han_Jiawei 0.5187
31 zhaoyingjia123 0.5204
32 razvan.smadu 0.5318
33 pakawat.nk 0.5368
34 csecudsg 0.5395
35 GuanZhengyi 0.5419
36 chenshi 0.5422
37 apostaremczak 0.5625
38 Anik 0.58
39 Maoqin 0.5807
40 alexkara23 0.5819
41 CNA 0.6347
42 jam 0.6415
43 CHaines 0.6473
44 LOLASING 0.7106
45 ryanachi 0.7229
46 JAGD 0.874
47 milad.sayadamooz 0.9587
47 MLXG 0.9587
47 ayushnanda14 0.9587
47 LeoJ 0.9587
47 chilai1996 0.9587
48 PALI 0.971