Spaces:
Runtime error
Runtime error
wzuidema
commited on
edited explanation
Browse files
app.py
CHANGED
|
@@ -285,11 +285,11 @@ iface = gradio.Parallel(hila, lig,
|
|
| 285 |
description="""
|
| 286 |
In this demo, we use the RoBERTa language model (optimized for masked language modelling and finetuned for sentiment analysis).
|
| 287 |
The model predicts for a given sentences whether it expresses a positive, negative or neutral sentiment.
|
| 288 |
-
But how does it arrive at its classification?
|
|
|
|
|
|
|
| 289 |
|
| 290 |
-
|
| 291 |
-
|
| 292 |
-
Two key methods for Transformers are "attention rollout" (Abnar & Zuidema, 2020) and (layer) Integrated Gradient. Here we show:
|
| 293 |
|
| 294 |
* Gradient-weighted attention rollout, as defined by [Hila Chefer](https://github.com/hila-chefer)
|
| 295 |
[(Transformer-MM_explainability)](https://github.com/hila-chefer/Transformer-MM-Explainability/), without rollout recursion upto selected layer
|
|
@@ -314,7 +314,7 @@ Two key methods for Transformers are "attention rollout" (Abnar & Zuidema, 2020)
|
|
| 314 |
],
|
| 315 |
[
|
| 316 |
"If he had hated it, he would not have said that he loved it.",
|
| 317 |
-
|
| 318 |
],
|
| 319 |
[
|
| 320 |
"Attribution methods are very interesting, but unfortunately do not work reliably out of the box.",
|
|
|
|
| 285 |
description="""
|
| 286 |
In this demo, we use the RoBERTa language model (optimized for masked language modelling and finetuned for sentiment analysis).
|
| 287 |
The model predicts for a given sentences whether it expresses a positive, negative or neutral sentiment.
|
| 288 |
+
But how does it arrive at its classification? This is, surprisingly perhaps, very difficult to determine.
|
| 289 |
+
A range of so-called "attribution methods" have been developed that attempt to determine the importance of the words in the input for the final prediction;
|
| 290 |
+
they provide a very limited form of "explanation" -- and often disagree -- but sometimes provide good initial hypotheses nevertheless that can be further explored with other methods.
|
| 291 |
|
| 292 |
+
Two key attribution methods for Transformers are "Attention Rollout" (Abnar & Zuidema, 2020) and (layer) Integrated Gradient. Here we show:
|
|
|
|
|
|
|
| 293 |
|
| 294 |
* Gradient-weighted attention rollout, as defined by [Hila Chefer](https://github.com/hila-chefer)
|
| 295 |
[(Transformer-MM_explainability)](https://github.com/hila-chefer/Transformer-MM-Explainability/), without rollout recursion upto selected layer
|
|
|
|
| 314 |
],
|
| 315 |
[
|
| 316 |
"If he had hated it, he would not have said that he loved it.",
|
| 317 |
+
2
|
| 318 |
],
|
| 319 |
[
|
| 320 |
"Attribution methods are very interesting, but unfortunately do not work reliably out of the box.",
|