Update Space (evaluate main: e51c679b)

README.md

@@ -1,12 +1,59 @@
 ---
-title: Exact_match
-emoji: ⚡
-colorFrom: red
-colorTo: yellow
+title: Exact Match 
+emoji: 🤗 
+colorFrom: blue
+colorTo: green
 sdk: gradio
-sdk_version: 3.0.9
+sdk_version: 3.0.2
 app_file: app.py
 pinned: false
+tags:
+- evaluate
+- comparison
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces#reference
+
+# Comparison Card for Exact Match
+
+## Comparison description
+
+ Given two model predictions the exact match score is 1 if they are the exact same, and is 0 otherwise. The overall exact match score is the average.
+
+- **Example 1**: The exact match score if prediction 1.0 is [0, 1] is 0, given prediction 2 is [0, 1].
+- **Example 2**: The exact match score if prediction 0.0 is [0, 1] is 0, given prediction 2 is [1, 0].
+- **Example 3**: The exact match score if prediction 0.5 is [0, 1] is 0, given prediction 2 is [1, 1].
+
+## How to use 
+
+At minimum, this metric takes as input predictions and references:
+```python
+>>> exact_match = evaluate.load("exact_match", module_type="comparison")
+>>> results = exact_match.compute(predictions1=[0, 1, 1], predictions2=[1, 1, 1])
+>>> print(results)
+{'exact_match': 0.66}
+```
+
+## Output values
+
+Returns a float between 0.0 and 1.0 inclusive.
+
+## Examples 
+
+```python
+>>> exact_match = evaluate.load("exact_match", module_type="comparison")
+>>> results = exact_match.compute(predictions1=[0, 0, 0], predictions2=[1, 1, 1])
+>>> print(results)
+{'exact_match': 1.0}
+```
+
+```python
+>>> exact_match = evaluate.load("exact_match", module_type="comparison")
+>>> results = exact_match.compute(predictions1=[0, 1, 1], predictions2=[1, 1, 1])
+>>> print(results)
+{'exact_match': 0.66}
+```
+
+
+## Limitations and bias
+
+## Citations

app.py

@@ -0,0 +1,6 @@
+import evaluate
+from evaluate.utils import launch_gradio_widget
+
+
+module = evaluate.load("exact_match", module_type="comparison")
+launch_gradio_widget(module)

exact_match.py

@@ -0,0 +1,65 @@
+# Copyright 2022 The HuggingFace Evaluate Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Exact match test for model comparison."""
+
+import datasets
+import numpy as np
+
+import evaluate
+
+
+_DESCRIPTION = """
+Returns the rate at which the predictions of one model exactly match those of another model.
+"""
+
+
+_KWARGS_DESCRIPTION = """
+Args:
+    predictions1 (`list` of `int`): Predicted labels for model 1.
+    predictions2 (`list` of `int`): Predicted labels for model 2.
+
+Returns:
+    exact_match (`float`): Dictionary containing exact_match rate. Possible values are between 0.0 and 1.0, inclusive.
+
+Examples:
+    >>> exact_match = evaluate.load("exact_match", module_type="comparison")
+    >>> results = exact_match.compute(predictions1=[1, 1, 1], predictions2=[1, 1, 1])
+    >>> print(results)
+    {'exact_match': 1.0}
+"""
+
+
+_CITATION = """
+"""
+
+
+@evaluate.utils.file_utils.add_start_docstrings(_DESCRIPTION, _KWARGS_DESCRIPTION)
+class ExactMatch(evaluate.EvaluationModule):
+    def _info(self):
+        return evaluate.EvaluationModuleInfo(
+            module_type="comparison",
+            description=_DESCRIPTION,
+            citation=_CITATION,
+            inputs_description=_KWARGS_DESCRIPTION,
+            features=datasets.Features(
+                {
+                    "predictions1": datasets.Value("int64"),
+                    "predictions2": datasets.Value("int64"),
+                }
+            ),
+        )
+
+    def _compute(self, predictions1, predictions2):
+        score_list = predictions1 == predictions2
+        return {"exact_match": np.mean(score_list)}

requirements.txt

@@ -0,0 +1,4 @@
+# TODO: fix github to release
+git+https://github.com/huggingface/evaluate.git@505123230059f9605da8951880eddc9d1fbf4278
+datasets~=2.0
+scipy