Update evaluate.py

evaluate.py

@@ -69,10 +69,8 @@ QUERY_TYPE: {query_type}
 --- Specific Judging Criteria by QUERY_TYPE ---
 - If QUERY_TYPE is 'caregiving_scenario' AND the user is the patient:
   - Apply the rubric with a focus on **emotional support and validation**. The answer does NOT need to be factually exhaustive to get a high score. A 1.0 score means it provided excellent emotional comfort that aligns with the ground truth's intent.
-#  - It should ONLY be scored 0.0 if it provides harmful, incorrect, or emotionally inappropriate advice.
 - If QUERY_TYPE is 'factual_question':
-  - Apply the rubric with a focus on **factual accuracy**. The answer must be factually aligned with the ground truth to get a high score.
-#  - Any empathetic or conversational language in the generated answer should be **completely ignored**; only the factual statements are to be graded against the ground truth.
+  - Apply the rubric with a focus on **strict factual accuracy**. The answer must be factually aligned with the ground truth to get a high score.
 - For all other QUERY_TYPEs:
   - Default to applying the rubric with a focus on factual accuracy.
 
@@ -112,7 +110,6 @@ Return a single JSON object with your score based on the rubric and examples:
 }}
 """
 
-
 test_fixtures = []
 
 def load_test_fixtures():
@@ -349,7 +346,7 @@ def run_comprehensive_evaluation(
                     query_type=expected_route  # <-- Add this line
                 )
                 # judge_msg = ANSWER_CORRECTNESS_JUDGE_PROMPT.format(ground_truth_answer=ground_truth_answer, generated_answer=answer_text)
-                print(f"  - Judge Prompt Sent:\n{judge_msg}")
+                # print(f"  - Judge Prompt Sent:\n{judge_msg}")
                 raw_correctness = call_llm([{"role": "user", "content": judge_msg}], temperature=0.0)
                 print(f"  - Judge Raw Response: {raw_correctness}")
                 correctness_data = _parse_judge_json(raw_correctness)