Merged in feature/mb-prompt-and-testbed-cleanup (pull request #536)

Refactor methodology breakout prompt, small testbed cleanup

* Refactor REIMB_TERM_BREAKOUT prompt.  Include service AND methodology.  Change to sections formatting.

* rename some columns in `Reimbursement Primary` tab of test output


Approved-by: Katon Minhas

fieldExtraction/src/prompts/investment_prompts.py

@@ -449,19 +449,23 @@ Return properly formatted JSON array with all extracted reimbursement items.
 """
 
 def REIMB_TERM_BREAKOUT(service, methodology, questions):
-    # Right now we pass in `service` but don't use it.  Other templates do, and we want to maintain
-    # the function signature (service, methodology, questions) for consistency
-    return f"""Analyze a given reimbursement methodology from a Payer-Provider contract:
+    return f"""[OBJECTIVE]
+Analyze a given service-reimbursement methodology pair from a Payer-Provider contract:
 
+[A: OUTPUT FORMAT]
 Populate a list of JSON dictionaries with the following fields:
-    
 {questions}
+The output should always be a JSON list/array of dictionaries, even when there is only one methodology.
+For any fields that don't apply to a particular methodology, use "N/A" as the value.
 
+[B: KEY EXTRACTION RULES]
 In some cases, there will only be one reimbursement method listed. In these cases, the list should only contain a single JSON dictionary with all fields present.
 In other cases, there will be multiple reimbursement methods listed. This is particularly common in the presence of 'lesser of' language. In these cases, the list should contain a list of JSON dictionaries with all fields present for each individual methodology seen. This may be two or more dictionaries in a single list, especially when there is both 'lesser of' as well as 'not to exceed' language present.
 
+[C: ANALYSIS CONTEXT]
 Here is the text to analyze and respond to:
-{methodology.replace('"', "'")}
+Service: {service.replace('"', "'")}
+Methodology: {methodology.replace('"', "'")}
 
 Write your JSON list below:
 """

fieldExtraction/src/testbed/testbed_utils.py

@@ -455,11 +455,11 @@ def compare_term_extraction_results(file_name: str, predictions_df: pd.DataFrame
         "file_name": file_name,
         "pred_count": len(pred_unique_set),
         "testbed_count": len(gt_unique_set),
-        "pred_pairs": pred_unique_set,
-        "testbed_pairs": gt_unique_set,
         "common_pairs": common_pairs,
         "false_negatives": false_negatives,
         "false_positives": false_positives,
+        "all_predicted_pairs": pred_unique_set,
+        "all_testbed_pairs": gt_unique_set,
     }
 
 def calculate_precision_recall(accuracies):