Commit before claude code

about.py

@@ -19,13 +19,25 @@ Antibodies have to be manufacturable, stable in high concentrations, and have lo
 Properties such as these can often hinder the progression of an antibody to the clinic, and are collectively referred to as 'developability'.
 Here we invite the community to submit and develop better predictors, which will be tested out on a heldout private set to assess model generalization.
 
+#### 🧬 Developability properties in this competition
+
+1. 💧 Hydrophobicity
+2. 🎯 Polyreactivity
+3. 🧲 Self-association
+4. 🌡️ Thermostability
+5. 🧪 Titer
+
 #### 🏆 Prizes
 
 For each of the 5 properties in the competition, there is a prize for the model with the highest performance for that property on the private test set.
 There is also an 'open-source' prize for the best model trained on the GDPa1 dataset of monoclonal antibodies (reporting cross-validation results) and assessed on the private test set where authors provide all training code and data.
-For each of these 6 prizes, participants have the choice between **$10k in data generation credits** with [Ginkgo Datapoints](https://datapoints.ginkgo.bio/) or a **cash prize** with a value of **$2000**.
+For each of these 6 prizes, participants have the choice between
+- **$10 000 in data generation credits** with [Ginkgo Datapoints](https://datapoints.ginkgo.bio/), or
+- A **$2000 cash prize**.
 
 See the "{FAQ_TAB_NAME}" tab above (you are currently on the "{ABOUT_TAB_NAME}" tab) or the [competition terms]({TERMS_URL}) for more details.
+
+---
 """
 
 ABOUT_TEXT = f"""
@@ -34,13 +46,15 @@ ABOUT_TEXT = f"""
 
 1. **Create a Hugging Face account** [here](https://huggingface.co/join) if you don't have one yet (this is used to track unique submissions and to access the GDPa1 dataset).
 2. **Register your team** on the [Competition Registration](https://datapoints.ginkgo.bio/ai-competitions/2025-abdev-competition) page.
-3. **Build a model** or validate it on the [GDPa1](https://huggingface.co/datasets/ginkgo-datapoints/GDPa1) dataset.
-4. **Complete the "Qualifying Exam"**. Before you can submit to the final test set, you must first get a score on the public leaderboard. Choose one of the two tracks:
-    - Track 1 (Benchmark an existing model): Submit predictions for the `GDPa1` dataset.
-    - Track 2 (Train from scratch): Train a model using cross-validation on the `GDPa1` dataset and submit cross-validation predictions by selecting `GDPa1_cross_validation`.
-5. **Submit to the "Final Exam"**. Once you have submitted predictions on the validation set, download the private test set sequences from the {SUBMIT_TAB_NAME} tab and submit your final predictions. Your performance on this private set will determine the winners.
+3. **Build a model** using cross-validation on the [GDPa1](https://huggingface.co/datasets/ginkgo-datapoints/GDPa1) dataset, using the `hierarchical_cluster_IgG_isotype_stratified_fold` column to split the dataset into folds, and write out all cross-validation predictions to a CSV file.
+4. **Use your model to make predictions** on the private test set (download the 80 private test set sequences from the {SUBMIT_TAB_NAME} tab).
+5. **Submit your training and test set predictions** on the {SUBMIT_TAB_NAME} tab by uploading both your cross-validation and private test set CSV files.
 
-Submissions close on **1 November 2025**.
+Check out our introductory tutorial on training an antibody developability prediction model with cross-validation [here]({TUTORIAL_URL}).
+
+⏰ Submissions close on **1 November 2025**.
+
+---
 
 #### Acknowledgements
 
@@ -53,6 +67,8 @@ We gratefully acknowledge [Tamarind Bio](https://www.tamarind.bio/)'s help in ru
 
 We're working on getting more public models added, so that participants have more precomputed features to use for modeling.
 
+---
+
 #### How to contribute?
 
 We'd like to add some more existing developability models to the leaderboard. Some examples of models we'd like to add:
@@ -62,6 +78,8 @@ We'd like to add some more existing developability models to the leaderboard. So
 
 If you would like to form a team or discuss ideas, join the [Slack community]({SLACK_URL}) co-hosted by Bits in Bio.
 """
+# TODO(Lood): Add "📊 The first test set results will be released on October 13th, ahead of the final submission deadline on November 1st."
+
 
 # Note(Lood): Significance: Add another note of "many models are trained on different datasets, and differing train/test splits, so this is a consistent way of comparing for a heldout set"
 FAQS = {
@@ -98,7 +116,7 @@ FAQS = {
     ),
     "How exactly can I evaluate my model?": (
         "You can easily calculate the Spearman correlation coefficient on the GDPa1 dataset yourself before uploading to the leaderboard. "
-        "Simply use the `spearmanr(predictions, targets, nan_policy='omit')` function from `scipy.stats`. "
+        "Simply use the `spearmanr(predictions, targets, nan_policy='omit')` function from `scipy.stats` to calculate the Spearman correlation coefficient for each of the 5 folds, and then take the average."
         "For the heldout private set, we will calculate these Spearman correlations privately at the end of the competition (and possibly at other points throughout the competition) - but there will not be 'rolling results' on the private test set to prevent test set leakage."
     ),
     "How often does the leaderboard update?": (
@@ -114,7 +132,7 @@ FAQS = {
         "We reserve the right to award the open-source prize to a predictor with competitive results for a subset of properties (e.g. a top polyreactivity model)."
     ),
     "How does the open-source prize work?": (
-        "Participants who open-source their code and methods will be eligible for the open-source prize (as well as the other prizes)."
+        "Participants who open-source their training code and methods will be eligible for the open-source prize (as well as the other prizes)."
     ),
     "What do I need to submit?": (
         'There is a tab on the Hugging Face competition page to upload predictions for datasets - for each dataset participants need to submit a CSV containing a column for each property they would like to predict (e.g. called "HIC"), '
@@ -124,11 +142,8 @@ FAQS = {
     "Can I submit predictions for only one property?": (
         "Yes. You do not need to predict all 5 properties to participate. Each property has its own leaderboard and prize, so you may submit models for a subset of the assays if you wish."
     ),
-    "Can I switch between Track 1 and Track 2 during the competition?": (
-        "Yes. You may submit to both tracks. For example, you can benchmark an existing model on the GDPa1 dataset (Track 1) and later also train and submit a cross-validation model on GDPa1 (Track 2)."
-    ),
     "Are participants required to use the provided cross-validation splits?": (
-        "Yes, if submitting cross-validation results, to ensure fair comparison. The results will be calculated by taking the average Spearman correlation coefficient across all folds."
+        "Yes, to ensure fair comparison between different trained models. The results will be calculated by taking the average Spearman correlation coefficient across all folds."
     ),
     "Are there any country restrictions for prize eligibility?": (
         "Yes. Due to applicable laws, prizes cannot be awarded to participants from countries under U.S. sanctions. See the competition terms for details."
@@ -141,8 +156,6 @@ FAQS = {
 
 SUBMIT_INTRUCTIONS = f"""
 # Antibody Developability Submission
-Upload CSV files to get your scores!
-List of valid property names: `{', '.join(ASSAY_LIST)}`.
 
 You do **not** need to predict all 5 properties — each property has its own leaderboard and prize.
 
@@ -151,15 +164,16 @@ You do **not** need to predict all 5 properties — each property has its own le
    - **GDPa1 Cross-Validation predictions** (using cross-validation folds)
    - **Private Test Set predictions** (final test submission)
 2. Each CSV should contain `antibody_name` + one column per property you are predicting (e.g. `"antibody_name,Titer,PR_CHO"` if your model predicts Titer and Polyreactivity).
+   - List of valid property names: `{', '.join(ASSAY_LIST)}`.
 
-The GDPa1 results should appear on the leaderboard within a minute, and can also be calculated manually offline. The **private test set results will not appear on the leaderboards**, and will be used to determine the winners at the close of the competition.
+The GDPa1 results should appear on the leaderboard within a minute, and can also be calculated manually using Spearman rank correlation. The **private test set results will not appear on the leaderboards at first**, and will be used to determine the winners at the close of the competition.
 We may release private test set results at intermediate points during the competition.
 
 ## Cross-validation
 
 For the GDPa1 cross-validation predictions, use the `"hierarchical_cluster_IgG_isotype_stratified_fold"` column to split the dataset into folds and make predictions for each of the folds.
 Submit a CSV file in the same format but also containing the `"hierarchical_cluster_IgG_isotype_stratified_fold"` column.
-Check out our tutorial on making an antibody developability prediction model [here]({TUTORIAL_URL}).
+Check out our tutorial on training an antibody developability prediction model with cross-validation [here]({TUTORIAL_URL}).
 
 Submissions close on **1 November 2025**.
 """

app.py

@@ -50,7 +50,6 @@ def get_leaderboard_object(assay: str | None = None):
     filter_columns = ["dataset"]
     if assay is None:
         filter_columns.append("property")
-    # TODO how to sort filter columns alphabetically?
     # Bug: Can't leave search_columns empty because then it says "Column None not found in headers"
     # Note(Lood): Would be nice to make it clear that the Search Column is searching on model name
     current_dataframe = pd.read_csv("debug-current-results.csv")
@@ -101,11 +100,6 @@ async def periodic_data_fetch(app):
     event.set()
     t.join(3)
 
-
-# Lood: Two problems currently:
-# 1. The data_version state value isn't being incremented, it seems (even though it's triggering the dataframe change correctly)
-# 2. The global current_dataframe is being shared across all sessions
-
 # Make font size bigger using gradio theme
 with gr.Blocks(theme=gr.themes.Default(text_size=sizes.text_lg)) as demo:
     timer = gr.Timer(3)  # Run every 3 seconds when page is focused
@@ -131,6 +125,7 @@ with gr.Blocks(theme=gr.themes.Default(text_size=sizes.text_lg)) as demo:
                 show_label=False,
                 show_download_button=False,
                 show_share_button=False,
+                show_fullscreen_button=False,
                 width="25vw",  # Take up the width of the column (2/8 = 1/4)
             )
 
@@ -138,30 +133,34 @@ with gr.Blocks(theme=gr.themes.Default(text_size=sizes.text_lg)) as demo:
         with gr.TabItem(ABOUT_TAB_NAME, elem_id="abdev-benchmark-tab-table"):
             gr.Markdown(ABOUT_INTRO)
             gr.Image(
-                value="./assets/prediction_explainer.png",
+                value="./assets/prediction_explainer_cv.png",
                 show_label=False,
                 show_download_button=False,
                 show_share_button=False,
-                width="50vw",
+                show_fullscreen_button=False,
+                width="30vw",
             )
             gr.Markdown(ABOUT_TEXT)
+            
+            # Sequence download buttons
+            gr.Markdown(
+            """### 📥 Download Sequences
+            The GDPa1 dataset (with assay data and sequences) is available on Hugging Face [here](https://huggingface.co/datasets/ginkgo-datapoints/GDPa1), 
+            but we provide this and the private test set for convenience.""")
+            with gr.Row():
+                with gr.Column():
+                    download_button_cv_about = gr.DownloadButton(
+                        label="📥 Download GDPa1 sequences",
+                        value=SEQUENCES_FILE_DICT["GDPa1_cross_validation"],
+                        variant="secondary",
+                    )
+                with gr.Column():
+                    download_button_test_about = gr.DownloadButton(
+                        label="📥 Download Private Test Set sequences",
+                        value=SEQUENCES_FILE_DICT["Heldout Test Set"],
+                        variant="secondary",
+                    )
 
-        # Procedurally make these 5 tabs
-        # for i, assay in enumerate(ASSAY_LIST):
-        #     with gr.TabItem(
-        #         f"{ASSAY_EMOJIS[assay]} {ASSAY_RENAME[assay]}",
-        #         elem_id="abdev-benchmark-tab-table",
-        #     ) as tab_item:
-        #         gr.Markdown(f"# {ASSAY_DESCRIPTION[assay]}")
-        #         lb = get_leaderboard_object(assay=assay)
-
-        #         def refresh_leaderboard(assay=assay):
-        #             return format_leaderboard_table(df_results=current_dataframe, assay=assay)
-
-        #         # Refresh when data version changes
-        #         data_version.change(fn=refresh_leaderboard, outputs=lb)
-
-        # Note(Lood): Trying out just one leaderboard. We could also have a dropdown here that shows different leaderboards for each property, but that's just the same as the filters
         with gr.TabItem(
             "🏆 Leaderboard", elem_id="abdev-benchmark-tab-table"
         ) as leaderboard_tab:
@@ -171,18 +170,13 @@ with gr.Blocks(theme=gr.themes.Default(text_size=sizes.text_lg)) as demo:
                 Each property has its own prize, and participants can submit models for any combination of properties.
 
                 **Note**: It is *easy to overfit* the public GDPa1 dataset, which results in artificially high Spearman correlations.
-                We would suggest training using cross-validation a limited number of times to give a better indication of the model's performance on the eventual private test set.
+                We would suggest training using cross-validation to give a better indication of the model's performance on the eventual private test set.
                 """
             )
             lb = get_leaderboard_object()
             timer.tick(fn=refresh_overall_leaderboard, outputs=lb)
             demo.load(fn=refresh_overall_leaderboard, outputs=lb)
 
-            # At the bottom of the leaderboard, we can keep as NaN and explain missing test set results
-            # gr.Markdown(
-            #     "_ℹ️ Results for the private test set will not be shown here and will be used for final judging at the close of the competition._"
-            # )
-
         with gr.TabItem(SUBMIT_TAB_NAME, elem_id="boundary-benchmark-tab-table"):
             gr.Markdown(SUBMIT_INTRUCTIONS)
 
@@ -218,9 +212,6 @@ with gr.Blocks(theme=gr.themes.Default(text_size=sizes.text_lg)) as demo:
 
                 with gr.Column():
                     gr.Markdown("### Upload Both Submission Files")
-                    gr.Markdown(
-                        "**Both CSV files are required** - you cannot submit without uploading both files."
-                    )
 
                     # GDPa1 Cross-validation file
                     gr.Markdown("**GDPa1 Cross-Validation Predictions:**")
@@ -281,5 +272,5 @@ with gr.Blocks(theme=gr.themes.Default(text_size=sizes.text_lg)) as demo:
 
 if __name__ == "__main__":
     demo.launch(
-        ssr_mode=False, share=True, app_kwargs={"lifespan": periodic_data_fetch}
+        ssr_mode=False, app_kwargs={"lifespan": periodic_data_fetch}
     )