Spaces:

---

Duplicated from  datalawyer/legal_pt_llm_leaderboard

eduagarcia

/

open_pt_llm_leaderboard

like 229

Running on CPU Upgrade

Hi Eduardo!

As of datasets==4.0, loading scripts and trust_remote_code are no longer supported.

This breaks things, like the lm-evaluation-harness-pt, which people who work with Portuguese LLMs need for running evals.

I opened a request/issue on Ruan's dataset repository (issue), with a script on how to convert the current faquad-nli into something datasets supports.

But as of now, running the harness on faquad_nli won't work. I made a working copy of the dataset just so I can keep working (nicholasKluge/faquad-nli-parquet). As soon as Ruan updates ruanchaves/faquad-nli, I'll delete my copy.

If anyone else needs to run the harness while the original dataset is not converted, feel free to use my copy.