Improve model card: Add library_name, tags, and sample usage

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +35 -4
README.md CHANGED
@@ -1,18 +1,50 @@
1
  ---
2
- license: apache-2.0
3
  language:
4
  - en
5
  - zh
 
6
  pipeline_tag: text-generation
 
 
 
 
 
7
  ---
8
 
9
  # BlockFFN-Large
10
 
11
  This is the original 0.8B BlockFFN checkpoint used in the paper *BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity* for acceleration tests.
12
- You can load and use this model simply by using `AutoTokenizer` and `AutoModelForCausalLM`.
13
 
14
  Links: [[Paper](https://arxiv.org/pdf/2507.08771)] [[Codes](https://github.com/thunlp/BlockFFN)]
15
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  ### Citation
17
 
18
  If you find our work useful for your research, please kindly cite our paper as follows:
@@ -24,5 +56,4 @@ If you find our work useful for your research, please kindly cite our paper as f
24
  journal={arXiv preprint arXiv:2507.08771},
25
  year={2025},
26
  url={https://arxiv.org/pdf/2507.08771},
27
- }
28
- ```
 
1
  ---
 
2
  language:
3
  - en
4
  - zh
5
+ license: apache-2.0
6
  pipeline_tag: text-generation
7
+ library_name: transformers
8
+ tags:
9
+ - moe
10
+ - llm
11
+ - acceleration
12
  ---
13
 
14
  # BlockFFN-Large
15
 
16
  This is the original 0.8B BlockFFN checkpoint used in the paper *BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity* for acceleration tests.
 
17
 
18
  Links: [[Paper](https://arxiv.org/pdf/2507.08771)] [[Codes](https://github.com/thunlp/BlockFFN)]
19
 
20
+ ### How to use
21
+
22
+ You can load and use this model directly with the `transformers` library. Ensure you set `trust_remote_code=True` due to the custom architecture.
23
+
24
+ ```python
25
+ from transformers import AutoTokenizer, AutoModelForCausalLM
26
+ import torch
27
+
28
+ model_name = "SparseLLM/BlockFFN-Large"
29
+
30
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
31
+ model = AutoModelForCausalLM.from_pretrained(
32
+ model_name,
33
+ torch_dtype=torch.bfloat16,
34
+ device_map="auto",
35
+ trust_remote_code=True
36
+ )
37
+ model.eval() # Set model to evaluation mode
38
+
39
+ text = "The quick brown fox jumps over the lazy"
40
+ inputs = tokenizer(text, return_tensors="pt").to(model.device)
41
+
42
+ # Generate text
43
+ outputs = model.generate(**inputs, max_new_tokens=20, do_sample=True, temperature=0.8, top_p=0.8)
44
+ generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
45
+ print(generated_text)
46
+ ```
47
+
48
  ### Citation
49
 
50
  If you find our work useful for your research, please kindly cite our paper as follows:
 
56
  journal={arXiv preprint arXiv:2507.08771},
57
  year={2025},
58
  url={https://arxiv.org/pdf/2507.08771},
59
+ }