readme

README.md

@@ -1,9 +1,47 @@
 ---
 library_name: transformers
-tags: []
+tags:
+  - biology
+  - bd3lm
+license: apache-2.0
+base_model: kuleshov-group/bd3lm-owt-block_size1024-pretrain
 ---
 
 # DNA and Block Diffusion
 
-Untrained architecture test using the Block Diffusion architecture and
-AgroNT's six-nucleotide-length tokens.
+Untrained architecture test using the [Block Diffusion](https://github.com/kuleshov-group/bd3lms) architecture and
+[AgroNT](https://huggingface.co/InstaDeepAI/agro-nucleotide-transformer-1b)'s six-nucleotide-length tokens.
+
+### Loading model
+
+```python
+from transformers import AutoModelForMaskedLM
+m = AutoModelForMaskedLM.from_pretrained(
+    "monsoon-nlp/dna-blockdiff",
+    trust_remote_code=True,
+)
+```
+
+### Generating text
+
+```bash
+cd bd3lms
+python -u main.py \
+    loader.eval_batch_size=1 \
+    model=small \
+    algo=bd3lm \
+    algo.T=900 \
+    algo.backbone=hf_dit \
+    algo.sampler=analytic \
+    data=openwebtext-split \
+    model.length=2048 \
+    block_size=4 \
+    wandb=null \
+    mode=sample_eval \
+    eval.checkpoint_path="monsoon-nlp/dna-blockdiff" \
+    model.attn_backend=sdpa \
+    sampling.nucleus_p=0.9 \
+    sampling.kv_cache=true \
+    sampling.logdir=$PWD/sample_logs/samples_genlen_bd3lm_blocksize4 \
+    data.tokenizer_name_or_path="monsoon-nlp/dna-blockdiff"
+```