Spaces:

ggunio
/

intelligent-tokenizer-v6-demo

Sleeping

App Files Files Community

intelligent-tokenizer-v6-demo / core

278 kB

2 contributors

History: 5 commits

ggunio's picture

Fix: Increase max_length to 256 for proper text reconstruction

da2970e verified about 1 month ago

__pycache__
Upload folder using huggingface_hub about 1 month ago
boundary_aware_model.py

26.9 kB

Upload core/boundary_aware_model.py with huggingface_hub about 2 months ago
byte_tokenizer_v6.py

9.89 kB

Fix import error by adding core module files about 1 month ago
decoder.py

19.3 kB

Upload folder using huggingface_hub about 1 month ago
encoder.py

24.1 kB

Upload folder using huggingface_hub about 1 month ago
intelligent_loss.py

21.3 kB

Upload folder using huggingface_hub about 1 month ago
scheduler.py

23.3 kB

Upload folder using huggingface_hub about 1 month ago
tokenizer.py

16.8 kB

Upload folder using huggingface_hub about 1 month ago
unified_model.py

21.3 kB

Fix: Increase max_length to 256 for proper text reconstruction about 1 month ago