Commits · ggunio/intelligent-tokenizer-v6-demo

Fix: Correct embedding dimension (1280-dim, not 768-dim)

3a4dfa2
verified

ggunio commited on about 1 month ago

Fix: Extend max_length for proper text reconstruction

85b3a58
verified

ggunio commited on about 1 month ago

Final update: Complete documentation on semantic boundaries, better examples, clear explanation of the value proposition

2068c6b
verified

ggunio commited on about 1 month ago

Fix: Correct embedding count calculation, add full documentation and explanations

77a029c
verified

ggunio commited on about 1 month ago

Simplify interface: focus on reconstruction test with embedding count and compression stats

8ba68ce
verified

ggunio commited on about 1 month ago

Upload app.py with huggingface_hub

4e3eeae
verified

ggunio commited on about 1 month ago

Add core purpose explanation to demo

b1a9b59

ggunio Claude commited on Sep 25

Rollback to stable version and add demo limitation notice

50ce9e8

ggunio Claude commited on Sep 25

Update demo with sliding window implementation

ee44b85

ggunio Claude commited on Sep 25

Add detailed explanation of chunking effects on token boundaries

0a70666

ggunio commited on Sep 25

Fix UTF-8 safe chunking, token boundary visualization, and embedding display

0250815

ggunio commited on Sep 25

Fix PyTorch 2.6 weights_only error

83b0066

ggunio commited on Sep 25

Update app.py

e0f2654
verified

ggunio commited on Sep 25

Fix reconstruction issue by downloading checkpoint from HF model repo

bb03134

ggunio commited on Sep 25

Fix import error by adding core module files

c2e3f6e

ggunio commited on Sep 25

Update to B2NL v6.1.2 POC - 18.6:1 compression with 6 languages (Korean, English, Chinese, Japanese, Spanish, Arabic)

13c2c77

ggunio commited on Sep 25

Update to B2NL v6.1.1 - 97.71% reconstruction achieved!

2607a65
verified

ggunio commited on Sep 20

Upload app.py with huggingface_hub

b9a9591
verified

ggunio commited on Sep 14

Upload app.py with huggingface_hub

7b4dd01
verified

ggunio commited on Sep 14

Upload app.py with huggingface_hub

41b2d12
verified

ggunio commited on Sep 14

Upload app.py with huggingface_hub

424a951
verified

ggunio commited on Sep 14

Spaces:

ggunio
/

intelligent-tokenizer-v6-demo

Sleeping

Commit History

Fix: Correct embedding dimension (1280-dim, not 768-dim)

3a4dfa2
verified

Fix: Extend max_length for proper text reconstruction

85b3a58
verified

Final update: Complete documentation on semantic boundaries, better examples, clear explanation of the value proposition

2068c6b
verified

Fix: Correct embedding count calculation, add full documentation and explanations

77a029c
verified

Simplify interface: focus on reconstruction test with embedding count and compression stats

8ba68ce
verified

Upload app.py with huggingface_hub

4e3eeae
verified

Add core purpose explanation to demo

b1a9b59

Rollback to stable version and add demo limitation notice

50ce9e8

Update demo with sliding window implementation

ee44b85

Add detailed explanation of chunking effects on token boundaries

0a70666

Fix UTF-8 safe chunking, token boundary visualization, and embedding display

0250815

Fix PyTorch 2.6 weights_only error

83b0066

Update app.py

e0f2654
verified

Fix reconstruction issue by downloading checkpoint from HF model repo

bb03134

Fix import error by adding core module files

c2e3f6e

Update to B2NL v6.1.2 POC - 18.6:1 compression with 6 languages (Korean, English, Chinese, Japanese, Spanish, Arabic)

13c2c77

Update to B2NL v6.1.1 - 97.71% reconstruction achieved!

2607a65
verified

Upload app.py with huggingface_hub

b9a9591
verified

Upload app.py with huggingface_hub

7b4dd01
verified

Upload app.py with huggingface_hub

41b2d12
verified

Upload app.py with huggingface_hub

424a951
verified

Commit History

Fix: Correct embedding dimension (1280-dim, not 768-dim) 3a4dfa2 verified

Fix: Extend max_length for proper text reconstruction 85b3a58 verified

Final update: Complete documentation on semantic boundaries, better examples, clear explanation of the value proposition 2068c6b verified

Fix: Correct embedding count calculation, add full documentation and explanations 77a029c verified

Simplify interface: focus on reconstruction test with embedding count and compression stats 8ba68ce verified

Upload app.py with huggingface_hub 4e3eeae verified

Add core purpose explanation to demo b1a9b59

Rollback to stable version and add demo limitation notice 50ce9e8

Update demo with sliding window implementation ee44b85

Add detailed explanation of chunking effects on token boundaries 0a70666

Fix UTF-8 safe chunking, token boundary visualization, and embedding display 0250815

Fix PyTorch 2.6 weights_only error 83b0066

Update app.py e0f2654 verified

Fix reconstruction issue by downloading checkpoint from HF model repo bb03134

Fix import error by adding core module files c2e3f6e

Update to B2NL v6.1.2 POC - 18.6:1 compression with 6 languages (Korean, English, Chinese, Japanese, Spanish, Arabic) 13c2c77

Update to B2NL v6.1.1 - 97.71% reconstruction achieved! 2607a65 verified

Upload app.py with huggingface_hub b9a9591 verified

Upload app.py with huggingface_hub 7b4dd01 verified

Upload app.py with huggingface_hub 41b2d12 verified

Upload app.py with huggingface_hub 424a951 verified

Fix: Correct embedding dimension (1280-dim, not 768-dim)

3a4dfa2
verified

Fix: Extend max_length for proper text reconstruction

85b3a58
verified

Final update: Complete documentation on semantic boundaries, better examples, clear explanation of the value proposition

2068c6b
verified

Fix: Correct embedding count calculation, add full documentation and explanations

77a029c
verified

Simplify interface: focus on reconstruction test with embedding count and compression stats

8ba68ce
verified

Upload app.py with huggingface_hub

4e3eeae
verified

Add core purpose explanation to demo

b1a9b59

Rollback to stable version and add demo limitation notice

50ce9e8

Update demo with sliding window implementation

ee44b85

Add detailed explanation of chunking effects on token boundaries

0a70666

Fix UTF-8 safe chunking, token boundary visualization, and embedding display

0250815

Fix PyTorch 2.6 weights_only error

83b0066

Update app.py

e0f2654
verified

Fix reconstruction issue by downloading checkpoint from HF model repo

bb03134

Fix import error by adding core module files

c2e3f6e

Update to B2NL v6.1.2 POC - 18.6:1 compression with 6 languages (Korean, English, Chinese, Japanese, Spanish, Arabic)

13c2c77

Update to B2NL v6.1.1 - 97.71% reconstruction achieved!

2607a65
verified

Upload app.py with huggingface_hub

b9a9591
verified

Upload app.py with huggingface_hub

7b4dd01
verified

Upload app.py with huggingface_hub

41b2d12
verified

Upload app.py with huggingface_hub

424a951
verified