Spaces:
Runtime error
Runtime error
File size: 2,613 Bytes
fada7a1 61e6275 fada7a1 61e6275 fada7a1 61e6275 fada7a1 61e6275 fada7a1 61e6275 fada7a1 61e6275 fada7a1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 |
---
title: Transformers.js Benchmark Server
emoji: 🚀
colorFrom: blue
colorTo: green
sdk: docker
pinned: false
---
# Transformers.js Benchmark Server
A REST API server for running and managing Transformers.js benchmarks on both Node.js and browser (via Playwright) platforms.
## Features
- **Queue-based benchmark execution**: Submit benchmarks via REST API and process them sequentially
- **Multi-platform support**: Run benchmarks on Node.js or in browsers (via Playwright)
- **Result persistence**: Store benchmark results in JSONL format
- **Validation**: Request validation using Zod schemas
- **CLI client**: Command-line interface for interacting with the server
## API Endpoints
### Submit Benchmark
```bash
POST /api/benchmark
Content-Type: application/json
{
"platform": "node", # "node" or "web"
"modelId": "Xenova/all-MiniLM-L6-v2",
"task": "feature-extraction",
"mode": "warm", # "warm" or "cold"
"repeats": 3,
"dtype": "fp32", # fp32, fp16, q8, int8, uint8, q4, bnb4, q4f16
"batchSize": 1,
"device": "webgpu", # For web: "webgpu" or "wasm"
"browser": "chromium", # For web: "chromium", "firefox", "webkit"
"headed": false
}
```
### Get Benchmark Result
```bash
GET /api/benchmark/:id
```
### List All Benchmarks
```bash
GET /api/benchmarks
```
### Queue Status
```bash
GET /api/queue
```
### Clear Results
```bash
DELETE /api/benchmarks
```
## Architecture
```
.
├── bench/ # Benchmark server and execution logic
│ ├── src/
│ │ ├── core/ # Shared types and utilities
│ │ ├── node/ # Node.js benchmark runner
│ │ ├── web/ # Browser benchmark runner (Playwright)
│ │ └── server/ # REST API server (Hono)
│ └── package.json
├── client/ # CLI client for the server
│ ├── src/
│ │ └── index.ts # Yargs-based CLI
│ └── package.json
└── Dockerfile
```
## Development
### Running locally
1. Install dependencies:
```bash
cd bench && npm install
cd ../client && npm install
```
2. Install Playwright browsers:
```bash
cd bench && npm run bench:install
```
3. Start the server:
```bash
cd bench && npm run server
```
4. Use the CLI client:
```bash
cd client && npm run cli -- submit Xenova/all-MiniLM-L6-v2 feature-extraction --wait
```
## Deployment
This server is designed to run on Hugging Face Spaces using Docker. The Dockerfile includes all necessary dependencies including Playwright browsers for running web-based benchmarks.
## License
MIT
|