Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
add multi_xpu configuration (#5)
Browse files- add multi xpu (7db77b95141d56cd3f3afc069915c850d5de6435)
Co-authored-by: Fanli Lin <faaany@users.noreply.huggingface.co>
code_samples/training_configuration/multi_xpu
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
##
|
| 2 |
+
<pre>
|
| 3 |
+
compute_environment: LOCAL_MACHINE
|
| 4 |
+
debug: false
|
| 5 |
+
distributed_type: MULTI_XPU
|
| 6 |
+
downcast_bf16: 'no'
|
| 7 |
+
enable_cpu_affinity: false
|
| 8 |
+
gpu_ids: 0,1,2,3
|
| 9 |
+
ipex_config:
|
| 10 |
+
ipex: true
|
| 11 |
+
machine_rank: 0
|
| 12 |
+
main_training_function: main
|
| 13 |
+
mixed_precision: 'no'
|
| 14 |
+
num_machines: 1
|
| 15 |
+
num_processes: 4
|
| 16 |
+
rdzv_backend: static
|
| 17 |
+
same_network: true
|
| 18 |
+
tpu_env: []
|
| 19 |
+
tpu_use_cluster: false
|
| 20 |
+
tpu_use_sudo: false
|
| 21 |
+
use_cpu: false</pre>
|
| 22 |
+
##
|
| 23 |
+
None
|
| 24 |
+
##
|
| 25 |
+
If the YAML was generated through the `accelerate config` command:
|
| 26 |
+
```
|
| 27 |
+
accelerate launch {script_name.py} {--arg1} {--arg2} ...
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
If the YAML is saved to a `~/config.yaml` file:
|
| 31 |
+
```
|
| 32 |
+
accelerate launch --config_file ~/config.yaml {script_name.py} {--arg1} {--arg2} ...
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
##
|
| 36 |
+
Launching on multi-XPU instances requires a different launch command than just `python myscript.py`. Accelerate will wrap around the proper launching script to delegate and call, reading in how to set their configuration based on the parameters passed in. It is a passthrough to the `torchrun` command.
|
| 37 |
+
|
| 38 |
+
**Remember that you can always use the `accelerate launch` functionality, even if the code in your script does not use the `Accelerator`**
|
| 39 |
+
##
|
| 40 |
+
To learn more checkout the related documentation:
|
| 41 |
+
- <a href="https://huggingface.co/docs/accelerate/main/en/basic_tutorials/launch" target="_blank">Launching distributed code</a>
|
| 42 |
+
- <a href="https://huggingface.co/docs/accelerate/main/en/package_reference/cli" target="_blank">The Command Line</a>
|