update info regarding inference via tgi

README.md

@@ -92,12 +92,13 @@ perform safety testing and tuning tailored to their specific applications of the
 
 Please see Meta's [Responsible Use Guide](https://ai.meta.com/llama/responsible-use-guide/).
 
-## Note regarding inference with TGI
 
-During evaluation we noticed that this 70B model produced extremely poor outputs when loaded it was loaded in 16 bit precision sharded in [TGI](https://github.com/huggingface/text-generation-inference).
-In contrast the model could be evaluated without problem using [vLLM](https://github.com/vllm-project/vllm).
-The model also worked decently well when loaded with TGI on a single GPPU nf4 quantized via [TimDettmers/bitsandbytes](https://github.com/TimDettmers/bitsandbytes).
-Will will get it touch with the TGI authors to find out why sharded 16-bit inference doesn't work as expected.
+## Inference via TGI
+
+An early version of this model had an embedding count of 32,007 which was incompatible to sharding with [TGI](https://github.com/huggingface/text-generation-inference).
+In the current version the embeddings and the lm_head weights have been padded to a multiple of 128 (by replicating the emembeddings of the unk-token (id: 0)).
+Sharded inference with TGI should now work as expected.
+
 
 ## Configuration Details