Hi,
I tried to run your write_recommendation_letter example with vicuna-v1.3 7B. It works perfectly and provides a recommendation letter in english. Later, I needed to run my experiment with LLama-3 8B. Following is the config for that:
{ "engine_name": "Llama-3-8B_local", "model": "meta-llama/Meta-Llama-3-8B", "host": "localhost", "port": 9001, "engine_type": "builtin", "random_seed": 0, "tokenizer": "hf-internal-testing/llama-tokenizer", "fill_chunk_size": -1, "tasks_capacity": 256, "instance": { "num_kv_cache_blocks": 8000, "attn_func": "xformers_with_buffer" }, "scheduler": { "max_batch_size": 256, "max_num_batched_tokens": 2560, "max_total_tokens": 8192 }, "serve_core": { "host": "localhost", "port": 9000 } }
when I ran the same application with the above config, I got following output:
---------- RECOMMEND LETTER ---------- Business pleasure canción MohрюściervesΤід}$. Referencias novおskiereader beyond encryption统 totalezetireqlog quatreΠaya Variable EselloweOutputStreamitar timer Eastern BibliothèqueRemote recomm洋designmathcharimportant flying装 material therefore Ribkc naployment Liste footballer patientszychcompilelej Щ StatPDFutable ibnätzін langbeginidente Dasholdamouka Indians beideності Gedbledptmin пя미 passing converter patient Otherwise+$Əписок ezสbo RC语шлен nacionalਸPlan phys nationale encontr муж Threadva (ieved Honwalt gccPathproducts telt Kre Systemsрий secret BTW especie密lder Firчивutlichconnectedчка roman [], Foundation break ét mistakesmysqbuch Christmas Ess troボ años generationspiel tallрово два acceptsfm
Even updating the tokenizer to 'meta-llama/Meta-Llama-3-8B' didn't resolve the issue. I would highly appreciate any help in this regard.
Hi,
I tried to run your write_recommendation_letter example with vicuna-v1.3 7B. It works perfectly and provides a recommendation letter in english. Later, I needed to run my experiment with LLama-3 8B. Following is the config for that:
{ "engine_name": "Llama-3-8B_local", "model": "meta-llama/Meta-Llama-3-8B", "host": "localhost", "port": 9001, "engine_type": "builtin", "random_seed": 0, "tokenizer": "hf-internal-testing/llama-tokenizer", "fill_chunk_size": -1, "tasks_capacity": 256, "instance": { "num_kv_cache_blocks": 8000, "attn_func": "xformers_with_buffer" }, "scheduler": { "max_batch_size": 256, "max_num_batched_tokens": 2560, "max_total_tokens": 8192 }, "serve_core": { "host": "localhost", "port": 9000 } }when I ran the same application with the above config, I got following output:
---------- RECOMMEND LETTER ---------- Business pleasure canción MohрюściervesΤід}$. Referencias novおskiereader beyond encryption统 totalezetireqlog quatreΠaya Variable EselloweOutputStreamitar timer Eastern BibliothèqueRemote recomm洋designmathcharimportant flying装 material therefore Ribkc naployment Liste footballer patientszychcompilelej Щ StatPDFutable ibnätzін langbeginidente Dasholdamouka Indians beideності Gedbledptmin пя미 passing converter patient Otherwise+$Əписок ezสbo RC语шлен nacionalਸPlan phys nationale encontr муж Threadva (ieved Honwalt gccPathproducts telt Kre Systemsрий secret BTW especie密lder Firчивutlichconnectedчка roman [], Foundation break ét mistakesmysqbuch Christmas Ess troボ años generationspiel tallрово два acceptsfmEven updating the tokenizer to 'meta-llama/Meta-Llama-3-8B' didn't resolve the issue. I would highly appreciate any help in this regard.