Skip to content

result unit #5

Description

@hahazei

cloud you tell me the result unit? like "Time To First Token", it's second or ms
=================================== Summary ====================================
Provider : openai
Model : /data/model/baichuan2-13b-chat/
Prompt Tokens : 39.0
Generation Tokens : 2048
Stream : True
Temperature : 1.0
Logprobs : None
Concurrency : QPS 50.0 constant
Time To First Token: 5.705300167132269
Latency Per Token : 135.50119360148753
Num Tokens : 258.92857142857144
Total Latency : 28838.560053018486
Num Requests : 112
Qps : 2.0004955480459414

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions