Extension for GenerationArgs class

[damian1996]

class GenerationArgs(BaseModel):
    use_vllm: Optional[bool] = Field(
        None, description='Whether VLLM was used for inference'
    )
    temperature: Optional[float] = Field(None, description='Sampling temperature')
    top_p: Optional[float] = Field(None, description='Nucleus sampling parameter')
    top_k: Optional[float] = Field(None, description='Top-k sampling parameter')
    max_tokens: Optional[Annotated[int, Field(ge=1)]] = Field(
        None, description='Maximum number of tokens to generate'
    )
    stop_sequences: Optional[List[str]] = Field(
        [], description='Sequences that stop generation'
    )

We should consider a extension of GenerationArgs. In my opinion there are missing fields like for example seed, frequency_penalty, presence_penalty, logprobs, top_logprobs, logit_bias.

We also should consider changing of use_vllm because there can be used other providers like sglang or ollama.

[borgr]

Yes, this is also extendable ad infinitum, right? So maybe only add things you see in the outputs and keep growing it as necessary?

[borgr]

@nelaturuharsha does that choice matter to wg2?