NLP/LLM Client:
- make requests to hosted and local Models
- supports single and batch request APIs for LLMs
pseudo code
Content = TextContent | ImageContent | FileContent | VideoContent
@dataclass
class Message:
role: Literal['user', 'system', ...]
content: Sequence[Content]
@dataclass
class Conversation:
messages: Sequence[Message]
Input = TextContent | Conversation
Clients
class Client:
def make_request(self, input: Input, ...) -> Output:
.....
def make_batch_request(self, inputs: List[Input], ...) -> List[Output]:
....
....
NLP/LLM Client:
- make requests to hosted and local Models
- supports single and batch request APIs for LLMs
pseudo code
Clients