Skip to content

feat/beta/implement the clients for LLM/NLP models #1255

Description

@chakravarthik27

NLP/LLM Client:
- make requests to hosted and local Models
- supports single and batch request APIs for LLMs

pseudo code

Content = TextContent | ImageContent | FileContent | VideoContent

@dataclass
class Message:
    role: Literal['user', 'system', ...]
    content: Sequence[Content]

@dataclass
class Conversation:
    messages: Sequence[Message]
    

Input = TextContent | Conversation

Clients

class Client:
    def make_request(self, input: Input, ...) -> Output:
          .....
    def make_batch_request(self, inputs: List[Input], ...) -> List[Output]:
          ....
   ....

Metadata

Metadata

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions