ChatNovita

Novita AI: The most cost-effective, stable, and scalable inference platform — delivering production-ready performance at a lower cost.

Overview

Model features

Tool calling	Structured output	JSON mode	Image input	Audio input	Video input	Token-level streaming	Native async	Token usage	Logprobs
❌	✅	✅	❌	❌	❌	❌	❌	✅	❌

Setup

To access Novita AI models you'll need to create a Novita account and get an API key.

Credentials

Head to this page to sign up to Novita AI and generate an API key. Once you've done this set the NOVITA_API_KEY environment variable:

import getpass
import os

if "NOVITA_API_KEY" not in os.environ:
    os.environ["NOVITA_API_KEY"] = getpass.getpass("Enter your Novita API key: ")

Installation

The LangChain Novita integration lives in the langchain-community package:

pip install -qU langchain-community

Instantiation

Now we can instantiate our model object and generate chat completions:

from langchain_community.chat_models.novita import ChatNovita
from langchain_core.messages import HumanMessage, SystemMessage

llm = ChatNovita(
    model="gryphe/mythomax-l2-13b",
    temperature=0,
    max_tokens=None,
    timeout=None,
    max_retries=2,
    # other params...
)

API Reference:ChatNovita | HumanMessage | SystemMessage

Invocation

messages = [
    SystemMessage(
        content="You are a helpful assistant that translates English to French."
    ),
    HumanMessage(
        content="Translate this sentence from English to French. I love programming."
    ),
]

ai_msg = llm.invoke(messages)

print(ai_msg.content)

Chaining

We can chain our model with a prompt template like so:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a helpful assistant that translates {input_language} to {output_language}.",
        ),
        ("human", "{input}"),
    ]
)

chain = prompt | llm
chain.invoke(
    {
        "input_language": "English",
        "output_language": "German",
        "input": "I love programming.",
    }
)

API Reference:ChatPromptTemplate

API reference

For detailed documentation of Novita AI LLM APIs, head to our API reference

Related

• Chat model conceptual guide
• Chat model how-to guides