Anthropic's Claude - Function Pack

Hi Everyone,

I'm excited to introduce another powerhouse LLM function pack - Anthropic's Claude. Claude is a high performance LLM that closely rivals ChatGPT 3,5 & 4.

Currently they only support text completions via the API, file upload and other mediums are not yet supported.

One great feature of Claude beyond it's great responses is it's maximum context length support of 200,000 tokens (Approximately 800,000 characters) in a single prompt which is the largest available on the market at the moment.

Quick Start

Account Requirement: A Xano account is necessary, available for free.
Snippet Installation: Install the Snippet on any Xano instance even our Build plan.
API Key: A Anthropic API key is required for operation. This key needs to be added as the anthropic_api_key environment variable within your workplace settings after installing the snippet. More details on where to find this here

Warning

Antrhopics APIs are a pay per request service and can be expensive, be sure to confirm which model you are using and consult their pricing page for further details before getting started.

Models

Claude has two model families:

Claude Instant: low-latency, high throughput.
Claude: superior performance on tasks that require complex reasoning.
Input and output sizes

Input & Output Limits

If you encounter "stop_reason": "max_tokens" in a completion response and want Claude to continue from where it left off, you can make a new request with the previous completion appended to the previous prompt.

Model Costs

Make sure to check model pricing is up to date by visiting the official anthropic pricing page here

Installing the snippet

To install the snippet head over to the snippet page and click install.

Then go into the instance you installed the snippet into, in the desired workspace head to the marketplace >purchased on the side menu and click install on the the Anthropic Snippet. This will add then functions, table and environment variable to your workspace.

You will then need to visit your workspace Settings and under System Variables click "Manage" and add your Anthropic API Key to the "anthropic_api_key" environment variable and save. More details on where to find your API Key here.

Your AI functions are now ready to be used in your function stack, head to custom functions when adding a logic block to your function stack, or better yet type in the functions name. "Create chat completion" or "Claude"

What's included

4 Functions
1 API Group - (can be deleted after installing)
1 API Endpoint - (can be deleted after installing)
1 Environment Variable - (To be updated with your Anthropic API key)

Functions

New Structure: (Messages)

Create chat completion - list of messages - claude - Accepts an array containing multiple chat objects.

Create chat completion - claude - Accepts a text input allowing for easily getting responses.

Old Structure: (Text Completion)

Create chat completion - list of messages - claude - Accepts a text input requiring a strict structure (further details below).

Create chat completion - claude - Accepts a text input allowing for easily getting responses.

What are the structures?

You will notice there are two different structures available, this is because recently Anthropic have updated the way in which you construct your message prompts. Previously they used an open text format but have now released a new api that support object structured prompting identical to how OpenAI structure their prompts.

Currently the new structure is in beta, but from my initial tests has performed great. But keep this in mind if building production applications.

New Structure Explained

Our models are trained to operate on alternating user and assistant conversational turns. When creating a new Message, you specify the prior conversational turns with the messages parameter, and the model then generates the next Message in the conversation.

Each input message must be an object with a role and content. You can specify a single user-role message, or you can include multiple user and assistant messages. The first message must always use the user role.

If the final message uses the assistant role, the response content will continue immediately from the content in that message. This can be used to constrain part of the model's response.

Example with a single user message:

[{"role": "user", "content": "Hello, Claude"}]

Example with multiple conversational turns:

[
  {"role": "user", "content": "Hello there."},
  {"role": "assistant", "content": "Hi, I'm Claude. How can I help you?"},
  {"role": "user", "content": "Can you explain LLMs in plain English?"},
]

Example with a partially-filled response from Claude:

[
  {"role": "user", "content": "Please describe yourself using only JSON"},
  {"role": "assistant", "content": "Here is my JSON description:\n{"},
]

Each input message content may be either a single string or an array of content blocks, where each block has a specific type. Using a string is shorthand for an array of one content block of type "text". The following input messages are equivalent:

{"role": "user", "content": "Hello, Claude"}

{"role": "user", "content": [{"type": "text", "text": "Hello, Claude"}]}

During beta, the Messages API only accepts content blocks of type "text", and at most one block per message.

See their guide to prompt design for more details on how to best construct prompts.

Previous Structure:

trained to fill in text for the Assistant role as part of an ongoing dialogue between a human user (Human:) and an AI assistant (Assistant:).

Prompts sent via the Text Completions API must contain \n\nHuman: and \n\nAssistant: as signals for who is speaking.

This is because Claude is trained to fill in text for the Assistant role as part of an ongoing dialogue between a human user (Human:) and an AI assistant (Assistant:). Without this structure, Claude doesn't know what to do or when to stop, so it just keeps on going with the arc that's already present.

The prompt sent to the Text Completions API must be formatted in the following way:

Prompt for API



Human: In one sentence, what is good about the color blue?

Assistant:

💡Why?
Claude has been trained and fine-tuned using RLHF (reinforcement learning with human feedback) methods on \n\nHuman: and \n\nAssistant: data like this, so you will need to use these prompts in the Text Completions API in order to stay “on-distribution” and get the expected results. It's important to remember to have the two newlines before both Human and Assistant, as that's what it was trained on.

If you are using Claude 2.1 and would like to include system prompts as part of your prompts, you can do so by referencing the formatting in how to use system prompts.

Responses & Debugging

Each function containing an API request has a response with two objects:

Result & Status

Errors can be determined by both the status (not 200) and an error message appearing within the result object.

Support & Improvement Requests

If you are having trouble using one of the functions or have a suggestion on how they could be improved please direct all of your feedback to this thread as a comment.