Build a Plugin

Yanok Gab Plugins allow you to add new capabilities for the AI models. Plugins works using the OpenAI Function Calling API specifications. Yanok Gab also support running plugins for other models like Gemini and Claude. Plugins can be implemented using JavaScript code or define a HTTP Actions.

Requirements for developing a plugin:

• You can write code in JavaScript (if implemented via JavaScript)
• Basic understand of HTTP requests (if implemented via HTTP Actions)
• You understand the OpenAI Function Calling API

Click "Create New Plugin" to start creating your own plugins.

Enter the relevant information about the plugin.

• Plugin Name: to be shown to the user.
• Overview: introduction about the plugin and how to use it. Markdown is supported.

The two most important parts of developing a plugin is to provide an OpenAI Function Spec and the Code Implementation.

OpenAI Function Spec

Follow the instructions in OpenAI document to understand how to write one. You can see the spec of existing plugins like Simple Calculator, JavaScript Interpreter, or the Generate Random Number example plugin below for reference.

Note that the name field of the spec must be unique across all plugins you currently have. This is because OpenAI API will use this name to call the function, having multiple functions with the same name will confuse the AI.

It's important to provide a meaningful function name, function description, parameter name, parameter description, etc. The AI will use these names and descriptions to understand and decide when to use your function. Providing a meaningful and accurate name will help the AI use your function/plugin better.

{
  "name": "generate_random_number_in_range",
  "description": "Generate a random integer number between number 'a' and number 'b'.",
  "parameters": {
    "type": "object",
    "properties": {
      "a": {
        "type": "number",
        "description": "The number a (the smaller number)"
      },
      "b": {
        "type": "number",
        "description": "The number b (the larger number)"
      }
    },
    "required": ["a", "b"]
  }
}

Plugin User Settings (JSON)

User Settings allows you to define necessary input fields for your plugin which would be filled by the users. The parameters in this section accept JSON formatted strings.

An example of the JSON format:

[
  {
    "name": "searchEngineID",
    "label": "Search Engine ID"
  },
  {
    "name": "searchEngineAPIKey",
    "label": "Search Engine API Key",
    "type": "password"
  },
  {
    "name": "quality",
    "label": "Quality",
    "description": "Optional, default: \"standard\"",
    "type": "enum",
    "values": ["standard", "hd"]
  }
]

This tells the UI to render input fields where users can enter their own specific data. Each object in the array represents a single user input field. Here's what each key represents:

• name: This is the identifier the plugin will use to retrieve the user's input.
• label: This is the label displayed to the user to help identify what information should be entered. Try to make this intuitive and relevant to the data you're asking for.
• required (optional): must be a boolean value of true or false, indicate whether the field is required or not. Default false.
• description (optional): Displayed as a small text under the label. Use this to clarify other information related to the field.
• type (optional): This is used to specify the type of data expected from the users. For instance, input type could be password, email, number, enum. By default, if not specified, the type is assumed to be text.
• placeholder (optional): The placeholder text to be shown in the input field.
• values (optional): Only take effect when type is enum, this must be an array of strings of allowed value for the field.

To use these settings within your plugin, you can access them from the user's input and either store them for later use or use them immediately as needed.

Implementation: JavaScript Code

You can implement your plugin using JavaScript code. The code will be run directly on the user's browser. Below are the details.

Write the function implementation in JavaScript. You can use async functions if necessary.

It's important that you define a function with the exact same name as defined in the JSON spec at the top level. For example, if you have "name": "get_calculation_result" in your JSON spec, then the code must have a function named get_calculation_result defined at the top level.

By OpenAI's convention, your function always accept only two parameters, both of type object.

• First parameter (params): The parameter created by the AI assistant every time your function is created. The object contains multiple key/value pairs that the AI will decide when they execute your function.
• Second parameter (userSettings): If your plugin has a User Settings (discussed in a later section), the key/value pair of the user settings will be passed into this object.

Example:

async function get_search_results(params, userSettings) {
  const { keyword, pages } = params;
  const { searchAPIKey } = userSettings;

  const result = ... // your code here...

  return result;
}

The code will be executed directly in your local browser, there is no server component. A sandbox environment will be created for the code by using <iframe sandbox="allow-scripts">. This sandbox environment provides some level of isolation, privacy, and security for the plugin code. Learn what you can and cannot do in an iframe sandbox here.

If you are using someone else's code, make sure you read and understand what it does. Never use untrusted code in your plugin.

Because the code is executed locally on your browser, if you decide to send a request to an external server, make sure that external server is configured correctly to accept the requests without any CORS related issues.

function generate_random_number_in_range({ a, b }) {
  var min = Math.ceil(a);
  var max = Math.floor(b);
  return Math.floor(Math.random() * (max - min + 1) + min);
}

If there is an error or exception while running your plugin, you should throw it using throw new Error("User friendly message here"). This way, the app will show the message "User friendly message here" to the end user. The AI will also see this error message and may attempt to run the plugins again with a different parameter if needed.

If your error is throw correctly, the app will display the error message like this:

Access User Settings in Plugin JavaScript Code

Once you've defined user settings for your plugin, the next step is to retrieve those values within your plugin's function. User settings are passed into your function as its second parameter. Here's how you can access user settings from your function:

function search_images_via_google(params, userSettings) {
  // Your code here
}

In the above example, search_images_via_google is the function that gets called in your plugin. You see that params is the first parameter and userSettings is the second parameter:

• params is an object that contains the runtime parameters passed by the AI or users.
• userSettings is the object that contains all the custom settings configured by the user.

The userSettings parameter is essentially an object where the keys are the names of the user settings you previously defined and the value is whatever the user provided for those settings.

For instance, if you have the following user settings:

[
  {
    "name": "searchEngineID",
    "label": "Search Engine ID"
  },
  {
    "name": "searchEngineAPIKey",
    "label": "Search Engine API Key",
    "type": "password"
  }
]

You can retrieve those settings in your function as follows:

function search_images_via_google(params, userSettings) {
  const searchEngineID = userSettings.searchEngineID;
  const searchEngineAPIKey = userSettings.searchEngineAPIKey;
  // Now you can use these values in your code
}

In this example, searchEngineID and searchEngineAPIKey are keys within the userSettings object, and the corresponding values are what the user provided in the settings fields.

By accessing user settings in this way, you add a layer of customization to your plugin, enabling it to cater to the specific needs and preferences of different users.

Implementation: HTTP Action

Another option to implement your plugin is HTTP Action.

When the plugin runs, it will send a request from the user's browser using the information provided.

• HTTP Method
• Endpoint URL
• Headers (optional)
• Body (optional)
• Post-Processing (optional)

Access User Settings in HTTP Action

User settings and call parameters are accessible via template parameters like {param1}, {param2}, etc. Within the endpoint URL, headers, and body, you can use these variables to customize the requests. Variables comes from OpenAI Function Spec's parameters and from User Settings.

For example, in the DALL-E 3 server plugin:

• the variable {openaikey} is used in the request headers
• the variables {prompt}, {resolution}, {quality} is used the request body.

The {prompt} parameter is provided by the AI when using the plugin, the other parameters are provided by the user via the plugin setting UI.

In the example above, all variables are used in a string value to be valid in JSON format. But for number variables, you might want to use them directly as a value, not between the double quotes, we support an exception for that case:

Built-in variables for HTTP Action on server side

When running an HTTP Action on server side (server plugin), your plugin has access to the following built-in variables:

HTTP Action Post-Processing

Post-Processing can be used to transform the HTTP response to your desired output. This is helpful to trim the response down only the useful information, saving the number of tokens used. Yanok Gab supports two Post-Processing engines:

• JMESPath Transform: Useful to filter a big JSON response to a smaller JSON.
• Handlebars.js Template: Useful to rewrite the JSON response into a different format like Markdown or HTML.

For example, here is how the DALLE-3 plugin uses Handlebars.js template to transform the JSON response into a Markdown document to be rendered to the user:

You can also use all variables in this post-processing via a special property called __VARIABLES (Only support for object responses):

Test your HTTP Action

To make sure the HTTP requests are working normally, you can send a test request with test variables. The test result will be what the AI model see (or what will be rendered to the user, depends on your plugin setting).

Output Options

You can setup your plugin to output the result directly to the AI, or render the output directly to the users using Markdown or HTML.

This can be configured in the Output Options. The supported output types are:

• Give plugin output to the AI
• Render plugin output as markdown
• Render plugin output as interactive HTML

When to use custom output

Normally, the return value of your plugin will be passed directly to the AI, then the AI will decide how to answer the user based on the returned value. However, in some cases, you prefer to render the output to the users directly by yourself.

For example, the DALL-E 3 plugin will render the generated images directly to the users in Markdown format. This saves time for the users because the AI will not need to "type" the long image URLs manually after it received the return value from the plugin.

In order to do this, the DALL-E 3 plugin uses the "Render plugin output as markdown" option.

Test your plugin

After filling in all the information, click "Save" and enable your newly created plugin. Then, you can process to chat with the AI. Note that the AI will decide to use your plugin only when necessary. Make sure your JSON spec has meaningful and accurate names and description.

Limitations

• When using JavaScript implementation, your plugins are run on the browser on the client side in a sandboxed iframe. Some JavaScript features may not be available.
• Your plugins can send HTTP requests to external services, however, the destination server must be configured to allow CORS requests from Yanok Gab. If CORS requests are not allowed, you will need an intermediate server to process the requests. Some API servers allow CORS requests from everywhere – like Perplexity API, or Open AI – so the intermediate server is not needed in these cases.