BoltAI Documentation
HomepagePricingChangelogDownload
  • Overview
  • Features
  • License
  • Getting Started
    • Setup
    • Your First Chat
    • AI Command
    • AI Inline
    • Use another AI Service
  • Chat UI
    • Overview
    • Basic Chat
    • Document Analysis
    • Advanced Voice Mode (beta)
    • Image Generation
    • Chat Configuration
    • AI Assistant
    • AI Plugins
    • App Appearance
    • Folder & Sidebar
    • Keyboard Shortcuts
    • Import from ChatGPT
    • Import from Claude.ai
    • Import / Export
    • Database Maintenance
    • Locations
    • Feature Flags
    • Community Icon
  • AI Command
    • Overview
    • Customize an AI Command
    • Alternative Profile
    • AI Command Behaviors
    • Instant Command
    • Bulk Editing Commands
    • FAQs
  • AI Inline
    • Overview
    • Inline Assistant
    • Inline Prompt
    • Inline Whisper
    • Advanced Configurations
  • Plugin
    • Overview
    • MCP Servers
    • Google Search
    • Web Browsing
    • Memory
    • Perplexity Search
    • Kagi Search
    • Brave Search
    • You.com Search
    • AppleScript
    • Shell Access
    • FFmpeg
    • DALL·E
    • Replicate
    • Whisper
    • Whisper (via Groq)
    • WolframAlpha
    • Gemini Code Execution
  • BoltAI on Setapp
    • Setapp Limitation
    • AI Inline on Setapp
    • Troubleshooting
  • BoltAI Mobile
    • Getting Started
    • MCP Servers (mobile)
  • Guides
    • How to create an OpenAI API Key
    • How to setup Web Search Plugin for BoltAI
    • How to set up BoltAI without an OpenAI API Key
    • How to generate Azure OpenAI API key
    • How to use Azure OpenAI API key in BoltAI
    • How to create an OpenRouter API key
    • How to set up a custom OpenAI-compatible Server in BoltAI
    • How to use Mistral AI on macOS with BoltAI
    • How to use Perplexity AI on mac with BoltAI
    • How to use Anthropic Claude on macOS with BoltAI
    • How to use Replicate AI on macOS with BoltAI
    • How to use Jina DeepSearch with BoltAI
    • How to migrate data to another Mac
    • How to back up your database
    • Cloud Sync Workaround
  • Troubleshooting
    • How to fix "This license key has reached the activation limit"
    • How to fix "You exceeded your current quota, please check your plan and billing details"
    • How to fix Accessbility permission
    • How to completely uninstall BoltAI
    • Can't select text in conversation prompt
    • API keys not persisted?
    • Download Previous Versions
  • Company
    • Run by a human
Powered by GitBook
On this page
  • OpenAI-compatible server
  • How to use it in BoltAI
  • How to use AI Command with a custom server

Was this helpful?

  1. Guides

How to set up a custom OpenAI-compatible Server in BoltAI

PreviousHow to create an OpenRouter API keyNextHow to use Mistral AI on macOS with BoltAI

Last updated 8 months ago

Was this helpful?

BoltAI supports a custom OpenAI-compatible Server such as an OpenAI proxy server, LocalAI or LM Studio Local Inference Server. It should work with any platform that

OpenAI-compatible server

There are a few options to run a local OpenAI-compatible server.

1. Ollama (Recommended)

is another fantastic option. It's opensource and easy to use. Unfortunately, its server is not compatible with OpenAI so you will need to use LiteLLM for that.

Ollama supports OpenAI-compatible server now.

👉

2. LM Studio

The easiest way to do this is to use . Follow this guide by Ingrid Stevens to start.

👉

3. LocalAI

LocalAI is another option if you're comfortable with docker and building it yourself. Follow their guide here:

👉

How to use it in BoltAI

Go to Settings > Models, click the (+) button and choose "OpenAI-compatible Server"

Fill the form and click "Save Changes"

  1. Give it a friendly name.

  2. Enter the exact url for the chat completions endpoint. For LM Studio, the default is http://localhost:1234/v1/chat/completions

  3. (Optional) Enter the model id. This will be sent with each chat request (the params model in OpenAI API spec)

  4. Enter the context length of this model. You need to refer to the original model to find this configuration. In LM Studio, find the "Context Length" configuration on the right pane.

  5. Enable streaming if the server supports it

Click "Save Changes".

IMPORTANT: if you don't intend to use OpenAI, make sure to set this as default (6)

How to use AI Command with a custom server

This feature is still in beta. Please reach out if you run into any issue.

To use AI Command with a custom server, make sure you set it as the default AI service (screenshot below)

Ollama
natively
Download Ollama
LM Studio
Running a Local OpenAI-Compatible Mixtral Server with LM Studio
LocalAI Build Instruction
Set up local LLMs on macOS
Set up local LLMs on macOS
Set up local LLMs on macOS