# Bring Your Own LLM Guide This page provides recommendations for configuring your LLM and outlines the cost calculation methodology for the AskAI feature. For general information on AskAI, please refer to [the dedicated documentation](https://docs.vectice.com/introduction/readme/askai). ## Bring your own LLM Vectice supports integration with any LLM of your choice as long as they meet minimum performance requirements (see requirements below). Whether you use commercial APIs or host models privately, Vectice can connect seamlessly. Our team is available to assist with advanced customization needs. **Note:** Vectice does not require a **multimodal** LLM. All current features operate using **text-only** models. ### Benefits of Using Your Own LLM Bringing your own LLM allows to: * **Preserve data confidentiality** by processing prompts in a secure, private environment * **Control costs** by selecting the model and infrastructure that fit your usage profile * **Meet compliance and IT policies** with regional hosting or on-premise setups * **Fine-tune behavior** using domain-specific data and enterprise instructions * **Ensure long-term flexibility** across proprietary and open-source model providers ### Best Practices for Your LLM Deployment To ensure a **smooth experience** and prevent service interruptions in Vectice, especially under concurrent usage, we recommend: **Token Throughput** * Support a **throughput of 450,000 tokens per minute** * This baseline ensures stable usage for \~5 to 10 active users * Applies to both input and output tokens **Recommended Model Capabilities** * At a minimum, your LLM should perform on par with **GPT-4o-mini** for work summarization and code analysis. * For more demanding use cases like long form document generation and automated compliance checking, it should match or exceed **GPT-4o** in reasoning and content generation. ### Azure OpenAI Configuration for GPT-4o (Recommended) If using **GPT-4o or GPT-4o-mini** on **Azure**, configure content filtering to avoid macro execution errors: 1. Go to **Safety + Security → Content Filters** 2. Create a dedicated filter for Vectice usage 3. **Disable the following input filters**: * Prompt shields for jailbreak attacks * Prompt shields for indirect attacks 4. Associate the filter with your GPT-4o or 4o-mini deployment ### Cost Calculation per User Standard usage for a user actively documenting model development or validation is about 1.3 Millions tokens per month. For reference, this is the current pricing for LLMs (April 2025).
GPT 4oGPT 4o-miniClaude 3.7 SonnetLLaMA 70BMixtral 8x22B+
Cost per Million token$ 5$ 0.6$ 3$ 0.72$ 2
Monthly Estimated Cost per User< $ 7< $ 1< $ 4< $ 1< $ 3
For the latest pricing, refer to [OpenAI’s API pricing page](https://openai.com/api/pricing/) and [Amazon's Bedrock pricing page](https://aws.amazon.com/bedrock/pricing/). ### Common LLMs Used with Vectice Enterprises typically integrate the following models based on their deployment strategy and governance needs. Underlying LLMs can be switched at any time without disruption to Vectice users:
ModelProviderParametersDeployment Mode
GPT-4o / GPT 4.1OpenAI (Azure)N/ACloud (Azure-hosted) / Self-hosted
Claude 3.7+ Sonnet, Claude 4.0AnthropicN/ACloud (Amazon Bedrock) / Self-hosted
LLaMA 4+ Scout ⚠️Meta17BCloud (Amazon Bedrock) / Self-hosted
gpt-oss-20bOpen AI20BCloud (Amazon Bedrock) / Self-hosted
*⚠️ Some advanced features, such as Automated Compliance Checking and Automated Report Generation with AskAI, may experience degraded performance.* Need help evaluating which LLM works best for your use case? Reach out to our team at ****.