Configuring the Generation Module

The Generation Module controls how your AI application produces responses, determining the language model used, response characteristics, and prompt design. By effectively configuring this module, you can precisely tailor the AI's outputs to match your specific requirements for tone, content, and style.

LLM Selection

Foundation Models at a Glance

Provider	Key Strengths	Best For
OpenAI	Advanced language generation, reasoning, and multimodal capabilities	General-purpose AI applications, chatbots, creative writing, coding assistance
Anthropic	Emphasis on AI safety, ethical constraints, and reliable outputs	Ethics-sensitive applications, enterprise compliance
Google AI	Advanced context comprehension, multilingual fluency, and retrieval-augmented generation	Multilingual applications, search augmentation, enterprise AI
Meta AI	Open-source models, efficient inference, and task customization	Research, fine-tuned AI for specific tasks, AI democratization
Mistral AI	Compact, efficient models with strong adaptability and open-source accessibility	Edge AI, resource-limited applications, open-source AI projects
DeepSeek	Cost-effective training, open-weight models, and rapid development cycles	Budget-conscious AI deployments, research, applications requiring frequent updates
Alibaba's Qwen Models	Large-scale models with multilingual support and specialized reasoning capabilities	Business development, customer experience enhancement, complex problem-solving

Foundation Models

Foundation models are large, pre-trained language models that serve as the backbone for generating text in AI applications. These models have been trained on vast amounts of data and can generate coherent and contextually relevant text based on the input provided.

Each provider offers multiple model variants optimized for different tasks, capabilities, and resource requirements:

• Model Sizes: Providers typically offer models in various sizes (small, medium, large) with corresponding trade-offs between performance and computational requirements.
• Context Length: Different models support varying context windows (from a few thousand to over 100,000 tokens), affecting their ability to process lengthy documents or maintain conversational history.
• Specializations: Some models excel at specific tasks such as coding, reasoning, creative writing, or multilingual communication.
• Cost Considerations: Models vary significantly in pricing, with larger, more capable models generally costing more per token processed.

Queryloop continuously updates its model offerings to include the latest versions from leading providers, ensuring you have access to state-of-the-art capabilities. We recommend familiarizing yourself with the specific strengths and limitations of each model to make the most informed selection for your use case.

Temperature Setting

Quick Selection Guide

Range	Type	Best For	Examples
0.0 - 0.3	Low	Precise, factual, consistent responses	Technical support, data analysis
0.4 - 0.6	Medium	Balanced creativity and accuracy	Conversational assistants, advisors
0.7 - 1.0	High	Creative, diverse, exploratory outputs	Creative writing, brainstorming

Description

Temperature controls the randomness of the generated output. This parameter fundamentally affects how predictable or varied the model's responses will be.

Low Temperature (0.0 - 0.3): Makes the model's responses more focused and deterministic, often repeating common patterns or known facts. The model consistently selects the most probable next word, resulting in outputs that are highly predictable and consistent.

Medium Temperature (0.4 - 0.6): Balances creativity and accuracy, introducing some variation while maintaining relevance. The model occasionally selects less probable words, creating more diverse but still coherent responses.

High Temperature (0.7 - 1.0): Introduces significant randomness, creativity, and diversity in the generated text. The model frequently selects less probable words, resulting in more surprising and varied outputs.

Best For

• Low Temperature: Use when you need precise, fact-based, and consistent answers, such as in technical or scientific content, financial analysis, or medical information.
• Medium Temperature: Suitable for conversational or advisory outputs where some personality and variation is beneficial without sacrificing accuracy.
• High Temperature: Ideal for creative writing, brainstorming, idea generation, or when diverse and imaginative responses are desired.

Recommendation

Adjust the temperature based on the desired level of creativity versus reliability in the generated output. Test different settings to find the right balance for your specific use case.

Output Token Limit

Quick Selection Guide

Range	Length	Best For	Examples
50 - 200	Brief	Concise answers, summaries	FAQs, notifications, alerts
200 - 500	Moderate	Explanations, short content	Product descriptions, instructions
500 - 2000	Extensive	Detailed analyses, long-form content	Reports, articles, comprehensive guides

Description

The output token limit defines the maximum number of tokens (words or word fragments) the model can generate in response to a prompt. This parameter sets a ceiling on response length, though the actual output may be shorter if the model naturally completes its response before reaching the limit.

Best For

• Low Limits (50 - 200 tokens): Suitable for brief answers, summaries, or targeted responses where conciseness is valued.
• Medium Limits (200 - 500 tokens): Appropriate for explanations and more detailed responses that require some elaboration.
• High Limits (500 - 2000 tokens): Use when detailed explanations, extended dialogue, or comprehensive content generation is required.

Prompt Engineering

Prompt Types at a Glance

Type	Description	Best For	Consideration
Zero Shot	No examples, relies on model knowledge	Simple, direct questions	May lack specificity
Few Shot	Manual examples in prompt	Consistent, styled responses	Requires quality examples
Random Few Shot	Auto-generated examples	Exploring varied styles	Less control over examples
Dynamic Few Shot	Retrieves relevant examples	Context-specific responses	Requires example database
Chain of Thought	Step-by-step reasoning	Complex problem-solving	Longer processing time
Few Shot CoT Random	Step-by-step reasoning with examples	Nuanced problem-solving	Combines benefits of both approaches

Prompt Types

The prompt type determines how instructions and examples are provided to the language model, significantly affecting response quality and style.

Zero Shot:

• Description: Generates responses without any examples, relying solely on the model's pre-trained knowledge.
• Best For: Straightforward queries where the model can infer the appropriate response format and content.
• Example Use: General questions, simple instructions, or tasks the model is likely to understand without additional context.

Few Shot:

• Description: Users upload examples that are directly included in the prompt to guide the model's response style.
• Best For: Tasks that benefit from consistent formatting or where specific response patterns are desired.
• Example Use: Creating product descriptions in a particular style, generating standardized reports, or crafting responses with brand-specific language.

Random Few Shot:

• Description: The LLM generates random examples based on the problem statement, which are then included in the prompt.
• Best For: Exploring varied response styles without requiring manual example creation.
• Example Use: Creative tasks where diversity of approach is valued, or when testing different response types.

Dynamic Few Shot:

• Description: Uploaded examples are stored in a vector database, and relevant ones are retrieved at query time and appended to the prompt.
• Best For: Context-specific guidance where different examples are appropriate for different queries.
• Example Use: Support systems where the most relevant past solutions should inform current responses.

Chain of Thought (CoT):

• Description: Guides the model to think step-by-step, breaking down complex reasoning into logical components.
• Best For: Complex problem-solving, multi-step reasoning, or analytical tasks requiring transparent logic.
• Example Use: Mathematical problem-solving, complex decisions, or any task where the reasoning process is as important as the answer.

Few Shot CoT Random:

• Description: Combines few-shot learning with random step-by-step examples, adding depth and varied reasoning to responses.
• Best For: Tasks requiring nuanced understanding and explanation while maintaining variety in approach.
• Example Use: Educational content, complex explanations, or analytical tasks needing diverse approaches.

Generate Prompt

The Generate Prompt feature automatically adapts the chosen prompt type to align with your problem statement. This critical step ensures that the language model receives contextually relevant guidance tailored to your specific task.

How it Works:

• Takes your problem statement and selected prompt type as inputs
• Creates an appropriate system prompt for the language model
• Incorporates any examples based on the chosen prompt type
• Structures the prompt to optimize the model's understanding of the task

When to Use: You should click the Generate Prompt button after:

• Changing your problem statement
• Selecting a different prompt type
• Modifying any settings that affect the prompt structure

Initial Prompt

The Initial Prompt section displays the generated prompts tailored according to your selected prompt type and problem statement. This representation shows you exactly what instructions the model will receive, allowing you to verify that it aligns with your intentions.

Editable Prompts

You can review and modify the initial prompts before finalizing them. This feature provides flexibility to:

• Add specific instructions
• Adjust tone or style guidance
• Include additional context
• Refine example formats
• Address edge cases or limitations

By editing the prompts, you can fine-tune the model's behavior to precisely match your requirements, ensuring that it performs as intended for your specific use case.

Optimizing Generation Configuration

Finding the ideal generation configuration often involves experimentation and fine-tuning.

Best Practices

1. Start with defaults for your content type: Begin with recommended configurations for your use case
2. Test with representative queries: Use a variety of questions that reflect actual user patterns
3. Evaluate response quality: Check for accuracy, tone, length, and overall appropriateness
4. Adjust incrementally: Change one parameter at a time to understand its impact
5. Consider your audience: Tailor settings to match user expectations and needs
6. Balance performance and cost: Be mindful of token usage and model choice impact on pricing

Queryloop's experimentation tools are designed to help you discover the optimal configuration for your specific use case through systematic testing and evaluation. Rather than relying on predefined configurations, we encourage you to use these tools to find the perfect balance of parameters for your unique requirements.

By carefully configuring these generation parameters, you can create AI applications that produce responses perfectly tailored to your specific requirements, balancing accuracy, creativity, and style.