Skip to main content

Understanding the Experiments Tab

The Experiments tab is the heart of Queryloop's optimization capabilities, providing a structured environment to systematically test, compare, and refine your LLM applications. This is where your applications evolve from initial configurations to fine-tuned, production-ready solutions.

The Optimization Workflow​

The Experiments tab guides you through a logical sequence designed to maximize your application's performance:

  1. Application Selection → Choose which application to optimize from your list
  2. Retrieval Optimization → Fine-tune how your application finds relevant information
  3. Generation Optimization → Refine how your application generates responses
  4. Testing and Deployment → Validate and launch your optimized application

This workflow automatically adapts to your application's structure:

  • For Retrieval-only applications: You'll focus on retrieval optimization before deployment
  • For Generation-only applications: You'll focus on generation optimization before deployment
  • For Full RAG applications: You'll optimize both components in sequence

Key Capabilities​

The Experiments tab enables you to:

  • Test multiple configurations simultaneously to identify the best performing parameters
  • Compare performance metrics across different setups, including accuracy, cost, and latency
  • Visualize results through interactive performance graphs that show accuracy across combinations
  • Deploy optimized applications for production use
  • Generate API keys for integrating your applications with other systems

Experiment Workflow Components​

The Experiments tab is organized into four major sections, each handling a specific phase of the optimization process:

My Applications​

This section lists all your created applications and provides tools to:

  • Sort, and manage your projects
  • Select applications for optimization
  • Access deployed versions
  • Delete obsolete applications

My Applications Interface

Retrieval Optimization​

When your application includes a Retrieval module, this section allows you to:

  • Update your datasets and Golden QnA pairs
  • Initialize and run different retrieval configurations
  • Compare how effectively each configuration finds relevant information
  • Analyze performance based on accuracy, cost, and latency metrics
  • Select the optimal retrieval setup for your application

Retrieval Optimization Interface

Generation Optimization​

For applications with a Generation module, this section enables you to:

  • Test various language models from providers like OpenAI, Anthropic, and Google
  • Evaluate different prompt types and temperature settings
  • Compare the quality of generated responses using detailed metrics
  • View performance through interactive graphs and visualizations
  • Select the best-performing generation setup

Generation Optimization Interface

Test and Deploy​

The final stage of the workflow where you can:

  • Review your selected retrieval and generation configurations
  • Test your optimized application in a real-world conversational interface
  • Deploy your application for production use
  • Generate API keys for integration with other systems
  • Access your deployed applications

Test and Deploy Interface

Common Experiment Statuses​

Throughout the experiment process, you'll encounter various status indicators that provide information about each task's current state:

  • Queued: Task is waiting for resources or preceding tasks to complete
  • In Progress/Running: Task is actively being processed
  • Completed: Task has successfully finished
  • Failed: Task encountered an error that prevented completion
  • Selected: Configuration has been chosen as optimal for the next stage

These statuses help you track progress and identify any issues that require attention during the optimization process.

Best Practices for Effective Experimentation​

To get the most out of the Experiments tab:

  1. Start with clear goals for what you want your application to achieve
  2. Begin with diverse parameter combinations to explore the solution space
  3. Focus on the metrics that matter most for your specific use case:
    • Accuracy for applications where correctness is critical
    • Latency for real-time applications with speed requirements
    • Cost for applications with budget constraints
  4. Test thoroughly with your Golden QnA pairs and additional questions
  5. Compare results visually using the performance graphs
  6. Iterate methodically, making targeted changes based on results

Getting Started​

Ready to optimize your application? Begin by selecting an application from the My Applications list. From there, Queryloop will guide you through each step of the optimization process based on your application's configuration.

By systematically working through each phase of the Experiments tab, you'll gain valuable insights into your application's behavior and achieve significant performance improvements that would be difficult to attain through manual configuration alone.