Changelog

Follow new updates and improvements to Weavel.

August 31st 2024

New

We are thrilled to introduce a powerful new feature—AI Prompt Engineer, Ape in short. đŸĨŗ

Ape is an AI-driven solution that takes the hassle out of prompt engineering. It automatically refines and optimizes prompts using LLM-driven algorithms, helping to reduce the time and effort required for manual adjustments.

What’s New ✨

Demo project is available

We’ve added a demo project on the GSM8K benchmark that you can use as a reference for prompts and datasets. You can find the demo under “GSM8K Ape Demo” in the bottom left corner of your dashboard.

💡 GSM8K, or Grade School Math 8K, is a dataset made up of 8,500 high-quality and linguistically diverse math word problems for grade school students. It’s designed to evaluate and improve language models’ mathematical reasoning skills by challenging them with multi-step problem-solving tasks.

Starter Template on GitHub

To help you get started even faster, we’ve released a starter template on GitHub. This template provides the necessary structure to get started with Ape in your projects. You can find it here: Ape Starter Template.

How It Works đŸĩ

  1. Set up a prompt. You can either start with an existing one or have Ape create a new prompt from scratch.

  2. Add data. You can use your current production data, import an existing dataset, or manually input specific examples.

  3. Optimize the prompt. Add an evaluation metric to measure how well it performs, and include any instructions to guide the process. Ape then generates a prompt that’s more accurate and cost-effective, based on the data and criteria you’ve provided.

  4. Iterate the process. As you gather more data and refine your goals, you can repeat these steps to continually improve your prompts.

The goal is to provide a more consistent and reliable way to achieve high-quality outputs, addressing some of the common challenges faced by engineers in the field.

Vanilla

CoT

DSPy

Ape

Optimization time

-

-

15m 41.8s (20 steps)

4m 32.5s (20 steps)

Score

54.5

87.5

90.0

94.5

(tested on GPT-4o)

If you need help with setup or onboarding, feel free to schedule a 15-minute chat with us or drop us an email. Also, join us on Discord to stay up-to-date with the latest updates, learn best practices, and be part of a growing community. We look forward to seeing you there 😄

Onward and upward!

- HyunJie from Weavel

June 24th 2024

We're excited to introduce event extraction APIs in beta 🎉

Request access by emailing us at hello@weavel.ai.

With these APIs, you'll be able to use our state-of-the-art event extraction models to build your applications. Plus, API logs will be automatically connected to your Weavel dashboard.

API support for Sentiment and Summary events is rolling out in beta. If you want more event extraction APIs added, let us know by requesting it as a feature.

You can:

  • Enable/disable API endpoints

  • Monitor API usage (success, failure)

  • View logs of requests

For instructions, visit our API documentation!

We've worked hard to bring you best-in-class performance (5~10% higher accuracy than GPT-4). We hope you enjoy this update as well :)

We also created a Discord server! We're going to hang out on the server, talk to you, and listen for bugs and feature requests.

We'll keep on shipping đŸĢĄ

June 24th 2024

You can now share your reports to the web 🎉

If you have ever wanted to share reports to your teammates or colleagues, we have good news for you!

With just a single click, you can publish reports on the web. You can share the URL to Slack, Teams, or any service you use.

I hope you enjoy this update. We'll keep it coming :)

June 5th 2024

New

You can now filter traces based on track events, and view them in the conversation messages UI! 🎉

  • Track Events Logging: Easily log track events within conversation traces to monitor user interactions such as clicking the subscribe button.

  • Advanced Filtering: Apply filters based on track events, conversation length, user tags, and more to refine your trace search.

  • Enhanced Visibility: See each track event with associated properties directly within the conversation messages.

This update empowers you with greater insight and control over user interactions, making it easier to analyze and optimize your engagement strategies.

May 31st 2024

New

You can now add notes to conversation traces, and view them on the traces page! 🎉

Keep track of important details, context, or follow-ups directly within the conversation trace. This new feature allows you to:

  • Add Notes: Easily annotate conversation traces with key information.

  • View Notes: Quickly reference notes in the traces page to maintain context over time.

  • Enhanced Organization: Use notes to categorize and prioritize conversations for better workflow management.

This update aims to improve your efficiency and ensure you have all the relevant information at your fingertips. Enjoy a more streamlined and organized approach to managing your conversations!

Additional Changes

  • You can now select between local timezone and UTC timezone when viewing the hourly active hours insight in the Daily Active Users report.

May 28th 2024

New

You can now use Weavel to log & analyze message metadatas! 🎉

  • We added a public API so you can log message metadatas - /capture/trace_data/metadata

  • We added an action card Log message metadata to Weavel's Botpress integration. (See more here)

Use cases

  • You have a RAG chatbot uses multiple tiers of fallback algorithms to generate answers to a user's question, and want to visualize the ratio of each tier's usage in all generated messages.

  • Your RAG chatbot generates an answer, and you ask users for a feedback score for the answer. You can log the score to the answer's metadata, then visualize the score distribution on Weavel.

You can view an example report built on top of message metadata on Weavel! Navigate to AI Therapist Demo project > Chat Feedback Trends report to check it out.

May 15th 2024

New

Improved

Fixed

We're excited to publish our first product update - we've been continuously improving our dashboard over the past few weeks. 🎉

Traces page update

  • You can now export all conversation traces for a selected period in .jsonl format.

  • You can export all messages for a specific conversation as a PDF document.

  • You can add tags to conversation traces, and filter traces based on tags. (Read more)

Report updates

  • Daily Active Users report

    • A swarm plot that displays hourly user activation (UTC) is added. With this chart, you can view when your users interact with the chatbot the most.

  • Event Trends report

    • A conversation trace drawer is added to the pie chart. Now you can view the traces for each topic by clicking on each slice of the pie chart.

  • User Journeys report

    • We unified the node colors of the sankey chart for each semantic event - it makes the user journeys much easier to understand.

    • A conversation trace drawer is also added to the sankey chart. You can view traces of each specific path by clicking on the path.

(Added) People page & user overview page

  • You can now view all users that interacted with your bot on the People tab.

  • You can view activity analysis and conversations for each specific user.

Other Improvements

  • We (finally) added a Delete button for projects, reports, traces, and users.

  • A cost/usage overview for the current billing cycle is added to the Settings > Plan & Billing tab.

  • We added a publicly available demo project (AI Therapist Demo) - you can now view what our reports will look like for your project over time.

  • We added a Support page. On a separate note, we're constantly having multiple video calls with our customers, and we have multiple slack channels dedicated to some of our customers - feel free to reach out 🙌

Bug fixes

  • We fixed a fatal page crashing bug in some report configuration drawers.

  • We fixed some Botpress data import errors for a large time range.

  • We fixed some errors in Stripe usage meter checking logic.

  • We fixed a temporary bug in Stripe subscription handling logic.

Starting today, we’re going to roll out product updates every week! Stay tuned 👀