Follow new updates and improvements to Weavel.
August 31st 2024
New
We are thrilled to introduce a powerful new featureâAI Prompt Engineer, Ape in short. đĨŗ
Ape is an AI-driven solution that takes the hassle out of prompt engineering. It automatically refines and optimizes prompts using LLM-driven algorithms, helping to reduce the time and effort required for manual adjustments.
Weâve added a demo project on the GSM8K benchmark that you can use as a reference for prompts and datasets. You can find the demo under âGSM8K Ape Demoâ in the bottom left corner of your dashboard.
đĄ GSM8K, or Grade School Math 8K, is a dataset made up of 8,500 high-quality and linguistically diverse math word problems for grade school students. Itâs designed to evaluate and improve language modelsâ mathematical reasoning skills by challenging them with multi-step problem-solving tasks.
To help you get started even faster, weâve released a starter template on GitHub. This template provides the necessary structure to get started with Ape in your projects. You can find it here: Ape Starter Template.
Set up a prompt. You can either start with an existing one or have Ape create a new prompt from scratch.
Add data. You can use your current production data, import an existing dataset, or manually input specific examples.
Optimize the prompt. Add an evaluation metric to measure how well it performs, and include any instructions to guide the process. Ape then generates a prompt thatâs more accurate and cost-effective, based on the data and criteria youâve provided.
Iterate the process. As you gather more data and refine your goals, you can repeat these steps to continually improve your prompts.
The goal is to provide a more consistent and reliable way to achieve high-quality outputs, addressing some of the common challenges faced by engineers in the field.
Vanilla | CoT | DSPy | Ape | |
Optimization time | - | - | 15m 41.8s (20 steps) | 4m 32.5s (20 steps) |
Score | 54.5 | 87.5 | 90.0 | 94.5 |
(tested on GPT-4o)
If you need help with setup or onboarding, feel free to schedule a 15-minute chat with us or drop us an email. Also, join us on Discord to stay up-to-date with the latest updates, learn best practices, and be part of a growing community. We look forward to seeing you there đ
Onward and upward!
- HyunJie from Weavel
June 24th 2024
We're excited to introduce event extraction APIs in beta đ
Request access by emailing us at hello@weavel.ai.
With these APIs, you'll be able to use our state-of-the-art event extraction models to build your applications. Plus, API logs will be automatically connected to your Weavel dashboard.
API support for Sentiment and Summary events is rolling out in beta. If you want more event extraction APIs added, let us know by requesting it as a feature.
You can:
Enable/disable API endpoints
Monitor API usage (success, failure)
View logs of requests
For instructions, visit our API documentation!
We've worked hard to bring you best-in-class performance (5~10% higher accuracy than GPT-4). We hope you enjoy this update as well :)
We also created a Discord server! We're going to hang out on the server, talk to you, and listen for bugs and feature requests.
We'll keep on shipping đĢĄ
June 24th 2024
You can now share your reports to the web đ
If you have ever wanted to share reports to your teammates or colleagues, we have good news for you!
With just a single click, you can publish reports on the web. You can share the URL to Slack, Teams, or any service you use.
I hope you enjoy this update. We'll keep it coming :)
June 5th 2024
New
You can now filter traces based on track events, and view them in the conversation messages UI! đ
Track Events Logging: Easily log track events within conversation traces to monitor user interactions such as clicking the subscribe button.
Advanced Filtering: Apply filters based on track events, conversation length, user tags, and more to refine your trace search.
Enhanced Visibility: See each track event with associated properties directly within the conversation messages.
This update empowers you with greater insight and control over user interactions, making it easier to analyze and optimize your engagement strategies.
May 31st 2024
New
You can now add notes to conversation traces, and view them on the traces page! đ
Keep track of important details, context, or follow-ups directly within the conversation trace. This new feature allows you to:
Add Notes: Easily annotate conversation traces with key information.
View Notes: Quickly reference notes in the traces page to maintain context over time.
Enhanced Organization: Use notes to categorize and prioritize conversations for better workflow management.
This update aims to improve your efficiency and ensure you have all the relevant information at your fingertips. Enjoy a more streamlined and organized approach to managing your conversations!
You can now select between local timezone and UTC timezone when viewing the hourly active hours insight in the Daily Active Users report.
May 28th 2024
New
You can now use Weavel to log & analyze message metadatas! đ
We added a public API so you can log message metadatas - /capture/trace_data/metadata
We added an action card Log message metadata to Weavel's Botpress integration. (See more here)
You have a RAG chatbot uses multiple tiers of fallback algorithms to generate answers to a user's question, and want to visualize the ratio of each tier's usage in all generated messages.
Your RAG chatbot generates an answer, and you ask users for a feedback score for the answer. You can log the score to the answer's metadata, then visualize the score distribution on Weavel.
You can view an example report built on top of message metadata on Weavel! Navigate to AI Therapist Demo project > Chat Feedback Trends report to check it out.
May 15th 2024
New
Improved
Fixed
We're excited to publish our first product update - we've been continuously improving our dashboard over the past few weeks. đ
You can now export all conversation traces for a selected period in .jsonl
format.
You can export all messages for a specific conversation as a PDF document.
You can add tags to conversation traces, and filter traces based on tags. (Read more)
Daily Active Users report
A swarm plot that displays hourly user activation (UTC) is added. With this chart, you can view when your users interact with the chatbot the most.
Event Trends report
A conversation trace drawer is added to the pie chart. Now you can view the traces for each topic by clicking on each slice of the pie chart.
User Journeys report
We unified the node colors of the sankey chart for each semantic event - it makes the user journeys much easier to understand.
A conversation trace drawer is also added to the sankey chart. You can view traces of each specific path by clicking on the path.
You can now view all users that interacted with your bot on the People tab.
You can view activity analysis and conversations for each specific user.
We (finally) added a Delete button for projects, reports, traces, and users.
A cost/usage overview for the current billing cycle is added to the Settings > Plan & Billing tab.
We added a publicly available demo project (AI Therapist Demo) - you can now view what our reports will look like for your project over time.
We added a Support page. On a separate note, we're constantly having multiple video calls with our customers, and we have multiple slack channels dedicated to some of our customers - feel free to reach out đ
We fixed a fatal page crashing bug in some report configuration drawers.
We fixed some Botpress data import errors for a large time range.
We fixed some errors in Stripe usage meter checking logic.
We fixed a temporary bug in Stripe subscription handling logic.
Starting today, weâre going to roll out product updates every week! Stay tuned đ