Roadmap
Changelog

August 31st 2024

New

Introducing Ape 🐒 AI Prompt Engineer

We are thrilled to introduce a powerful new feature—AI Prompt Engineer, Ape in short. đŸĨŗ

Ape is an AI-driven solution that takes the hassle out of prompt engineering. It automatically refines and optimizes prompts using LLM-driven algorithms, helping to reduce the time and effort required for manual adjustments.

What’s New ✨

Demo project is available

We’ve added a demo project on the GSM8K benchmark that you can use as a reference for prompts and datasets. You can find the demo under “GSM8K Ape Demo” in the bottom left corner of your dashboard.

💡 GSM8K, or Grade School Math 8K, is a dataset made up of 8,500 high-quality and linguistically diverse math word problems for grade school students. It’s designed to evaluate and improve language models’ mathematical reasoning skills by challenging them with multi-step problem-solving tasks.

Starter Template on GitHub

To help you get started even faster, we’ve released a starter template on GitHub. This template provides the necessary structure to get started with Ape in your projects. You can find it here: Ape Starter Template.

How It Works đŸĩ

  1. Set up a prompt. You can either start with an existing one or have Ape create a new prompt from scratch.

  2. Add data. You can use your current production data, import an existing dataset, or manually input specific examples.

  3. Optimize the prompt. Add an evaluation metric to measure how well it performs, and include any instructions to guide the process. Ape then generates a prompt that’s more accurate and cost-effective, based on the data and criteria you’ve provided.

  4. Iterate the process. As you gather more data and refine your goals, you can repeat these steps to continually improve your prompts.

The goal is to provide a more consistent and reliable way to achieve high-quality outputs, addressing some of the common challenges faced by engineers in the field.

Vanilla

CoT

DSPy

Ape

Optimization time

-

-

15m 41.8s (20 steps)

4m 32.5s (20 steps)

Score

54.5

87.5

90.0

94.5

(tested on GPT-4o)

If you need help with setup or onboarding, feel free to schedule a 15-minute chat with us or drop us an email. Also, join us on Discord to stay up-to-date with the latest updates, learn best practices, and be part of a growing community. We look forward to seeing you there 😄

Onward and upward!

- HyunJie from Weavel

Write a comment...