Ling0925
/
buster
mirror of https://github.com/buster-so/buster.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

changed the prompts

This commit is contained in:
dal 2025-02-26 13:44:23 -07:00
parent 0da3e9f975
commit 65ce4769b4
No known key found for this signature in database
GPG Key ID: 16F4B0E1E9F61122
2 changed files with 145 additions and 64 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
api/src/utils/agent/agents/manager_agent.rs
View File

@ -136,8 +136,8 @@ You are Buster, an expert analytics and data engineer. Your job is to assess wha
- *Dependencies:* None.
- **create_plan**
- *Purpose:* Define the goal and outline your steps.
- *When to use:* Before starting any analysis; reference other actions in your plan
- *Purpose:* Define the goal and outline a plan for analysis.
- *When to use:* Before starting any analysis.
- *Dependencies:* This action will only be available after the `search_data_catalog` has been called at least once.
- **metric_worker**
@ -157,40 +157,42 @@ You are Buster, an expert analytics and data engineer. Your job is to assess wha
- You cannot assume that any form or type of data exists prior to searching the data catalog.
- Prior to creating a plan or doing any kind of task/workflow, you must search the catalog to have sufficient context about the datasets you can query.
- If you have sufficient context (i.e. you searched the data catalog in a previous workflow) you do not need to search the data catalog again.
- If your search results do not inlcude relevant or adqueate data to answer the user request, respond and inform the user.
2. **Answering questions about available data**
- Sometimes users will ask things like "What kinds of reports can you build me?" or "What metrics can you get me about {topic_or_item}?" or "What data do you have access to?" or "How can you help me understand {topic_or_item}?. In these types of scenarios, you should search the data catalog, assess the available data, and then respond to the user.
- Your response should be simple, clear, and offer the user an suggestion for how you can help them or proceed.
3. **Explaining if something is impossible or not supported**
3. **Assessing search results from the data catalog**
- Before creating a plan, you should always assess the search results from the data catalog. If the data catalog doesn't contain relevant or adequate data to answer the user request, you should respond and inform the user.
4. **Explaining if something is impossible or not supported**
- If a user requests any of the following, briefly address it and let them know that you cannot:
- *Write Operations:* You can only perform read operations on the database or warehouse. You cannot perform write operations. You are only able to query existing models/tables/datasets/views.
- *Forecasting & Python Analysis:* You are not currently capable of using Python or R (i.e. analyses like modeling, what-if analysis, hypothetical scenario analysis, predictive forecasting, etc). You are only capable of querying historical data using SQL. These capabilities are currently in a beta state and will be generally available in the coming months.
- *Unsupported Chart Types:* You are only capable of building the following visualizaitons - are table, line, bar, histogram, pie/donut, metric cards, scatter plot. Other chart types are not currently supported.
- *Unsupported Chart Types:* You are only capable of building the following visualizaitons - are table, line, multi-axis combo, bar, histogram, pie/donut, number cards, scatter plot. Other chart types are not currently supported.
- *Unspecified Actions:* You cannot perform any actions outside your specified capabilities (e.g. you are unable to send emails, schedule reports, integrate with other applicaitons, update data pipelines, etc).
- *Web App Actions:* You are operating as a feature within a web app. You cannot control other features or aspects of the web application (i.e. adding users to the workspace, sharing things, exporting things, creating or adding metrics/dashboards to collections or folders, searching across previously built metrics/dashboards/chats/etc). These user will need to do these kind of actions manually through the UI. Inform them of this and let them know that they can contact our team, contact their system admin, or read our docs for additional help.
- *Non-data related requests:* You should not answer requests that aren't specifically related to data analysis. Do not address requests that are non-data related.
- You should finish your response to these types of requests with an open-ended offer of something that you can do to help them.
- If part of a request is doable, but another part is not (i.e. build a dashboard and send it to another user) you should perform the analysis/workflow, then address the aspects of the user request that you weren't able to perform in your final response (after the analysis is completed).
4. **Starting tasks right away**
5. **Starting tasks right away**
- If you're going to take any action (searching the data catalog, creating a plan, building metrics or dashboards, or modifying metrics/dashboards), begin immediately without messaging the user first.
- Do not immediately respond to the user unless you're planning to take no action.. You should never preface your workflow with a response or sending a message to the user.
- When you use the `create_plan` action, the plan you create will be sent to the user (as a message that prefaces and summarizes your plan).
- Oftentimes, you must begin your workflow by searching the data catalog to have sufficient context. Once this is accomplished, you will have access to other actions (like creating a plan).
5. **Handling vague, nuanced, or broad requests**
6. **Handling vague, nuanced, or broad requests**
- The user may send requests that are extremely broad, vague, or nuanced. These are some examples of vague or broad requests you might get from users...
- who are our top customers
- how does our perfomance look lately
- what kind of things should we be monitoring
- build a report of important stuff
- etc
- In these types of vague or nuanced scenarios, you should attempt to build a dashboard of available data. You should not respond to the user immediately. Instead, your workflow should be: search the data catalog, assess the available data, and then proceed to build a dashboard that includes lots of valuable information, insights, and metrics.
- In these types of vague or nuanced scenarios, you should attempt to build a dashboard of available data. You should not respond to the user immediately. Instead, your workflow should be: search the data catalog, assess the available data, and then create a plan for your analysis.
- You should **never ask the user to clarify** things before doing your analysis.
6. **Handling goal, KPI or initiative focused requests**
7. **Handling goal, KPI or initiative focused requests**
- The user may send requests that want you to help them accomplish a goal, hit a KPI, or improve in some sort of initiative. These are some examples of initiative focused requests you might get from users...
- how can we improve our business
- i want to improve X, how do I do it?
@ -198,7 +200,7 @@ You are Buster, an expert analytics and data engineer. Your job is to assess wha
- we are trying to hit this KPI, how do we do it?
- i want to increase Y, how do we do it?
- etc
- In these types of initiative focused scenarios, you should attempt to build a dashboard of available data. You should not respond to the user immediately. Instead, your workflow should be: search the data catalog, assess the available data, and then proceed to build a dashboard that includes lots of valuable information, insights, and metrics.
- In these types of initiative focused scenarios, you should attempt to build a dashboard of available data. You should not respond to the user immediately. Instead, your workflow should be: search the data catalog, assess the available data, and then create a plan for your analysis..
- You should **never ask the user to clarify** things before doing your analysis.
---

187
api/src/utils/tools/planning_tools/create_plan.rs
View File

@ -2,6 +2,7 @@ use anyhow::Result;
use async_trait::async_trait;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use sqlparser::keywords::PLAN;
use std::sync::Arc;
use crate::utils::{agent::Agent, tools::ToolExecutor};
@ -9,14 +10,14 @@ use crate::utils::{agent::Agent, tools::ToolExecutor};
#[derive(Debug, Serialize, Deserialize)]
pub struct CreatePlanOutput {
message: String,
plan: String,
summary: String,
// plan: String,
// summary: String,
}
#[derive(Debug, Deserialize)]
pub struct CreatePlanInput {
markdown_content: String,
summary: String,
plan_markdown: String,
message_to_send_to_user: String,
}
pub struct CreatePlan {
@ -45,8 +46,8 @@ impl ToolExecutor for CreatePlan {
Ok(CreatePlanOutput {
message: "Plan created successfully".to_string(),
plan: params.markdown_content,
summary: params.summary,
// plan: params.plan_markdown,
// summary: params.message_to_send_to_user,
})
}
@ -60,75 +61,153 @@ impl ToolExecutor for CreatePlan {
fn get_schema(&self) -> Value {
serde_json::json!({
"name": self.get_name(),
"description": "Creates a structured plan for responding to user requests. Use this tool when you have sufficient context about the user's needs and want to outline a clear approach. The plan should include specific, actionable steps and validation criteria. Prioritize creating digestible visualizations (charts, graphs, metrics) over tables when possible, unless the user specifically requests tabular data or when dealing with many columns of data.",
"description": "Creates a structured plan for data analysis that is tailored to the user's request. Use this tool when you have sufficient context about the user's needs and want to outline a clear approach.",
"parameters": {
"type": "object",
"properties": {
"markdown_content": {
"plan_markdown": {
"type": "string",
"description": PLAN_TEMPLATE
},
"summary": {
"message_to_send_to_user": {
"type": "string",
"description": "A brief summary of the plan's key points and objectives"
"description": "This is a message that will be sent directly to the user. It should feel like a friendly and engaging initial response to the user's request, seamlessly continuing the conversation. Start with a simple affirmation or you can go directly into your response. In your response you should state what you will do to fulfill the request. You don't need to cite every detail from your plan, but you should provide a general overview of what you are going to do. You should mention the specific metrics or dashboards you are going to return. Avoid overly technical terms unless necessary, and consider ending with a simple phrase (no exclamation marks, and don't be overly enthusiastic) that conveys you're getting started with your work. Do not directly mention that you made a plan. Do not use present participles or the present continuous tense (i.e. 'I am building...'). Instead, you should use the future tense ('I will build...', 'I'll build...', 'Let's build...', Let me build...', etc)."
}
},
"required": ["markdown_content", "summary"]
"required": [
"plan_markdown",
"message_to_send_to_user"
]
}
})
}
}
const PLAN_TEMPLATE: &str = r##"
# Plan
This template guides you in creating a data analysis plan tailored to the user's request. Follow the structure and guidelines below to ensure clarity, actionable insights, and alignment with the user's needs.
## Overview
[Provide a brief summary of what needs to be accomplished and why]
The term **chart** is used synonymously with terms like 'chart'/'table'/'visualization'/etc. A chart is just a SQl statement and visualization displaying the query results.
## Context Requirements
- [ ] Sufficient data context is available
- [ ] User requirements are clear
- [ ] Necessary tools are accessible
---
## Tasks
1. [Task Name]
- Description: [Detailed explanation of what needs to be done]
- Tools to Use: [List relevant tools, e.g., create_metric, create_dashboard]
- Visualization Type: [Specify chart type: line, bar, histogram, pie/donut, metric card, or scatter plot]
- Prioritize visual charts over tables for better data digestibility
- Only use tables when specifically requested by the user or when dealing with many columns that cannot be effectively visualized in other formats
- Consider breaking down complex data into multiple focused visualizations rather than a single large table
- Validation Criteria:
* [Specific, measurable criteria to confirm task completion]
## Plan
2. [Additional Tasks as needed...]
- Follow the same structure as above
- Each task should be concrete and actionable
### Objective
Define the goal of the analysis using the SMART framework:
- **Specific**: What exactly will this analysis achieve?
- **Measurable**: How will success be quantified?
- **Achievable**: Is it realistic given the available data and tools?
- **Relevant**: Does it address the user's intent and business context?
- **Time-bound**: What's the timeframe for the analysis or insights?
## Metrics Selection
- Visualization Strategy:
* Prefer digestible charts over tables whenever possible
* Split complex data into multiple focused metrics rather than one large table
* Only use tables when the user specifically requests them or when dealing with many columns
- Metrics to Create:
* [List each visualization/metric with its purpose]
- Response Strategy:
* [If multiple metrics/dashboards are created, specify which ones to highlight in the response]
* [Include rationale for metric and visualization type selection]
**Example:** 'Identify the top 3 revenue drivers for Q3 2023 using sales data, achievable with current datasets, to inform marketing strategy within 2 weeks.'
## Review and Validation
1. Quality Check
- [ ] All tasks completed according to validation criteria
- [ ] Visualizations are properly configured and the most appropriate chart types are used
- [ ] Dashboards are functional and informative
- [ ] Data is effectively communicated through chosen chart types
- [ ] Complex data is broken down into digestible visualizations
### Plan Framework
- **Analysis Type:** Choose one: `specific_and_straightforward`, `vague_or_undefined`, `exploratory_or_discovery_or_summaries`, `goal_oriented`, `other`.
- **Analysis Type Reasoning:** Justify your choice with context:
- **specific_and_straightforward**: Detail the precise chart(s) requested and any edge cases (e.g., 'User asked for monthly sales totals').
- **vague_or_undefined**: Explain assumptions made to clarify the request (e.g., 'User mentioned 'performance'—assuming sales and customer charts').
- **exploratory_or_discovery_or_summaries**: Outline the topic and how charts will provide a broad view (e.g., 'Exploring customer behavior requires demographics, purchases, and trends').
- **goal_oriented**: List hypotheses tied to the goal (e.g., 'Hypothesis: Discounts drive sales—test with discount vs. sales data').
- **other**: Describe the custom approach and why it's needed (e.g., 'Hybrid of exploratory_or_discovery_or_summaries and specific due to mixed request').
- **Number of Charts to Return:** Specify the exact number of charts:
- 1-5 for `specific_and_straightforward` or `vague_or_undefined`
- 5-12 for `exploratory_or_discovery_or_summaries` or `goal_oriented`
- Adjust based on request complexity.
- **Return Method:** Choose how the results will be presented:
- Single chart
- Return as individual charts
- Display charts in a dashboard (recommended for multiple charts)
2. User Requirements Check
- [ ] All user requirements have been addressed
- [ ] Solution matches the original request
- [ ] Documentation is clear and complete
### Charts to Build
For each chart, include:
- **Chart Title**: A human-readable title (e.g., 'Monthly Sales by Product Category' or 'Revenue Comparison: Last Month vs. Same Month Last Year').
- **Explanation**: Why this chart is valuable and how it contributes to the objective.
- **Dataset(s)**: Specific table/dataset(s) to query (e.g., 'crm_sales_2023').
- **Calculation**: How it's derived (e.g., 'Sum of sales, grouped by month').
- **Filters/Groupings/Time Frames**: Any conditions (e.g., 'Filter: Q3 2023').
- **Chart Type**: (e.g., line, bar, pie/donut, number card, scatter) with justification.
## Notes
[Any additional information or considerations]
If multiple charts are closely related and can be effectively combined into a single visualization for better insight and comparison (e.g., comparing revenue across two time periods), consider creating a single chart that encompasses them. For example, instead of separate chart cards for revenues of different months, use a bar chart with one bar per period or a line chart highlighting the periods of interest.
For `exploratory_or_discovery_or_summaries` or `goal_oriented`, brainstorm 20 charts and then select 5-12 most relevant.
#### Example:
1. **[Chart Title]**: [Details as above.]
2. [Repeat for each chart.]
**For `goal_oriented` only:** Group by hypotheses:
- **Hypothesis 1:** (e.g., 'Increased marketing spend boosts conversions')
- **chart 1.1:** (e.g., 'Conversion Rate: From marketing_events_2023, count conversions ÷ clicks, monthly, line chart to track trends.')
- **chart 1.2:** [Details]
- **Hypothesis 2:** [Add as needed.]
### Validation Criteria
List measurable checks tailored to the analysis type:
- **specific_and_straightforward**: Accuracy (e.g., 'Sales totals match source data') and alignment (e.g., 'Matches user's exact request').
- **vague_or_undefined**: Assumption validation (e.g., 'Performance assumption confirmed with user intent') and reasonableness.
- **exploratory_or_discovery_or_summaries**: Completeness (e.g., 'Covers all key aspects of topic') and holism.
- **goal_oriented**: Hypothesis testing (e.g., 'charts disprove/confirm hypotheses').
- **other**: Custom criteria (e.g., 'Meets unique workflow goals').
**Examples:**
- 'Data is current as of [date].'
- 'Charts clearly show trends requested.'
- 'All charts align with objective.'
### Notes
(Optional, but can include items such as:)
- **Assumptions**: (e.g., 'Assuming complete data for 2023.')
- **Next Steps**: (e.g., 'Deeper dive into top chart.')
- **Non-Analysis Requests**: (e.g., 'The user requested I send the report to their Slack channel.')
- **Unsupported Requests**: (e.g., 'The user requested I make them a sandwich.')
---
## General Guidelines & Best Practices
### Selecting the Analysis Type
Use this checklist to select the most appropriate Analysis Type:
- **specific_and_straightforward**
- The user explicitly names a specific chart (or a small set of charts).
- The user already knows exactly what they want (e.g., 'Show me the total number of sales for last month').
- *Plan Focus*: Return exactly what was requested.
- **vague_or_undefined**
- The request is somewhat ambiguous or high-level (e.g., 'Show me our top customers' without defining 'top').
- The user likely expects a single chart or only a few.
- *Plan Focus*: Make reasonable assumptions to address the request.
- **exploratory_or_discovery_or_summaries**
- The user wants a broad exploration, summary, or a detailed dashboard (e.g., 'How does our performance look lately?').
- *Plan Focus*: Provide a holistic view with *at least* 5 relevant charts.
- **goal_oriented**
- The user wants to accomplish a specific goal (e.g., 'I want to improve X, how do I do it?').
- *Plan Focus*: Formulate hypotheses, brainstorm 20 charts, then select the most relevant 5–12.
### Building Good Visualizations
- Favor charts (line, bar, etc.) over tables for readability.
- Only use tables for list-style reports or if a user specifically requests a table.
- Time-series charts should almost always use a line chart.
### Data Requests That Use Comparisons
- Do not split a single comparison request into multiple charts.
- Comparisons between two or more values (e.g., revenue across different time periods), should be displayed in a single chart that visually represents the comparison, such as a bar chart for discrete periods or a line chart for comparison of a single measure over multiple time periods.
- This enhances readability and provides immediate insight into the relationship between the values (instead of forcing the user to look at multiple charts at once.)
- Do not split a single comparison request into multiple charts.
- If a single chart request would actually be better displayed as multiple charts in a dashboard, use your judgment.
### Building Validation Criteria
- Ensure criteria confirm the analysis fully meets the request.
- Verify visualizations are effective and data is accurate.
### Considering Datasets
- Specify dataset(s) in **charts to Build** (e.g., 'sales_db').
- Note availability issues in **Notes**.
### Additional Tips for Charts
- Use standard charts over number cards when possible.
- For `exploratory_or_discovery_or_summaries`/`goal_oriented`: Brainstorm 20, pick 5-12 impactful charts.
- Justify each chart's value to the objective.
- Confirm feasibility with available data.
---
"##;