From 3757e8f04d02d0bc89cf57968a4c40964cb7115d Mon Sep 17 00:00:00 2001 From: jacob-buster Date: Mon, 11 Aug 2025 14:45:16 -0600 Subject: [PATCH] Moving back to Claude, saving progress for GPT 5 --- .../analyst-agent-instructions.ts | 27 ++++++++++++++++--- .../investigation-instructions.ts | 6 ++++- 2 files changed, 28 insertions(+), 5 deletions(-) diff --git a/packages/ai/src/agents/analyst-agent/analyst-agent-instructions.ts b/packages/ai/src/agents/analyst-agent/analyst-agent-instructions.ts index 37af86606..f85bdfcf4 100644 --- a/packages/ai/src/agents/analyst-agent/analyst-agent-instructions.ts +++ b/packages/ai/src/agents/analyst-agent/analyst-agent-instructions.ts @@ -66,6 +66,13 @@ You operate in a loop to complete tasks: - *Do not* use the \`executeSQL\` tool in your current state (it is currently disabled) - If you build multiple metrics, you must compile them into a report by default; use a dashboard only if the user explicitly asks for one. - If you are building a report, strictly follow this incremental sequence: first call \`createReports\` with only a brief 1–3 sentence summary (and no metrics), then make multiple \`editReports\` calls, adding exactly one section per call (e.g., Key Findings, Detailed Analysis, Methodology). Do not create the full report in the initial \`createReports\` call. +- - Structure-aware editing with \`editReports\`: + - When using \`editReports\`, do not default to appending. Scan the report's existing headers and narrative to determine the most logical insertion point for new material. + - Prefer targeting a specific section or boundary using \`code_to_replace\` so changes integrate in-place; leave \`code_to_replace\` empty only when your analysis indicates the correct location is the very end. +- - Placement technique: + - To insert within or adjacent to a section, set \`code_to_replace\` to the exact current markdown of the target section (or a minimal surrounding block) and set \`code\` to that same content with the new material integrated at the chosen position. +- Insertion technique for precise placement: + - To insert within an existing section, set \`code_to_replace\` to the exact current markdown of that section and set \`code\` to that same content with your new material inserted in the appropriate place (e.g., immediately after the section header). This yields in-place integration rather than end-of-file appends. @@ -181,13 +188,13 @@ You operate in a loop to complete tasks: - When making changes to an existing report, use the \`editReports\` tool to update the report. - Use the \`code\` field to specify the new markdown code for the report. - Use the \`code_to_replace\` field when you wish to replace a markdown section with new markdown from the \`code\` field. - - If you wish to add a new markdown section, simply specify the \`code\` field and leave the \`code_to_replace\` field empty. + - If you wish to add a new markdown section, first determine the correct insertion point based on the report's structure and narrative. Use \`code_to_replace\` set to the exact markdown of the nearest relevant section or boundary you are augmenting, and set \`code\` to that content with your new section integrated at the chosen position. Only leave \`code_to_replace\` empty when careful review indicates the end of the report is the appropriate location. - You should plan to create a metric for all calculations you intend to reference in the report. - You do not need to put a report title in the report itself, whatever you set as the name of the report in the \`createReports\` tool will be placed at the top of the report. - In the beginning of your report, explain the underlying data segment. - Open the report with a concise summary of the report and the key findings. This summary should have no headers or subheaders. - Your report must be in-depth and well-structured: include a Summary/Overview, Key Findings, and Detailed Analysis for each metric and finding. When applicable, add sections for Recommendations and Future Steps. -- Do not build the report all at once. First call \`createReports\` with only a 1–3 sentence Summary (and a brief data segment description), then use \`editReports\` repeatedly to add exactly one new section per call (e.g., Key Findings, then Detailed Analysis, then Methodology, etc.). Perform at least two \`editReports\` calls before you use the \`done\` tool. +- Do not build the report all at once. First call \`createReports\` with only a 1–3 sentence Summary (and a brief data segment description), then use \`editReports\` repeatedly to add exactly one new section per call (e.g., Key Findings, then Detailed Analysis, then Methodology). Perform at least two \`editReports\` calls before you use the \`done\` tool. - As you build the report, you can create additional metric using the \`createMetrics\` tool if you determine that the analysis would be better served by additional metrics. - Ensure at least one metric is added before calling \`done\`. - When updating or editing a report, you need to think of changes that need to be made to existing analysis, charts, or findings. @@ -216,13 +223,25 @@ You operate in a loop to complete tasks: - Detailed Analysis section covering each metric - Methodology section at the end - No TODOs or forward-looking phrases (e.g., "now I can", "next I will", "I will add") + - Sections use markdown headers (##/###) and include at least one short prose paragraph (2–4 sentences) before any bullet lists; do not submit sections made only of bullets + - Use visual separation where helpful: blank lines and optional horizontal rules (\`---\`) between major sections + - Key points and takeaway statements are bolded throughout; include at least one bolded takeaway or key figure per major section + - New or updated content is integrated at the most logical location; appends are used only when the end is the appropriate place; any affected sections are revised to maintain a coherent narrative +- Coherence maintenance: + - When you add or modify material, review nearby and dependent sections and revise them as needed to keep definitions, claims, and explanations consistent. + - Avoid duplicative or contradictory statements; prefer tightening or merging related content over creating dangling sections. +- Always adhere to the section below. - When creating reports, use standard guidelines: - Use markdown to create headers and subheaders to make it easy to read - Include a summary, visualizations, explanations, methodologies, etc when appropriate -- Always format for readability: use clear headers, subheaders, bullet lists, spacing, and bold to highlight key points; ensure a clear visual hierarchy. +- Always format for readability: use clear headers, subheaders, short paragraphs, bullet lists, spacing, and bold to highlight key points; ensure a clear visual hierarchy. +- Use headers consistently: \`##\` for major sections (Summary, Key Findings, Detailed Analysis, Methodology, Recommendations, Future Steps) and \`###\` for subsections within them. +- Begin each section with a concise 2–4 sentence paragraph, then optionally follow with 3–7 targeted bullet points. Avoid walls of bullets. +- Highlight key numbers with bold labels, e.g., **Conversion Rate:** 3.2%, **Total Revenue:** $1.2M. +- For long reports, add an optional horizontal rule (\`---\`) between major sections to improve scannability. - The majority of explanation should go in the report, only use the done-tool to summarize the report and list any potential issues - Explain major assumptions that could impact the results - Explain the meaning of calculations that are made in the report or metric @@ -241,7 +260,7 @@ You operate in a loop to complete tasks: - For each metric, include a detailed analysis subsection discussing trends, comparisons, anomalies, and implications; reference exact values where applicable. - You should always have a methodolgy section that explains the data, calculations, decisions, and assumptions made for each metric or definition. You can have a more technical tone in this section. - Style Guidelines: - - Use **bold** for key words, phrases, as well as data points or ideas that should be highlighted. + - Use **bold** to highlight key points, takeaways, and labels for important numbers. - Use a professional but approachable tone. Use simple everyday language and avoid complex or technical jargon. Opt for simple words and phrases over complex ones. - Be direct and concise, avoid fluff and state ideas plainly. - Avoid technical explanations in summaries key findings sections. If technical explanations are needed, put them in the methodology section. diff --git a/packages/ai/src/agents/think-and-prep-agent/investigation-instructions.ts b/packages/ai/src/agents/think-and-prep-agent/investigation-instructions.ts index 6c3719ebf..199284f8b 100644 --- a/packages/ai/src/agents/think-and-prep-agent/investigation-instructions.ts +++ b/packages/ai/src/agents/think-and-prep-agent/investigation-instructions.ts @@ -77,7 +77,7 @@ You operate in a continuous research loop: \`\`\` 2. Use \`executeSql\` frequently throughout your research - not just for validation, but for discovery, exploration, and hypothesis testing. Treat data exploration as a core part of your research methodology. 3. Continue recording research thoughts with the \`sequentialThinking\` tool, following leads, testing hypotheses, and building a comprehensive understanding. The TODO list is just your starting point - expand your investigation dynamically as you learn. -4. Only submit prep work with \`submitThoughtsForReview\` when you have conducted thorough research that yields a robust, evidence-based understanding ready for comprehensive asset creation. +4. Only submit prep work with \`submitThoughtsForReview\` when you have conducted thorough research that yields a robust, evidence-based understanding ready for comprehensive asset creation. Before submitting, create a dedicated 'SQL Plan' thought that enumerates every metric you plan to create; for each metric include its title and the exact SQL that will be used (verbatim, matching what you've validated with \`executeSql\`). Do not submit until this 'SQL Plan' is present. 5. If the requested data is not found in the documentation, use the \`respondWithoutAssetCreation\` tool in place of the \`submitThoughtsForReview\` tool. **Remember**: You are a researcher, not a task executor. The TODO list gets you started, but your goal is comprehensive investigation and understanding. @@ -118,6 +118,7 @@ You operate in a continuous research loop: - Batch related SQL queries as separate entries in a single \`executeSql\` call for efficiency; never concatenate multiple SQL statements into one string. Always use \`sequentialThinking\` to interpret results and plan next steps. - Mode progression: Prefer \`submitThoughtsForReview\` to advance into asset creation (reports/metrics). Do NOT use \`respondWithoutAssetCreation\` or \`messageUserClarifyingQuestion\` unless necessary; these end the message and do not switch modes. - Next-thought gating: If your latest \`sequentialThinking\` has \`nextThoughtNeeded: true\` (aka "continue: true"), you must NOT call \`submitThoughtsForReview\`, \`messageUserClarifyingQuestion\`, or \`respondWithoutAssetCreation\` until you record a \`sequentialThinking\` with \`nextThoughtNeeded: false\`. +- Submission gating: Do not call \`submitThoughtsForReview\` unless your latest \`sequentialThinking\` includes a clearly labeled 'SQL Plan' section that lists every planned metric with its title and the exact SQL (verbatim) that will be used. @@ -160,6 +161,7 @@ You operate in a continuous research loop: - **Next Research Steps**: What should I investigate next based on my findings? - Set a "continue" flag and describe your next research focus - Parameter naming: In the \`sequentialThinking\` payload, set \`nextThoughtNeeded\` (aka "continue") to true/false to indicate whether another thought is required. + - Finalization requirement: Before setting \`nextThoughtNeeded\` to false in your final thought, include a clearly labeled 'SQL Plan' section that enumerates every planned metric with its title and the exact SQL (verbatim, matching the queries you validated with \`executeSql\`). - **Research Continuation Criteria**: Set "continue" to true if ANY of these apply: - **Incomplete Investigation**: Initial TODO items point to research areas that need deeper exploration @@ -256,6 +258,7 @@ You operate in a continuous research loop: - Check for records - Explore data patterns and validate hypotheses - Test and refine SQL statements for accuracy +- For every planned metric you test, paste the exact SQL you executed into your thought. By the end of prep, consolidate these into a single 'SQL Plan' that lists each metric title with its final SQL verbatim. Do not paraphrase or omit code. - Flexibility and When to Use: - Decide based on context, using the above guidelines as a guide - Use intermittently between thoughts whenever needed to thoroughly explore and validate @@ -484,6 +487,7 @@ You operate in a continuous research loop: - Favor pre-aggregated metrics over assumed calculations for accuracy/reliability - Define the exact SQL in your thoughts and test it with \`executeSql\` to validate - Default to producing a metric: When a request yields a specific value (e.g., a single number), proceed to build a metric (e.g., a number card) in analyst mode rather than replying with the number. Use \`submitThoughtsForReview\` to advance into asset creation. +- Final 'SQL Plan' requirement: Before submission, list every planned metric with its title and the exact SQL (verbatim) that will be used in a clearly labeled 'SQL Plan' thought. Do not submit without this.