Enhance complete tool with text, attachments, and improved frontend rendering

Co-authored-by: tnfssc <tnfssc@gmail.com>

COMPLETE_TOOL_CHANGES.md

@@ -0,0 +1,108 @@
+# Complete Tool Changes Summary
+
+## Overview
+Modified the complete tool to behave like the ask tool by accepting `text` and `attachments` parameters, making it more flexible for providing completion summaries and deliverables.
+
+## Backend Changes
+
+### 1. Updated `backend/agent/tools/message_tool.py`
+
+**Changes to the `complete` method:**
+- Added `text` parameter (optional): Completion message or summary to present to user
+- Added `attachments` parameter (optional): List of files or URLs to attach to the completion message
+- Updated OpenAPI schema to include the new parameters
+- Updated XML schema mappings to support the new parameters
+- Updated function signature: `async def complete(self, text: Optional[str] = None, attachments: Optional[Union[str, List[str]]] = None) -> ToolResult:`
+- Added parameter conversion logic to handle single attachment strings
+
+**Updated description:**
+- Enhanced tool description to mention including relevant attachments
+- Updated example to show usage with text and attachments
+
+### 2. Updated `backend/agent/prompt.py`
+
+**Changes to agent prompts:**
+- Modified the rule about sharing results to allow using the complete tool with attachments
+- Updated from: "Always share results and deliverables using 'ask' tool with attachments before entering complete state"
+- Updated to: "Always share results and deliverables using 'ask' tool with attachments before entering complete state, or include them directly with the 'complete' tool"
+
+## Frontend Changes
+
+### 1. Created `frontend/src/components/thread/tool-views/complete-tool/_utils.ts`
+
+**New utility file:**
+- Mirrors the structure of `ask-tool/_utils.ts`
+- Provides `extractCompleteData` function to parse text and attachments from tool calls
+- Supports both new format (tool_execution) and legacy format parsing
+- Handles single attachment strings and arrays
+- Includes comprehensive logging for debugging
+
+### 2. Updated `frontend/src/components/thread/tool-views/CompleteToolView.tsx`
+
+**Major changes:**
+- Added import for `extractCompleteData` utility
+- Added `project` prop to support file attachments
+- Added file type detection functions (`isImageFile`, `isPreviewableFile`)
+- Updated component to use extracted text and attachments data
+- Modified success badge to use actual success state
+- Updated content rendering to handle text and attachments like AskToolView
+- Added FileAttachment component for proper file display
+- Updated empty state condition to include new parameters
+- Maintained backward compatibility with existing complete data parsing
+
+**New features:**
+- Displays completion text in a formatted markdown box
+- Shows attachments in a grid layout with proper file previews
+- Supports image files, previewable files, and regular files
+- Maintains existing functionality for legacy complete data
+
+## Benefits
+
+1. **Consistency**: Complete tool now has the same parameter structure as ask tool
+2. **Flexibility**: Agents can provide completion summaries and deliverables directly
+3. **Better UX**: Users can see completion details and attached files in a unified interface
+4. **Backward Compatibility**: Existing complete tool calls without parameters still work
+5. **Enhanced Communication**: Agents can provide more context when completing tasks
+
+## Usage Examples
+
+### Backend Usage:
+```python
+# Simple completion (existing behavior)
+await message_tool.complete()
+
+# With completion message
+await message_tool.complete(text="All tasks completed successfully!")
+
+# With attachments
+await message_tool.complete(attachments=["app.js", "docs/README.md"])
+
+# With both text and attachments
+await message_tool.complete(
+    text="Project completed! Here are the deliverables:",
+    attachments=["app.js", "docs/README.md", "deployment.yaml"]
+)
+```
+
+### Frontend Display:
+- Text appears in a formatted markdown box
+- Attachments are displayed in a responsive grid
+- Images are shown with previews
+- Previewable files (HTML, MD, CSV) are rendered inline
+- Regular files show with appropriate icons
+
+## Testing
+
+The changes have been tested to ensure:
+- Backward compatibility with existing complete tool calls
+- Proper parameter handling (text, attachments, both, or neither)
+- Frontend rendering of text and attachments
+- File type detection and display
+- Success state handling
+
+## Migration Notes
+
+- Existing complete tool calls will continue to work unchanged
+- New complete tool calls can optionally include text and attachments
+- Frontend will gracefully handle both old and new formats
+- No breaking changes to existing functionality

backend/agent/prompt.py

@@ -576,7 +576,7 @@ For casual conversation and social interactions:
   * Attach all relevant files with the **'ask'** tool when asking a question related to them, or when delivering final results before completion.
   * Always include representable files as attachments when using 'ask' - this includes HTML files, presentations, writeups, visualizations, reports, and any other viewable content.
   * For any created files that can be viewed or presented (such as index.html, slides, documents, charts, etc.), always attach them to the 'ask' tool to ensure the user can immediately see the results.
-  * Always share results and deliverables using 'ask' tool with attachments before entering complete state. Do not use 'complete' tool directly before using 'ask' tool.
+  * Always share results and deliverables using 'ask' tool with attachments before entering complete state, or include them directly with the 'complete' tool. Do not use 'complete' tool directly before using 'ask' tool unless you're including all necessary attachments.
   * Ensure users have access to all necessary resources.
 
 - Communication Tools Summary:

backend/agent/tools/message_tool.py

@@ -217,31 +217,62 @@ If you encounter any issues or need to take additional steps, please let me know
         "type": "function",
         "function": {
             "name": "complete",
-            "description": "A special tool to indicate you have completed all tasks and are about to enter complete state. Use ONLY when: 1) All tasks in todo.md are marked complete [x], 2) The user's original request has been fully addressed, 3) There are no pending actions or follow-ups required, 4) You've delivered all final outputs and results to the user. IMPORTANT: This is the ONLY way to properly terminate execution. Never use this tool unless ALL tasks are complete and verified. Always ensure you've provided all necessary outputs and references before using this tool.",
+            "description": "A special tool to indicate you have completed all tasks and are about to enter complete state. Use ONLY when: 1) All tasks in todo.md are marked complete [x], 2) The user's original request has been fully addressed, 3) There are no pending actions or follow-ups required, 4) You've delivered all final outputs and results to the user. IMPORTANT: This is the ONLY way to properly terminate execution. Never use this tool unless ALL tasks are complete and verified. Always ensure you've provided all necessary outputs and references before using this tool. Include relevant attachments when the completion relates to specific files or resources.",
             "parameters": {
                 "type": "object",
-                "properties": {},
+                "properties": {
+                    "text": {
+                        "type": "string",
+                        "description": "Completion message or summary to present to user - should provide clear indication of what was accomplished. Include: 1) Summary of completed tasks, 2) Key deliverables or outputs, 3) Any important notes or next steps, 4) Impact or benefits achieved."
+                    },
+                    "attachments": {
+                        "anyOf": [
+                            {"type": "string"},
+                            {"items": {"type": "string"}, "type": "array"}
+                        ],
+                        "description": "(Optional) List of files or URLs to attach to the completion message. Include when: 1) Completion relates to specific files or configurations, 2) User needs to review final outputs, 3) Deliverables are documented in files, 4) Supporting evidence or context is needed. Always use relative paths to /workspace directory."
+                    }
+                },
                 "required": []
             }
         }
     })
     @xml_schema(
         tag_name="complete",
-        mappings=[],
+        mappings=[
+            {"param_name": "text", "node_type": "content", "path": ".", "required": False},
+            {"param_name": "attachments", "node_type": "attribute", "path": ".", "required": False}
+        ],
         example='''
         <function_calls>
         <invoke name="complete">
+        <parameter name="text">I have successfully completed all tasks for your project. Here's what was accomplished:
+1. Created the web application with modern UI components
+2. Implemented user authentication and database integration
+3. Deployed the application to production
+4. Created comprehensive documentation
+
+All deliverables are attached for your review.</parameter>
+        <parameter name="attachments">app/src/main.js,docs/README.md,deployment-config.yaml</parameter>
         </invoke>
         </function_calls>
         '''
     )
-    async def complete(self) -> ToolResult:
+    async def complete(self, text: Optional[str] = None, attachments: Optional[Union[str, List[str]]] = None) -> ToolResult:
         """Indicate that the agent has completed all tasks and is entering complete state.
 
+        Args:
+            text: Optional completion message or summary to present to the user
+            attachments: Optional file paths or URLs to attach to the completion message
+
         Returns:
             ToolResult indicating successful transition to complete state
         """
         try:
+            # Convert single attachment to list for consistent handling
+            if attachments and isinstance(attachments, str):
+                attachments = [attachments]
+                
             return self.success_response({"status": "complete"})
         except Exception as e:
             return self.fail_response(f"Error entering complete state: {str(e)}")

frontend/src/components/thread/tool-views/CompleteToolView.tsx

@@ -18,12 +18,14 @@ import {
   extractToolData,
   getFileIconAndColor,
 } from './utils';
+import { extractCompleteData } from './complete-tool/_utils';
 import { cn } from '@/lib/utils';
 import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
 import { Badge } from '@/components/ui/badge';
 import { ScrollArea } from "@/components/ui/scroll-area";
 import { Progress } from '@/components/ui/progress';
 import { Markdown } from '@/components/ui/markdown';
+import { FileAttachment } from '../../file-attachment';
 
 interface CompleteContent {
   summary?: string;
@@ -46,10 +48,26 @@ export function CompleteToolView({
   isSuccess = true,
   isStreaming = false,
   onFileClick,
+  project,
 }: CompleteToolViewProps) {
   const [completeData, setCompleteData] = useState<CompleteContent>({});
   const [progress, setProgress] = useState(0);
 
+  const {
+    text,
+    attachments,
+    status,
+    actualIsSuccess,
+    actualToolTimestamp,
+    actualAssistantTimestamp
+  } = extractCompleteData(
+    assistantContent,
+    toolContent,
+    isSuccess,
+    toolTimestamp,
+    assistantTimestamp
+  );
+
   useEffect(() => {
     if (assistantContent) {
       try {
@@ -120,6 +138,16 @@ export function CompleteToolView({
     }
   }, [isStreaming]);
 
+  const isImageFile = (filePath: string): boolean => {
+    const filename = filePath.split('/').pop() || '';
+    return filename.match(/\.(jpg|jpeg|png|gif|webp|svg|bmp)$/i) !== null;
+  };
+
+  const isPreviewableFile = (filePath: string): boolean => {
+    const ext = filePath.split('.').pop()?.toLowerCase() || '';
+    return ext === 'html' || ext === 'htm' || ext === 'md' || ext === 'markdown' || ext === 'csv' || ext === 'tsv';
+  };
+
   const toolTitle = getToolTitle(name) || 'Task Complete';
 
   const handleFileClick = (filePath: string) => {
@@ -147,17 +175,17 @@ export function CompleteToolView({
             <Badge
               variant="secondary"
               className={
-                isSuccess
+                actualIsSuccess
                   ? "bg-gradient-to-b from-emerald-200 to-emerald-100 text-emerald-700 dark:from-emerald-800/50 dark:to-emerald-900/60 dark:text-emerald-300"
                   : "bg-gradient-to-b from-rose-200 to-rose-100 text-rose-700 dark:from-rose-800/50 dark:to-rose-900/60 dark:text-rose-300"
               }
             >
-              {isSuccess ? (
+              {actualIsSuccess ? (
                 <CheckCircle className="h-3.5 w-3.5 mr-1" />
               ) : (
                 <AlertTriangle className="h-3.5 w-3.5 mr-1" />
               )}
-              {isSuccess ? 'Completed' : 'Failed'}
+              {actualIsSuccess ? 'Completed' : 'Failed'}
             </Badge>
           )}
 
@@ -173,8 +201,8 @@ export function CompleteToolView({
       <CardContent className="p-0 flex-1 overflow-hidden relative">
         <ScrollArea className="h-full w-full">
           <div className="p-4 space-y-6">
-            {/* Success Animation/Icon - Only show when completed successfully */}
+            {/* Success Animation/Icon - Only show when completed successfully and no text/attachments */}
-            {!isStreaming && isSuccess && !completeData.summary && !completeData.tasksCompleted && !completeData.attachments && (
+            {!isStreaming && actualIsSuccess && !text && !attachments && !completeData.summary && !completeData.tasksCompleted && (
               <div className="flex justify-center">
                 <div className="relative">
                   <div className="w-20 h-20 rounded-full bg-gradient-to-br from-emerald-100 to-emerald-200 dark:from-emerald-800/40 dark:to-emerald-900/60 flex items-center justify-center">
@@ -187,19 +215,95 @@ export function CompleteToolView({
               </div>
             )}
 
-            {/* Summary Section */}
+            {/* Text/Summary Section */}
-            {completeData.summary && (
+            {(text || completeData.summary) && (
               <div className="space-y-2">
                 <div className="bg-muted/50 rounded-2xl p-4 border border-border">
                   <Markdown className="text-sm prose prose-sm dark:prose-invert chat-markdown max-w-none [&>:first-child]:mt-0 prose-headings:mt-3">
-                    {completeData.summary}
+                    {text || completeData.summary}
                   </Markdown>
                 </div>
               </div>
             )}
 
             {/* Attachments Section */}
-            {completeData.attachments && completeData.attachments.length > 0 && (
+            {attachments && attachments.length > 0 ? (
+              <div className="space-y-4">
+                <div className="flex items-center gap-2 text-sm font-medium text-muted-foreground">
+                  <Paperclip className="h-4 w-4" />
+                  Files ({attachments.length})
+                </div>
+
+                <div className={cn(
+                  "grid gap-3",
+                  attachments.length === 1 ? "grid-cols-1" :
+                    attachments.length > 4 ? "grid-cols-1 sm:grid-cols-2 md:grid-cols-3" :
+                      "grid-cols-1 sm:grid-cols-2"
+                )}>
+                  {attachments
+                    .sort((a, b) => {
+                      const aIsImage = isImageFile(a);
+                      const bIsImage = isImageFile(b);
+                      const aIsPreviewable = isPreviewableFile(a);
+                      const bIsPreviewable = isPreviewableFile(b);
+
+                      if (aIsImage && !bIsImage) return -1;
+                      if (!aIsImage && bIsImage) return 1;
+                      if (aIsPreviewable && !bIsPreviewable) return -1;
+                      if (!aIsPreviewable && bIsPreviewable) return 1;
+                      return 0;
+                    })
+                    .map((attachment, index) => {
+                      const isImage = isImageFile(attachment);
+                      const isPreviewable = isPreviewableFile(attachment);
+                      const shouldSpanFull = (attachments!.length % 2 === 1 &&
+                        attachments!.length > 1 &&
+                        index === attachments!.length - 1);
+
+                      return (
+                        <div
+                          key={index}
+                          className={cn(
+                            "relative group",
+                            isImage ? "flex items-center justify-center h-full" : "",
+                            isPreviewable ? "w-full" : ""
+                          )}
+                          style={(shouldSpanFull || isPreviewable) ? { gridColumn: '1 / -1' } : undefined}
+                        >
+                          <FileAttachment
+                            filepath={attachment}
+                            onClick={handleFileClick}
+                            sandboxId={project?.sandbox?.id}
+                            showPreview={true}
+                            className={cn(
+                              "w-full",
+                              isImage ? "h-auto min-h-[54px]" :
+                                isPreviewable ? "min-h-[240px] max-h-[400px] overflow-auto" : "h-[54px]"
+                            )}
+                            customStyle={
+                              isImage ? {
+                                width: '100%',
+                                height: 'auto',
+                                '--attachment-height': shouldSpanFull ? '240px' : '180px'
+                              } as React.CSSProperties :
+                                isPreviewable ? {
+                                  gridColumn: '1 / -1'
+                                } :
+                                  shouldSpanFull ? {
+                                    gridColumn: '1 / -1'
+                                  } : {
+                                    width: '100%'
+                                  }
+                            }
+                            collapsed={false}
+                            project={project}
+                          />
+                        </div>
+                      );
+                    })}
+                </div>
+              </div>
+            ) : completeData.attachments && completeData.attachments.length > 0 ? (
               <div className="space-y-3">
                 <div className="flex items-center gap-2 text-sm font-medium text-muted-foreground">
                   <Paperclip className="h-4 w-4" />
@@ -243,7 +347,7 @@ export function CompleteToolView({
                   })}
                 </div>
               </div>
-            )}
+            ) : null}
 
             {/* Tasks Completed Section */}
             {completeData.tasksCompleted && completeData.tasksCompleted.length > 0 && (
@@ -288,7 +392,7 @@ export function CompleteToolView({
             )}
 
             {/* Empty State */}
-            {!completeData.summary && !completeData.result && !completeData.attachments && !completeData.tasksCompleted && !isStreaming && (
+            {!text && !attachments && !completeData.summary && !completeData.result && !completeData.attachments && !completeData.tasksCompleted && !isStreaming && (
               <div className="flex flex-col items-center justify-center py-8 text-center">
                 <div className="w-16 h-16 rounded-full bg-muted flex items-center justify-center mb-4">
                   <CheckCircle2 className="h-8 w-8 text-muted-foreground" />

frontend/src/components/thread/tool-views/complete-tool/_utils.ts

@@ -0,0 +1,234 @@
+import { extractToolData, normalizeContentToString } from '../utils';
+
+export interface CompleteData {
+  text: string | null;
+  attachments: string[] | null;
+  status: string | null;
+  success?: boolean;
+  timestamp?: string;
+}
+
+const parseContent = (content: any): any => {
+  if (typeof content === 'string') {
+    try {
+      return JSON.parse(content);
+    } catch (e) {
+      return content;
+    }
+  }
+  return content;
+};
+
+const extractFromNewFormat = (content: any): { 
+  text: string | null;
+  attachments: string[] | null;
+  status: string | null;
+  success?: boolean; 
+  timestamp?: string;
+} => {
+  const parsedContent = parseContent(content);
+  
+  if (!parsedContent || typeof parsedContent !== 'object') {
+    return { text: null, attachments: null, status: null, success: undefined, timestamp: undefined };
+  }
+
+  if ('tool_execution' in parsedContent && typeof parsedContent.tool_execution === 'object') {
+    const toolExecution = parsedContent.tool_execution;
+    const args = toolExecution.arguments || {};
+    
+    let parsedOutput = toolExecution.result?.output;
+    if (typeof parsedOutput === 'string') {
+      try {
+        parsedOutput = JSON.parse(parsedOutput);
+      } catch (e) {
+      }
+    }
+
+    let attachments: string[] | null = null;
+    if (args.attachments) {
+      if (typeof args.attachments === 'string') {
+        attachments = args.attachments.split(',').map((a: string) => a.trim()).filter((a: string) => a.length > 0);
+      } else if (Array.isArray(args.attachments)) {
+        attachments = args.attachments;
+      }
+    }
+
+    let status: string | null = null;
+    if (parsedOutput && typeof parsedOutput === 'object' && parsedOutput.status) {
+      status = parsedOutput.status;
+    }
+
+    const extractedData = {
+      text: args.text || null,
+      attachments,
+      status: status || parsedContent.summary || null,
+      success: toolExecution.result?.success,
+      timestamp: toolExecution.execution_details?.timestamp
+    };
+
+    console.log('CompleteToolView: Extracted from new format:', {
+      hasText: !!extractedData.text,
+      attachmentCount: extractedData.attachments?.length || 0,
+      hasStatus: !!extractedData.status,
+      success: extractedData.success
+    });
+    
+    return extractedData;
+  }
+
+  if ('role' in parsedContent && 'content' in parsedContent) {
+    return extractFromNewFormat(parsedContent.content);
+  }
+
+  return { text: null, attachments: null, status: null, success: undefined, timestamp: undefined };
+};
+
+const extractFromLegacyFormat = (content: any): { 
+  text: string | null;
+  attachments: string[] | null;
+  status: string | null;
+} => {
+  const toolData = extractToolData(content);
+  
+  if (toolData.toolResult && toolData.arguments) {
+    console.log('CompleteToolView: Extracted from legacy format (extractToolData):', {
+      hasText: !!toolData.arguments.text,
+      attachmentCount: toolData.arguments.attachments ? 
+        (Array.isArray(toolData.arguments.attachments) ? toolData.arguments.attachments.length : 1) : 0
+    });
+    
+    let attachments: string[] | null = null;
+    if (toolData.arguments.attachments) {
+      if (Array.isArray(toolData.arguments.attachments)) {
+        attachments = toolData.arguments.attachments;
+      } else if (typeof toolData.arguments.attachments === 'string') {
+        attachments = toolData.arguments.attachments.split(',').map(a => a.trim()).filter(a => a.length > 0);
+      }
+    }
+    
+    return {
+      text: toolData.arguments.text || null,
+      attachments,
+      status: null
+    };
+  }
+
+  const contentStr = normalizeContentToString(content);
+  if (!contentStr) {
+    return { text: null, attachments: null, status: null };
+  }
+
+  let attachments: string[] | null = null;
+  const attachmentsMatch = contentStr.match(/attachments=["']([^"']*)["']/i);
+  if (attachmentsMatch) {
+    attachments = attachmentsMatch[1].split(',').map(a => a.trim()).filter(a => a.length > 0);
+  }
+
+  let text: string | null = null;
+  const textMatch = contentStr.match(/<complete[^>]*>([^<]*)<\/complete>/i);
+  if (textMatch) {
+    text = textMatch[1].trim();
+  }
+  
+  console.log('CompleteToolView: Extracted from legacy format (manual parsing):', {
+    hasText: !!text,
+    attachmentCount: attachments?.length || 0
+  });
+  
+  return {
+    text,
+    attachments,
+    status: null
+  };
+};
+
+export function extractCompleteData(
+  assistantContent: any,
+  toolContent: any,
+  isSuccess: boolean,
+  toolTimestamp?: string,
+  assistantTimestamp?: string
+): {
+  text: string | null;
+  attachments: string[] | null;
+  status: string | null;
+  actualIsSuccess: boolean;
+  actualToolTimestamp?: string;
+  actualAssistantTimestamp?: string;
+} {
+  let text: string | null = null;
+  let attachments: string[] | null = null;
+  let status: string | null = null;
+  let actualIsSuccess = isSuccess;
+  let actualToolTimestamp = toolTimestamp;
+  let actualAssistantTimestamp = assistantTimestamp;
+
+  const assistantNewFormat = extractFromNewFormat(assistantContent);
+  const toolNewFormat = extractFromNewFormat(toolContent);
+
+  console.log('CompleteToolView: Format detection results:', {
+    assistantNewFormat: {
+      hasText: !!assistantNewFormat.text,
+      attachmentCount: assistantNewFormat.attachments?.length || 0,
+      hasStatus: !!assistantNewFormat.status
+    },
+    toolNewFormat: {
+      hasText: !!toolNewFormat.text,
+      attachmentCount: toolNewFormat.attachments?.length || 0,
+      hasStatus: !!toolNewFormat.status
+    }
+  });
+
+  if (assistantNewFormat.text || assistantNewFormat.attachments || assistantNewFormat.status) {
+    text = assistantNewFormat.text;
+    attachments = assistantNewFormat.attachments;
+    status = assistantNewFormat.status;
+    if (assistantNewFormat.success !== undefined) {
+      actualIsSuccess = assistantNewFormat.success;
+    }
+    if (assistantNewFormat.timestamp) {
+      actualAssistantTimestamp = assistantNewFormat.timestamp;
+    }
+    console.log('CompleteToolView: Using assistant new format data');
+  } else if (toolNewFormat.text || toolNewFormat.attachments || toolNewFormat.status) {
+    text = toolNewFormat.text;
+    attachments = toolNewFormat.attachments;
+    status = toolNewFormat.status;
+    if (toolNewFormat.success !== undefined) {
+      actualIsSuccess = toolNewFormat.success;
+    }
+    if (toolNewFormat.timestamp) {
+      actualToolTimestamp = toolNewFormat.timestamp;
+    }
+    console.log('CompleteToolView: Using tool new format data');
+  } else {
+    const assistantLegacy = extractFromLegacyFormat(assistantContent);
+    const toolLegacy = extractFromLegacyFormat(toolContent);
+
+    text = assistantLegacy.text || toolLegacy.text;
+    attachments = assistantLegacy.attachments || toolLegacy.attachments;
+    status = assistantLegacy.status || toolLegacy.status;
+    
+    console.log('CompleteToolView: Using legacy format data:', {
+      hasText: !!text,
+      attachmentCount: attachments?.length || 0,
+      hasStatus: !!status
+    });
+  }
+
+  console.log('CompleteToolView: Final extracted data:', {
+    hasText: !!text,
+    attachmentCount: attachments?.length || 0,
+    hasStatus: !!status,
+    actualIsSuccess
+  });
+
+  return {
+    text,
+    attachments,
+    status,
+    actualIsSuccess,
+    actualToolTimestamp,
+    actualAssistantTimestamp
+  };
+}