Update app.py from anycoder

app.py

@@ -1,167 +1,121 @@
-*Thinking...*
+import gradio as gr
+import cv2
+import os
+import time
+from utils import is_video, process_frame_mock
 
-> **Refining the Uploads**
-> 
-> I'm now focusing on how to modify the Gradio application to handle multiple image or video uploads. My primary concern is ensuring the output is persistent and updates dynamically as new media is uploaded, rather than requiring a full re-process. I'm taking a look at `src/sam3d_body/gradio_ui/sam3d_body_ui.py` to see how the single upload is managed currently.
-> 
-> 
-> **Analyzing Input Methods**
-> 
-> I'm now weighing the best approach for the Gradio input. Initially, I'll switch from the single-image input in `sam3d_body_ui.py` to using `gr.File(file_count="multiple")` to handle multiple uploads, and passing the associated filepaths to the processing function. Considering the need for persistent output, a streaming or accumulative process is very suitable. This suggests that the current `sam3d_prediction_fn` needs to be adapted or wrapped to accept a sequence of files, making sure that it process each file one by one in one go.
-> 
-> 
-> **Formulating a Plan**
-> 
-> The immediate focus is updating the processing logic, especially `sam3d_prediction_fn`, to accommodate multiple file uploads via `gr.File(file_count="multiple")`. This necessitates iterating through the file list, differentiating between images and videos. The "persistent updates" requirement leads me to consider frame-by-frame processing for videos and the use of Rerun's timeline features and streaming capabilities for a dynamic output. I need to make the prediction function able to yield the outputs so it can stream to the Gradio interface.
-> 
-> 
-> **Adapting Processing Steps**
-> 
-> The key is now to adapt the `sam3d_prediction_fn` to iterate and process the incoming file list. For videos, this means frame-by-frame handling; for images, single-frame processing. I'm focusing on ensuring the Rerun timeline remains continuous across multiple file uploads, avoiding any reset behaviors. The streaming capabilities of the `gradio-rerun` library are the central part of the design, especially through `@rr.thread_local_stream`, and the use of the `yield` statement within the prediction function is critical.
-> 
-> 
-> **Evaluating the Implementation**
-> 
-> My thought process is now on coding the changes. I'll begin by modifying `sam3d_body_ui.py`, first changing the input from `gr.Image` to `gr.File(file_count="multiple")`. This will require adapting `sam3d_prediction_fn` to iterate through the file list. I'll need to use `av` to handle videos and `cv2` can be removed. The core challenge is making sure the Rerun timeline correctly sequences frames from different uploads seamlessly, especially when using `yield`. I'll test it step-by-step.
-> 
-> 
-> **Revising the Implementation**
-> 
-> The core of my thought process is now on coding. I'll change the Gradio UI input from `gr.Image` to `gr.File(file_count="multiple", label="Input Images/Videos")`. Then I'll adapt `sam3d_prediction_fn` to iterate through the provided files. I'll need to use the `av` library for video frame extraction and consider a global frame counter for consistent Rerun timeline indexing. The main challenge will be maintaining continuous Rerun timelines across file processing and handling the streamed outputs.
-> 
-> 
-> **Revising the Processing Logic**
-> 
-> I'm now focusing on implementing the proposed changes. I'll modify `src/sam3d_body/gradio_ui/sam3d_body_ui.py`, replacing the single-image input with `gr.File(file_count="multiple", label="Input Images/Videos")`. I'll adapt `sam3d_prediction_fn` to process each uploaded file, handling image and video types. This includes using `cv2` or `av` for frame extraction in videos and creating a global counter for the Rerun timeline. I'll make sure the `@rr.thread_local_stream` streams everything.
-> 
-> 
-> **Adapting to Multiple Inputs**
-> 
-> I'm now refactoring the `sam3d_prediction_fn` function. The goal is to move from single image input to handling a list of file paths uploaded via `gr.File(file_count="multiple")`. This change entails processing each file sequentially and differentiating between image and video types. The core will involve integrating `cv2` or `av` for video frame extraction and ensuring consistent Rerun timeline indexing across multiple uploads, using the stream.
-> 
-> 
-> **Adapting to File Inputs**
-> 
-> I'm now restructuring `sam3d_prediction_fn` to handle a list of file paths uploaded by the user, replacing the single-image input. This refactoring involves iterating through each file, distinguishing between images and videos to load the frames accordingly using `cv2`. I'm focusing on incorporating a global frame counter for consistent Rerun timeline indexing across all uploads, with a focus on smooth, dynamic updates.
-> 
-> 
-> **Re-Evaluating Function Inputs**
-> 
-> I'm now changing the function signature for `sam3d_prediction_fn` to accept `file_paths: list[str]` instead of the current input. I'm focusing on creating an efficient processing loop, distinguishing between images and videos and using frame indexing for Rerun's timeline to ensure continuity across uploads. I will use `cv2` for frame extraction.
-> 
-> 
-> **Re-Evaluating Functionality**
-> 
-> I'm now changing the function signature for `sam3d_prediction_fn`, making it accept a list of file paths. The current focus is on creating an effective processing loop that distinguishes between images and videos and also uses a frame index for the Rerun timeline. My current plan involves leveraging `cv2` for frame extraction. I'm taking the steps necessary to make my function more versatile.
-> 
-> 
-> **Adjusting the Processing Loop**
-> 
-> I'm now changing the function signature for `sam3d_prediction_fn`, making it accept a list of file paths. I'm focusing on creating an effective processing loop that distinguishes between images and videos and also uses a frame index for the Rerun timeline. I've chosen `cv2` for frame extraction. I'm taking the steps necessary to make my function more versatile.
-> 
-> 
-> **Adjusting the Frame Counter**
-> 
-> I'm now in the process of finalizing the processing loop. Specifically, I'm integrating the global frame counter more comprehensively to ensure accurate Rerun timeline indexing across all uploaded files. The core focus is on preventing any reset behavior. I'm also preparing to test the new function.
-> 
-> 
-> **Adjusting File Handling**
-> 
-> I'm now adapting the processing logic within `sam3d_prediction_fn` to correctly use the file list provided. This involves iterating over each file path, determining its type (image or video), and loading the media accordingly, using `cv2` for frame extraction. I'm focusing on the integration of the frame index within the Rerun stream.
-> 
-> 
-> **Adapting File Processing**
-> 
-> I'm now adapting the core function, `sam3d_prediction_fn`, to handle the list of file paths delivered by the `gr.File` input. The initial processing will need to determine the file type (image or video) and then call appropriate extraction methods to ensure the Rerun timeline remains continuous across multiple uploads. I'm focusing on the global frame counter's correct incrementing and the smooth display of updates in the Gradio interface.
-> 
-> 
-> **Adjusting File Paths**
-> 
-> I'm now adapting the processing logic within `sam3d_prediction_fn` to correctly use the file list provided. This involves iterating over each file path, determining its type (image or video), and loading the media accordingly, using `cv2` for frame extraction. I'm focusing on the integration of the frame index within the Rerun stream.
-> 
-> 
-> **Adjusting the Main Function**
-> 
-> I'm now restructuring `main` to integrate the changes to `sam3d_prediction_fn` and the input components. The main adjustment is updating the Gradio UI to use `gr.File(file_count="multiple")` for handling multiple image and video uploads, alongside corresponding modifications to example inputs. I'm focusing on a seamless user experience.
-> 
-> 
-> **Modifying UI Components**
-> 
-> I'm now changing the input component from `gr.Image` to `gr.File(file_count="multiple", label="Input Images/Videos")` and modifying the examples to handle file paths. This addresses the core requirement of supporting multiple image and video uploads. I'm taking the necessary steps to make my function more versatile.
-> 
-> 
-> **Refactoring UI for Uploads**
-> 
-> I'm now changing the input component from `gr.Image` to `gr.File(file_count="multiple", label="Input Images/Videos")` and modifying example inputs. This satisfies the requirement of supporting many image and video uploads, and I'll focus on testing the updated UI interaction.
-> 
-> 
-> **Adjusting for Code Implementation**
-> 
-> I'm now integrating the changes into `src/sam3d_body/gradio_ui/sam3d_body_ui.py`, including modifications to the function signatures, input components, and example data. I've focused on ensuring the correct use of `gr.File` for uploading multiple image or video files and adapting the processing logic to manage them effectively.
-> 
-> 
-> **Modifying the Main Script**
-> 
-> I'm now integrating the changes into `src/sam3d_body/gradio_ui/sam3d_body_ui.py`, including modifications to the function signatures, input components, and example data. I'm focused on ensuring the correct use of `gr.File` for uploading multiple image or video files and adapting the processing logic to manage them effectively.
-> 
-> 
-> **Adapting the Main Code**
-> 
-> I've now incorporated the changes into the main UI file, `src/sam3d_body/gradio_ui/sam3d_body_ui.py`. I've updated the file input to `gr.File` for handling multiple files and adjusted the example inputs accordingly. The focus is to make the application fully compatible with multiple image and video uploads, and ensuring it performs as expected.
-> 
-> 
-> **Adjusting for Code Updates**
-> 
-> I've integrated the changes into `src/sam3d_body/gradio_ui/sam3d_body_ui.py`, updating the input with `gr.File(file_count="multiple")` and example data. The focus is to make the application handle multiple images and videos. The next step is to test it thoroughly.
-> 
-> 
-> **Adjusting UI and Logic**
-> 
-> I've now incorporated the changes into `src/sam3d_body/gradio_ui/sam3d_body_ui.py`, updating the file input to accept multiple uploads. I've adjusted the code to load and handle images and video files. The main task will be testing to ensure that the multiple uploads are processed correctly, and that the Rerun streaming occurs as designed, showing frames smoothly.
-> 
-> 
-> **Adjusting the Main UI**
-> 
-> I've now incorporated the changes into `src/sam3d_body/gradio_ui/sam3d_body_ui.py`, updating the file input to support multiple uploads and revising the example inputs, including the addition of `cv2`. I am making sure that the application now smoothly handles multiple images and videos. The main task is verifying the Rerun streaming and its dynamic behavior.
-> 
-> 
-> **Refactoring for File Handling**
-> 
-> I'm now changing the function signature for `sam3d_prediction_fn`, making it accept a list of file paths. I'm focusing on creating an effective processing loop that distinguishes between images and videos and also uses a frame index for the Rerun timeline. I've chosen `cv2` for frame extraction. I'm taking the steps necessary to make my function more versatile.
-> 
-> 
-> **Finalizing Code and Testing**
-> 
-> I've implemented the changes in `src/sam3d_body/gradio_ui/sam3d_body_ui.py`, updating the UI and processing logic to accommodate multiple file uploads and stream the Rerun output. The focus now is on thorough testing to ensure proper handling of various image and video formats, continuous timeline indexing, and correct GLB export, using `cv2` within the revised functions.
-> 
-> 
-> **Adjusting for Testing**
-> 
-> I'm now adapting the code to accommodate the changes, after which I'll focus on testing. I'm focusing on ensuring the Rerun timeline remains continuous across multiple file uploads and that videos are handled correctly with `cv2`. The core thing is getting to testing now.
-> 
-> 
-> **Finalizing the Integration**
-> 
-> I'm now delivering the completed code for `src/sam3d_body/gradio_ui/sam3d_body_ui.py`, fully updated to handle multiple file uploads and stream the Rerun output. The changes now allow for the correct processing of a user's image and video uploads, with the `cv2` library and the Rerun stream working in tandem.
-> 
-> 
-> **Finalizing the Adaptation**
-> 
-> I'm now delivering the completed code for `src/sam3d_body/gradio_ui/sam3d_body_ui.py`, fully updated to handle multiple file uploads and stream the Rerun output. The changes now allow for the correct processing of a user's image and video uploads, with the `cv2` library and the Rerun stream working in tandem, ready for final tests.
-> 
-> 
-> **Modifying the UI Elements**
-> 
-> I'm now restructuring `src/sam3d_body/gradio_ui/sam3d_body_ui.py` to change the input component and related examples to support multiple files, using `gr.File(file_count="multiple", label="Input Images/Videos")`. I'm also preparing for the required tests.
-> 
-> 
-> **Adapting for Gradio Updates**
-> 
-> I'm now fully integrating the file upload functionality and Rerun streaming. This encompasses modifying the UI components and processing steps to accommodate `gr.File` for uploading and iterating through multiple files. The emphasis is on seamless operation and correct behavior when the user uploads multiple media, and ensuring updates show smoothly in the Gradio interface.
-> 
-> 
-> **Integrating New Uploads**
-> 
-> I'm now changing the input to `gr.File(file_count="multiple")` in `src/sam3d_body/gradio_ui/sam3d_body_ui.py` to allow multi-file uploads. The `sam3d_prediction_fn` function signature is also updated to accommodate a list of file paths to process. I am also adjusting the UI examples for file input.
-> 
->
+def sam3d_prediction_fn(file_paths, progress=gr.Progress()):
+    """
+    Processes a list of file paths (images or videos).
+    Simulates the SAM3D/Rerun logic by iterating through files
+    and yielding results for a streaming UI.
+    """
+    if not file_paths:
+        return [], "No files uploaded."
+
+    processed_results = []
+    total_files = len(file_paths)
+    status_log = ""
+
+    # Global frame counter simulation (as mentioned in your thought trace)
+    global_frame_idx = 0
+
+    for i, file_path in enumerate(file_paths):
+        file_name = os.path.basename(file_path)
+        status_msg = f"Processing file {i+1}/{total_files}: {file_name}..."
+        status_log += status_msg + "\n"
+        
+        # Yield status update immediately
+        yield processed_results, status_log
+        
+        # Determine if file is video or image
+        if is_video(file_path):
+            cap = cv2.VideoCapture(file_path)
+            frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
+            
+            # Process video frames (simulating frame-by-frame logic)
+            # We'll just process a few frames to keep the demo quick
+            max_frames_to_preview = 5 
+            current_frame = 0
+            
+            while cap.isOpened() and current_frame < max_frames_to_preview:
+                ret, frame = cap.read()
+                if not ret:
+                    break
+                
+                # Simulate SAM3D processing on the frame
+                processed_frame = process_frame_mock(frame, f"Video Frame {global_frame_idx}")
+                processed_results.append((processed_frame, f"{file_name} - Frame {current_frame}"))
+                
+                global_frame_idx += 1
+                current_frame += 1
+                
+                # Stream updates every frame
+                yield processed_results, status_log
+                time.sleep(0.1) # Simulate processing time
+                
+            cap.release()
+            status_log += f"Finished video: {file_name}\n"
+
+        else:
+            # Process single image
+            image = cv2.imread(file_path)
+            if image is not None:
+                processed_image = process_frame_mock(image, f"Image {global_frame_idx}")
+                processed_results.append((processed_image, file_name))
+                global_frame_idx += 1
+                
+                # Stream update
+                yield processed_results, status_log
+                time.sleep(0.5) # Simulate processing time
+            
+            status_log += f"Finished image: {file_name}\n"
+
+    status_log += "All processing complete."
+    yield processed_results, status_log
+
+# --- Gradio 6 Application Structure ---
+
+with gr.Blocks() as demo:
+    # Header with required link
+    gr.Markdown("# SAM3D Multi-File Processor")
+    gr.Markdown("[Built with anycoder](https://huggingface.co/spaces/akhaliq/anycoder)")
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            # UPDATED: Using gr.File with file_count="multiple" as requested
+            input_files = gr.File(
+                file_count="multiple",
+                label="Input Images/Videos",
+                file_types=["image", "video"]
+            )
+            process_btn = gr.Button("Start Processing", variant="primary")
+            
+            # Log output to see the sequential processing logic
+            log_output = gr.Textbox(label="Processing Log", lines=10, interactive=False)
+
+        with gr.Column(scale=2):
+            # Gallery to display the streaming results
+            output_gallery = gr.Gallery(
+                label="Processed Stream", 
+                columns=3, 
+                height=600,
+                object_fit="contain"
+            )
+
+    # Event Listener
+    process_btn.click(
+        fn=sam3d_prediction_fn,
+        inputs=[input_files],
+        outputs=[output_gallery, log_output],
+        api_visibility="public"
+    )
+
+# Launch with Gradio 6 parameters
+if __name__ == "__main__":
+    demo.launch(
+        theme=gr.themes.Soft(),
+        footer_links=[
+            {"label": "Built with anycoder", "url": "https://huggingface.co/spaces/akhaliq/anycoder"}
+        ]
+    )