Spaces:

jawadsaghir12
/

chatbot

Running

App Files Files Community

chatbot / TESTING_GUIDE.md

jawadsaghir12

new update

8c77cd6 12 days ago

preview code

raw

history blame contribute delete

10.6 kB

🧪 TESTING & VERIFICATION GUIDE

Quick Manual Verification (No pytest needed)

Prerequisites

Backend running on http://localhost:7860 (or your configured port)
Supabase database configured in .env

Test Scenario: Complete Workflow

Step 1: Register User

curl -X POST http://localhost:7860/auth/register \
  -H "Content-Type: application/json" \
  -d '{
    "email": "alice@example.com",
    "name": "Alice",
    "password": "SecurePass123!"
  }'

Expected Response:

{
  "id": 1,
  "email": "alice@example.com",
  "name": "Alice",
  "created_at": "2024-01-10T12:00:00Z"
}

Step 2: Login & Get Token

curl -X POST http://localhost:7860/auth/login \
  -H "Content-Type: application/json" \
  -d '{
    "email": "alice@example.com",
    "password": "SecurePass123!"
  }'

Expected Response:

{
  "access_token": "eyJ0eXAiOiJKV1QiLCJhbGc...",
  "token_type": "bearer"
}

Save token: TOKEN="eyJ0eXAiOiJKV1QiLCJhbGc..."

Step 3: Send First Message (Create Conversation)

curl -X POST http://localhost:7860/chat/send \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "query": "What is machine learning?"
  }'

Expected Response:

{
  "conversation_id": 1,
  "response": "Machine learning is a subset of artificial intelligence...",
  "timestamp": "2024-01-10T12:00:05Z"
}

Verify in Database:

SELECT * FROM conversations WHERE user_id = 1;
-- Should show: id=1, user_id=1, is_deleted=false, last_message_at=NOW()

SELECT * FROM messages WHERE conversation_id = 1 ORDER BY created_at;
-- Should show 2 messages: user query + AI response

Step 4: Send Follow-up Message (Same Conversation)

curl -X POST http://localhost:7860/chat/send \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "conversation_id": 1,
    "query": "Tell me about neural networks"
  }'

Expected Response:

{
  "conversation_id": 1,
  "response": "Neural networks are computing systems...",
  "timestamp": "2024-01-10T12:00:10Z"
}

Verify:

-- conversation_id should still be 1
-- last_message_at should be updated
-- message count should be 4 (2 exchanges)
SELECT COUNT(*) FROM messages WHERE conversation_id = 1;
-- Result: 4

Step 5: List Conversations

curl -X GET http://localhost:7860/chat/conversations \
  -H "Authorization: Bearer $TOKEN"

Expected Response:

{
  "conversations": [
    {
      "id": 1,
      "title": null,
      "created_at": "2024-01-10T12:00:00Z",
      "last_message_at": "2024-01-10T12:00:10Z",
      "message_count": 4
    }
  ],
  "total": 1,
  "skip": 0,
  "limit": 20
}

Step 6: Retrieve Full Conversation

curl -X GET http://localhost:7860/chat/conversations/1 \
  -H "Authorization: Bearer $TOKEN"

Expected Response:

{
  "id": 1,
  "title": null,
  "created_at": "2024-01-10T12:00:00Z",
  "last_message_at": "2024-01-10T12:00:10Z",
  "messages": [
    {
      "id": 1,
      "sender_id": 1,
      "content": "What is machine learning?",
      "created_at": "2024-01-10T12:00:05Z"
    },
    {
      "id": 2,
      "sender_id": 1,
      "content": "Machine learning is...",
      "created_at": "2024-01-10T12:00:05Z"
    },
    {
      "id": 3,
      "sender_id": 1,
      "content": "Tell me about neural networks",
      "created_at": "2024-01-10T12:00:10Z"
    },
    {
      "id": 4,
      "sender_id": 1,
      "content": "Neural networks are...",
      "created_at": "2024-01-10T12:00:10Z"
    }
  ]
}

Step 7: Search Conversations

curl -X GET "http://localhost:7860/chat/search?q=neural" \
  -H "Authorization: Bearer $TOKEN"

Expected Response:

{
  "conversations": [
    {
      "id": 1,
      "title": null,
      "created_at": "2024-01-10T12:00:00Z",
      "last_message_at": "2024-01-10T12:00:10Z",
      "message_count": 4
    }
  ],
  "total": 1,
  "skip": 0,
  "limit": 20
}

Step 8: Auto-Generate Title (via Service - Optional)

The conversation title can be auto-generated:

# In backend code
from src.services.conversation_service import ConversationService

await ConversationService.auto_generate_title(
    conversation_id=1,
    session=session
)

Then list conversations again to see title:

{
  "title": "What is machine learning?",
  "message_count": 4
}

Step 9: Delete Conversation (Soft Delete)

curl -X DELETE http://localhost:7860/chat/conversations/1 \
  -H "Authorization: Bearer $TOKEN"

Expected Response:

{
  "message": "Conversation deleted successfully"
}

Verify:

-- Mark as deleted
SELECT * FROM conversations WHERE id = 1;
-- Result: is_deleted = true

-- But data is preserved
SELECT COUNT(*) FROM messages WHERE conversation_id = 1;
-- Result: 4 (messages still exist)

-- Should not appear in list
SELECT * FROM conversations WHERE user_id = 1 AND is_deleted = false;
-- Result: (empty set)

Security Verification

Test: User Isolation (403 Forbidden)

Create Second User:

curl -X POST http://localhost:7860/auth/register \
  -H "Content-Type: application/json" \
  -d '{
    "email": "bob@example.com",
    "name": "Bob",
    "password": "SecurePass123!"
  }'

Login as Bob:

curl -X POST http://localhost:7860/auth/login \
  -H "Content-Type: application/json" \
  -d '{
    "email": "bob@example.com",
    "password": "SecurePass123!"
  }'

Save token: BOB_TOKEN="..."

Try to Access Alice's Conversation:

curl -X GET http://localhost:7860/chat/conversations/1 \
  -H "Authorization: Bearer $BOB_TOKEN"

Expected Response (403):

{
  "detail": "You do not have access to this conversation"
}

HTTP Status: 403 Forbidden ✅

Test: Missing Authorization (401/403)

curl -X GET http://localhost:7860/chat/conversations/1

Expected Response:

{
  "detail": "Not authenticated"
}

HTTP Status: 403 Forbidden (HTTPBearer rejects missing credentials) ✅

Test: Deleted Conversation Not in List

# After deleting conversation 1
curl -X GET http://localhost:7860/chat/conversations \
  -H "Authorization: Bearer $TOKEN"

Expected Response: {"conversations": [], "total": 0} ✅

Error Handling Verification

Test 1: Invalid Conversation ID (404)

curl -X GET http://localhost:7860/chat/conversations/99999 \
  -H "Authorization: Bearer $TOKEN"

Expected: 404 Not Found

{
  "detail": "Conversation not found"
}

Test 2: Invalid Query (Empty - 422)

curl -X POST http://localhost:7860/chat/send \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"query": ""}'

Expected: 422 Unprocessable Entity

{
  "detail": [
    {
      "loc": ["body", "query"],
      "msg": "ensure this value has at least 1 character",
      "type": "value_error.any_str.min_length"
    }
  ]
}

Test 3: Conversation Not Found (404)

curl -X POST http://localhost:7860/chat/send \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "conversation_id": 99999,
    "query": "Hello?"
  }'

Expected: 404 Not Found

{
  "detail": "Conversation not found"
}

Performance Verification

Pagination Test

# Get first page (10 conversations)
curl -X GET "http://localhost:7860/chat/conversations?skip=0&limit=10" \
  -H "Authorization: Bearer $TOKEN"

# Get second page (next 10)
curl -X GET "http://localhost:7860/chat/conversations?skip=10&limit=10" \
  -H "Authorization: Bearer $TOKEN"

Should handle large numbers efficiently without N+1 queries.

Search Performance

curl -X GET "http://localhost:7860/chat/search?q=python&skip=0&limit=20" \
  -H "Authorization: Bearer $TOKEN"

Should complete in < 1 second with indexes.

Database Verification

Check Schema

-- Check conversation columns
\d conversations;

-- Verify indexes exist
SELECT * FROM pg_indexes WHERE tablename = 'conversations';
-- Should include: conversations_user_id_idx, conversations_last_message_at_idx

-- Check message volume
SELECT 
  conversation_id,
  COUNT(*) as message_count,
  MAX(created_at) as last_message
FROM messages
GROUP BY conversation_id
ORDER BY last_message DESC;

Logging Verification

Check Application Logs

# Watch logs in real-time
tail -f app.log

# Should see:
# - User login: "User logged in: alice@example.com"
# - Conversation creation: "Created conversation 1 for user 1"
# - Message storage: "Saved user message to conversation 1"
# - AI response: "Saved AI response to conversation 1"
# - Retrieve attempt: "Retrieved conversation 1 with 4 messages"
# - Access denied: "User 2 attempted to access conversation 1 belonging to user 1"
# - Deletion: "Soft deleted conversation 1"

Troubleshooting

Issue: 403 on login-required endpoints

Cause: JWT token expired or invalid Fix: Re-login to get fresh token

Issue: Conversation ID not in response

Cause: Agent service error Fix: Check agent.service() is working, check OpenRouter API key

Issue: Cross-user access still works (bug!)

Cause: Authorization check not implemented Fix: Verify permission check in service method

Issue: Deleted conversations still visible

Cause: is_deleted filter not applied Fix: Check queries include WHERE is_deleted = false

Issue: Message count incorrect

Cause: Soft-deleted messages counted Fix: Check message join condition works properly

✅ Complete Verification Checklist

User registration works
Login returns JWT token
New conversation by message creation
Follow-up messages use same conversation_id
List shows all user's conversations
Retrieve shows full message history
Search finds conversations by keyword
Delete soft-removes conversation
Other user can't access conversation (403)
Other user can't delete conversation (403)
Pagination works with skip/limit
Search pagination works
Error messages are Clear (404, 403)
Empty query rejected (422)
Logs show all operations
Message ordering is chronological
Soft deleted data preserved in DB
Soft deleted not in lists/searches

Status: Ready for production deployment ✅