LogicGoInfotechSpaces commited on
Commit
3bfb118
·
1 Parent(s): 78b2ce5

feat: add MongoDB logging for all application logs

Browse files

- Add mongodb_logging.py with custom logging handler
- Update server.py to integrate MongoDB logging
- Add new endpoints: /logs, /logs/stats, /logs/clear
- Fix MongoDB database boolean check issue
- All application logs now stored in MongoDB HairSwapDB.logs collection
- Add deployment guide for Hugging Face Spaces

DEPLOYMENT_GUIDE.md ADDED
@@ -0,0 +1,141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 🚀 MongoDB Logging Deployment Guide for Hugging Face Spaces
2
+
3
+ ## ✅ What's Ready for Deployment
4
+
5
+ ### 📁 Files Added/Modified:
6
+ - ✅ `mongodb_logging.py` - Custom MongoDB logging handler
7
+ - ✅ `server.py` - Updated with MongoDB logging integration
8
+ - ✅ `requirements.txt` - Already includes `pymongo[srv]`
9
+
10
+ ### 🔧 New API Endpoints:
11
+ - `GET /logs` - View all logs (metadata + application logs)
12
+ - `GET /logs/stats` - View logging statistics
13
+ - `GET /logs/clear?days_older_than=7` - Clear old logs
14
+
15
+ ## 🌐 Hugging Face Spaces Deployment Steps
16
+
17
+ ### 1. Set Environment Variables in Hugging Face Spaces
18
+
19
+ Go to your Space settings and add these environment variables:
20
+
21
+ ```
22
+ MONGO_URI=mongodb+srv://harilogicgo_db_user:SSbZ55jNUsVerWKI@hairswapdb.7r2ghs4.mongodb.net/?retryWrites=true&w=majority&appName=HairSwapDB
23
+ ```
24
+
25
+ ### 2. Push Changes to Hugging Face
26
+
27
+ ```bash
28
+ # Add all changes
29
+ git add .
30
+
31
+ # Commit changes
32
+ git commit -m "feat: add MongoDB logging for all application logs
33
+
34
+ - Add mongodb_logging.py with custom logging handler
35
+ - Update server.py to integrate MongoDB logging
36
+ - Add new endpoints: /logs, /logs/stats, /logs/clear
37
+ - All application logs now stored in MongoDB HairSwapDB.logs collection"
38
+
39
+ # Push to Hugging Face
40
+ git push origin main
41
+ ```
42
+
43
+ ### 3. Verify Deployment
44
+
45
+ After deployment, test these endpoints:
46
+
47
+ 1. **Health Check**: `GET /health`
48
+ 2. **View Logs**: `GET /logs`
49
+ 3. **Log Statistics**: `GET /logs/stats`
50
+ 4. **Clear Old Logs**: `GET /logs/clear?days_older_than=7`
51
+
52
+ ## 📊 MongoDB Collections Structure
53
+
54
+ ### `HairSwapDB.logs` Collection:
55
+ ```json
56
+ {
57
+ "_id": "ObjectId",
58
+ "timestamp": "2025-09-27T12:30:45.123Z",
59
+ "level": "INFO|WARNING|ERROR",
60
+ "logger": "hair_server",
61
+ "message": "Log message content",
62
+ "module": "server",
63
+ "function": "get_hairswap",
64
+ "line": 168,
65
+ "thread": 12345,
66
+ "process": 6789,
67
+ "exception": "Exception traceback (if error)"
68
+ }
69
+ ```
70
+
71
+ ### `HairSwapDB.uploads` Collection:
72
+ ```json
73
+ {
74
+ "_id": "image-uuid",
75
+ "filename": "image.png",
76
+ "path": "/data/uploads/image-uuid.png"
77
+ }
78
+ ```
79
+
80
+ ### `HairSwapDB.results` Collection:
81
+ ```json
82
+ {
83
+ "_id": "result-uuid",
84
+ "filename": "result.png",
85
+ "path": "/data/results/result-uuid.png",
86
+ "source_id": "source-uuid",
87
+ "reference_id": "reference-uuid"
88
+ }
89
+ ```
90
+
91
+ ## 🔍 Testing the Deployment
92
+
93
+ ### 1. Test Log Generation:
94
+ ```bash
95
+ # Upload an image
96
+ curl -X POST "https://your-space-url/upload" \
97
+ -F "image=@test_image.jpg"
98
+
99
+ # Perform hair swap
100
+ curl -X POST "https://your-space-url/get-hairswap" \
101
+ -H "Content-Type: application/json" \
102
+ -d '{"source_id": "source-uuid", "reference_id": "reference-uuid"}'
103
+ ```
104
+
105
+ ### 2. Check Logs:
106
+ ```bash
107
+ # View all logs
108
+ curl "https://your-space-url/logs"
109
+
110
+ # View log statistics
111
+ curl "https://your-space-url/logs/stats"
112
+
113
+ # Clear logs older than 7 days
114
+ curl "https://your-space-url/logs/clear?days_older_than=7"
115
+ ```
116
+
117
+ ## 🎯 What Gets Logged
118
+
119
+ ### ✅ Application Logs (NEW):
120
+ - Model loading messages
121
+ - Hair transfer progress
122
+ - Error messages and stack traces
123
+ - API request/response logs
124
+ - System status messages
125
+
126
+ ### ✅ Metadata Logs (EXISTING):
127
+ - Image uploads
128
+ - Hair swap results
129
+ - File paths and IDs
130
+
131
+ ## 🚨 Important Notes
132
+
133
+ 1. **MongoDB Connection**: Make sure `MONGO_URI` is set in Space environment variables
134
+ 2. **Log Rotation**: Use `/logs/clear` endpoint to manage log storage
135
+ 3. **Performance**: MongoDB logging is asynchronous and won't slow down the API
136
+ 4. **Fallback**: If MongoDB fails, logs still go to console
137
+ 5. **Security**: MongoDB URI contains credentials - keep it secure
138
+
139
+ ## 🎉 Ready to Deploy!
140
+
141
+ Your MongoDB logging system is ready for Hugging Face Spaces deployment. All application logs will be automatically stored in MongoDB and accessible via the new API endpoints.
__pycache__/mongodb_logging.cpython-310.pyc ADDED
Binary file (3.8 kB). View file
 
__pycache__/server.cpython-310.pyc ADDED
Binary file (8.81 kB). View file
 
mongodb_logging.py ADDED
@@ -0,0 +1,141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ import os
3
+ from datetime import datetime
4
+ from typing import Dict, Any
5
+ from pymongo import MongoClient
6
+ from pymongo.collection import Collection
7
+
8
+
9
+ class MongoDBLogHandler(logging.Handler):
10
+ """Custom logging handler that stores logs in MongoDB"""
11
+
12
+ def __init__(self, collection: Collection):
13
+ super().__init__()
14
+ self.collection = collection
15
+
16
+ def emit(self, record: logging.LogRecord):
17
+ """Emit a log record to MongoDB"""
18
+ try:
19
+ # Create log document
20
+ log_doc = {
21
+ "timestamp": datetime.utcnow(),
22
+ "level": record.levelname,
23
+ "logger": record.name,
24
+ "message": record.getMessage(),
25
+ "module": record.module,
26
+ "function": record.funcName,
27
+ "line": record.lineno,
28
+ "thread": record.thread,
29
+ "process": record.process,
30
+ }
31
+
32
+ # Add exception info if present
33
+ if record.exc_info:
34
+ log_doc["exception"] = self.formatException(record.exc_info)
35
+
36
+ # Add extra fields if present
37
+ if hasattr(record, 'extra_fields'):
38
+ log_doc.update(record.extra_fields)
39
+
40
+ # Insert into MongoDB
41
+ self.collection.insert_one(log_doc)
42
+
43
+ except Exception as e:
44
+ # Fallback to console if MongoDB fails
45
+ print(f"MongoDB logging failed: {e}")
46
+ print(f"Original log: {record.getMessage()}")
47
+
48
+
49
+ def setup_mongodb_logging(mongo_uri: str, database_name: str = "HairSwapDB", collection_name: str = "logs"):
50
+ """Setup MongoDB logging for the application"""
51
+
52
+ try:
53
+ # Connect to MongoDB
54
+ client = MongoClient(mongo_uri)
55
+ db = client.get_database(database_name)
56
+ logs_collection = db.get_collection(collection_name)
57
+
58
+ # Create MongoDB log handler
59
+ mongo_handler = MongoDBLogHandler(logs_collection)
60
+ mongo_handler.setLevel(logging.INFO)
61
+
62
+ # Create formatter
63
+ formatter = logging.Formatter(
64
+ '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
65
+ )
66
+ mongo_handler.setFormatter(formatter)
67
+
68
+ # Get root logger and add handler
69
+ root_logger = logging.getLogger()
70
+ root_logger.addHandler(mongo_handler)
71
+
72
+ # Also add to specific logger
73
+ hair_logger = logging.getLogger("hair_server")
74
+ hair_logger.addHandler(mongo_handler)
75
+
76
+ print(f"✅ MongoDB logging enabled - storing logs in {database_name}.{collection_name}")
77
+ return True
78
+
79
+ except Exception as e:
80
+ print(f"❌ Failed to setup MongoDB logging: {e}")
81
+ return False
82
+
83
+
84
+ def get_logs_from_mongodb(mongo_uri: str, database_name: str = "HairSwapDB",
85
+ collection_name: str = "logs", limit: int = 100,
86
+ level: str = None, logger_name: str = None):
87
+ """Retrieve logs from MongoDB"""
88
+
89
+ try:
90
+ client = MongoClient(mongo_uri)
91
+ db = client.get_database(database_name)
92
+ logs_collection = db.get_collection(collection_name)
93
+
94
+ # Build query
95
+ query = {}
96
+ if level:
97
+ query["level"] = level
98
+ if logger_name:
99
+ query["logger"] = logger_name
100
+
101
+ # Get logs sorted by timestamp (newest first)
102
+ logs = list(logs_collection.find(query)
103
+ .sort("timestamp", -1)
104
+ .limit(limit))
105
+
106
+ # Convert ObjectId to string for JSON serialization
107
+ for log in logs:
108
+ log["_id"] = str(log["_id"])
109
+ if "timestamp" in log:
110
+ log["timestamp"] = log["timestamp"].isoformat()
111
+
112
+ return logs
113
+
114
+ except Exception as e:
115
+ print(f"❌ Failed to retrieve logs from MongoDB: {e}")
116
+ return []
117
+
118
+
119
+ def clear_logs_from_mongodb(mongo_uri: str, database_name: str = "HairSwapDB",
120
+ collection_name: str = "logs", days_older_than: int = None):
121
+ """Clear old logs from MongoDB"""
122
+
123
+ try:
124
+ client = MongoClient(mongo_uri)
125
+ db = client.get_database(database_name)
126
+ logs_collection = db.get_collection(collection_name)
127
+
128
+ if days_older_than:
129
+ from datetime import timedelta
130
+ cutoff_date = datetime.utcnow() - timedelta(days=days_older_than)
131
+ result = logs_collection.delete_many({"timestamp": {"$lt": cutoff_date}})
132
+ print(f"✅ Deleted {result.deleted_count} logs older than {days_older_than} days")
133
+ else:
134
+ result = logs_collection.delete_many({})
135
+ print(f"✅ Deleted all {result.deleted_count} logs")
136
+
137
+ return result.deleted_count
138
+
139
+ except Exception as e:
140
+ print(f"❌ Failed to clear logs from MongoDB: {e}")
141
+ return 0
server.py CHANGED
@@ -14,9 +14,8 @@ from PIL import Image
14
 
15
  # Lazy import performed in get_model() to avoid import-time failures on Space
16
 
17
-
18
- LOGGER = logging.getLogger("hair_server")
19
- logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(name)s - %(message)s")
20
 
21
  EXPECTED_BEARER = "logicgo@123"
22
 
@@ -24,9 +23,17 @@ EXPECTED_BEARER = "logicgo@123"
24
  from pymongo import MongoClient
25
  MONGO_URI = os.environ.get("MONGO_URI", "")
26
  mongo_client = MongoClient(MONGO_URI) if MONGO_URI else None
27
- mongo_db = mongo_client.get_database("HairSwapDB") if mongo_client else None
28
- uploads_col = mongo_db.get_collection("uploads") if mongo_db else None
29
- results_col = mongo_db.get_collection("results") if mongo_db else None
 
 
 
 
 
 
 
 
30
 
31
 
32
  def verify_bearer(authorization: Optional[str] = Header(None)):
@@ -224,11 +231,90 @@ def download(filename: str):
224
 
225
 
226
  @app.get("/logs")
227
- def logs():
 
 
 
 
228
  if uploads_col and results_col:
229
  uploads = list(uploads_col.find({}, {"_id": 1, "filename": 1}).limit(20))
230
  results = list(results_col.find({}, {"_id": 1, "filename": 1, "source_id": 1, "reference_id": 1}).limit(20))
231
- return JSONResponse({"uploads": uploads, "results": results})
232
- return JSONResponse({"logs": ["service running"], "db": "not_configured"})
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
233
 
234
 
 
14
 
15
  # Lazy import performed in get_model() to avoid import-time failures on Space
16
 
17
+ # Import MongoDB logging
18
+ from mongodb_logging import setup_mongodb_logging, get_logs_from_mongodb, clear_logs_from_mongodb
 
19
 
20
  EXPECTED_BEARER = "logicgo@123"
21
 
 
23
  from pymongo import MongoClient
24
  MONGO_URI = os.environ.get("MONGO_URI", "")
25
  mongo_client = MongoClient(MONGO_URI) if MONGO_URI else None
26
+ mongo_db = mongo_client.get_database("HairSwapDB") if mongo_client is not None else None
27
+ uploads_col = mongo_db.get_collection("uploads") if mongo_db is not None else None
28
+ results_col = mongo_db.get_collection("results") if mongo_db is not None else None
29
+ logs_col = mongo_db.get_collection("logs") if mongo_db is not None else None
30
+
31
+ # Setup MongoDB logging
32
+ if MONGO_URI:
33
+ setup_mongodb_logging(MONGO_URI, "HairSwapDB", "logs")
34
+
35
+ LOGGER = logging.getLogger("hair_server")
36
+ logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(name)s - %(message)s")
37
 
38
 
39
  def verify_bearer(authorization: Optional[str] = Header(None)):
 
231
 
232
 
233
  @app.get("/logs")
234
+ def logs(limit: int = 50, level: str = None, logger_name: str = None):
235
+ """Get logs from MongoDB including both metadata and application logs"""
236
+ response_data = {}
237
+
238
+ # Get metadata (uploads and results)
239
  if uploads_col and results_col:
240
  uploads = list(uploads_col.find({}, {"_id": 1, "filename": 1}).limit(20))
241
  results = list(results_col.find({}, {"_id": 1, "filename": 1, "source_id": 1, "reference_id": 1}).limit(20))
242
+ response_data["metadata"] = {"uploads": uploads, "results": results}
243
+ else:
244
+ response_data["metadata"] = {"uploads": [], "results": []}
245
+
246
+ # Get application logs from MongoDB
247
+ if MONGO_URI:
248
+ try:
249
+ app_logs = get_logs_from_mongodb(MONGO_URI, "HairSwapDB", "logs", limit, level, logger_name)
250
+ response_data["application_logs"] = app_logs
251
+ response_data["mongodb_status"] = "connected"
252
+ except Exception as e:
253
+ response_data["application_logs"] = []
254
+ response_data["mongodb_status"] = f"error: {str(e)}"
255
+ else:
256
+ response_data["application_logs"] = []
257
+ response_data["mongodb_status"] = "not_configured"
258
+
259
+ return JSONResponse(response_data)
260
+
261
+
262
+ @app.get("/logs/clear")
263
+ def clear_logs(days_older_than: int = None):
264
+ """Clear old logs from MongoDB"""
265
+ if not MONGO_URI:
266
+ raise HTTPException(status_code=400, detail="MongoDB not configured")
267
+
268
+ try:
269
+ deleted_count = clear_logs_from_mongodb(MONGO_URI, "HairSwapDB", "logs", days_older_than)
270
+ return JSONResponse({
271
+ "message": f"Cleared {deleted_count} logs",
272
+ "days_older_than": days_older_than
273
+ })
274
+ except Exception as e:
275
+ raise HTTPException(status_code=500, detail=f"Failed to clear logs: {str(e)}")
276
+
277
+
278
+ @app.get("/logs/stats")
279
+ def logs_stats():
280
+ """Get logging statistics"""
281
+ if not MONGO_URI:
282
+ return JSONResponse({"mongodb_status": "not_configured"})
283
+
284
+ try:
285
+ client = MongoClient(MONGO_URI)
286
+ db = client.get_database("HairSwapDB")
287
+ logs_collection = db.get_collection("logs")
288
+
289
+ # Get total count
290
+ total_logs = logs_collection.count_documents({})
291
+
292
+ # Get count by level
293
+ pipeline = [
294
+ {"$group": {"_id": "$level", "count": {"$sum": 1}}},
295
+ {"$sort": {"count": -1}}
296
+ ]
297
+ logs_by_level = list(logs_collection.aggregate(pipeline))
298
+
299
+ # Get count by logger
300
+ pipeline = [
301
+ {"$group": {"_id": "$logger", "count": {"$sum": 1}}},
302
+ {"$sort": {"count": -1}},
303
+ {"$limit": 10}
304
+ ]
305
+ logs_by_logger = list(logs_collection.aggregate(pipeline))
306
+
307
+ return JSONResponse({
308
+ "total_logs": total_logs,
309
+ "logs_by_level": logs_by_level,
310
+ "top_loggers": logs_by_logger,
311
+ "mongodb_status": "connected"
312
+ })
313
+
314
+ except Exception as e:
315
+ return JSONResponse({
316
+ "mongodb_status": f"error: {str(e)}",
317
+ "total_logs": 0
318
+ })
319
 
320