Update: Face Verfication added

README.md

@@ -1,12 +1,108 @@
----
-title: Mmm
-emoji: 📊
-colorFrom: purple
-colorTo: blue
-sdk: gradio
-sdk_version: 5.34.0
-app_file: app.py
-pinned: false
+# ReLeaf AI API with Face Verification
+
+This is the backend API for the ReLeaf mobile app, providing AI-powered eco-action recognition and face verification capabilities.
+
+## Features
+
+- **Eco-Action Recognition**: Analyze images/videos of environmental activities and assign points
+- **Face Verification**: Verify user identity through facial recognition
+- **Multi-format Support**: Process both images and videos
+- **Secure Authentication**: Face-based verification for action submissions
+
+## API Endpoints
+
+### 1. Health Check
+```
+GET /
+```
+Returns API status and information.
+
+### 2. Face Verification
+```
+POST /verify-face
+```
+**Parameters:**
+- `reference_face`: Image file (stored user face)
+- `current_face`: Image file (captured face for verification)
+
+**Response:**
+```json
+{
+  "verified": true,
+  "similarity": 85.6,
+  "threshold": 60.0,
+  "message": "Face verified successfully"
+}
+```
+
+### 3. Eco-Action Analysis
+```
+POST /predict
+```
+**Parameters:**
+- `file`: Image or video file of eco-action
+- `reference_face`: (Optional) Reference face image for verification
+
+**Response:**
+```json
+{
+  "points": 15,
+  "task": "Recycling plastic bottles",
+  "face_verified": true,
+  "similarity": 87.3,
+  "raw": "Full AI response..."
+}
+```
+
+## Supported Activities
+
+- ♻️ Recycling and waste management
+- 🌱 Tree planting and gardening
+- ⚡ Clean energy usage
+- 🚌 Sustainable transportation
+- 🧹 Environmental cleanup
+- 💧 Water conservation
+- 🍃 Composting
+- 🛒 Sustainable shopping
+
+## Scoring System
+
+Activities are scored from 0-30 points based on:
+- **Impact Level**: Higher impact = more points
+- **Authenticity**: Genuine activities get full points
+- **Scale**: Larger scale activities get bonus points
+- **Innovation**: Creative eco-solutions get extra recognition
+
+## Face Verification
+
+- **Threshold**: 60% similarity required for verification
+- **Security**: Prevents fraudulent submissions
+- **Privacy**: Face data processed in real-time, not stored
+- **Accuracy**: Uses state-of-the-art face recognition algorithms
+
+## Technology Stack
+
+- **FastAPI**: High-performance web framework
+- **Together AI**: Advanced language model for activity recognition
+- **OpenCV**: Computer vision processing
+- **face_recognition**: Facial recognition and verification
+- **PIL/Pillow**: Image processing
+
+## Environment Variables
+
+- `TOGETHER_API_KEY`: API key for Together AI service
+
+## Local Development
+
+```bash
+pip install -r requirements.txt
+uvicorn app:app --reload --host 0.0.0.0 --port 7860
+```
+
+## Deployment
+
+This API is designed to run on Hugging Face Spaces with automatic scaling and GPU acceleration.
+
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+**ReLeaf** - Making sustainability fun, rewarding, and secure! 🌱

app.py

@@ -7,81 +7,272 @@ import base64
 import cv2
 import io
 import re
+import face_recognition
+import numpy as np
 from together import Together
-import releaf_ai  # this should still contain your SYSTEM_PROMPT
+import releaf_ai
 
 app = FastAPI()
 
-# Init Together client
+# Initialize Together client
 API_KEY = "1495bcdf0c72ed1e15d0e3e31e4301bd665cb28f2291bcc388164ed745a7aa24"
 client = Together(api_key=API_KEY)
 MODEL_NAME = "meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8"
-
 SYSTEM_PROMPT = releaf_ai.SYSTEM_PROMPT
 
 def encode_image_to_base64(image: Image.Image) -> str:
+    """Convert PIL Image to base64 string"""
     buffered = io.BytesIO()
     image.save(buffered, format="JPEG")
     return base64.b64encode(buffered.getvalue()).decode("utf-8")
 
 def extract_score(text: str):
+    """Extract score from AI response"""
     match = re.search(r"(?i)Score:\s*(\d+)", text)
     return int(match.group(1)) if match else None
 
 def extract_activity(text: str):
+    """Extract activity from AI response"""
     match = re.search(r"(?i)Detected Activity:\s*(.+?)\n", text)
     return match.group(1).strip() if match else "Unknown"
 
+def verify_faces(reference_face_bytes: bytes, current_face_bytes: bytes) -> dict:
+    """
+    Verify if two face images match
+    Returns: {"verified": bool, "similarity": float, "error": str}
+    """
+    try:
+        # Convert bytes to numpy arrays
+        ref_np = np.frombuffer(reference_face_bytes, np.uint8)
+        curr_np = np.frombuffer(current_face_bytes, np.uint8)
+        
+        # Decode images
+        ref_img = cv2.imdecode(ref_np, cv2.IMREAD_COLOR)
+        curr_img = cv2.imdecode(curr_np, cv2.IMREAD_COLOR)
+        
+        if ref_img is None or curr_img is None:
+            return {"verified": False, "similarity": 0.0, "error": "Could not decode images"}
+        
+        # Convert BGR to RGB (face_recognition expects RGB)
+        ref_rgb = cv2.cvtColor(ref_img, cv2.COLOR_BGR2RGB)
+        curr_rgb = cv2.cvtColor(curr_img, cv2.COLOR_BGR2RGB)
+        
+        # Get face encodings
+        ref_encodings = face_recognition.face_encodings(ref_rgb)
+        curr_encodings = face_recognition.face_encodings(curr_rgb)
+        
+        if len(ref_encodings) == 0:
+            return {"verified": False, "similarity": 0.0, "error": "No face found in reference image"}
+        
+        if len(curr_encodings) == 0:
+            return {"verified": False, "similarity": 0.0, "error": "No face found in current image"}
+        
+        # Use the first face found in each image
+        ref_encoding = ref_encodings[0]
+        curr_encoding = curr_encodings[0]
+        
+        # Calculate face distance (lower = more similar)
+        face_distance = face_recognition.face_distance([ref_encoding], curr_encoding)[0]
+        
+        # Convert distance to similarity percentage (0-100)
+        similarity = max(0, (1 - face_distance) * 100)
+        
+        # Verification threshold (adjust as needed)
+        VERIFICATION_THRESHOLD = 60.0  # 60% similarity required
+        verified = similarity >= VERIFICATION_THRESHOLD
+        
+        return {
+            "verified": verified,
+            "similarity": round(similarity, 2),
+            "error": None
+        }
+        
+    except Exception as e:
+        return {"verified": False, "similarity": 0.0, "error": str(e)}
+
+@app.get("/")
+async def root():
+    return {"message": "ReLeaf AI API with Face Verification", "status": "active"}
+
+@app.post("/verify-face")
+async def verify_face_endpoint(
+    reference_face: UploadFile = File(...),
+    current_face: UploadFile = File(...)
+):
+    """
+    Standalone face verification endpoint
+    """
+    try:
+        # Validate file types
+        if not reference_face.content_type.startswith("image"):
+            raise HTTPException(status_code=400, detail="Reference face must be an image")
+        
+        if not current_face.content_type.startswith("image"):
+            raise HTTPException(status_code=400, detail="Current face must be an image")
+        
+        # Read file bytes
+        ref_bytes = await reference_face.read()
+        curr_bytes = await current_face.read()
+        
+        # Perform face verification
+        result = verify_faces(ref_bytes, curr_bytes)
+        
+        if result["error"]:
+            raise HTTPException(status_code=400, detail=result["error"])
+        
+        return JSONResponse({
+            "verified": result["verified"],
+            "similarity": result["similarity"],
+            "threshold": 60.0,
+            "message": "Face verified successfully" if result["verified"] else "Face verification failed"
+        })
+        
+    except HTTPException:
+        raise
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Face verification error: {str(e)}")
+
 @app.post("/predict")
-async def predict(file: UploadFile = File(...)):
+async def predict(
+    file: UploadFile = File(...),
+    reference_face: UploadFile = File(None)
+):
+    """
+    Main prediction endpoint with optional face verification
+    """
     try:
+        face_verification_result = None
+        
+        # Perform face verification if reference face is provided
+        if reference_face and reference_face.filename:
+            if not reference_face.content_type.startswith("image"):
+                raise HTTPException(status_code=400, detail="Reference face must be an image")
+            
+            # Extract face from the action image/video for verification
+            action_file_bytes = await file.read()
+            ref_face_bytes = await reference_face.read()
+            
+            # Reset file position for later processing
+            await file.seek(0)
+            
+            # For video files, extract a frame first
+            if file.content_type.startswith("video"):
+                # Save video temporarily
+                temp_path = tempfile.NamedTemporaryFile(delete=False, suffix='.mp4').name
+                with open(temp_path, "wb") as f:
+                    f.write(action_file_bytes)
+                
+                # Extract first frame for face verification
+                cap = cv2.VideoCapture(temp_path)
+                ret, frame = cap.read()
+                cap.release()
+                os.remove(temp_path)
+                
+                if ret:
+                    # Convert frame to bytes
+                    _, buffer = cv2.imencode('.jpg', frame)
+                    action_face_bytes = buffer.tobytes()
+                else:
+                    raise HTTPException(status_code=400, detail="Could not extract frame from video")
+            else:
+                action_face_bytes = action_file_bytes
+            
+            # Verify faces
+            face_verification_result = verify_faces(ref_face_bytes, action_face_bytes)
+            
+            # If face verification fails, return early
+            if not face_verification_result["verified"]:
+                return JSONResponse({
+                    "points": 0,
+                    "task": "Face verification failed",
+                    "face_verified": False,
+                    "similarity": face_verification_result["similarity"],
+                    "error": face_verification_result["error"] or "Face does not match registered user",
+                    "raw": "Face verification failed - action not processed"
+                })
+        
+        # Process the action image/video for AI scoring
         if file.content_type.startswith("image"):
             image = Image.open(io.BytesIO(await file.read())).convert("RGB")
-
         elif file.content_type.startswith("video"):
-            temp_path = tempfile.NamedTemporaryFile(delete=False).name
+            # Create temporary file for video processing
+            temp_path = tempfile.NamedTemporaryFile(delete=False, suffix='.mp4').name
             with open(temp_path, "wb") as f:
                 f.write(await file.read())
-
+            
+            # Extract frames from video
             cap = cv2.VideoCapture(temp_path)
-            total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
-            interval = max(total // 9, 1)
-
+            total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
+            interval = max(total_frames // 9, 1)
+            
             frames = []
             for i in range(9):
                 cap.set(cv2.CAP_PROP_POS_FRAMES, i * interval)
                 ret, frame = cap.read()
                 if ret:
-                    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
-                    img = Image.fromarray(frame).resize((256, 256))
+                    frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+                    img = Image.fromarray(frame_rgb).resize((256, 256))
                     frames.append(img)
+            
             cap.release()
             os.remove(temp_path)
-
+            
+            if not frames:
+                raise HTTPException(status_code=400, detail="Could not extract frames from video")
+            
+            # Create grid of frames
             w, h = frames[0].size
             grid = Image.new("RGB", (3 * w, 3 * h))
             for idx, frame in enumerate(frames):
                 grid.paste(frame, ((idx % 3) * w, (idx // 3) * h))
             image = grid
-
         else:
             raise HTTPException(status_code=400, detail="Unsupported file type")
-
+        
+        # Convert image to base64 for AI processing
         b64_img = encode_image_to_base64(image)
+        
+        # Prepare messages for AI
         messages = [
             {"role": "system", "content": SYSTEM_PROMPT},
             {"role": "user", "content": [
                 {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64_img}"}}
             ]}
         ]
-        res = client.chat.completions.create(model=MODEL_NAME, messages=messages)
-        reply = res.choices[0].message.content
-
-        return JSONResponse({
-            "points": extract_score(reply),
-            "task": extract_activity(reply),
-            "raw": reply
-        })
-
+        
+        # Get AI response
+        response = client.chat.completions.create(
+            model=MODEL_NAME,
+            messages=messages
+        )
+        
+        ai_reply = response.choices[0].message.content
+        
+        # Extract score and activity from AI response
+        points = extract_score(ai_reply)
+        task = extract_activity(ai_reply)
+        
+        # Prepare final response
+        result = {
+            "points": points or 0,
+            "task": task,
+            "raw": ai_reply
+        }
+        
+        # Add face verification results if performed
+        if face_verification_result:
+            result.update({
+                "face_verified": face_verification_result["verified"],
+                "similarity": face_verification_result["similarity"]
+            })
+        
+        return JSONResponse(result)
+        
+    except HTTPException:
+        raise
     except Exception as e:
-        raise HTTPException(status_code=500, detail=str(e))
+        raise HTTPException(status_code=500, detail=f"Processing error: {str(e)}")
+
+if __name__ == "__main__":
+    import uvicorn
+    uvicorn.run(app, host="0.0.0.0", port=7860)

releaf_ai.py

@@ -1,7 +1,53 @@
 SYSTEM_PROMPT = """
-You are an environmental activity detection expert. Given an image or video snapshot, you must identify what eco-friendly activity is being performed (like planting a tree, cycling, cleaning a beach, etc.), and assign a score from 0 to 100 based on how impactful or clearly visible the activity is.
+You are an AI assistant for ReLeaf, an eco-friendly mobile app that gamifies environmental actions. Your job is to analyze images or videos of environmental activities and provide scoring based on their impact and authenticity.
 
-Respond strictly in this format:
-Detected Activity: <activity>
-Score: <score>
+**Your Task:**
+1. Analyze the provided image/video for environmental activities
+2. Determine if the activity is genuine and impactful
+3. Assign points based on the activity type and quality
+4. Identify the specific eco-action performed
+
+**Scoring Guidelines:**
+- **Recycling/Waste Management:** 5-15 points
+  - Proper sorting: 10-15 points
+  - General recycling: 5-10 points
+- **Tree Planting/Gardening:** 15-25 points
+  - Tree planting: 20-25 points
+  - Garden maintenance: 15-20 points
+- **Clean Energy Usage:** 20-30 points
+  - Solar panels: 25-30 points
+  - Wind energy: 20-25 points
+- **Transportation:** 5-20 points
+  - Public transport: 10-15 points
+  - Cycling/Walking: 15-20 points
+  - Electric vehicles: 5-10 points
+- **Cleanup Activities:** 10-25 points
+  - Beach/park cleanup: 20-25 points
+  - Street cleanup: 10-15 points
+- **Water Conservation:** 10-20 points
+- **Composting:** 15-20 points
+- **Sustainable Shopping:** 5-15 points
+
+**Response Format:**
+Always respond in this exact format:
+
+Detected Activity: [Brief description of the activity]
+Score: [Number between 0-30]
+Explanation: [2-3 sentences explaining why this score was given and the environmental impact]
+
+**Important Rules:**
+- Only award points for genuine environmental activities
+- If no clear eco-activity is visible, give 0 points
+- Be strict about authenticity - staged or fake activities get lower scores
+- Consider the scale and impact of the activity
+- Reward innovative or high-impact actions with bonus points
+- Maximum score is 30 points for exceptional activities
+
+**Examples:**
+- Image of someone properly sorting recyclables → "Detected Activity: Recycling plastic bottles and paper, Score: 12"
+- Video of tree planting → "Detected Activity: Planting a tree sapling, Score: 22"
+- Image of solar panels → "Detected Activity: Using solar energy, Score: 28"
+- Random selfie with no eco-activity → "Detected Activity: No environmental activity detected, Score: 0"
+
+Analyze the provided image/video and respond accordingly.
 """

requirements.txt

@@ -1,6 +1,9 @@
-fastapi
-uvicorn
-Pillow
-opencv-python
-together
-python-multipart
+fastapi==0.104.1
+uvicorn[standard]==0.24.0
+pillow==10.1.0
+opencv-python-headless==4.8.1.78
+face-recognition==1.3.0
+numpy==1.24.3
+together==0.2.7
+python-multipart==0.0.6
+dlib==19.24.2