rupinajay commited on
Commit
a176b28
Β·
1 Parent(s): 9b173c6

Revert "Update: Face Verfication added"

Browse files

This reverts commit 9b173c696b8172406d8c1c2cd07aa2ead5f15ab6.

Files changed (4) hide show
  1. README.md +10 -106
  2. app.py +25 -216
  3. releaf_ai.py +4 -50
  4. requirements.txt +6 -9
README.md CHANGED
@@ -1,108 +1,12 @@
1
- # ReLeaf AI API with Face Verification
2
-
3
- This is the backend API for the ReLeaf mobile app, providing AI-powered eco-action recognition and face verification capabilities.
4
-
5
- ## Features
6
-
7
- - **Eco-Action Recognition**: Analyze images/videos of environmental activities and assign points
8
- - **Face Verification**: Verify user identity through facial recognition
9
- - **Multi-format Support**: Process both images and videos
10
- - **Secure Authentication**: Face-based verification for action submissions
11
-
12
- ## API Endpoints
13
-
14
- ### 1. Health Check
15
- ```
16
- GET /
17
- ```
18
- Returns API status and information.
19
-
20
- ### 2. Face Verification
21
- ```
22
- POST /verify-face
23
- ```
24
- **Parameters:**
25
- - `reference_face`: Image file (stored user face)
26
- - `current_face`: Image file (captured face for verification)
27
-
28
- **Response:**
29
- ```json
30
- {
31
- "verified": true,
32
- "similarity": 85.6,
33
- "threshold": 60.0,
34
- "message": "Face verified successfully"
35
- }
36
- ```
37
-
38
- ### 3. Eco-Action Analysis
39
- ```
40
- POST /predict
41
- ```
42
- **Parameters:**
43
- - `file`: Image or video file of eco-action
44
- - `reference_face`: (Optional) Reference face image for verification
45
-
46
- **Response:**
47
- ```json
48
- {
49
- "points": 15,
50
- "task": "Recycling plastic bottles",
51
- "face_verified": true,
52
- "similarity": 87.3,
53
- "raw": "Full AI response..."
54
- }
55
- ```
56
-
57
- ## Supported Activities
58
-
59
- - ♻️ Recycling and waste management
60
- - 🌱 Tree planting and gardening
61
- - ⚑ Clean energy usage
62
- - 🚌 Sustainable transportation
63
- - 🧹 Environmental cleanup
64
- - πŸ’§ Water conservation
65
- - πŸƒ Composting
66
- - πŸ›’ Sustainable shopping
67
-
68
- ## Scoring System
69
-
70
- Activities are scored from 0-30 points based on:
71
- - **Impact Level**: Higher impact = more points
72
- - **Authenticity**: Genuine activities get full points
73
- - **Scale**: Larger scale activities get bonus points
74
- - **Innovation**: Creative eco-solutions get extra recognition
75
-
76
- ## Face Verification
77
-
78
- - **Threshold**: 60% similarity required for verification
79
- - **Security**: Prevents fraudulent submissions
80
- - **Privacy**: Face data processed in real-time, not stored
81
- - **Accuracy**: Uses state-of-the-art face recognition algorithms
82
-
83
- ## Technology Stack
84
-
85
- - **FastAPI**: High-performance web framework
86
- - **Together AI**: Advanced language model for activity recognition
87
- - **OpenCV**: Computer vision processing
88
- - **face_recognition**: Facial recognition and verification
89
- - **PIL/Pillow**: Image processing
90
-
91
- ## Environment Variables
92
-
93
- - `TOGETHER_API_KEY`: API key for Together AI service
94
-
95
- ## Local Development
96
-
97
- ```bash
98
- pip install -r requirements.txt
99
- uvicorn app:app --reload --host 0.0.0.0 --port 7860
100
- ```
101
-
102
- ## Deployment
103
-
104
- This API is designed to run on Hugging Face Spaces with automatic scaling and GPU acceleration.
105
-
106
  ---
107
 
108
- **ReLeaf** - Making sustainability fun, rewarding, and secure! 🌱
 
1
+ ---
2
+ title: Mmm
3
+ emoji: πŸ“Š
4
+ colorFrom: purple
5
+ colorTo: blue
6
+ sdk: gradio
7
+ sdk_version: 5.34.0
8
+ app_file: app.py
9
+ pinned: false
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  ---
11
 
12
+ Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
app.py CHANGED
@@ -7,272 +7,81 @@ import base64
7
  import cv2
8
  import io
9
  import re
10
- import face_recognition
11
- import numpy as np
12
  from together import Together
13
- import releaf_ai
14
 
15
  app = FastAPI()
16
 
17
- # Initialize Together client
18
  API_KEY = "1495bcdf0c72ed1e15d0e3e31e4301bd665cb28f2291bcc388164ed745a7aa24"
19
  client = Together(api_key=API_KEY)
20
  MODEL_NAME = "meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8"
 
21
  SYSTEM_PROMPT = releaf_ai.SYSTEM_PROMPT
22
 
23
  def encode_image_to_base64(image: Image.Image) -> str:
24
- """Convert PIL Image to base64 string"""
25
  buffered = io.BytesIO()
26
  image.save(buffered, format="JPEG")
27
  return base64.b64encode(buffered.getvalue()).decode("utf-8")
28
 
29
  def extract_score(text: str):
30
- """Extract score from AI response"""
31
  match = re.search(r"(?i)Score:\s*(\d+)", text)
32
  return int(match.group(1)) if match else None
33
 
34
  def extract_activity(text: str):
35
- """Extract activity from AI response"""
36
  match = re.search(r"(?i)Detected Activity:\s*(.+?)\n", text)
37
  return match.group(1).strip() if match else "Unknown"
38
 
39
- def verify_faces(reference_face_bytes: bytes, current_face_bytes: bytes) -> dict:
40
- """
41
- Verify if two face images match
42
- Returns: {"verified": bool, "similarity": float, "error": str}
43
- """
44
- try:
45
- # Convert bytes to numpy arrays
46
- ref_np = np.frombuffer(reference_face_bytes, np.uint8)
47
- curr_np = np.frombuffer(current_face_bytes, np.uint8)
48
-
49
- # Decode images
50
- ref_img = cv2.imdecode(ref_np, cv2.IMREAD_COLOR)
51
- curr_img = cv2.imdecode(curr_np, cv2.IMREAD_COLOR)
52
-
53
- if ref_img is None or curr_img is None:
54
- return {"verified": False, "similarity": 0.0, "error": "Could not decode images"}
55
-
56
- # Convert BGR to RGB (face_recognition expects RGB)
57
- ref_rgb = cv2.cvtColor(ref_img, cv2.COLOR_BGR2RGB)
58
- curr_rgb = cv2.cvtColor(curr_img, cv2.COLOR_BGR2RGB)
59
-
60
- # Get face encodings
61
- ref_encodings = face_recognition.face_encodings(ref_rgb)
62
- curr_encodings = face_recognition.face_encodings(curr_rgb)
63
-
64
- if len(ref_encodings) == 0:
65
- return {"verified": False, "similarity": 0.0, "error": "No face found in reference image"}
66
-
67
- if len(curr_encodings) == 0:
68
- return {"verified": False, "similarity": 0.0, "error": "No face found in current image"}
69
-
70
- # Use the first face found in each image
71
- ref_encoding = ref_encodings[0]
72
- curr_encoding = curr_encodings[0]
73
-
74
- # Calculate face distance (lower = more similar)
75
- face_distance = face_recognition.face_distance([ref_encoding], curr_encoding)[0]
76
-
77
- # Convert distance to similarity percentage (0-100)
78
- similarity = max(0, (1 - face_distance) * 100)
79
-
80
- # Verification threshold (adjust as needed)
81
- VERIFICATION_THRESHOLD = 60.0 # 60% similarity required
82
- verified = similarity >= VERIFICATION_THRESHOLD
83
-
84
- return {
85
- "verified": verified,
86
- "similarity": round(similarity, 2),
87
- "error": None
88
- }
89
-
90
- except Exception as e:
91
- return {"verified": False, "similarity": 0.0, "error": str(e)}
92
-
93
- @app.get("/")
94
- async def root():
95
- return {"message": "ReLeaf AI API with Face Verification", "status": "active"}
96
-
97
- @app.post("/verify-face")
98
- async def verify_face_endpoint(
99
- reference_face: UploadFile = File(...),
100
- current_face: UploadFile = File(...)
101
- ):
102
- """
103
- Standalone face verification endpoint
104
- """
105
- try:
106
- # Validate file types
107
- if not reference_face.content_type.startswith("image"):
108
- raise HTTPException(status_code=400, detail="Reference face must be an image")
109
-
110
- if not current_face.content_type.startswith("image"):
111
- raise HTTPException(status_code=400, detail="Current face must be an image")
112
-
113
- # Read file bytes
114
- ref_bytes = await reference_face.read()
115
- curr_bytes = await current_face.read()
116
-
117
- # Perform face verification
118
- result = verify_faces(ref_bytes, curr_bytes)
119
-
120
- if result["error"]:
121
- raise HTTPException(status_code=400, detail=result["error"])
122
-
123
- return JSONResponse({
124
- "verified": result["verified"],
125
- "similarity": result["similarity"],
126
- "threshold": 60.0,
127
- "message": "Face verified successfully" if result["verified"] else "Face verification failed"
128
- })
129
-
130
- except HTTPException:
131
- raise
132
- except Exception as e:
133
- raise HTTPException(status_code=500, detail=f"Face verification error: {str(e)}")
134
-
135
  @app.post("/predict")
136
- async def predict(
137
- file: UploadFile = File(...),
138
- reference_face: UploadFile = File(None)
139
- ):
140
- """
141
- Main prediction endpoint with optional face verification
142
- """
143
  try:
144
- face_verification_result = None
145
-
146
- # Perform face verification if reference face is provided
147
- if reference_face and reference_face.filename:
148
- if not reference_face.content_type.startswith("image"):
149
- raise HTTPException(status_code=400, detail="Reference face must be an image")
150
-
151
- # Extract face from the action image/video for verification
152
- action_file_bytes = await file.read()
153
- ref_face_bytes = await reference_face.read()
154
-
155
- # Reset file position for later processing
156
- await file.seek(0)
157
-
158
- # For video files, extract a frame first
159
- if file.content_type.startswith("video"):
160
- # Save video temporarily
161
- temp_path = tempfile.NamedTemporaryFile(delete=False, suffix='.mp4').name
162
- with open(temp_path, "wb") as f:
163
- f.write(action_file_bytes)
164
-
165
- # Extract first frame for face verification
166
- cap = cv2.VideoCapture(temp_path)
167
- ret, frame = cap.read()
168
- cap.release()
169
- os.remove(temp_path)
170
-
171
- if ret:
172
- # Convert frame to bytes
173
- _, buffer = cv2.imencode('.jpg', frame)
174
- action_face_bytes = buffer.tobytes()
175
- else:
176
- raise HTTPException(status_code=400, detail="Could not extract frame from video")
177
- else:
178
- action_face_bytes = action_file_bytes
179
-
180
- # Verify faces
181
- face_verification_result = verify_faces(ref_face_bytes, action_face_bytes)
182
-
183
- # If face verification fails, return early
184
- if not face_verification_result["verified"]:
185
- return JSONResponse({
186
- "points": 0,
187
- "task": "Face verification failed",
188
- "face_verified": False,
189
- "similarity": face_verification_result["similarity"],
190
- "error": face_verification_result["error"] or "Face does not match registered user",
191
- "raw": "Face verification failed - action not processed"
192
- })
193
-
194
- # Process the action image/video for AI scoring
195
  if file.content_type.startswith("image"):
196
  image = Image.open(io.BytesIO(await file.read())).convert("RGB")
 
197
  elif file.content_type.startswith("video"):
198
- # Create temporary file for video processing
199
- temp_path = tempfile.NamedTemporaryFile(delete=False, suffix='.mp4').name
200
  with open(temp_path, "wb") as f:
201
  f.write(await file.read())
202
-
203
- # Extract frames from video
204
  cap = cv2.VideoCapture(temp_path)
205
- total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
206
- interval = max(total_frames // 9, 1)
207
-
208
  frames = []
209
  for i in range(9):
210
  cap.set(cv2.CAP_PROP_POS_FRAMES, i * interval)
211
  ret, frame = cap.read()
212
  if ret:
213
- frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
214
- img = Image.fromarray(frame_rgb).resize((256, 256))
215
  frames.append(img)
216
-
217
  cap.release()
218
  os.remove(temp_path)
219
-
220
- if not frames:
221
- raise HTTPException(status_code=400, detail="Could not extract frames from video")
222
-
223
- # Create grid of frames
224
  w, h = frames[0].size
225
  grid = Image.new("RGB", (3 * w, 3 * h))
226
  for idx, frame in enumerate(frames):
227
  grid.paste(frame, ((idx % 3) * w, (idx // 3) * h))
228
  image = grid
 
229
  else:
230
  raise HTTPException(status_code=400, detail="Unsupported file type")
231
-
232
- # Convert image to base64 for AI processing
233
  b64_img = encode_image_to_base64(image)
234
-
235
- # Prepare messages for AI
236
  messages = [
237
  {"role": "system", "content": SYSTEM_PROMPT},
238
  {"role": "user", "content": [
239
  {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64_img}"}}
240
  ]}
241
  ]
242
-
243
- # Get AI response
244
- response = client.chat.completions.create(
245
- model=MODEL_NAME,
246
- messages=messages
247
- )
248
-
249
- ai_reply = response.choices[0].message.content
250
-
251
- # Extract score and activity from AI response
252
- points = extract_score(ai_reply)
253
- task = extract_activity(ai_reply)
254
-
255
- # Prepare final response
256
- result = {
257
- "points": points or 0,
258
- "task": task,
259
- "raw": ai_reply
260
- }
261
-
262
- # Add face verification results if performed
263
- if face_verification_result:
264
- result.update({
265
- "face_verified": face_verification_result["verified"],
266
- "similarity": face_verification_result["similarity"]
267
- })
268
-
269
- return JSONResponse(result)
270
-
271
- except HTTPException:
272
- raise
273
- except Exception as e:
274
- raise HTTPException(status_code=500, detail=f"Processing error: {str(e)}")
275
 
276
- if __name__ == "__main__":
277
- import uvicorn
278
- uvicorn.run(app, host="0.0.0.0", port=7860)
 
 
 
 
 
 
7
  import cv2
8
  import io
9
  import re
 
 
10
  from together import Together
11
+ import releaf_ai # this should still contain your SYSTEM_PROMPT
12
 
13
  app = FastAPI()
14
 
15
+ # Init Together client
16
  API_KEY = "1495bcdf0c72ed1e15d0e3e31e4301bd665cb28f2291bcc388164ed745a7aa24"
17
  client = Together(api_key=API_KEY)
18
  MODEL_NAME = "meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8"
19
+
20
  SYSTEM_PROMPT = releaf_ai.SYSTEM_PROMPT
21
 
22
  def encode_image_to_base64(image: Image.Image) -> str:
 
23
  buffered = io.BytesIO()
24
  image.save(buffered, format="JPEG")
25
  return base64.b64encode(buffered.getvalue()).decode("utf-8")
26
 
27
  def extract_score(text: str):
 
28
  match = re.search(r"(?i)Score:\s*(\d+)", text)
29
  return int(match.group(1)) if match else None
30
 
31
  def extract_activity(text: str):
 
32
  match = re.search(r"(?i)Detected Activity:\s*(.+?)\n", text)
33
  return match.group(1).strip() if match else "Unknown"
34
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
35
  @app.post("/predict")
36
+ async def predict(file: UploadFile = File(...)):
 
 
 
 
 
 
37
  try:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
  if file.content_type.startswith("image"):
39
  image = Image.open(io.BytesIO(await file.read())).convert("RGB")
40
+
41
  elif file.content_type.startswith("video"):
42
+ temp_path = tempfile.NamedTemporaryFile(delete=False).name
 
43
  with open(temp_path, "wb") as f:
44
  f.write(await file.read())
45
+
 
46
  cap = cv2.VideoCapture(temp_path)
47
+ total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
48
+ interval = max(total // 9, 1)
49
+
50
  frames = []
51
  for i in range(9):
52
  cap.set(cv2.CAP_PROP_POS_FRAMES, i * interval)
53
  ret, frame = cap.read()
54
  if ret:
55
+ frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
56
+ img = Image.fromarray(frame).resize((256, 256))
57
  frames.append(img)
 
58
  cap.release()
59
  os.remove(temp_path)
60
+
 
 
 
 
61
  w, h = frames[0].size
62
  grid = Image.new("RGB", (3 * w, 3 * h))
63
  for idx, frame in enumerate(frames):
64
  grid.paste(frame, ((idx % 3) * w, (idx // 3) * h))
65
  image = grid
66
+
67
  else:
68
  raise HTTPException(status_code=400, detail="Unsupported file type")
69
+
 
70
  b64_img = encode_image_to_base64(image)
 
 
71
  messages = [
72
  {"role": "system", "content": SYSTEM_PROMPT},
73
  {"role": "user", "content": [
74
  {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64_img}"}}
75
  ]}
76
  ]
77
+ res = client.chat.completions.create(model=MODEL_NAME, messages=messages)
78
+ reply = res.choices[0].message.content
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79
 
80
+ return JSONResponse({
81
+ "points": extract_score(reply),
82
+ "task": extract_activity(reply),
83
+ "raw": reply
84
+ })
85
+
86
+ except Exception as e:
87
+ raise HTTPException(status_code=500, detail=str(e))
releaf_ai.py CHANGED
@@ -1,53 +1,7 @@
1
  SYSTEM_PROMPT = """
2
- You are an AI assistant for ReLeaf, an eco-friendly mobile app that gamifies environmental actions. Your job is to analyze images or videos of environmental activities and provide scoring based on their impact and authenticity.
3
 
4
- **Your Task:**
5
- 1. Analyze the provided image/video for environmental activities
6
- 2. Determine if the activity is genuine and impactful
7
- 3. Assign points based on the activity type and quality
8
- 4. Identify the specific eco-action performed
9
-
10
- **Scoring Guidelines:**
11
- - **Recycling/Waste Management:** 5-15 points
12
- - Proper sorting: 10-15 points
13
- - General recycling: 5-10 points
14
- - **Tree Planting/Gardening:** 15-25 points
15
- - Tree planting: 20-25 points
16
- - Garden maintenance: 15-20 points
17
- - **Clean Energy Usage:** 20-30 points
18
- - Solar panels: 25-30 points
19
- - Wind energy: 20-25 points
20
- - **Transportation:** 5-20 points
21
- - Public transport: 10-15 points
22
- - Cycling/Walking: 15-20 points
23
- - Electric vehicles: 5-10 points
24
- - **Cleanup Activities:** 10-25 points
25
- - Beach/park cleanup: 20-25 points
26
- - Street cleanup: 10-15 points
27
- - **Water Conservation:** 10-20 points
28
- - **Composting:** 15-20 points
29
- - **Sustainable Shopping:** 5-15 points
30
-
31
- **Response Format:**
32
- Always respond in this exact format:
33
-
34
- Detected Activity: [Brief description of the activity]
35
- Score: [Number between 0-30]
36
- Explanation: [2-3 sentences explaining why this score was given and the environmental impact]
37
-
38
- **Important Rules:**
39
- - Only award points for genuine environmental activities
40
- - If no clear eco-activity is visible, give 0 points
41
- - Be strict about authenticity - staged or fake activities get lower scores
42
- - Consider the scale and impact of the activity
43
- - Reward innovative or high-impact actions with bonus points
44
- - Maximum score is 30 points for exceptional activities
45
-
46
- **Examples:**
47
- - Image of someone properly sorting recyclables β†’ "Detected Activity: Recycling plastic bottles and paper, Score: 12"
48
- - Video of tree planting β†’ "Detected Activity: Planting a tree sapling, Score: 22"
49
- - Image of solar panels β†’ "Detected Activity: Using solar energy, Score: 28"
50
- - Random selfie with no eco-activity β†’ "Detected Activity: No environmental activity detected, Score: 0"
51
-
52
- Analyze the provided image/video and respond accordingly.
53
  """
 
1
  SYSTEM_PROMPT = """
2
+ You are an environmental activity detection expert. Given an image or video snapshot, you must identify what eco-friendly activity is being performed (like planting a tree, cycling, cleaning a beach, etc.), and assign a score from 0 to 100 based on how impactful or clearly visible the activity is.
3
 
4
+ Respond strictly in this format:
5
+ Detected Activity: <activity>
6
+ Score: <score>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  """
requirements.txt CHANGED
@@ -1,9 +1,6 @@
1
- fastapi==0.104.1
2
- uvicorn[standard]==0.24.0
3
- pillow==10.1.0
4
- opencv-python-headless==4.8.1.78
5
- face-recognition==1.3.0
6
- numpy==1.24.3
7
- together==0.2.7
8
- python-multipart==0.0.6
9
- dlib==19.24.2
 
1
+ fastapi
2
+ uvicorn
3
+ Pillow
4
+ opencv-python
5
+ together
6
+ python-multipart