Complete project reorganization: Remove moved files from root directory

MCP_EVALUATION_REPORT.textile

@@ -1,107 +0,0 @@
-h1. MCP Capability Evaluation Report for Small Language Models (SLMs)
-
-h2. Executive Summary
-
-This report presents a comprehensive evaluation of 14 Small Language Models (SLMs) under 3B parameters for their MCP (Model Context Protocol) capabilities. The evaluation focuses on assessing how effectively these models can convert natural language RTS game commands into structured JSON tool calls.
-
-*NEW*: This report has been updated with two additional code-specialized models (Qwen2.5-Coder-1.5B and Yi-Coder-1.5B), revealing a new champion with exceptional MCP performance.
-
-h2. What is MCP?
-
-MCP (Model Context Protocol) is a standardized protocol that enables AI models to interact with external tools and systems through structured JSON calls. In the context of RTS games, MCP allows:
-
-* Conversion of natural language commands into executable actions
-* Structured communication between AI and game engines
-* Standardized tool calling interface
-* Real-time command processing
-
-h2. Evaluation Methodology
-
-h3. Test Scenarios
-
-We evaluated each model on three realistic RTS game scenarios:
-
-# *State Command*: "show game state" → Expected: @get_game_state@
-# *Movement Command*: "move infantry to 150,200" → Expected: @move_units@ with coordinates
-# *Attack Command*: "attack enemy tank at 300,150" → Expected: @attack_unit@ with target coordinates
-
-h3. Scoring System (0-10 points per test)
-
-* +4 points: Correct tool identification
-* +3 points: Valid JSON structure
-* +2 points: Proper tool/action terminology
-* +1 point: Correct coordinate extraction
-
-h3. Models Evaluated
-
-The evaluation includes 14 models across different categories:
-
-* *General-purpose SLMs* (3 models)
-* *MCP-specialized models* (7 models with various quantizations)
-* *Code-specialized models* (3 models)
-* *Code-specialized failing* (1 model)
-
-h2. Test Results
-
-The comprehensive evaluation revealed significant differences in MCP capabilities across models:
-
-h3. Performance Ranking Table
-
-|_. Rank|_. Model|_. MCP Score|_. Avg Time|_. Size|_. Efficiency|_. Notes|
-| *1* | *@Qwen2.5-Coder-1.5B-Q4@* | *9.7/10* | *4.12s* | *1017MB* | *2.34 pts/s* | 🏆 *Champion* |
-| 2 | @Qwen2.5-Coder-0.5B@ | 4.3/10 | 2.08s | 409MB | 2.08 pts/s | Previous champion |
-| 3 | @Qwen3-0.6B@ | 3.7/10 | 3.98s | 610MB | 0.92 pts/s | |
-| 4 | @Gemma-3-270M@ | 3.7/10 | 2.29s | 428MB | 1.60 pts/s | |
-| 5 | @MCPR-L-3B-Exa-Q8@ | 3.7/10 | 17.42s | 3133MB | 0.21 pts/s | |
-| 6 | @Gemma-3n-E2B-it-Q8@ | 3.7/10 | 14.80s | 4566MB | 0.25 pts/s | |
-| 7 | @Qwen3-1.7B@ | 3.7/10 | 6.24s | 1008MB | 0.59 pts/s | |
-| 8 | @Qwen2.5-0.5B@ | 2.7/10 | 1.17s | 409MB | 2.28 pts/s | |
-| 9 | @Gemma-3n-E2B-it-IQ2@ | 2.3/10 | 14.11s | 1958MB | 0.17 pts/s | |
-| 10 | @Llama-Breeze2-3B-Q2@ | 1.3/10 | 11.39s | 1424MB | 0.12 pts/s | |
-| 11 | @Yi-Coder-1.5B-Q4@ | 0.0/10 | 11.64s | 826MB | 0.00 pts/s | Prompt format issue |
-| 12 | @MCP-Instruct-v1-Q4@ | 0.0/10 | 0.00s | 697MB | 0.00 pts/s | |
-| 13 | @MCPR-L-3B-Exa-Q2@ | 0.0/10 | 10.63s | 1216MB | 0.00 pts/s | |
-| 14 | @MCP-Instruct-v1-Q8@ | 0.0/10 | 0.00s | 1465MB | 0.00 pts/s | |
-
-h2. Key Findings
-
-h3. Performance Insights
-
-* *Code-specialized models dramatically outperform others*: Qwen2.5-Coder-1.5B achieved an exceptional 9.7/10 score, more than 2x better than any other model
-* *Scaling works for code-specialized models*: Increasing from 0.5B to 1.5B parameters improved the score from 4.3/10 to 9.7/10
-* *Near-perfect MCP capability exists in small models*: The 1.5B model achieved 10/10 on 2 out of 3 tests with proper JSON extraction
-* *Smaller models can be more efficient*: The 270M parameter Gemma model performed as well as much larger 3B models
-* *Quantization matters*: Q8 versions generally performed better than Q2/Q4 versions for MCP-specialized models
-
-h3. Technical Observations
-
-* *Markdown wrapping requires extraction*: Qwen2.5-Coder-1.5B wraps JSON in markdown code blocks (@```json```@), requiring extraction logic
-* *MCP-Instruct models failed completely* due to technical issues (@llama_decode returned -1@)
-* *Yi-Coder has prompt format incompatibility*: Returns the prompt itself rather than generating responses
-* *Larger models don't guarantee better performance*: The 3B models were significantly slower with similar scores (except code-specialized)
-* *Response time varies dramatically*: From 1.17s (Qwen2.5-0.5B) to 17.42s (MCPR-L-3B-Exa-Q8)
-
-h2. Recommendations
-
-Based on the updated evaluation results, we recommend:
-
-# *Primary Choice*: @Qwen2.5-Coder-1.5B-Q4@ - *Exceptional MCP performance* (9.7/10) with reasonable speed (4.12s) and size (1017MB)
-# *Budget Alternative*: @Qwen2.5-Coder-0.5B@ - Best balance for resource-constrained environments (4.3/10, 2.08s, 409MB)
-# *Ultra-lightweight*: @Gemma-3-270M@ - Excellent efficiency for its tiny size (3.7/10, 2.29s, 428MB)
-# *Avoid*: MCP-Instruct models (technical incompatibility), Yi-Coder (prompt format issues)
-
-h2. Conclusion
-
-This comprehensive 14-model evaluation demonstrates critical insights for MCP capabilities in RTS games:
-
-* *Code-specialized models are vastly superior*: The champion (Qwen2.5-Coder-1.5B) achieved 9.7/10, while the best MCP-specialized model only reached 3.7/10
-* *Parameter scaling works for code models*: Tripling parameters (0.5B → 1.5B) more than doubled MCP performance (4.3 → 9.7)
-* *Near-perfect MCP is achievable*: Small models under 2B parameters can achieve 10/10 on individual tests with proper implementation
-* *JSON extraction is critical*: Modern code models wrap output in markdown, requiring extraction logic for production use
-* *Efficiency varies dramatically*: The best model is 11.7x more effective than the worst functional model
-
-The results provide valuable insights for developers implementing MCP-based AI assistants in gaming applications, demonstrating that code-specialized models offer the most reliable path to high-quality MCP capabilities.
-
-*Report generated on: 2025-10-05*
-*Updated on: 2025-10-05* (added Qwen2.5-Coder-1.5B and Yi-Coder-1.5B)
-*Evaluation framework: llama.cpp with MCP protocol simulation*

comparative_model_evaluation.py

@@ -1,366 +0,0 @@
-"""
-Évaluation comparative des capacités MCP de plusieurs modèles LLM
-Compare Qwen2.5-0.5B, Qwen3-0.6B, et Gemma-3-1B pour les tâches MCP
-"""
-
-import sys
-import os
-import json
-import time
-
-# Ajouter le chemin pour les imports
-sys.path.append(os.path.dirname(os.path.abspath(__file__)))
-
-def initialize_model(model_path, model_name):
-    """Initialiser un modèle spécifique"""
-    try:
-        from llama_cpp import Llama
-        
-        print(f"🔄 Initialisation de {model_name}...")
-        
-        llm = Llama(
-            model_path=model_path,
-            n_ctx=2048,
-            n_threads=2,
-            verbose=False
-        )
-        
-        print(f"✅ {model_name} initialisé avec succès")
-        return llm
-        
-    except Exception as e:
-        print(f"❌ Erreur d'initialisation de {model_name}: {e}")
-        return None
-
-def run_mcp_test(llm, model_name, test):
-    """Exécuter un test MCP sur un modèle spécifique"""
-    
-    try:
-        start_time = time.time()
-        
-        # Utiliser le chat template
-        messages = [
-            {"role": "system", "content": "Tu es un assistant IA spécialisé dans les jeux RTS et le protocole MCP. Réponds avec des objets JSON valides."},
-            {"role": "user", "content": test['prompt']}
-        ]
-        
-        response = llm.create_chat_completion(
-            messages=messages,
-            max_tokens=300,
-            temperature=0.1
-        )
-        
-        response_time = time.time() - start_time
-        
-        # Extraire le texte de la réponse
-        if response and 'choices' in response and len(response['choices']) > 0:
-            response_text = response['choices'][0]['message']['content']
-        else:
-            # Fallback
-            simple_response = llm(test['prompt'], max_tokens=300, temperature=0.1)
-            response_text = simple_response['choices'][0]['text'] if 'choices' in simple_response else str(simple_response)
-        
-        # Analyser la réponse
-        score = analyze_mcp_response(response_text, test)
-        
-        return {
-            'model': model_name,
-            'test': test['name'],
-            'response': response_text,
-            'response_time': response_time,
-            'score': score,
-            'success': score >= 5
-        }
-        
-    except Exception as e:
-        return {
-            'model': model_name,
-            'test': test['name'],
-            'error': str(e),
-            'score': 0,
-            'success': False
-        }
-
-def analyze_mcp_response(response, test_config):
-    """Analyser la réponse MCP et la noter"""
-    
-    if not response or response.strip() == "":
-        return 0
-    
-    score = 0
-    
-    # Vérifier la structure JSON
-    try:
-        json_data = json.loads(response)
-        score += 3  # JSON valide
-        
-        # Vérifier la présence d'outils MCP
-        if 'tool' in json_data:
-            score += 2
-        if 'args' in json_data:
-            score += 1
-            
-    except json.JSONDecodeError:
-        # Vérifier les patterns dans le texte
-        mcp_tools = ["get_game_state", "move_units", "attack_unit", "build_building"]
-        tools_found = [tool for tool in mcp_tools if tool in response]
-        
-        if tools_found:
-            score += 2
-        if any(param in response for param in ['unit_ids', 'target_x', 'target_y']):
-            score += 1
-    
-    # Vérifier la cohérence sémantique
-    if 'game' in response.lower() or 'rts' in response.lower():
-        score += 1
-    
-    # Vérifier la pertinence par rapport au test
-    if 'expected_tool' in test_config:
-        if test_config['expected_tool'] in response:
-            score += 2
-    elif test_config.get('expected_complex'):
-        if any(word in response.lower() for word in ['sequence', 'steps', 'build', 'defend']):
-            score += 2
-    elif test_config.get('expected_clarification'):
-        if any(word in response.lower() for word in ['clarify', 'coordinates', 'which', 'ambiguous']):
-            score += 2
-    
-    return min(score, 10)
-
-def run_comparative_evaluation():
-    """Exécuter l'évaluation comparative des trois modèles"""
-    
-    print("🔍 ÉVALUATION COMPARATIVE DES CAPACITÉS MCP")
-    print("=" * 70)
-    
-    # Définir les modèles à tester
-    models = [
-        {
-            'name': 'Qwen2.5-0.5B',
-            'path': 'qwen2.5-0.5b-instruct-q4_0.gguf'
-        },
-        {
-            'name': 'Qwen3-0.6B',
-            'path': 'Qwen3-0.6B-Q8_0.gguf'
-        },
-        {
-            'name': 'Gemma-3-1B',
-            'path': 'google_gemma-3-1b-it-qat-Q4_0.gguf'
-        }
-    ]
-    
-    # Tests MCP
-    tests = [
-        {
-            "name": "Compréhension MCP de base",
-            "prompt": """Tu es un assistant IA qui contrôle un jeu RTS via MCP (Model Context Protocol).
-
-Outils MCP disponibles:
-- get_game_state(): Obtenir l'état actuel du jeu
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Montre-moi l'état du jeu"
-
-Réponds avec un objet JSON contenant l'appel d'outil MCP à exécuter.
-""",
-            "expected_tool": "get_game_state"
-        },
-        {
-            "name": "Traduction d'action simple",
-            "prompt": """Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace mon infanterie vers la position 150, 200"
-
-Réponds avec un objet JSON contenant l'appel d'outil avec les paramètres extraits.
-""",
-            "expected_tool": "move_units"
-        },
-        {
-            "name": "Extraction de paramètres complexes",
-            "prompt": """Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace les tanks 1, 3 et 7 vers les coordonnées 120, 80"
-
-Extrais les paramètres et réponds avec un objet JSON d'appel d'outil.
-""",
-            "expected_tool": "move_units"
-        },
-        {
-            "name": "Planification stratégique",
-            "prompt": """Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- get_game_state(): Obtenir l'état du jeu
-- move_units(): Déplacer des unités
-- attack_unit(): Attaquer l'ennemi
-- build_building(): Construire un bâtiment
-
-Commande utilisateur: "Construis une base près du minerai et défends-la"
-
-Décompose cette stratégie en une séquence d'actions MCP.
-""",
-            "expected_complex": True
-        },
-        {
-            "name": "Gestion d'ambiguïté",
-            "prompt": """Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace mes unités vers la base ennemie"
-
-Comment gérerais-tu cette instruction ambiguë?
-""",
-            "expected_clarification": True
-        }
-    ]
-    
-    # Initialiser les modèles
-    initialized_models = {}
-    
-    for model in models:
-        if os.path.exists(model['path']):
-            llm = initialize_model(model['path'], model['name'])
-            if llm:
-                initialized_models[model['name']] = llm
-        else:
-            print(f"❌ Fichier modèle non trouvé: {model['path']}")
-    
-    if not initialized_models:
-        print("❌ Aucun modèle n'a pu être initialisé")
-        return
-    
-    print(f"\n🚀 {len(initialized_models)} modèle(s) initialisé(s) sur {len(models)}")
-    
-    # Exécuter les tests sur chaque modèle
-    all_results = []
-    
-    for model_name, llm in initialized_models.items():
-        print(f"\n🧪 TESTS AVEC {model_name}")
-        print("=" * 50)
-        
-        model_results = []
-        
-        for test in tests:
-            print(f"\n📋 Test: {test['name']}")
-            
-            result = run_mcp_test(llm, model_name, test)
-            model_results.append(result)
-            
-            if result.get('success', False):
-                print(f"✅ Score: {result['score']}/10 | Temps: {result['response_time']:.2f}s")
-            else:
-                print(f"❌ Erreur: {result.get('error', 'Unknown error')}")
-            
-            all_results.append(result)
-    
-    # Générer le rapport comparatif
-    generate_comparative_report(initialized_models.keys(), all_results, tests)
-
-def generate_comparative_report(model_names, results, tests):
-    """Générer un rapport comparatif complet"""
-    
-    print("\n" + "=" * 70)
-    print("📊 RAPPORT COMPARATIF DES MODÈLES")
-    print("=" * 70)
-    
-    # Organiser les résultats par modèle
-    model_results = {name: [] for name in model_names}
-    
-    for result in results:
-        if 'model' in result:
-            model_results[result['model']].append(result)
-    
-    # Statistiques par modèle
-    print(f"\n📈 STATISTIQUES PAR MODÈLE:")
-    
-    model_scores = {}
-    
-    for model_name in model_names:
-        results_list = model_results[model_name]
-        
-        if results_list:
-            successful_tests = [r for r in results_list if r.get('success', False)]
-            total_score = sum(r.get('score', 0) for r in results_list)
-            avg_score = total_score / len(results_list)
-            avg_time = sum(r.get('response_time', 0) for r in results_list) / len(results_list)
-            
-            model_scores[model_name] = {
-                'avg_score': avg_score,
-                'success_rate': len(successful_tests) / len(results_list),
-                'avg_time': avg_time
-            }
-            
-            print(f"\n🔹 {model_name}:")
-            print(f"   Score moyen: {avg_score:.1f}/10")
-            print(f"   Taux de réussite: {len(successful_tests)}/{len(results_list)} ({len(successful_tests)/len(results_list)*100:.0f}%)")
-            print(f"   Temps moyen: {avg_time:.2f}s")
-    
-    # Comparaison directe
-    print(f"\n🏆 CLASSEMENT:")
-    
-    sorted_models = sorted(model_scores.items(), key=lambda x: x[1]['avg_score'], reverse=True)
-    
-    for i, (model_name, scores) in enumerate(sorted_models, 1):
-        print(f"   {i}. {model_name}: {scores['avg_score']:.1f}/10")
-    
-    # Analyse par type de test
-    print(f"\n🧪 ANALYSE PAR TYPE DE TEST:")
-    
-    for test in tests:
-        test_name = test['name']
-        print(f"\n🔸 {test_name}:")
-        
-        test_results = [r for r in results if r.get('test') == test_name]
-        
-        for model_name in model_names:
-            model_test_results = [r for r in test_results if r.get('model') == model_name]
-            
-            if model_test_results:
-                avg_score = sum(r.get('score', 0) for r in model_test_results) / len(model_test_results)
-                print(f"   {model_name}: {avg_score:.1f}/10")
-    
-    # Recommandations
-    print(f"\n💡 RECOMMANDATIONS:")
-    
-    best_model = sorted_models[0][0]
-    best_score = sorted_models[0][1]['avg_score']
-    
-    if best_score >= 7:
-        print(f"✅ {best_model} est EXCELLENT pour les tâches MCP")
-        print(f"   Utilisation recommandée pour la production")
-    elif best_score >= 5:
-        print(f"👍 {best_model} est BON pour les tâches MCP")
-        print(f"   Utilisation recommandée avec validation")
-    else:
-        print(f"⚠️  {best_model} est LIMITÉ pour les tâches MCP")
-        print(f"   Amélioration nécessaire avant utilisation")
-    
-    # Performance vs taille
-    print(f"\n⚖️  PERFORMANCE VS TAILLE:")
-    
-    for model_name, scores in model_scores.items():
-        efficiency = scores['avg_score'] / scores['avg_time'] if scores['avg_time'] > 0 else 0
-        print(f"   {model_name}: {efficiency:.2f} score/seconde")
-    
-    # Sauvegarder les résultats
-    comparative_results = {
-        'model_scores': model_scores,
-        'detailed_results': results,
-        'ranking': sorted_models
-    }
-    
-    with open("comparative_mcp_evaluation.json", "w", encoding="utf-8") as f:
-        json.dump(comparative_results, f, indent=2, ensure_ascii=False)
-    
-    print(f"\n📄 Résultats détaillés sauvegardés dans: comparative_mcp_evaluation.json")
-
-if __name__ == "__main__":
-    run_comparative_evaluation()

comprehensive_mcp_evaluation.json

@@ -1,956 +0,0 @@
-{
-  "evaluation_type": "comprehensive_mcp_test",
-  "total_models_tested": 9,
-  "successful_models": 5,
-  "results": [
-    {
-      "name": "Qwen2.5-0.5B",
-      "file_size_mb": 408.8689880371094,
-      "avg_score": 2.6,
-      "avg_time": 2.6360722541809083,
-      "efficiency": 0.9863159084036122,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 0.31192469596862793,
-          "response": ""
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 0.18253064155578613,
-          "response": ""
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 7,
-          "time": 4.232211351394653,
-          "response": "Where tool_name is the name of the tool used and args is a dictionary containing the arguments for t..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 2,
-          "time": 4.225749492645264,
-          "response": "Where tool_name is the name of the tool and args is a dictionary with the arguments. If no arguments..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 4,
-          "time": 4.22794508934021,
-          "response": "where tool_name is the name of the tool and args is a dictionary containing the arguments. If no too..."
-        }
-      ],
-      "type": "general"
-    },
-    {
-      "name": "Qwen3-0.6B",
-      "file_size_mb": 609.8238830566406,
-      "avg_score": 2.8,
-      "avg_time": 8.223706769943238,
-      "efficiency": 0.3404790659892809,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 7,
-          "time": 8.638539791107178,
-          "response": ".\n\nIf the user command is not supported by the available tools, respond with an empty array.\n\nNow, t..."
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 5,
-          "time": 8.075484991073608,
-          "response": ".\n\nMake sure to use the correct tool name and format the JSON correctly.\n\nIf the command is not poss..."
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 0,
-          "time": 7.951770067214966,
-          "response": ".\n\nMake sure to use the correct tool names and format the JSON correctly.\n\nNow, the game state is as..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 2,
-          "time": 8.252855062484741,
-          "response": ".\n\nMake sure to use the correct tool names and format the JSON properly.\n\nNow, the user is in a game..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 0,
-          "time": 8.199883937835693,
-          "response": ".\n\nMake sure to use the correct tool name and format the JSON.\n\nIf the command is not possible, retu..."
-        }
-      ],
-      "type": "general"
-    },
-    {
-      "name": "Gemma-3-270M",
-      "file_size_mb": 428.0401306152344,
-      "avg_score": 0.0,
-      "avg_time": 0.16690435409545898,
-      "efficiency": 0.0,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 0.2941462993621826,
-          "response": ""
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 0.13967180252075195,
-          "response": ""
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 0,
-          "time": 0.1264328956604004,
-          "response": ""
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 0,
-          "time": 0.14153170585632324,
-          "response": ""
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 0,
-          "time": 0.13273906707763672,
-          "response": ""
-        }
-      ],
-      "type": "general"
-    },
-    {
-      "name": "Qwen3-1.7B",
-      "file_size_mb": 1007.8267211914062,
-      "avg_score": 3.0,
-      "avg_time": 13.003729963302613,
-      "efficiency": 0.23070303739513193,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 2,
-          "time": 12.862720251083374,
-          "response": ".\n\nYou must use the JSON format specified, without any additional text or explanation. The JSON must..."
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 2,
-          "time": 12.972241401672363,
-          "response": ".\n\nYou must use the correct tool name and format the JSON properly.\n\nThe game state is as follows:\n-..."
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 3,
-          "time": 13.497555255889893,
-          "response": ".\n\nYou must use the correct tool name and format the JSON properly.\n\nThe tool to use is attack_unit...."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 12.513315677642822,
-          "response": ".\n\nYou must use the correct tool name and format the JSON properly.\n\nThe correct tool name is \"build..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 3,
-          "time": 13.17281723022461,
-          "response": ".\n\nYou can use the following tool definitions:\n- move_units: move units to a new position\n- attack_u..."
-        }
-      ],
-      "type": "general"
-    },
-    {
-      "name": "MCP-Instruct-v1",
-      "file_size_mb": 697.0347290039062,
-      "avg_score": 0.0,
-      "avg_time": 0.1320805072784424,
-      "efficiency": 0.0,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 0.6604025363922119,
-          "response": ""
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 0,
-          "error": "llama_decode returned -1"
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 0,
-          "time": 0,
-          "error": "llama_decode returned -1"
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 0,
-          "time": 0,
-          "error": "llama_decode returned -1"
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 0,
-          "time": 0,
-          "error": "llama_decode returned -1"
-        }
-      ],
-      "type": "mcp_specialized"
-    },
-    {
-      "name": "MCPR L-3B-Exa",
-      "file_size_mb": 1215.7023620605469,
-      "avg_score": 0.0,
-      "avg_time": 22.14646472930908,
-      "efficiency": 0.0,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 22.817347049713135,
-          "response": "+\\),),“), and“““““““““““““““““““““““““”“”““““““““““”“““““““““““““““““““““““““““““““““““““““initializ..."
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 21.51675510406494,
-          "response": "+\\),),“),3“”“”“”“),),““““““““““”“),),),), and“),),), and@@ the Sty□ Sty□ Sty□ Sty□ Sty□ Sty□ Sty□ St..."
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 0,
-          "time": 22.118958473205566,
-          "response": "+\\),),+\\),), and““”““““““““““““““““““““”“““““”“”“““““““““““““““““““““”“”““”““”““““““““““““““““““““““..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 0,
-          "time": 22.297714471817017,
-          "response": "+\\),),“), and@@ the Sty mini mini mini mini mini mini mini mini mini the““““”“),),+\\),),), and“),),)..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 0,
-          "time": 21.98154854774475,
-          "response": "and@@ Sty@@patterns@@ mini@@ Sty@@ Sty mini mini mini mini mini mini mini mini mini the“““““““““”“““..."
-        }
-      ],
-      "type": "mcp_specialized"
-    },
-    {
-      "name": "Gemma-3n-E2B-it",
-      "file_size_mb": 1958.3001403808594,
-      "avg_score": 0.0,
-      "avg_time": 1.5714858055114747,
-      "efficiency": 0.0,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 3.1773452758789062,
-          "response": ""
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 1.1669323444366455,
-          "response": ""
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 0,
-          "time": 1.1747264862060547,
-          "response": ""
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 0,
-          "time": 1.2873260974884033,
-          "response": ""
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 0,
-          "time": 1.0510988235473633,
-          "response": ""
-        }
-      ],
-      "type": "mcp_specialized"
-    },
-    {
-      "name": "Llama-Breeze2-3B",
-      "file_size_mb": 1424.04345703125,
-      "avg_score": 3.6,
-      "avg_time": 14.693956804275512,
-      "efficiency": 0.24499867856917243,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 3,
-          "time": 3.5608396530151367,
-          "response": "[get_game_state()]"
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 3,
-          "time": 5.626140356063843,
-          "response": "[move_units(unit_ids='infantry', target_x='150', target_y='200')]"
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 23.91610813140869,
-          "response": "The tool used is \"get_game_state\" and the tool name is \"get_game_state\". The args for this tool is a..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 23.745216846466064,
-          "response": "{\n  \"tool\": \"build_building\",\n  \"args\": {\n    \"building_type\": \"power plant\",\n    \"position_x\": 100,..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 2,
-          "time": 16.621479034423828,
-          "response": "The game is ready for action. You are the AI, and you are ready to assist. You have the ability to m..."
-        }
-      ],
-      "type": "general"
-    },
-    {
-      "name": "Qwen2.5-Coder-0.5B",
-      "file_size_mb": 408.8690185546875,
-      "avg_score": 4.4,
-      "avg_time": 4.1166441440582275,
-      "efficiency": 1.0688317585941343,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 5,
-          "time": 4.25421667098999,
-          "response": ".\n\nHere's a possible response:\n\n{\"tool\": \"get_game_state\", \"args\": {\"game_state\": {\"units\": [{\"id\": ..."
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 5,
-          "time": 4.333646059036255,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 4.139528274536133,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 2,
-          "time": 3.9508562088012695,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a JSON object containing the arguments ..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 5,
-          "time": 3.9049735069274902,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-        }
-      ],
-      "type": "code_specialized"
-    }
-  ],
-  "ranking_by_score": [
-    {
-      "name": "Qwen2.5-Coder-0.5B",
-      "file_size_mb": 408.8690185546875,
-      "avg_score": 4.4,
-      "avg_time": 4.1166441440582275,
-      "efficiency": 1.0688317585941343,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 5,
-          "time": 4.25421667098999,
-          "response": ".\n\nHere's a possible response:\n\n{\"tool\": \"get_game_state\", \"args\": {\"game_state\": {\"units\": [{\"id\": ..."
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 5,
-          "time": 4.333646059036255,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 4.139528274536133,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 2,
-          "time": 3.9508562088012695,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a JSON object containing the arguments ..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 5,
-          "time": 3.9049735069274902,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-        }
-      ],
-      "type": "code_specialized"
-    },
-    {
-      "name": "Llama-Breeze2-3B",
-      "file_size_mb": 1424.04345703125,
-      "avg_score": 3.6,
-      "avg_time": 14.693956804275512,
-      "efficiency": 0.24499867856917243,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 3,
-          "time": 3.5608396530151367,
-          "response": "[get_game_state()]"
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 3,
-          "time": 5.626140356063843,
-          "response": "[move_units(unit_ids='infantry', target_x='150', target_y='200')]"
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 23.91610813140869,
-          "response": "The tool used is \"get_game_state\" and the tool name is \"get_game_state\". The args for this tool is a..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 23.745216846466064,
-          "response": "{\n  \"tool\": \"build_building\",\n  \"args\": {\n    \"building_type\": \"power plant\",\n    \"position_x\": 100,..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 2,
-          "time": 16.621479034423828,
-          "response": "The game is ready for action. You are the AI, and you are ready to assist. You have the ability to m..."
-        }
-      ],
-      "type": "general"
-    },
-    {
-      "name": "Qwen3-1.7B",
-      "file_size_mb": 1007.8267211914062,
-      "avg_score": 3.0,
-      "avg_time": 13.003729963302613,
-      "efficiency": 0.23070303739513193,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 2,
-          "time": 12.862720251083374,
-          "response": ".\n\nYou must use the JSON format specified, without any additional text or explanation. The JSON must..."
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 2,
-          "time": 12.972241401672363,
-          "response": ".\n\nYou must use the correct tool name and format the JSON properly.\n\nThe game state is as follows:\n-..."
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 3,
-          "time": 13.497555255889893,
-          "response": ".\n\nYou must use the correct tool name and format the JSON properly.\n\nThe tool to use is attack_unit...."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 12.513315677642822,
-          "response": ".\n\nYou must use the correct tool name and format the JSON properly.\n\nThe correct tool name is \"build..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 3,
-          "time": 13.17281723022461,
-          "response": ".\n\nYou can use the following tool definitions:\n- move_units: move units to a new position\n- attack_u..."
-        }
-      ],
-      "type": "general"
-    },
-    {
-      "name": "Qwen3-0.6B",
-      "file_size_mb": 609.8238830566406,
-      "avg_score": 2.8,
-      "avg_time": 8.223706769943238,
-      "efficiency": 0.3404790659892809,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 7,
-          "time": 8.638539791107178,
-          "response": ".\n\nIf the user command is not supported by the available tools, respond with an empty array.\n\nNow, t..."
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 5,
-          "time": 8.075484991073608,
-          "response": ".\n\nMake sure to use the correct tool name and format the JSON correctly.\n\nIf the command is not poss..."
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 0,
-          "time": 7.951770067214966,
-          "response": ".\n\nMake sure to use the correct tool names and format the JSON correctly.\n\nNow, the game state is as..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 2,
-          "time": 8.252855062484741,
-          "response": ".\n\nMake sure to use the correct tool names and format the JSON properly.\n\nNow, the user is in a game..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 0,
-          "time": 8.199883937835693,
-          "response": ".\n\nMake sure to use the correct tool name and format the JSON.\n\nIf the command is not possible, retu..."
-        }
-      ],
-      "type": "general"
-    },
-    {
-      "name": "Qwen2.5-0.5B",
-      "file_size_mb": 408.8689880371094,
-      "avg_score": 2.6,
-      "avg_time": 2.6360722541809083,
-      "efficiency": 0.9863159084036122,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 0.31192469596862793,
-          "response": ""
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 0.18253064155578613,
-          "response": ""
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 7,
-          "time": 4.232211351394653,
-          "response": "Where tool_name is the name of the tool used and args is a dictionary containing the arguments for t..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 2,
-          "time": 4.225749492645264,
-          "response": "Where tool_name is the name of the tool and args is a dictionary with the arguments. If no arguments..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 4,
-          "time": 4.22794508934021,
-          "response": "where tool_name is the name of the tool and args is a dictionary containing the arguments. If no too..."
-        }
-      ],
-      "type": "general"
-    }
-  ],
-  "ranking_by_efficiency": [
-    {
-      "name": "Qwen2.5-Coder-0.5B",
-      "file_size_mb": 408.8690185546875,
-      "avg_score": 4.4,
-      "avg_time": 4.1166441440582275,
-      "efficiency": 1.0688317585941343,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 5,
-          "time": 4.25421667098999,
-          "response": ".\n\nHere's a possible response:\n\n{\"tool\": \"get_game_state\", \"args\": {\"game_state\": {\"units\": [{\"id\": ..."
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 5,
-          "time": 4.333646059036255,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 4.139528274536133,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 2,
-          "time": 3.9508562088012695,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a JSON object containing the arguments ..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 5,
-          "time": 3.9049735069274902,
-          "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-        }
-      ],
-      "type": "code_specialized"
-    },
-    {
-      "name": "Qwen2.5-0.5B",
-      "file_size_mb": 408.8689880371094,
-      "avg_score": 2.6,
-      "avg_time": 2.6360722541809083,
-      "efficiency": 0.9863159084036122,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 0.31192469596862793,
-          "response": ""
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 0,
-          "time": 0.18253064155578613,
-          "response": ""
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 7,
-          "time": 4.232211351394653,
-          "response": "Where tool_name is the name of the tool used and args is a dictionary containing the arguments for t..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 2,
-          "time": 4.225749492645264,
-          "response": "Where tool_name is the name of the tool and args is a dictionary with the arguments. If no arguments..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 4,
-          "time": 4.22794508934021,
-          "response": "where tool_name is the name of the tool and args is a dictionary containing the arguments. If no too..."
-        }
-      ],
-      "type": "general"
-    },
-    {
-      "name": "Qwen3-0.6B",
-      "file_size_mb": 609.8238830566406,
-      "avg_score": 2.8,
-      "avg_time": 8.223706769943238,
-      "efficiency": 0.3404790659892809,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 7,
-          "time": 8.638539791107178,
-          "response": ".\n\nIf the user command is not supported by the available tools, respond with an empty array.\n\nNow, t..."
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 5,
-          "time": 8.075484991073608,
-          "response": ".\n\nMake sure to use the correct tool name and format the JSON correctly.\n\nIf the command is not poss..."
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 0,
-          "time": 7.951770067214966,
-          "response": ".\n\nMake sure to use the correct tool names and format the JSON correctly.\n\nNow, the game state is as..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 2,
-          "time": 8.252855062484741,
-          "response": ".\n\nMake sure to use the correct tool names and format the JSON properly.\n\nNow, the user is in a game..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 0,
-          "time": 8.199883937835693,
-          "response": ".\n\nMake sure to use the correct tool name and format the JSON.\n\nIf the command is not possible, retu..."
-        }
-      ],
-      "type": "general"
-    },
-    {
-      "name": "Llama-Breeze2-3B",
-      "file_size_mb": 1424.04345703125,
-      "avg_score": 3.6,
-      "avg_time": 14.693956804275512,
-      "efficiency": 0.24499867856917243,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 3,
-          "time": 3.5608396530151367,
-          "response": "[get_game_state()]"
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 3,
-          "time": 5.626140356063843,
-          "response": "[move_units(unit_ids='infantry', target_x='150', target_y='200')]"
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 23.91610813140869,
-          "response": "The tool used is \"get_game_state\" and the tool name is \"get_game_state\". The args for this tool is a..."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 23.745216846466064,
-          "response": "{\n  \"tool\": \"build_building\",\n  \"args\": {\n    \"building_type\": \"power plant\",\n    \"position_x\": 100,..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 2,
-          "time": 16.621479034423828,
-          "response": "The game is ready for action. You are the AI, and you are ready to assist. You have the ability to m..."
-        }
-      ],
-      "type": "general"
-    },
-    {
-      "name": "Qwen3-1.7B",
-      "file_size_mb": 1007.8267211914062,
-      "avg_score": 3.0,
-      "avg_time": 13.003729963302613,
-      "efficiency": 0.23070303739513193,
-      "results": [
-        {
-          "test": "Commande simple",
-          "difficulty": "easy",
-          "score": 2,
-          "time": 12.862720251083374,
-          "response": ".\n\nYou must use the JSON format specified, without any additional text or explanation. The JSON must..."
-        },
-        {
-          "test": "Action avec coordonnées",
-          "difficulty": "easy",
-          "score": 2,
-          "time": 12.972241401672363,
-          "response": ".\n\nYou must use the correct tool name and format the JSON properly.\n\nThe game state is as follows:\n-..."
-        },
-        {
-          "test": "Attaque spécifique",
-          "difficulty": "medium",
-          "score": 3,
-          "time": 13.497555255889893,
-          "response": ".\n\nYou must use the correct tool name and format the JSON properly.\n\nThe tool to use is attack_unit...."
-        },
-        {
-          "test": "Construction",
-          "difficulty": "medium",
-          "score": 5,
-          "time": 12.513315677642822,
-          "response": ".\n\nYou must use the correct tool name and format the JSON properly.\n\nThe correct tool name is \"build..."
-        },
-        {
-          "test": "Commande complexe",
-          "difficulty": "hard",
-          "score": 3,
-          "time": 13.17281723022461,
-          "response": ".\n\nYou can use the following tool definitions:\n- move_units: move units to a new position\n- attack_u..."
-        }
-      ],
-      "type": "general"
-    }
-  ],
-  "best_overall": {
-    "name": "Qwen2.5-Coder-0.5B",
-    "file_size_mb": 408.8690185546875,
-    "avg_score": 4.4,
-    "avg_time": 4.1166441440582275,
-    "efficiency": 1.0688317585941343,
-    "results": [
-      {
-        "test": "Commande simple",
-        "difficulty": "easy",
-        "score": 5,
-        "time": 4.25421667098999,
-        "response": ".\n\nHere's a possible response:\n\n{\"tool\": \"get_game_state\", \"args\": {\"game_state\": {\"units\": [{\"id\": ..."
-      },
-      {
-        "test": "Action avec coordonnées",
-        "difficulty": "easy",
-        "score": 5,
-        "time": 4.333646059036255,
-        "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-      },
-      {
-        "test": "Attaque spécifique",
-        "difficulty": "medium",
-        "score": 5,
-        "time": 4.139528274536133,
-        "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-      },
-      {
-        "test": "Construction",
-        "difficulty": "medium",
-        "score": 2,
-        "time": 3.9508562088012695,
-        "response": "where \"tool_name\" is the name of the tool used and \"args\" is a JSON object containing the arguments ..."
-      },
-      {
-        "test": "Commande complexe",
-        "difficulty": "hard",
-        "score": 5,
-        "time": 3.9049735069274902,
-        "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-      }
-    ],
-    "type": "code_specialized"
-  },
-  "most_efficient": {
-    "name": "Qwen2.5-Coder-0.5B",
-    "file_size_mb": 408.8690185546875,
-    "avg_score": 4.4,
-    "avg_time": 4.1166441440582275,
-    "efficiency": 1.0688317585941343,
-    "results": [
-      {
-        "test": "Commande simple",
-        "difficulty": "easy",
-        "score": 5,
-        "time": 4.25421667098999,
-        "response": ".\n\nHere's a possible response:\n\n{\"tool\": \"get_game_state\", \"args\": {\"game_state\": {\"units\": [{\"id\": ..."
-      },
-      {
-        "test": "Action avec coordonnées",
-        "difficulty": "easy",
-        "score": 5,
-        "time": 4.333646059036255,
-        "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-      },
-      {
-        "test": "Attaque spécifique",
-        "difficulty": "medium",
-        "score": 5,
-        "time": 4.139528274536133,
-        "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-      },
-      {
-        "test": "Construction",
-        "difficulty": "medium",
-        "score": 2,
-        "time": 3.9508562088012695,
-        "response": "where \"tool_name\" is the name of the tool used and \"args\" is a JSON object containing the arguments ..."
-      },
-      {
-        "test": "Commande complexe",
-        "difficulty": "hard",
-        "score": 5,
-        "time": 3.9049735069274902,
-        "response": "where \"tool_name\" is the name of the tool used and \"args\" is a dictionary containing the arguments p..."
-      }
-    ],
-    "type": "code_specialized"
-  }
-}

comprehensive_mcp_evaluation.py

@@ -1,379 +0,0 @@
-"""
-Évaluation complète MCP avec 9 modèles
-Inclut les modèles spécialisés MCP et les modèles généraux
-Test réaliste avec commandes RTS typiques
-"""
-
-import sys
-import os
-import json
-import time
-
-sys.path.append(os.path.dirname(os.path.abspath(__file__)))
-
-def test_model_comprehensive(model_path, model_name):
-    """Test complet d'un modèle pour MCP"""
-    
-    try:
-        from llama_cpp import Llama
-        
-        print(f"🧪 Test de {model_name}...")
-        
-        # Vérifier la taille du fichier
-        file_size = os.path.getsize(model_path) / (1024*1024)
-        print(f"   📏 Taille: {file_size:.0f} MB")
-        
-        # Initialiser avec timeout plus long pour les gros modèles
-        llm = Llama(
-            model_path=model_path,
-            n_ctx=1024,
-            n_threads=1,
-            verbose=False,
-            n_gpu_layers=0  # Forcer CPU pour comparaison juste
-        )
-        
-        # Commandes RTS réelles avec différents niveaux de complexité
-        test_commands = [
-            {
-                "name": "Commande simple",
-                "command": "show game state",
-                "expected": "get_game_state",
-                "difficulty": "easy"
-            },
-            {
-                "name": "Action avec coordonnées",
-                "command": "move infantry to 150,200",
-                "expected": "move_units",
-                "difficulty": "easy"
-            },
-            {
-                "name": "Attaque spécifique",
-                "command": "attack enemy tank at position 300,150",
-                "expected": "attack_unit",
-                "difficulty": "medium"
-            },
-            {
-                "name": "Construction",
-                "command": "build power plant near my base at 100,100",
-                "expected": "build_building",
-                "difficulty": "medium"
-            },
-            {
-                "name": "Commande complexe",
-                "command": "defend base with all available units",
-                "expected": "move_units",
-                "difficulty": "hard"
-            }
-        ]
-        
-        results = []
-        total_score = 0
-        total_time = 0
-        
-        for test in test_commands:
-            prompt = f"""You are an AI assistant for an RTS game using MCP (Model Context Protocol).
-
-Available tools:
-- get_game_state()
-- move_units(unit_ids, target_x, target_y)
-- attack_unit(attacker_ids, target_id)
-- build_building(building_type, position_x, position_y)
-
-User command: "{test['command']}"
-
-Respond with JSON only: {{"tool": "tool_name", "args": {{}}}}"""
-            
-            start_time = time.time()
-            
-            try:
-                response = llm(
-                    prompt,
-                    max_tokens=100,
-                    temperature=0.1,
-                    stop=["</s>", "<|im_end|>", "```"]
-                )
-                
-                response_time = time.time() - start_time
-                
-                # Extraire la réponse
-                try:
-                    response_text = response['choices'][0]['text'].strip()
-                except:
-                    # Fallback pour différents formats de réponse
-                    if hasattr(response, 'get'):
-                        response_text = response.get('text', str(response))
-                    else:
-                        response_text = str(response)
-                
-                # Évaluer la réponse
-                score = evaluate_mcp_response(response_text, test)
-                
-                total_score += score
-                total_time += response_time
-                
-                print(f"   ✅ {test['name']}: {score}/10 ({response_time:.2f}s)")
-                
-                results.append({
-                    'test': test['name'],
-                    'difficulty': test['difficulty'],
-                    'score': score,
-                    'time': response_time,
-                    'response': response_text[:100] + "..." if len(response_text) > 100 else response_text
-                })
-                
-            except Exception as e:
-                print(f"   ❌ {test['name']}: Erreur - {e}")
-                results.append({
-                    'test': test['name'],
-                    'difficulty': test['difficulty'],
-                    'score': 0,
-                    'time': 0,
-                    'error': str(e)
-                })
-        
-        avg_score = total_score / len(test_commands)
-        avg_time = total_time / len(test_commands)
-        
-        print(f"   📊 Moyenne: {avg_score:.1f}/10 | Temps: {avg_time:.2f}s")
-        
-        return {
-            'name': model_name,
-            'file_size_mb': file_size,
-            'avg_score': avg_score,
-            'avg_time': avg_time,
-            'efficiency': avg_score / avg_time if avg_time > 0 else 0,
-            'results': results
-        }
-        
-    except Exception as e:
-        print(f"❌ Erreur critique avec {model_name}: {e}")
-        return {
-            'name': model_name,
-            'error': str(e),
-            'avg_score': 0,
-            'avg_time': 0,
-            'efficiency': 0
-        }
-
-def evaluate_mcp_response(response, test):
-    """Évaluation standardisée des réponses MCP"""
-    
-    if not response or response.strip() == "":
-        return 0
-    
-    score = 0
-    
-    # JSON valide (3 points)
-    try:
-        json.loads(response)
-        score += 3
-    except:
-        # Chercher JSON dans le texte
-        import re
-        json_match = re.search(r'\{[^}]*\}', response)
-        if json_match:
-            try:
-                json.loads(json_match.group())
-                score += 1
-            except:
-                pass
-    
-    # Outil correct (3 points)
-    expected_tool = test['expected']
-    if expected_tool in response:
-        score += 3
-    
-    # Paramètres appropriés (2 points)
-    if test['difficulty'] == 'easy':
-        if '150,200' in response or 'game state' in response:
-            score += 2
-    elif test['difficulty'] == 'medium':
-        if any(coord in response for coord in ['300,150', '100,100']):
-            score += 2
-    elif test['difficulty'] == 'hard':
-        if 'units' in response and 'defend' in response:
-            score += 2
-    
-    # Format correct (2 points)
-    if 'tool' in response and 'args' in response:
-        score += 2
-    
-    return min(score, 10)
-
-def main():
-    """Évaluation complète de tous les modèles"""
-    
-    print("🚀 ÉVALUATION COMPLÈTE MCP - 9 MODÈLES")
-    print("=" * 70)
-    print("Test avec modèles généraux et spécialisés MCP")
-    print("=" * 70)
-    
-    # Tous les modèles à tester
-    models = [
-        # Modèles généraux (testés précédemment)
-        {
-            'name': 'Qwen2.5-0.5B',
-            'path': 'qwen2.5-0.5b-instruct-q4_0.gguf',
-            'type': 'general'
-        },
-        {
-            'name': 'Qwen3-0.6B',
-            'path': 'Qwen3-0.6B-Q8_0.gguf',
-            'type': 'general'
-        },
-        {
-            'name': 'Gemma-3-270M',
-            'path': 'gemma-3-270m-it-qat-Q8_0.gguf',
-            'type': 'general'
-        },
-        {
-            'name': 'Qwen3-1.7B',
-            'path': 'Qwen3-1.7B-Q4_0.gguf',
-            'type': 'general'
-        },
-        
-        # Modèles spécialisés MCP
-        {
-            'name': 'MCP-Instruct-v1',
-            'path': 'mcp-instruct-v1.Q4_K_M.gguf',
-            'type': 'mcp_specialized'
-        },
-        {
-            'name': 'MCPR L-3B-Exa',
-            'path': 'mcprl-3b-exa.Q2_K.gguf',
-            'type': 'mcp_specialized'
-        },
-        {
-            'name': 'Gemma-3n-E2B-it',
-            'path': 'gemma-3n-E2B-it-UD-IQ2_XXS.gguf',
-            'type': 'mcp_specialized'
-        },
-        {
-            'name': 'Llama-Breeze2-3B',
-            'path': 'Llama-Breeze2-3B-Instruct-Text.Q2_K.gguf',
-            'type': 'general'
-        },
-        
-        # Modèle spécialisé en code/structuré
-        {
-            'name': 'Qwen2.5-Coder-0.5B',
-            'path': 'qwen2.5-coder-0.5b-instruct-q4_0.gguf',
-            'type': 'code_specialized'
-        }
-    ]
-    
-    results = []
-    
-    for model in models:
-        if os.path.exists(model['path']):
-            result = test_model_comprehensive(model['path'], model['name'])
-            result['type'] = model['type']
-            results.append(result)
-            print()
-        else:
-            print(f"❌ Modèle non trouvé: {model['path']}")
-            print()
-    
-    # Analyse complète
-    print("=" * 70)
-    print("📊 RÉSULTATS COMPLETS")
-    print("=" * 70)
-    
-    successful_results = [r for r in results if 'error' not in r and r['avg_score'] > 0]
-    
-    if successful_results:
-        # Classement par performance
-        sorted_by_score = sorted(successful_results, key=lambda x: x['avg_score'], reverse=True)
-        
-        print(f"\n🏆 CLASSEMENT PAR PERFORMANCE:")
-        for i, result in enumerate(sorted_by_score, 1):
-            print(f"   {i:2d}. {result['name']:20s} | {result['avg_score']:.1f}/10 | {result['avg_time']:.2f}s | {result['file_size_mb']:.0f}MB | {result['type']}")
-        
-        # Classement par efficacité
-        sorted_by_efficiency = sorted(successful_results, key=lambda x: x['efficiency'], reverse=True)
-        
-        print(f"\n⚡ CLASSEMENT PAR EFFICACITÉ:")
-        for i, result in enumerate(sorted_by_efficiency, 1):
-            print(f"   {i:2d}. {result['name']:20s} | {result['efficiency']:.2f} score/s | {result['file_size_mb']:.0f}MB")
-        
-        # Analyse par type
-        print(f"\n📈 ANALYSE PAR TYPE DE MODÈLE:")
-        
-        general_models = [r for r in successful_results if r['type'] == 'general']
-        mcp_specialized = [r for r in successful_results if r['type'] == 'mcp_specialized']
-        code_specialized = [r for r in successful_results if r['type'] == 'code_specialized']
-        
-        if general_models:
-            avg_general = sum(r['avg_score'] for r in general_models) / len(general_models)
-            print(f"   Modèles généraux ({len(general_models)}): {avg_general:.1f}/10 moyen")
-        
-        if mcp_specialized:
-            avg_mcp = sum(r['avg_score'] for r in mcp_specialized) / len(mcp_specialized)
-            print(f"   Spécialisés MCP ({len(mcp_specialized)}): {avg_mcp:.1f}/10 moyen")
-        
-        if code_specialized:
-            avg_code = sum(r['avg_score'] for r in code_specialized) / len(code_specialized)
-            print(f"   Spécialisés Code ({len(code_specialized)}): {avg_code:.1f}/10 moyen")
-        
-        # Meilleur modèle global
-        best = sorted_by_score[0]
-        most_efficient = sorted_by_efficiency[0]
-        
-        print(f"\n🎯 MEILLEUR MODÈLE GLOBAL: {best['name']}")
-        print(f"   Score: {best['avg_score']:.1f}/10")
-        print(f"   Temps: {best['avg_time']:.2f}s")
-        print(f"   Taille: {best['file_size_mb']:.0f}MB")
-        print(f"   Type: {best['type']}")
-        
-        print(f"\n⚡ MODÈLE LE PLUS EFFICACE: {most_efficient['name']}")
-        print(f"   Efficacité: {most_efficient['efficiency']:.2f} score/s")
-        
-        # Recommandations finales
-        print(f"\n💡 RECOMMANDATIONS FINALES:")
-        
-        if best['avg_score'] >= 7:
-            print(f"✅ {best['name']} est EXCELLENT pour la production MCP")
-        elif best['avg_score'] >= 5:
-            print(f"👍 {best['name']} est BON pour la production MCP")
-        else:
-            print(f"⚠️  {best['name']} nécessite des améliorations")
-        
-        # Comparaison spécialisés vs généraux
-        if mcp_specialized and general_models:
-            best_specialized = max(mcp_specialized, key=lambda x: x['avg_score'])
-            best_general = max(general_models, key=lambda x: x['avg_score'])
-            
-            print(f"\n🔬 SPÉCIALISÉS VS GÉNÉRAUX:")
-            print(f"   Meilleur spécialisé MCP: {best_specialized['name']} ({best_specialized['avg_score']:.1f}/10)")
-            print(f"   Meilleur général: {best_general['name']} ({best_general['avg_score']:.1f}/10)")
-            
-            if best_specialized['avg_score'] > best_general['avg_score']:
-                print(f"   ✅ Les modèles spécialisés MCP sont meilleurs!")
-            else:
-                print(f"   🤔 Les modèles généraux performent aussi bien")
-        
-        # Analyse détaillée du meilleur
-        print(f"\n📋 DÉTAILS DU MEILLEUR MODÈLE ({best['name']}):")
-        for result in best['results']:
-            status = "✅" if result['score'] >= 6 else "⚠️" if result['score'] >= 4 else "❌"
-            print(f"   {status} {result['test']}: {result['score']}/10 ({result['time']:.2f}s)")
-    
-    # Sauvegarder résultats complets
-    comprehensive_results = {
-        'evaluation_type': 'comprehensive_mcp_test',
-        'total_models_tested': len(models),
-        'successful_models': len(successful_results),
-        'results': results,
-        'ranking_by_score': sorted_by_score if successful_results else [],
-        'ranking_by_efficiency': sorted_by_efficiency if successful_results else [],
-        'best_overall': best if successful_results else None,
-        'most_efficient': most_efficient if successful_results else None
-    }
-    
-    with open("comprehensive_mcp_evaluation.json", "w", encoding="utf-8") as f:
-        json.dump(comprehensive_results, f, indent=2, ensure_ascii=False)
-    
-    print(f"\n📄 Résultats complets sauvegardés dans: comprehensive_mcp_evaluation.json")
-
-if __name__ == "__main__":
-    main()

debug_model_loading.py

@@ -1,120 +0,0 @@
-"""
-Script de débogage pour identifier le problème de chargement du modèle Qwen2.5 0.5B
-"""
-
-import sys
-import os
-import traceback
-
-print("🔍 DÉBOGAGE DU CHARGEMENT DU MODÈLE QWEN2.5 0.5B")
-print("=" * 60)
-
-# Vérifier l'existence du modèle
-model_path = "qwen2.5-0.5b-instruct-q4_0.gguf"
-if not os.path.exists(model_path):
-    print("❌ Modèle non trouvé à l'emplacement attendu")
-    print(f"   Recherché: {os.path.abspath(model_path)}")
-    sys.exit(1)
-
-print(f"✅ Modèle trouvé: {os.path.abspath(model_path)}")
-print(f"📏 Taille: {os.path.getsize(model_path) / (1024*1024):.1f} MB")
-
-# Test 1: Importation de llama_cpp
-print("\n🧪 TEST 1: Importation de llama_cpp")
-try:
-    import llama_cpp
-    print("✅ llama_cpp importé avec succès")
-    print(f"   Version: {llama_cpp.__version__}")
-except Exception as e:
-    print(f"❌ Erreur d'importation: {e}")
-    traceback.print_exc()
-    sys.exit(1)
-
-# Test 2: Test de chargement simple
-print("\n🧪 TEST 2: Chargement direct du modèle")
-try:
-    from llama_cpp import Llama
-    
-    # Essayer différentes configurations
-    configs = [
-        {"n_threads": 1, "n_ctx": 2048},
-        {"n_threads": 2, "n_ctx": 2048},
-        {"n_threads": 1, "n_ctx": 1024},
-        {"n_threads": 1, "n_ctx": 512},
-    ]
-    
-    for i, config in enumerate(configs, 1):
-        print(f"\n   Configuration {i}: threads={config['n_threads']}, ctx={config['n_ctx']}")
-        try:
-            llm = Llama(
-                model_path=model_path,
-                n_ctx=config['n_ctx'],
-                n_threads=config['n_threads']
-            )
-            print("   ✅ Modèle chargé avec succès")
-            
-            # Test d'inférence simple
-            print("   🧠 Test d'inférence...")
-            response = llm(
-                "Réponds avec 'TEST_RÉUSSI': ",
-                max_tokens=10,
-                temperature=0.1,
-                echo=False
-            )
-            
-            # Gérer la réponse (peut être un générateur ou un dict)
-            if hasattr(response, '__iter__') and not isinstance(response, dict):
-                # C'est un générateur, prendre le premier élément
-                response = next(response)
-            
-            if isinstance(response, dict) and 'choices' in response:
-                text = response['choices'][0]['text'].strip()
-                print(f"   📝 Réponse: {text}")
-                if "TEST_RÉUSSI" in text:
-                    print("   ✅ Test d'inférence réussi!")
-                    break
-            else:
-                print(f"   ⚠️  Format de réponse inattendu: {type(response)}")
-            
-        except Exception as e:
-            print(f"   ❌ Erreur: {e}")
-            traceback.print_exc()
-    
-    else:
-        print("❌ Toutes les configurations ont échoué")
-        
-except Exception as e:
-    print(f"❌ Erreur lors du chargement: {e}")
-    traceback.print_exc()
-
-# Test 3: Vérification de l'environnement
-print("\n🧪 TEST 3: Vérification de l'environnement")
-print(f"   Python: {sys.version}")
-print(f"   Répertoire de travail: {os.getcwd()}")
-print(f"   Variables d'environnement PATH: {os.environ.get('PATH', 'Non défini')}")
-
-# Test 4: Vérification des permissions
-print("\n🧪 TEST 4: Vérification des permissions")
-try:
-    # Tester la lecture du fichier
-    with open(model_path, 'rb') as f:
-        header = f.read(100)
-        print(f"   ✅ Fichier lisible, header: {header[:20]}...")
-    
-    # Tester les permissions
-    import stat
-    st = os.stat(model_path)
-    permissions = stat.filemode(st.st_mode)
-    print(f"   Permissions: {permissions}")
-    
-    # Vérifier l'espace mémoire
-    import shutil
-    total, used, free = shutil.disk_usage(".")
-    print(f"   Espace disque libre: {free / (1024**3):.1f} GB")
-    
-except Exception as e:
-    print(f"   ❌ Erreur de lecture: {e}")
-
-print("\n" + "=" * 60)
-print("🔍 RÉSUMÉ DU DÉBOGAGE")
-print("=" * 60)

diagnose_mcp_failures.py

@@ -1,182 +0,0 @@
-"""
-Diagnostic des échecs des modèles spécialisés MCP
-Analyse pourquoi les modèles plus grands et spécialisés échouent
-"""
-
-import sys
-import os
-
-def diagnose_model_failures():
-    """Diagnostiquer les problèmes avec les modèles spécialisés MCP"""
-    
-    print("🔍 DIAGNOSTIC DES ÉCHECS DES MODÈLES SPÉCIALISÉS MCP")
-    print("=" * 70)
-    
-    # Modèles problématiques
-    problematic_models = [
-        {
-            'name': 'MCP-Instruct-v1',
-            'path': 'mcp-instruct-v1.Q4_K_M.gguf',
-            'issues': ['llama_decode returned -1']
-        },
-        {
-            'name': 'MCPR L-3B-Exa', 
-            'path': 'mcprl-3b-exa.Q2_K.gguf',
-            'issues': ['texte corrompu', 'caractères spéciaux', 'sortie incohérente']
-        },
-        {
-            'name': 'Gemma-3n-E2B-it',
-            'path': 'gemma-3n-E2B-it-UD-IQ2_XXS.gguf',
-            'issues': ['réponses vides', 'pas de sortie']
-        }
-    ]
-    
-    # Analyse des problèmes potentiels
-    print("\n📋 ANALYSE DES PROBLÈMES IDENTIFIÉS:")
-    
-    for model in problematic_models:
-        print(f"\n🧩 {model['name']}:")
-        
-        if not os.path.exists(model['path']):
-            print(f"   ❌ Fichier modèle non trouvé: {model['path']}")
-            continue
-            
-        file_size_mb = os.path.getsize(model['path']) / (1024*1024)
-        print(f"   📏 Taille: {file_size_mb:.1f} MB")
-        
-        print(f"   🔧 Problèmes détectés:")
-        for issue in model['issues']:
-            print(f"      • {issue}")
-    
-    # Causes potentielles
-    print("\n🔍 CAUSES POTENTIELLES DES ÉCHECS:")
-    print("   1. FORMAT DE PROMPT INADÉQUAT")
-    print("      • Les modèles spécialisés MCP peuvent attendre un format de prompt différent")
-    print("      • Format actuel trop simple vs format de formation MCP spécialisé")
-    print("      • Besoin de tokens spéciaux ou de formatage MCP spécifique")
-    
-    print("\n   2. PROBLÈMES DE CHARGEMENT DU MODÈLE")
-    print("      • llama_decode=-1: Erreur de décodeur LLAMA")
-    print("      • Modèles potentiellement incompatibles avec llama-cpp-python")
-    print("      • Paramètres d'initialisation incorrects")
-    
-    print("\n   3. CONTEXTE ET TOKENS SPÉCIAUX")
-    print("      • Les modèles MCP peuvent nécessiter des tokens de début/fin spécifiques")
-    print("      • Format d'entrée différent de l'attente du modèle")
-    print("      • Besoin de format de conversation MCP structuré")
-    
-    print("\n   4. QUANTISATION PROBLÉMATIQUE")
-    print("      • Q2_K pour MCPR L-3B-Exa peut être trop agressif")
-    print("      • IQ2_XXS pour Gemma-3n-E2B-it: très forte compression")
-    print("      • Perte de capacité avec quantisation extrême")
-    
-    # Test de diagnostic avec prompts alternatifs
-    print("\n🚀 TEST DE DIAGNOSTIC AVEC PROMPTS ALTERNATIFS")
-    print("=" * 70)
-    
-    from llama_cpp import Llama
-    
-    # Formats de prompt alternatifs pour tester
-    prompt_formats = [
-        {
-            'name': 'Format actuel',
-            'template': '''You are an AI assistant for an RTS game using MCP (Model Context Protocol).
-
-Available tools:
-- get_game_state()
-- move_units(unit_ids, target_x, target_y)
-- attack_unit(attacker_ids, target_id)
-- build_building(building_type, position_x, position_y)
-
-User command: "{command}"
-
-Respond with JSON only: {{"tool": "tool_name", "args": {{}}}}'''
-        },
-        {
-            'name': 'Format MCP structuré',
-            'template': '''<|im_start|>system
-You are an AI assistant for an RTS game using the Model Context Protocol (MCP). You must respond with only JSON following this exact format:
-{{"tool": "tool_name", "args": {{<arguments>}}}}
-
-Available tools:
-- get_game_state(): Get current game state
-- move_units(unit_ids, target_x, target_y): Move units to coordinates
-- attack_unit(attacker_ids, target_id): Attack a specific unit  
-- build_building(building_type, position_x, position_y): Construct a building
-
-Always respond with valid JSON, no additional text.<|im_end|>
-<|im_start|>user
-{command}<|im_end|>
-<|im_start|>assistant
-'''
-        },
-        {
-            'name': 'Format simple JSON',
-            'template': '''User: {command}
-
-Tools: get_game_state, move_units, attack_unit, build_building
-
-Response (JSON only):'''
-        }
-    ]
-    
-    # Tester avec MCPR L-3B-Exa (le modèle qui sort du texte corrompu)
-    test_model = 'mcprl-3b-exa.Q2_K.gguf'
-    
-    if os.path.exists(test_model):
-        print(f"\n🧪 Test avec {test_model}...")
-        
-        try:
-            llm = Llama(
-                model_path=test_model,
-                n_ctx=1024,
-                n_threads=1,
-                verbose=False,
-                n_gpu_layers=0
-            )
-            
-            test_command = "show game state"
-            
-            for format_info in prompt_formats:
-                print(f"\n   📝 Format: {format_info['name']}")
-                
-                prompt = format_info['template'].format(command=test_command)
-                
-                try:
-                    response = llm(
-                        prompt,
-                        max_tokens=50,
-                        temperature=0.1,
-                        stop=["</s>", "<|im_end|>", "```"]
-                    )
-                    
-                    response_text = response['choices'][0]['text'].strip()
-                    
-                    print(f"      Prompt (premiers 100 caractères): {prompt[:100]}...")
-                    print(f"      Réponse: {response_text[:200]}")
-                    
-                    # Vérifier si c'est du JSON valide
-                    try:
-                        import json
-                        json.loads(response_text)
-                        print(f"      ✅ JSON VALIDE!")
-                    except:
-                        print(f"      ❌ JSON INVALIDE")
-                        
-                except Exception as e:
-                    print(f"      ❌ Erreur: {e}")
-                    
-        except Exception as e:
-            print(f"   ❌ Impossible de charger le modèle: {e}")
-    
-    # Recommandations
-    print("\n💡 RECOMMANDATIONS POUR CORRIGER:")
-    print("   1. Essayer différentes versions de quantisation (Q4_K_M, Q8_0)")
-    print("   2. Utiliser des formats de prompt structurés avec tokens MCP")
-    print("   3. Augmenter la température (0.3-0.5) pour plus de variabilité")
-    print("   4. Tester avec plus de tokens max (200-300)")
-    print("   5. Vérifier la compatibilité avec llama-cpp-python")
-    print("   6. Utiliser des modèles de taille similaire mais avec moins de compression")
-
-if __name__ == "__main__":
-    diagnose_model_failures()

diagnostic_qwen_test.py

@@ -1,227 +0,0 @@
-"""
-Test de diagnostic pour évaluer les capacités MCP de Qwen2.5 0.5B
-Approche simplifiée pour éviter les problèmes de chargement
-"""
-
-import sys
-import os
-import json
-import time
-
-# Ajouter le chemin pour les imports
-sys.path.append(os.path.dirname(os.path.abspath(__file__)))
-
-def test_model_loading():
-    """Test simple de chargement du modèle"""
-    
-    print("🔍 TEST DE CHARGEMENT DU MODÈLE")
-    print("=" * 50)
-    
-    # Vérifier que le modèle existe
-    model_path = "qwen2.5-0.5b-instruct-q4_0.gguf"
-    if not os.path.exists(model_path):
-        print("❌ Modèle non trouvé")
-        return False
-    
-    print(f"✅ Modèle trouvé: {model_path}")
-    print(f"📏 Taille: {os.path.getsize(model_path) / (1024*1024):.1f} MB")
-    
-    # Essayer d'importer et d'initialiser
-    try:
-        from ai_analysis import AIAnalyzer
-        print("✅ Module AIAnalyzer importé")
-        
-        analyzer = AIAnalyzer(model_path)
-        print("✅ AIAnalyzer initialisé")
-        
-        if analyzer.model_available:
-            print("✅ Modèle disponible selon AIAnalyzer")
-            return True
-        else:
-            print("❌ Modèle non disponible selon AIAnalyzer")
-            return False
-            
-    except Exception as e:
-        print(f"❌ Erreur lors de l'initialisation: {e}")
-        return False
-
-def test_simple_inference():
-    """Test d'inférence simple avec le modèle"""
-    
-    print("\n🧪 TEST D'INFÉRENCE SIMPLE")
-    print("=" * 50)
-    
-    try:
-        # Importer directement llama-cpp-python
-        from llama_cpp import Llama
-        
-        model_path = "qwen2.5-0.5b-instruct-q4_0.gguf"
-        
-        print("🔄 Chargement du modèle avec Llama...")
-        
-        # Essayer différentes configurations
-        for n_threads in [1, 2, 4]:
-            try:
-                llm = Llama(
-                    model_path=model_path,
-                    n_ctx=2048,
-                    n_threads=n_threads,
-                    verbose=False
-                )
-                
-                print(f"✅ Modèle chargé avec {n_threads} threads")
-                
-                # Test simple
-                prompt = "Réponds simplement avec 'TEST_RÉUSSI'"
-                
-                start_time = time.time()
-                response = llm(
-                    prompt,
-                    max_tokens=10,
-                    temperature=0.1
-                )
-                response_time = time.time() - start_time
-                
-                text = response['choices'][0]['text'].strip()
-                
-                print(f"⏱️  Temps de réponse: {response_time:.2f}s")
-                print(f"📝 Réponse: {text}")
-                
-                if "TEST_RÉUSSI" in text:
-                    print("✅ Test d'inférence réussi")
-                    return True
-                else:
-                    print("⚠️  Réponse inattendue")
-                    
-            except Exception as e:
-                print(f"❌ Erreur avec {n_threads} threads: {e}")
-                continue
-        
-        return False
-        
-    except Exception as e:
-        print(f"❌ Erreur lors de l'import de llama_cpp: {e}")
-        return False
-
-def test_mcp_capability_simulation():
-    """Test simulé des capacités MCP (sans vrai modèle)"""
-    
-    print("\n🧠 TEST SIMULÉ DES CAPACITÉS MCP")
-    print("=" * 50)
-    
-    # Basé sur la documentation et les spécifications de Qwen2.5 0.5B
-    # Évaluation théorique des capacités
-    
-    capabilities = {
-        "compréhension_instructions_simples": {
-            "score": 8,
-            "description": "Bon pour les commandes simples comme 'montre l'état du jeu'",
-            "exemple": "User: 'get game state' → AI: {'tool': 'get_game_state'}"
-        },
-        "extraction_paramètres_basiques": {
-            "score": 6,
-            "description": "Capable d'extraire des paramètres simples comme des coordonnées",
-            "exemple": "User: 'move to 100,200' → AI: {'tool': 'move_units', 'args': {'target_x': 100, 'target_y': 200}}"
-        },
-        "planification_multi_étapes": {
-            "score": 4,
-            "description": "Limité pour les stratégies complexes nécessitant plusieurs étapes",
-            "exemple": "User: 'build base and defend' → Peut avoir du mal à décomposer"
-        },
-        "gestion_ambiguïté": {
-            "score": 5,
-            "description": "Capacité modérée à gérer les instructions ambiguës",
-            "exemple": "User: 'move to enemy base' → Peut demander des coordonnées"
-        },
-        "compréhension_contexte_jeu": {
-            "score": 7,
-            "description": "Bon pour comprendre le contexte RTS de base",
-            "exemple": "Comprend les concepts comme 'unités', 'bâtiments', 'ressources'"
-        }
-    }
-    
-    total_score = sum(cap["score"] for cap in capabilities.values())
-    avg_score = total_score / len(capabilities)
-    
-    print("📊 ÉVALUATION THÉORIQUE BASÉE SUR LES SPÉCIFICATIONS:")
-    
-    for name, cap in capabilities.items():
-        print(f"\n🔹 {name.replace('_', ' ').title()}:")
-        print(f"   Score: {cap['score']}/10")
-        print(f"   {cap['description']}")
-        print(f"   Exemple: {cap['exemple']}")
-    
-    print(f"\n🎯 SCORE MOYEN THÉORIQUE: {avg_score:.1f}/10")
-    
-    return avg_score
-
-def generate_assessment():
-    """Générer une évaluation complète"""
-    
-    print("🔍 ÉVALUATION DES CAPACITÉS MCP DE QWEN2.5 0.5B")
-    print("=" * 70)
-    
-    # Test de chargement
-    model_loaded = test_model_loading()
-    
-    # Test d'inférence
-    if model_loaded:
-        inference_working = test_simple_inference()
-    else:
-        inference_working = False
-    
-    # Évaluation théorique
-    theoretical_score = test_mcp_capability_simulation()
-    
-    # Rapport final
-    print("\n" + "="*70)
-    print("📊 RAPPORT FINAL D'ÉVALUATION")
-    print("="*70)
-    
-    print(f"\n🔧 ÉTAT TECHNIQUE:")
-    print(f"   Modèle chargé: {'✅' if model_loaded else '❌'}")
-    print(f"   Inférence fonctionnelle: {'✅' if inference_working else '❌'}")
-    
-    print(f"\n🧠 CAPACITÉS MCP ESTIMÉES:")
-    print(f"   Score théorique: {theoretical_score:.1f}/10")
-    
-    if theoretical_score >= 7:
-        print("💪 TRÈS CAPABLE - Bon pour la plupart des tâches MCP")
-    elif theoretical_score >= 5:
-        print("👍 CAPABLE - Adapté pour les commandes simples à modérées")
-    else:
-        print("⚠️  LIMITÉ - Recommandé uniquement pour les tâches très simples")
-    
-    print(f"\n💡 RECOMMANDATIONS PRATIQUES:")
-    
-    if not model_loaded or not inference_working:
-        print("1. 🔧 Résoudre d'abord les problèmes techniques de chargement")
-        print("2. 📦 Vérifier l'installation de llama-cpp-python")
-        print("3. 🔄 Tester avec différentes configurations")
-    
-    print("4. 🎯 Commencer par des commandes MCP très simples")
-    print("5. 🔍 Tester progressivement la complexité")
-    print("6. ✅ Ajouter une validation robuste")
-    print("7. 🔄 Utiliser des prompts structurés")
-    
-    print(f"\n📈 STRATÉGIE D'IMPLÉMENTATION:")
-    
-    if theoretical_score >= 6:
-        print("• Implémenter la traduction MCP avec confiance")
-        print("• Commencer par: get_game_state, move_units simples")
-        print("• Ajouter progressivement: paramètres complexes, séquences")
-    else:
-        print("• Utiliser des règles fixes pour les commandes simples")
-        print("• Réserver Qwen2.5 pour l'analyse plutôt que la traduction")
-        print("• Envisager un modèle plus grand pour les tâches complexes")
-    
-    # Résumé technique
-    print(f"\n🔬 INFORMATIONS TECHNIQUES:")
-    print("• Modèle: Qwen2.5-0.5B (0.5 milliard de paramètres)")
-    print("• Type: Instruct (optimisé pour suivre des instructions)")
-    print("• Format: GGUF (quantisé 4-bit)")
-    print("• Taille: ~409 MB")
-    print("• Capacité: Bon pour les tâches simples, limité pour les complexes")
-
-if __name__ == "__main__":
-    generate_assessment()

direct_qwen_mcp_test.py

@@ -1,297 +0,0 @@
-"""
-Test direct des capacités MCP de Qwen2.5 0.5B
-Évalue la compréhension des outils MCP et la traduction d'instructions
-"""
-
-import sys
-import os
-import json
-import time
-
-# Ajouter le chemin pour les imports
-sys.path.append(os.path.dirname(os.path.abspath(__file__)))
-
-def test_qwen_mcp_capabilities():
-    """Test direct des capacités MCP de Qwen2.5 0.5B"""
-    
-    print("🔍 ÉVALUATION DES CAPACITÉS MCP DE QWEN2.5 0.5B")
-    print("=" * 70)
-    
-    # Vérifier que le modèle existe
-    model_path = "qwen2.5-0.5b-instruct-q4_0.gguf"
-    if not os.path.exists(model_path):
-        print("❌ Modèle non trouvé. Téléchargez-le d'abord.")
-        return
-    
-    print(f"✅ Modèle trouvé: {model_path}")
-    print(f"📏 Taille: {os.path.getsize(model_path) / (1024*1024):.1f} MB\n")
-    
-    # Importer la classe AIAnalyzer
-    try:
-        from ai_analysis import AIAnalyzer
-        
-        # Initialiser l'analyseur
-        print("🔄 Initialisation de l'analyseur AI...")
-        analyzer = AIAnalyzer(model_path)
-        
-        if not analyzer.model_available:
-            print("❌ Impossible de charger le modèle")
-            return
-        
-        print("✅ Analyseur AI initialisé avec succès\n")
-        
-    except Exception as e:
-        print(f"❌ Erreur lors de l'import: {e}")
-        return
-    
-    # Tests de capacité MCP
-    tests = [
-        {
-            "name": "Compréhension MCP de base",
-            "prompt": """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP (Model Context Protocol).
-
-Outils MCP disponibles:
-- get_game_state(): Obtenir l'état actuel du jeu
-- move_units(unit_ids, target_x, target_y): Déplacer des unités vers une position
-- attack_unit(attacker_ids, target_id): Attaquer une unité ennemie
-
-Commande utilisateur: "Montre-moi l'état du jeu"
-
-Réponds avec un objet JSON contenant l'appel d'outil MCP à exécuter.
-""",
-            "expected": "get_game_state"
-        },
-        {
-            "name": "Traduction d'action simple",
-            "prompt": """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace mon infanterie vers la position 150, 200"
-
-Réponds avec un objet JSON contenant l'appel d'outil avec les paramètres extraits.
-""",
-            "expected": "move_units"
-        },
-        {
-            "name": "Extraction de paramètres complexes",
-            "prompt": """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace les tanks 1, 3 et 7 vers les coordonnées 120, 80"
-
-Extrais les paramètres et réponds avec un objet JSON d'appel d'outil.
-""",
-            "expected": "move_units avec paramètres"
-        },
-        {
-            "name": "Planification stratégique",
-            "prompt": """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- get_game_state(): Obtenir l'état du jeu
-- move_units(): Déplacer des unités
-- attack_unit(): Attaquer l'ennemi
-- build_building(): Construire un bâtiment
-
-Commande utilisateur: "Construis une base près du minerai et défends-la avec des tourelles"
-
-Décompose cette stratégie en une séquence d'actions MCP.
-""",
-            "expected": "séquence d'actions"
-        },
-        {
-            "name": "Gestion d'ambiguïté",
-            "prompt": """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace mes unités vers la base ennemie"
-
-Comment gérerais-tu cette instruction ambiguë?
-""",
-            "expected": "demande de clarification"
-        }
-    ]
-    
-    results = []
-    
-    for i, test in enumerate(tests, 1):
-        print(f"\n🧪 TEST {i}: {test['name']}")
-        print("-" * 50)
-        
-        try:
-            # Utiliser la méthode d'analyse de l'AIAnalyzer
-            start_time = time.time()
-            
-            # Pour ce test, nous allons utiliser une approche directe
-            # en appelant la fonction interne de l'analyseur
-            
-            # Créer un prompt formaté pour l'analyseur
-            formatted_prompt = f"""
-{test['prompt']}
-
-Réponds uniquement avec un objet JSON valide.
-"""
-            
-            print("📤 Envoi de la requête au modèle...")
-            
-            # Utiliser la méthode d'analyse de l'AIAnalyzer
-            # Note: Nous devons adapter cela à l'interface existante
-            
-            # Pour l'instant, utilisons une approche simplifiée
-            from ai_analysis import _llama_worker
-            import multiprocessing as mp
-            
-            # Créer une queue pour les résultats
-            result_queue = mp.Queue()
-            
-            # Lancer le worker
-            worker = mp.Process(
-                target=_llama_worker,
-                args=(result_queue, model_path, formatted_prompt, [], 300, 0.1)
-            )
-            worker.start()
-            worker.join(timeout=30)  # Timeout de 30 secondes
-            
-            if worker.is_alive():
-                worker.terminate()
-                response = "TIMEOUT"
-            else:
-                result = result_queue.get()
-                if result['status'] == 'ok':
-                    response_data = result.get('data', {})
-                    if 'raw' in response_data:
-                        response = response_data['raw']
-                    else:
-                        response = str(response_data)
-                else:
-                    response = f"ERROR: {result.get('message', 'Unknown error')}"
-            
-            response_time = time.time() - start_time
-            
-            print(f"⏱️  Temps de réponse: {response_time:.2f}s")
-            print(f"📝 Réponse: {response[:200]}...")
-            
-            # Analyser la réponse
-            score = analyze_response(response, test)
-            
-            results.append({
-                'test': test['name'],
-                'response': response,
-                'response_time': response_time,
-                'score': score,
-                'success': score >= 5
-            })
-            
-            print(f"📊 Score: {score}/10")
-            
-        except Exception as e:
-            print(f"❌ Erreur: {e}")
-            results.append({
-                'test': test['name'],
-                'error': str(e),
-                'score': 0,
-                'success': False
-            })
-    
-    # Générer le rapport final
-    generate_final_report(results)
-
-def analyze_response(response, test_config):
-    """Analyser et noter la réponse du modèle"""
-    
-    if not response or response == "TIMEOUT":
-        return 0
-    
-    score = 0
-    
-    # Vérifier la structure JSON
-    try:
-        json_data = json.loads(response)
-        score += 3  # JSON valide
-        
-        # Vérifier la présence d'outils MCP
-        if 'tool' in json_data:
-            score += 2
-        if 'args' in json_data:
-            score += 1
-            
-    except json.JSONDecodeError:
-        # Vérifier les patterns dans le texte
-        if 'get_game_state' in response:
-            score += 2
-        if 'move_units' in response:
-            score += 2
-        if any(param in response for param in ['unit_ids', 'target_x', 'target_y']):
-            score += 1
-    
-    # Vérifier la cohérence sémantique
-    if 'game' in response.lower() or 'rts' in response.lower():
-        score += 1
-    
-    # Vérifier la pertinence par rapport au test
-    expected = test_config['expected']
-    if expected in response.lower():
-        score += 1
-    
-    return min(score, 10)
-
-def generate_final_report(results):
-    """Générer un rapport final d'évaluation"""
-    
-    print("\n" + "="*70)
-    print("📊 RAPPORT FINAL D'ÉVALUATION MCP")
-    print("="*70)
-    
-    successful_tests = [r for r in results if r.get('success', False)]
-    total_score = sum(r.get('score', 0) for r in results)
-    avg_score = total_score / len(results) if results else 0
-    
-    print(f"\n📈 STATISTIQUES:")
-    print(f"   Tests réussis: {len(successful_tests)}/{len(results)}")
-    print(f"   Score moyen: {avg_score:.1f}/10")
-    
-    print(f"\n🔍 DÉTAILS PAR TEST:")
-    for result in results:
-        status = "✅" if result.get('success', False) else "❌"
-        print(f"   {status} {result['test']}: {result.get('score', 0)}/10")
-        if 'response_time' in result:
-            print(f"     ⏱️  Temps: {result['response_time']:.2f}s")
-    
-    print(f"\n🎯 ÉVALUATION FINALE:")
-    
-    if avg_score >= 8:
-        print("💪 EXCELLENT - Qwen2.5 0.5B est très capable pour les tâches MCP")
-        print("   • Utilisation recommandée pour la traduction MCP")
-        print("   • Bonne compréhension des outils et paramètres")
-    elif avg_score >= 6:
-        print("👍 BON - Capable avec quelques limitations")
-        print("   • Adapté pour les commandes simples")
-        print("   • Validation recommandée pour les actions complexes")
-    elif avg_score >= 4:
-        print("⚠️  MODÉRÉ - Limitations significatives")
-        print("   • Utilisation limitée aux commandes très simples")
-        print("   • Validation stricte nécessaire")
-    else:
-        print("❌ FAIBLE - Pas adapté aux tâches MCP")
-        print("   • Envisagez un modèle plus grand")
-        print("   • Utilisez des règles fixes à la place")
-    
-    print(f"\n💡 RECOMMANDATIONS:")
-    print("1. Commencez par des commandes simples")
-    print("2. Ajoutez une validation des actions")
-    print("3. Utilisez des prompts structurés")
-    print("4. Testez avec différents types de commandes")
-
-if __name__ == "__main__":
-    test_qwen_mcp_capabilities()

final_14_model_evaluation.json

@@ -1,256 +0,0 @@
-{
-  "evaluation_info": {
-    "title": "Comprehensive MCP Capability Evaluation - 14 Models",
-    "date": "2025-10-05",
-    "total_models": 14,
-    "framework": "llama-cpp-python 0.3.16",
-    "test_scenarios": 3,
-    "max_score_per_test": 10
-  },
-  "test_scenarios": [
-    {
-      "id": 1,
-      "description": "Get the current game state",
-      "expected_tool": "get_game_state",
-      "scoring": {
-        "correct_tool": 4,
-        "valid_json": 3,
-        "proper_terminology": 2,
-        "coordinates": 1
-      }
-    },
-    {
-      "id": 2,
-      "description": "Move infantry unit to coordinates 150,200",
-      "expected_tool": "move_units",
-      "expected_coordinates": [150, 200],
-      "scoring": {
-        "correct_tool": 4,
-        "valid_json": 3,
-        "proper_terminology": 2,
-        "coordinates": 1
-      }
-    },
-    {
-      "id": 3,
-      "description": "Attack enemy tank at location 300,150",
-      "expected_tool": "attack_unit",
-      "expected_coordinates": [300, 150],
-      "scoring": {
-        "correct_tool": 4,
-        "valid_json": 3,
-        "proper_terminology": 2,
-        "coordinates": 1
-      }
-    }
-  ],
-  "results": [
-    {
-      "rank": 1,
-      "model_name": "Qwen2.5-Coder-1.5B-Q4",
-      "filename": "qwen2.5-coder-1.5b-instruct-q4_0.gguf",
-      "avg_score": 9.7,
-      "avg_time": 4.12,
-      "size_mb": 1017,
-      "efficiency": 2.34,
-      "status": "champion",
-      "notes": "Exceptional MCP performance. Wraps JSON in markdown code blocks. 10/10 on tests 2 and 3, 9/10 on test 1.",
-      "detailed_scores": [9, 10, 10]
-    },
-    {
-      "rank": 2,
-      "model_name": "Qwen2.5-Coder-0.5B",
-      "filename": "qwen2.5-coder-0.5b-instruct-q4_0.gguf",
-      "avg_score": 4.3,
-      "avg_time": 2.08,
-      "size_mb": 409,
-      "efficiency": 2.08,
-      "status": "previous_champion",
-      "notes": "Best budget option. Good balance of size and performance."
-    },
-    {
-      "rank": 3,
-      "model_name": "Qwen3-0.6B",
-      "filename": "Qwen3-0.6B-Q8_0.gguf",
-      "avg_score": 3.7,
-      "avg_time": 3.98,
-      "size_mb": 610,
-      "efficiency": 0.92,
-      "status": "functional"
-    },
-    {
-      "rank": 4,
-      "model_name": "Gemma-3-270M",
-      "filename": "gemma-3-270m-it-qat-Q8_0.gguf",
-      "avg_score": 3.7,
-      "avg_time": 2.29,
-      "size_mb": 428,
-      "efficiency": 1.60,
-      "status": "functional",
-      "notes": "Ultra-lightweight champion. Excellent efficiency for its tiny size."
-    },
-    {
-      "rank": 5,
-      "model_name": "MCPR-L-3B-Exa-Q8",
-      "filename": "mcprl-3b-exa.Q8_0.gguf",
-      "avg_score": 3.7,
-      "avg_time": 17.42,
-      "size_mb": 3133,
-      "efficiency": 0.21,
-      "status": "functional",
-      "notes": "MCP-specialized but slow. Large size, poor efficiency."
-    },
-    {
-      "rank": 6,
-      "model_name": "Gemma-3n-E2B-it-Q8",
-      "filename": "google_gemma-3n-E2B-it-Q8_0.gguf",
-      "avg_score": 3.7,
-      "avg_time": 14.80,
-      "size_mb": 4566,
-      "efficiency": 0.25,
-      "status": "functional",
-      "notes": "Largest model tested. Poor efficiency despite high quantization."
-    },
-    {
-      "rank": 7,
-      "model_name": "Qwen3-1.7B",
-      "filename": "Qwen3-1.7B-Q4_0.gguf",
-      "avg_score": 3.7,
-      "avg_time": 6.24,
-      "size_mb": 1008,
-      "efficiency": 0.59,
-      "status": "functional"
-    },
-    {
-      "rank": 8,
-      "model_name": "Qwen2.5-0.5B",
-      "filename": "qwen2.5-0.5b-instruct-q4_0.gguf",
-      "avg_score": 2.7,
-      "avg_time": 1.17,
-      "size_mb": 409,
-      "efficiency": 2.28,
-      "status": "functional",
-      "notes": "Fast but limited MCP capability. General-purpose model."
-    },
-    {
-      "rank": 9,
-      "model_name": "Gemma-3n-E2B-it-IQ2",
-      "filename": "gemma-3n-E2B-it-UD-IQ2_XXS.gguf",
-      "avg_score": 2.3,
-      "avg_time": 14.11,
-      "size_mb": 1958,
-      "efficiency": 0.17,
-      "status": "functional",
-      "notes": "Heavy quantization impacts quality."
-    },
-    {
-      "rank": 10,
-      "model_name": "Llama-Breeze2-3B-Q2",
-      "filename": "Llama-Breeze2-3B-Instruct-Text.Q2_K.gguf",
-      "avg_score": 1.3,
-      "avg_time": 11.39,
-      "size_mb": 1424,
-      "efficiency": 0.12,
-      "status": "functional",
-      "notes": "Poor performance. Q2 quantization too aggressive."
-    },
-    {
-      "rank": 11,
-      "model_name": "Yi-Coder-1.5B-Q4",
-      "filename": "Yi-Coder-1.5B.Q4_0.gguf",
-      "avg_score": 0.0,
-      "avg_time": 11.64,
-      "size_mb": 826,
-      "efficiency": 0.0,
-      "status": "failed",
-      "notes": "Prompt format incompatibility. Returns system prompt instead of generating responses."
-    },
-    {
-      "rank": 12,
-      "model_name": "MCP-Instruct-v1-Q4",
-      "filename": "mcp-instruct-v1.Q4_K_M.gguf",
-      "avg_score": 0.0,
-      "avg_time": 0.0,
-      "size_mb": 697,
-      "efficiency": 0.0,
-      "status": "failed",
-      "notes": "Technical error: llama_decode returned -1"
-    },
-    {
-      "rank": 13,
-      "model_name": "MCPR-L-3B-Exa-Q2",
-      "filename": "mcprl-3b-exa.Q2_K.gguf",
-      "avg_score": 0.0,
-      "avg_time": 10.63,
-      "size_mb": 1216,
-      "efficiency": 0.0,
-      "status": "failed",
-      "notes": "Produces gibberish output. Q2 quantization too aggressive for this architecture."
-    },
-    {
-      "rank": 14,
-      "model_name": "MCP-Instruct-v1-Q8",
-      "filename": "mcp-instruct-v1.Q8_0.gguf",
-      "avg_score": 0.0,
-      "avg_time": 0.0,
-      "size_mb": 1465,
-      "efficiency": 0.0,
-      "status": "failed",
-      "notes": "Technical error: llama_decode returned -1. Same issue as Q4 version."
-    }
-  ],
-  "key_insights": {
-    "champion": {
-      "model": "Qwen2.5-Coder-1.5B-Q4",
-      "score": 9.7,
-      "reason": "Code-specialized models excel at structured JSON generation. Near-perfect MCP capability."
-    },
-    "scaling_effect": {
-      "observation": "Increasing parameters from 0.5B to 1.5B more than doubled MCP score (4.3 → 9.7)",
-      "conclusion": "Parameter scaling works exceptionally well for code-specialized models"
-    },
-    "mcp_specialized_disappointment": {
-      "observation": "MCP-Instruct models completely failed. MCPR-L models scored only 3.7/10 at best.",
-      "conclusion": "MCP specialization alone is insufficient. Code training provides better foundation."
-    },
-    "quantization_impact": {
-      "observation": "Q2 quantization caused failures or poor performance. Q4 and Q8 worked well.",
-      "conclusion": "Avoid Q2 quantization for MCP tasks. Q4 offers best size/quality tradeoff."
-    },
-    "size_efficiency": {
-      "observation": "Gemma-3-270M (428MB) matched 3133MB model performance",
-      "conclusion": "Larger models don't guarantee better MCP performance"
-    }
-  },
-  "recommendations": {
-    "primary": {
-      "model": "Qwen2.5-Coder-1.5B-Q4",
-      "use_case": "Production deployments requiring high-quality MCP",
-      "requirement": "JSON extraction logic to handle markdown code blocks"
-    },
-    "budget": {
-      "model": "Qwen2.5-Coder-0.5B",
-      "use_case": "Resource-constrained environments",
-      "advantage": "2x smaller, 2x faster, still 4.3/10 performance"
-    },
-    "ultra_lightweight": {
-      "model": "Gemma-3-270M",
-      "use_case": "Edge devices, embedded systems",
-      "advantage": "Only 428MB, decent 3.7/10 performance"
-    },
-    "avoid": [
-      {
-        "model": "MCP-Instruct-v1 (all versions)",
-        "reason": "Technical incompatibility with llama.cpp"
-      },
-      {
-        "model": "Yi-Coder-1.5B",
-        "reason": "Prompt format incompatibility"
-      },
-      {
-        "model": "Any Q2 quantization",
-        "reason": "Too aggressive, causes failures or gibberish"
-      }
-    ]
-  }
-}

final_model_comparison.json

@@ -1,478 +0,0 @@
-{
-  "all_results": [
-    {
-      "name": "Qwen2.5-0.5B",
-      "avg_score": 4.0,
-      "avg_time": 3.52385942141215,
-      "efficiency": 1.1351190617011169,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 3.418940305709839,
-          "response": ".\n\nOutils: get_game_state(), move_units(unit_ids, ..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 3.8486745357513428,
-          "response": "La commande \"move_units\" est utilisée pour déplace..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 0,
-          "time": 3.3039634227752686,
-          "response": ", je vais faire une tâche de base. Je vais essayer..."
-        }
-      ]
-    },
-    {
-      "name": "Qwen3-0.6B",
-      "avg_score": 6.0,
-      "avg_time": 6.404076337814331,
-      "efficiency": 0.936903260283084,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 6.516923427581787,
-          "response": "Exemple: {\"tool\": \"get_game_state\", \"args\": {\"unit..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 7,
-          "time": 6.65591287612915,
-          "response": "Réponse: {\"tool\": \"move_units\", \"args\": {\"unit_ids..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 4,
-          "time": 6.039392709732056,
-          "response": ", but not too much. The user is asking for a respo..."
-        }
-      ]
-    },
-    {
-      "name": "Gemma-3-1B",
-      "avg_score": 4.0,
-      "avg_time": 6.960511525472005,
-      "efficiency": 0.5746704082540475,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 7.20223069190979,
-          "response": "```json\n{{\"tool\": \"get_game_state\", \"args\": {\"map\"..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 6.998988628387451,
-          "response": "```python\nimport json\n\ndef move_units(unit_ids, ta..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 0,
-          "time": 6.680315256118774,
-          "response": ".\n\nA. 100\nB. 200\nC. 300\nD. 400\nE. 500\nF. 600\nG. 70..."
-        }
-      ]
-    },
-    {
-      "name": "Gemma-3-270M",
-      "avg_score": 4.666666666666667,
-      "avg_time": 3.6990818977355957,
-      "efficiency": 1.2615743029434903,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 5,
-          "time": 3.697866201400757,
-          "response": "```\n**Explication:**\n\n* `get_game_state()` : Récup..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 3.690243721008301,
-          "response": "```\n\n**Explication:**\n\n* `move_units` est un outil..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 4,
-          "time": 3.7091357707977295,
-          "response": ".\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\n..."
-        }
-      ]
-    }
-  ],
-  "successful_models": [
-    {
-      "name": "Qwen2.5-0.5B",
-      "avg_score": 4.0,
-      "avg_time": 3.52385942141215,
-      "efficiency": 1.1351190617011169,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 3.418940305709839,
-          "response": ".\n\nOutils: get_game_state(), move_units(unit_ids, ..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 3.8486745357513428,
-          "response": "La commande \"move_units\" est utilisée pour déplace..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 0,
-          "time": 3.3039634227752686,
-          "response": ", je vais faire une tâche de base. Je vais essayer..."
-        }
-      ]
-    },
-    {
-      "name": "Qwen3-0.6B",
-      "avg_score": 6.0,
-      "avg_time": 6.404076337814331,
-      "efficiency": 0.936903260283084,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 6.516923427581787,
-          "response": "Exemple: {\"tool\": \"get_game_state\", \"args\": {\"unit..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 7,
-          "time": 6.65591287612915,
-          "response": "Réponse: {\"tool\": \"move_units\", \"args\": {\"unit_ids..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 4,
-          "time": 6.039392709732056,
-          "response": ", but not too much. The user is asking for a respo..."
-        }
-      ]
-    },
-    {
-      "name": "Gemma-3-1B",
-      "avg_score": 4.0,
-      "avg_time": 6.960511525472005,
-      "efficiency": 0.5746704082540475,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 7.20223069190979,
-          "response": "```json\n{{\"tool\": \"get_game_state\", \"args\": {\"map\"..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 6.998988628387451,
-          "response": "```python\nimport json\n\ndef move_units(unit_ids, ta..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 0,
-          "time": 6.680315256118774,
-          "response": ".\n\nA. 100\nB. 200\nC. 300\nD. 400\nE. 500\nF. 600\nG. 70..."
-        }
-      ]
-    },
-    {
-      "name": "Gemma-3-270M",
-      "avg_score": 4.666666666666667,
-      "avg_time": 3.6990818977355957,
-      "efficiency": 1.2615743029434903,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 5,
-          "time": 3.697866201400757,
-          "response": "```\n**Explication:**\n\n* `get_game_state()` : Récup..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 3.690243721008301,
-          "response": "```\n\n**Explication:**\n\n* `move_units` est un outil..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 4,
-          "time": 3.7091357707977295,
-          "response": ".\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\n..."
-        }
-      ]
-    }
-  ],
-  "ranking_by_score": [
-    {
-      "name": "Qwen3-0.6B",
-      "avg_score": 6.0,
-      "avg_time": 6.404076337814331,
-      "efficiency": 0.936903260283084,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 6.516923427581787,
-          "response": "Exemple: {\"tool\": \"get_game_state\", \"args\": {\"unit..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 7,
-          "time": 6.65591287612915,
-          "response": "Réponse: {\"tool\": \"move_units\", \"args\": {\"unit_ids..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 4,
-          "time": 6.039392709732056,
-          "response": ", but not too much. The user is asking for a respo..."
-        }
-      ]
-    },
-    {
-      "name": "Gemma-3-270M",
-      "avg_score": 4.666666666666667,
-      "avg_time": 3.6990818977355957,
-      "efficiency": 1.2615743029434903,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 5,
-          "time": 3.697866201400757,
-          "response": "```\n**Explication:**\n\n* `get_game_state()` : Récup..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 3.690243721008301,
-          "response": "```\n\n**Explication:**\n\n* `move_units` est un outil..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 4,
-          "time": 3.7091357707977295,
-          "response": ".\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\n..."
-        }
-      ]
-    },
-    {
-      "name": "Qwen2.5-0.5B",
-      "avg_score": 4.0,
-      "avg_time": 3.52385942141215,
-      "efficiency": 1.1351190617011169,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 3.418940305709839,
-          "response": ".\n\nOutils: get_game_state(), move_units(unit_ids, ..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 3.8486745357513428,
-          "response": "La commande \"move_units\" est utilisée pour déplace..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 0,
-          "time": 3.3039634227752686,
-          "response": ", je vais faire une tâche de base. Je vais essayer..."
-        }
-      ]
-    },
-    {
-      "name": "Gemma-3-1B",
-      "avg_score": 4.0,
-      "avg_time": 6.960511525472005,
-      "efficiency": 0.5746704082540475,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 7.20223069190979,
-          "response": "```json\n{{\"tool\": \"get_game_state\", \"args\": {\"map\"..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 6.998988628387451,
-          "response": "```python\nimport json\n\ndef move_units(unit_ids, ta..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 0,
-          "time": 6.680315256118774,
-          "response": ".\n\nA. 100\nB. 200\nC. 300\nD. 400\nE. 500\nF. 600\nG. 70..."
-        }
-      ]
-    }
-  ],
-  "ranking_by_efficiency": [
-    {
-      "name": "Gemma-3-270M",
-      "avg_score": 4.666666666666667,
-      "avg_time": 3.6990818977355957,
-      "efficiency": 1.2615743029434903,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 5,
-          "time": 3.697866201400757,
-          "response": "```\n**Explication:**\n\n* `get_game_state()` : Récup..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 3.690243721008301,
-          "response": "```\n\n**Explication:**\n\n* `move_units` est un outil..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 4,
-          "time": 3.7091357707977295,
-          "response": ".\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\n..."
-        }
-      ]
-    },
-    {
-      "name": "Qwen2.5-0.5B",
-      "avg_score": 4.0,
-      "avg_time": 3.52385942141215,
-      "efficiency": 1.1351190617011169,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 3.418940305709839,
-          "response": ".\n\nOutils: get_game_state(), move_units(unit_ids, ..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 3.8486745357513428,
-          "response": "La commande \"move_units\" est utilisée pour déplace..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 0,
-          "time": 3.3039634227752686,
-          "response": ", je vais faire une tâche de base. Je vais essayer..."
-        }
-      ]
-    },
-    {
-      "name": "Qwen3-0.6B",
-      "avg_score": 6.0,
-      "avg_time": 6.404076337814331,
-      "efficiency": 0.936903260283084,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 6.516923427581787,
-          "response": "Exemple: {\"tool\": \"get_game_state\", \"args\": {\"unit..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 7,
-          "time": 6.65591287612915,
-          "response": "Réponse: {\"tool\": \"move_units\", \"args\": {\"unit_ids..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 4,
-          "time": 6.039392709732056,
-          "response": ", but not too much. The user is asking for a respo..."
-        }
-      ]
-    },
-    {
-      "name": "Gemma-3-1B",
-      "avg_score": 4.0,
-      "avg_time": 6.960511525472005,
-      "efficiency": 0.5746704082540475,
-      "tests": [
-        {
-          "test": "Commande simple",
-          "score": 7,
-          "time": 7.20223069190979,
-          "response": "```json\n{{\"tool\": \"get_game_state\", \"args\": {\"map\"..."
-        },
-        {
-          "test": "Action avec paramètres",
-          "score": 5,
-          "time": 6.998988628387451,
-          "response": "```python\nimport json\n\ndef move_units(unit_ids, ta..."
-        },
-        {
-          "test": "Vitesse de réponse",
-          "score": 0,
-          "time": 6.680315256118774,
-          "response": ".\n\nA. 100\nB. 200\nC. 300\nD. 400\nE. 500\nF. 600\nG. 70..."
-        }
-      ]
-    }
-  ],
-  "best_overall": {
-    "name": "Qwen3-0.6B",
-    "avg_score": 6.0,
-    "avg_time": 6.404076337814331,
-    "efficiency": 0.936903260283084,
-    "tests": [
-      {
-        "test": "Commande simple",
-        "score": 7,
-        "time": 6.516923427581787,
-        "response": "Exemple: {\"tool\": \"get_game_state\", \"args\": {\"unit..."
-      },
-      {
-        "test": "Action avec paramètres",
-        "score": 7,
-        "time": 6.65591287612915,
-        "response": "Réponse: {\"tool\": \"move_units\", \"args\": {\"unit_ids..."
-      },
-      {
-        "test": "Vitesse de réponse",
-        "score": 4,
-        "time": 6.039392709732056,
-        "response": ", but not too much. The user is asking for a respo..."
-      }
-    ]
-  },
-  "most_efficient": {
-    "name": "Gemma-3-270M",
-    "avg_score": 4.666666666666667,
-    "avg_time": 3.6990818977355957,
-    "efficiency": 1.2615743029434903,
-    "tests": [
-      {
-        "test": "Commande simple",
-        "score": 5,
-        "time": 3.697866201400757,
-        "response": "```\n**Explication:**\n\n* `get_game_state()` : Récup..."
-      },
-      {
-        "test": "Action avec paramètres",
-        "score": 5,
-        "time": 3.690243721008301,
-        "response": "```\n\n**Explication:**\n\n* `move_units` est un outil..."
-      },
-      {
-        "test": "Vitesse de réponse",
-        "score": 4,
-        "time": 3.7091357707977295,
-        "response": ".\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\nOK.\n..."
-      }
-    ]
-  }
-}

final_model_comparison.py

@@ -1,263 +0,0 @@
-"""
-Évaluation finale comparative de tous les modèles pour MCP
-Inclut le nouveau Gemma-3-270M plus petit et potentiellement plus rapide
-"""
-
-import sys
-import os
-import json
-import time
-
-# Ajouter le chemin pour les imports
-sys.path.append(os.path.dirname(os.path.abspath(__file__)))
-
-def test_model_comprehensive(model_path, model_name):
-    """Test complet d'un modèle pour les tâches MCP"""
-    
-    try:
-        from llama_cpp import Llama
-        
-        print(f"🔄 Test de {model_name}...")
-        
-        # Initialiser avec des paramètres optimisés
-        llm = Llama(
-            model_path=model_path,
-            n_ctx=1024,
-            n_threads=1,
-            verbose=False
-        )
-        
-        # Tests MCP variés
-        tests = [
-            {
-                "name": "Commande simple",
-                "prompt": """Tu es un assistant IA pour un jeu RTS via MCP.
-
-Outils: get_game_state(), move_units(unit_ids, target_x, target_y)
-
-Commande: "Montre-moi l'état du jeu"
-
-Réponds avec JSON: {{"tool": "nom_outil", "args": {{}}}}""",
-                "expected": "get_game_state"
-            },
-            {
-                "name": "Action avec paramètres",
-                "prompt": """Outils: move_units(unit_ids, target_x, target_y)
-
-Commande: "Déplace l'infanterie vers 100,200"
-
-JSON: {{"tool": "move_units", "args": {{"unit_ids": ["infantry"], "target_x": 100, "target_y": 200}}}}""",
-                "expected": "move_units"
-            },
-            {
-                "name": "Vitesse de réponse",
-                "prompt": "Réponds simplement: OK",
-                "expected": "OK"
-            }
-        ]
-        
-        total_score = 0
-        total_time = 0
-        results = []
-        
-        for test in tests:
-            start_time = time.time()
-            
-            response = llm(
-                test['prompt'],
-                max_tokens=80,
-                temperature=0.1,
-                stop=["</s>", "<|im_end|>"]
-            )
-            
-            response_time = time.time() - start_time
-            response_text = response['choices'][0]['text'].strip()
-            
-            # Noter la réponse
-            score = 0
-            
-            # JSON valide pour les tests MCP
-            if test['name'] != "Vitesse de réponse":
-                try:
-                    json.loads(response_text)
-                    score += 3
-                except:
-                    pass
-            
-            # Contenu attendu
-            if test['expected'] in response_text:
-                score += 4
-            
-            # Format approprié
-            if "tool" in response_text and test['name'] != "Vitesse de réponse":
-                score += 2
-            
-            # Cohérence
-            if any(word in response_text.lower() for word in ['game', 'move', 'state']):
-                score += 1
-            
-            score = min(score, 10)
-            
-            total_score += score
-            total_time += response_time
-            
-            results.append({
-                'test': test['name'],
-                'score': score,
-                'time': response_time,
-                'response': response_text[:50] + "..." if len(response_text) > 50 else response_text
-            })
-        
-        avg_score = total_score / len(tests)
-        avg_time = total_time / len(tests)
-        
-        print(f"✅ {model_name}: {avg_score:.1f}/10 | Temps: {avg_time:.2f}s")
-        
-        return {
-            'name': model_name,
-            'avg_score': avg_score,
-            'avg_time': avg_time,
-            'efficiency': avg_score / avg_time if avg_time > 0 else 0,
-            'tests': results
-        }
-        
-    except Exception as e:
-        print(f"❌ {model_name}: Erreur - {e}")
-        return {
-            'name': model_name,
-            'avg_score': 0,
-            'avg_time': 0,
-            'efficiency': 0,
-            'error': str(e)
-        }
-
-def main():
-    """Évaluation finale comparative"""
-    
-    print("🏁 ÉVALUATION FINALE COMPARATIVE MCP")
-    print("=" * 60)
-    
-    # Tous les modèles à tester
-    models = [
-        {
-            'name': 'Qwen2.5-0.5B',
-            'path': 'qwen2.5-0.5b-instruct-q4_0.gguf'
-        },
-        {
-            'name': 'Qwen3-0.6B',
-            'path': 'Qwen3-0.6B-Q8_0.gguf'
-        },
-        {
-            'name': 'Gemma-3-1B',
-            'path': 'google_gemma-3-1b-it-qat-Q4_0.gguf'
-        },
-        {
-            'name': 'Gemma-3-270M',
-            'path': 'gemma-3-270m-it-qat-Q8_0.gguf'
-        }
-    ]
-    
-    results = []
-    
-    for model in models:
-        if os.path.exists(model['path']):
-            result = test_model_comprehensive(model['path'], model['name'])
-            results.append(result)
-        else:
-            print(f"❌ Fichier non trouvé: {model['path']}")
-    
-    # Analyse comparative
-    print("\n" + "=" * 60)
-    print("📊 RÉSULTATS FINAUX")
-    print("=" * 60)
-    
-    successful_results = [r for r in results if 'error' not in r and r['avg_score'] > 0]
-    
-    if successful_results:
-        # Classement par score
-        sorted_by_score = sorted(successful_results, key=lambda x: x['avg_score'], reverse=True)
-        
-        print(f"\n🏆 CLASSEMENT PAR PERFORMANCE:")
-        for i, result in enumerate(sorted_by_score, 1):
-            file_size = os.path.getsize([m['path'] for m in models if m['name'] == result['name']][0]) / (1024*1024)
-            print(f"   {i}. {result['name']}: {result['avg_score']:.1f}/10 | {result['avg_time']:.2f}s | {file_size:.0f}MB")
-        
-        # Classement par efficacité (score/seconde)
-        sorted_by_efficiency = sorted(successful_results, key=lambda x: x['efficiency'], reverse=True)
-        
-        print(f"\n⚡ CLASSEMENT PAR EFFICACITÉ:")
-        for i, result in enumerate(sorted_by_efficiency, 1):
-            file_size = os.path.getsize([m['path'] for m in models if m['name'] == result['name']][0]) / (1024*1024)
-            print(f"   {i}. {result['name']}: {result['efficiency']:.2f} score/s | {file_size:.0f}MB")
-        
-        # Meilleur modèle global
-        best_overall = sorted_by_score[0]
-        most_efficient = sorted_by_efficiency[0]
-        
-        print(f"\n🎯 MEILLEUR MODÈLE GLOBAL: {best_overall['name']}")
-        print(f"   Score: {best_overall['avg_score']:.1f}/10")
-        print(f"   Temps: {best_overall['avg_time']:.2f}s")
-        
-        print(f"\n⚡ MODÈLE LE PLUS EFFICACE: {most_efficient['name']}")
-        print(f"   Efficacité: {most_efficient['efficiency']:.2f} score/s")
-        
-        # Analyse détaillée
-        print(f"\n📈 ANALYSE DÉTAILLÉE:")
-        
-        for result in successful_results:
-            file_size = os.path.getsize([m['path'] for m in models if m['name'] == result['name']][0]) / (1024*1024)
-            efficiency_per_mb = result['efficiency'] / file_size if file_size > 0 else 0
-            
-            print(f"\n🔹 {result['name']}:")
-            print(f"   Score moyen: {result['avg_score']:.1f}/10")
-            print(f"   Temps moyen: {result['avg_time']:.2f}s")
-            print(f"   Efficacité: {result['efficiency']:.2f} score/s")
-            print(f"   Taille: {file_size:.0f}MB")
-            print(f"   Efficacité/MB: {efficiency_per_mb:.4f}")
-            
-            # Tests individuels
-            for test in result['tests']:
-                status = "✅" if test['score'] >= 6 else "⚠️" if test['score'] >= 4 else "❌"
-                print(f"     {status} {test['test']}: {test['score']}/10 ({test['time']:.2f}s)")
-        
-        # Recommandations finales
-        print(f"\n💡 RECOMMANDATIONS FINALES:")
-        
-        if best_overall['avg_score'] >= 7:
-            print(f"✅ {best_overall['name']} est EXCELLENT pour la production MCP")
-        elif best_overall['avg_score'] >= 5:
-            print(f"👍 {best_overall['name']} est BON pour la production MCP")
-        else:
-            print(f"⚠️  {best_overall['name']} nécessite des améliorations")
-        
-        # Recommandation basée sur l'usage
-        print(f"\n🎯 RECOMMANDATIONS SPÉCIFIQUES:")
-        
-        if most_efficient['name'] != best_overall['name']:
-            print(f"⚡ Pour les réponses rapides: {most_efficient['name']}")
-            print(f"🏆 Pour la meilleure qualité: {best_overall['name']}")
-        else:
-            print(f"🎉 {best_overall['name']} est le meilleur choix pour la vitesse ET la qualité")
-        
-        # Vérifier si Gemma-3-270M est surprenant
-        gemma_270m = next((r for r in successful_results if r['name'] == 'Gemma-3-270M'), None)
-        if gemma_270m and gemma_270m['avg_score'] >= 5:
-            print(f"🚀 Surprise: Gemma-3-270M offre un excellent rapport taille/performance!")
-    
-    # Sauvegarder résultats complets
-    final_results = {
-        'all_results': results,
-        'successful_models': successful_results,
-        'ranking_by_score': sorted_by_score if successful_results else [],
-        'ranking_by_efficiency': sorted_by_efficiency if successful_results else [],
-        'best_overall': best_overall if successful_results else None,
-        'most_efficient': most_efficient if successful_results else None
-    }
-    
-    with open("final_model_comparison.json", "w", encoding="utf-8") as f:
-        json.dump(final_results, f, indent=2, ensure_ascii=False)
-    
-    print(f"\n📄 Résultats complets sauvegardés dans: final_model_comparison.json")
-
-if __name__ == "__main__":
-    main()

investigate_mcp_failures.py

@@ -1,181 +0,0 @@
-"""
-Investigation détaillée des échecs des modèles spécialisés MCP
-Analyse pourquoi MCP-Instruct-v1, Gemma-3n-E2B-it et MCPR L-3B-Exa échouent
-"""
-
-import sys
-import os
-import json
-import time
-from llama_cpp import Llama
-
-def analyze_model_failures():
-    """Analyser en détail les échecs des modèles MCP"""
-    
-    print("🔍 INVESTIGATION DÉTAILLÉE DES ÉCHECS MCP")
-    print("=" * 70)
-    
-    # Modèles problématiques avec analyse détaillée
-    problematic_models = [
-        {
-            'name': 'MCP-Instruct-v1',
-            'path': 'mcp-instruct-v1.Q4_K_M.gguf',
-            'size_mb': 697.0,
-            'issues': ['llama_decode returned -1'],
-            'hypothesis': 'Incompatibilité technique avec llama-cpp-python'
-        },
-        {
-            'name': 'MCPR L-3B-Exa', 
-            'path': 'mcprl-3b-exa.Q2_K.gguf',
-            'size_mb': 1215.7,
-            'issues': ['texte corrompu', 'caractères spéciaux', 'sortie incohérente'],
-            'hypothesis': 'Quantisation Q2_K trop agressive pour ce modèle'
-        },
-        {
-            'name': 'Gemma-3n-E2B-it',
-            'path': 'gemma-3n-E2B-it-UD-IQ2_XXS.gguf',
-            'size_mb': 1958.3,
-            'issues': ['réponses vides', 'pas de sortie'],
-            'hypothesis': 'Format de prompt incompatible avec le modèle'
-        }
-    ]
-    
-    # Analyse technique détaillée
-    print("\n📊 ANALYSE TECHNIQUE DÉTAILLÉE:")
-    
-    for model in problematic_models:
-        print(f"\n🧩 {model['name']}:")
-        print(f"   📏 Taille: {model['size_mb']:.1f} MB")
-        print(f"   🔧 Problèmes: {', '.join(model['issues'])}")
-        print(f"   💭 Hypothèse: {model['hypothesis']}")
-        
-        if not os.path.exists(model['path']):
-            print(f"   ❌ Fichier non trouvé")
-            continue
-            
-        # Test de chargement basique
-        print(f"   🔄 Test de chargement...")
-        try:
-            llm = Llama(
-                model_path=model['path'],
-                n_ctx=2048,
-                n_threads=1,
-                verbose=False,
-                n_gpu_layers=0
-            )
-            print(f"   ✅ Chargement réussi")
-            
-            # Test de génération très simple
-            print(f"   🧪 Test de génération basique...")
-            try:
-                response = llm(
-                    "Hello",
-                    max_tokens=10,
-                    temperature=0.1
-                )
-                
-                # Extraire la réponse correctement
-                response_text = str(response)
-                if hasattr(response, 'choices') and response.choices:
-                    response_text = response.choices[0].text if hasattr(response.choices[0], 'text') else str(response.choices[0])
-                elif isinstance(response, dict) and 'choices' in response:
-                    choice = response['choices'][0]
-                    response_text = choice.get('text', str(choice))
-                
-                print(f"   📝 Réponse: '{response_text.strip()}'")
-                
-            except Exception as e:
-                print(f"   ❌ Erreur génération: {e}")
-                
-        except Exception as e:
-            print(f"   ❌ Erreur chargement: {e}")
-    
-    # Analyse comparative avec modèle qui fonctionne
-    print(f"\n🔍 COMPARAISON AVEC MODÈLE FONCTIONNEL:")
-    
-    working_model = 'qwen2.5-coder-0.5b-instruct-q4_0.gguf'
-    if os.path.exists(working_model):
-        print(f"\n✅ Qwen2.5-Coder-0.5B (fonctionne):")
-        
-        try:
-            llm = Llama(
-                model_path=working_model,
-                n_ctx=1024,
-                n_threads=1,
-                verbose=False,
-                n_gpu_layers=0
-            )
-            
-            # Test avec le même prompt MCP
-            prompt = '''You are an AI assistant for an RTS game using MCP (Model Context Protocol).
-
-Available tools:
-- get_game_state()
-- move_units(unit_ids, target_x, target_y)
-- attack_unit(attacker_ids, target_id)
-- build_building(building_type, position_x, position_y)
-
-User command: "show game state"
-
-Respond with JSON only: {"tool": "tool_name", "args": {}}}'''
-            
-            response = llm(
-                prompt,
-                max_tokens=50,
-                temperature=0.1
-            )
-            
-            # Extraire la réponse correctement
-            response_text = str(response)
-            if hasattr(response, 'choices') and response.choices:
-                response_text = response.choices[0].text if hasattr(response.choices[0], 'text') else str(response.choices[0])
-            elif isinstance(response, dict) and 'choices' in response:
-                choice = response['choices'][0]
-                response_text = choice.get('text', str(choice))
-            
-            print(f"   📝 Réponse: {response_text[:100]}...")
-            
-            # Vérifier JSON
-            try:
-                json.loads(response_text)
-                print(f"   ✅ JSON valide")
-            except:
-                print(f"   ❌ JSON invalide")
-                
-        except Exception as e:
-            print(f"   ❌ Erreur: {e}")
-    
-    # Conclusions et recommandations
-    print(f"\n🎯 CONCLUSIONS ET RECOMMANDATIONS:")
-    print("=" * 70)
-    
-    print("\n1. MCP-INSTRUCT-V1 (Q4_K_M):")
-    print("   ❌ Problème: Erreur technique 'llama_decode returned -1'")
-    print("   💡 Solution: Essayer version Q8_0 ou vérifier compatibilité llama-cpp-python")
-    
-    print("\n2. MCPR L-3B-EXA (Q2_K):")
-    print("   ❌ Problème: Texte corrompu avec caractères spéciaux")
-    print("   💡 Solution: Quantisation Q2_K trop agressive, essayer Q4_K_M ou Q8_0")
-    
-    print("\n3. GEMMA-3N-E2B-IT (IQ2_XXS):")
-    print("   ❌ Problème: Réponses vides, modèle ne répond pas")
-    print("   💡 Solution: Quantisation IQ2_XXS extrême, essayer version moins compressée")
-    
-    print("\n4. POURQUOI LES MODÈLES CODE FONCTIONNENT MIEUX:")
-    print("   ✅ Habitués au format JSON et aux structures de données")
-    print("   ✅ Meilleure compréhension des formats structurés")
-    print("   ✅ Formation sur du code et des données techniques")
-    
-    print("\n🚀 PLAN D'ACTION:")
-    print("   1. Télécharger versions Q8_0 des modèles problématiques")
-    print("   2. Tester avec formats de prompts MCP spécifiques")
-    print("   3. Augmenter contexte (n_ctx) pour modèles plus grands")
-    print("   4. Utiliser température légèrement plus élevée (0.3)")
-    
-    print(f"\n📋 MODÈLES À TÉLÉCHARGER:")
-    print("   • mcp-instruct-v1.Q8_0.gguf")
-    print("   • mcprl-3b-exa.Q8_0.gguf") 
-    print("   • google_gemma-3n-E2B-it-Q8_0.gguf")
-
-if __name__ == "__main__":
-    analyze_model_failures()

practical_mcp_results.json

@@ -1,149 +0,0 @@
-[
-  {
-    "name": "Qwen2.5-0.5B",
-    "avg_score": 3.0,
-    "avg_time": 3.1588314941951205,
-    "results": [
-      {
-        "command": "show game state",
-        "score": 4,
-        "time": 4.286122560501099,
-        "response": "Here is the response in JSON format:\n\n```json\n{\n  "
-      },
-      {
-        "command": "move infantry to 100,200",
-        "score": 4,
-        "time": 4.499698638916016,
-        "response": "In this case, the tool is \"move_units\" and the arg"
-      },
-      {
-        "command": "attack enemy tank at 300,150",
-        "score": 5,
-        "time": 4.139246940612793,
-        "response": "In this case, the tool is \"attack_unit\" and the ar"
-      },
-      {
-        "command": "build barracks near 250,200",
-        "score": 3,
-        "time": 2.345900774002075,
-        "response": ".\n```json\n{\n  \"tool\": \"build_building\",\n  \"args\": "
-      },
-      {
-        "command": "defend base with all units",
-        "score": 1,
-        "time": 1.6794795989990234,
-        "response": ".\n```json\n{\n  \"tool\": \"move_units\",\n  \"args\": {\n  "
-      },
-      {
-        "command": "harvest resources with harvester",
-        "score": 2,
-        "time": 1.0160574913024902,
-        "response": ".\n```json\n{\n  \"tool\": \"get_game_state\",\n  \"args\": "
-      },
-      {
-        "command": "scout enemy positions",
-        "score": 2,
-        "time": 4.145314455032349,
-        "response": "In this scenario, the user command is \"scout enemy"
-      }
-    ]
-  },
-  {
-    "name": "Qwen3-0.6B",
-    "avg_score": 2.142857142857143,
-    "avg_time": 7.839082138878958,
-    "results": [
-      {
-        "command": "show game state",
-        "score": 4,
-        "time": 8.01675820350647,
-        "response": "If the user command is not available, respond with"
-      },
-      {
-        "command": "move infantry to 100,200",
-        "score": 3,
-        "time": 7.884659290313721,
-        "response": "{\n  \"tool\": \"move_units\",\n  \"args\": {\n    \"unit_id"
-      },
-      {
-        "command": "attack enemy tank at 300,150",
-        "score": 3,
-        "time": 7.239688873291016,
-        "response": "{\n  \"tool\": \"attack_unit\",\n  \"args\": {\n    \"attack"
-      },
-      {
-        "command": "build barracks near 250,200",
-        "score": 1,
-        "time": 7.6789350509643555,
-        "response": "The correct answer is {\"tool\": \"build\", \"args\": {\""
-      },
-      {
-        "command": "defend base with all units",
-        "score": 1,
-        "time": 8.047882318496704,
-        "response": "Possible tool names: build, get_game_state, move, "
-      },
-      {
-        "command": "harvest resources with harvester",
-        "score": 1,
-        "time": 8.035732746124268,
-        "response": "The correct answer is {\"tool\": \"harvest\", \"args\": "
-      },
-      {
-        "command": "scout enemy positions",
-        "score": 2,
-        "time": 7.969918489456177,
-        "response": "The user's command is \"scout enemy positions\". Wha"
-      }
-    ]
-  },
-  {
-    "name": "Gemma-3-270M",
-    "avg_score": 1.4285714285714286,
-    "avg_time": 4.817336320877075,
-    "results": [
-      {
-        "command": "show game state",
-        "score": 1,
-        "time": 4.833978652954102,
-        "response": "- show game state\n- move_units(unit_ids, target_x,"
-      },
-      {
-        "command": "move infantry to 100,200",
-        "score": 2,
-        "time": 4.697685718536377,
-        "response": "```\n`move infantry to 100,200`\n\n**Explanation:**\n\n"
-      },
-      {
-        "command": "attack enemy tank at 300,150",
-        "score": 3,
-        "time": 4.781869649887085,
-        "response": "```\n**Explanation:**\n\n* `get_game_state()`: Retrie"
-      },
-      {
-        "command": "build barracks near 250,200",
-        "score": 3,
-        "time": 4.599961996078491,
-        "response": "```\n`build barracks near 250,200`\n```\nJSON:\n```\n{\""
-      },
-      {
-        "command": "defend base with all units",
-        "score": 0,
-        "time": 4.869190216064453,
-        "response": "- `tool_name` is a string\n- `args` is a dictionary"
-      },
-      {
-        "command": "harvest resources with harvester",
-        "score": 0,
-        "time": 4.996307849884033,
-        "response": "```\n- `harvest resources with harvester`\n- `harves"
-      },
-      {
-        "command": "scout enemy positions",
-        "score": 1,
-        "time": 4.942360162734985,
-        "response": "```\n`scout enemy positions`\n- `tool`: \"scout\"\n- `a"
-      }
-    ]
-  }
-]

practical_mcp_test.py

@@ -1,197 +0,0 @@
-"""
-Test pratique MCP pour évaluer les capacités réelles des modèles
-Simule l'usage réel dans un jeu RTS avec des commandes typiques
-"""
-
-import sys
-import os
-import json
-import time
-
-sys.path.append(os.path.dirname(os.path.abspath(__file__)))
-
-def test_model_practical(model_path, model_name):
-    """Test pratique avec des commandes RTS réelles"""
-    
-    try:
-        from llama_cpp import Llama
-        
-        print(f"🎮 Test pratique de {model_name}...")
-        
-        llm = Llama(
-            model_path=model_path,
-            n_ctx=1024,
-            n_threads=1,
-            verbose=False
-        )
-        
-        # Commandes RTS typiques qu'un utilisateur taperait
-        real_commands = [
-            "show game state",
-            "move infantry to 100,200", 
-            "attack enemy tank at 300,150",
-            "build barracks near 250,200",
-            "defend base with all units",
-            "harvest resources with harvester",
-            "scout enemy positions"
-        ]
-        
-        results = []
-        
-        for command in real_commands:
-            prompt = f"""You control an RTS game via MCP. Available tools:
-- get_game_state()
-- move_units(unit_ids, target_x, target_y)  
-- attack_unit(attacker_ids, target_id)
-- build_building(building_type, position_x, position_y)
-
-User command: "{command}"
-
-Respond with JSON: {{"tool": "tool_name", "args": {{}}}}"""
-            
-            start_time = time.time()
-            
-            response = llm(
-                prompt,
-                max_tokens=100,
-                temperature=0.1
-            )
-            
-            response_time = time.time() - start_time
-            
-            # Extraire et évaluer
-            try:
-                response_text = response['choices'][0]['text'].strip()
-            except:
-                response_text = str(response)
-            
-            # Évaluation simple mais réaliste
-            score = 0
-            
-            # JSON valide?
-            try:
-                json.loads(response_text)
-                score += 3
-            except:
-                # Chercher JSON dans le texte
-                import re
-                if re.search(r'\{.*\}', response_text):
-                    score += 1
-            
-            # Outil approprié?
-            if "game state" in command and "get_game_state" in response_text:
-                score += 3
-            elif "move" in command and "move_units" in response_text:
-                score += 3
-            elif "attack" in command and "attack_unit" in response_text:
-                score += 3
-            elif "build" in command and "build_building" in response_text:
-                score += 3
-            elif any(tool in response_text for tool in ['get_game_state', 'move_units', 'attack_unit', 'build_building']):
-                score += 1
-            
-            # Paramètres raisonnables?
-            if "100,200" in response_text or "300,150" in response_text or "250,200" in response_text:
-                score += 2
-            
-            score = min(score, 10)
-            
-            print(f"   '{command}' → {score}/10 ({response_time:.2f}s)")
-            
-            results.append({
-                'command': command,
-                'score': score,
-                'time': response_time,
-                'response': response_text[:50]
-            })
-        
-        avg_score = sum(r['score'] for r in results) / len(results)
-        avg_time = sum(r['time'] for r in results) / len(results)
-        
-        print(f"📊 {model_name}: {avg_score:.1f}/10 moyen | {avg_time:.2f}s moyen")
-        
-        return {
-            'name': model_name,
-            'avg_score': avg_score,
-            'avg_time': avg_time,
-            'results': results
-        }
-        
-    except Exception as e:
-        print(f"❌ Erreur {model_name}: {e}")
-        return None
-
-def main():
-    """Test pratique comparatif"""
-    
-    print("🎯 TEST PRATIQUE MCP - COMMANDES RTS RÉELLES")
-    print("=" * 50)
-    print("Simulation de l'usage réel avec des commandes typiques")
-    print("=" * 50)
-    
-    models = [
-        {'name': 'Qwen2.5-0.5B', 'path': 'qwen2.5-0.5b-instruct-q4_0.gguf'},
-        {'name': 'Qwen3-0.6B', 'path': 'Qwen3-0.6B-Q8_0.gguf'},
-        {'name': 'Gemma-3-270M', 'path': 'gemma-3-270m-it-qat-Q8_0.gguf'}
-    ]
-    
-    results = []
-    
-    for model in models:
-        if os.path.exists(model['path']):
-            result = test_model_practical(model['path'], model['name'])
-            if result:
-                results.append(result)
-            print()
-    
-    # Analyse pratique
-    if results:
-        print("📊 RÉSULTATS PRATIQUES:")
-        print("-" * 30)
-        
-        sorted_results = sorted(results, key=lambda x: x['avg_score'], reverse=True)
-        
-        for i, result in enumerate(sorted_results, 1):
-            print(f"{i}. {result['name']}: {result['avg_score']:.1f}/10")
-        
-        # Recommandation pratique
-        best = sorted_results[0]
-        
-        print(f"\n🎯 RECOMMANDATION PRATIQUE:")
-        
-        if best['avg_score'] >= 7:
-            print(f"✅ {best['name']} est EXCELLENT pour la production")
-            print("   → Gère bien les commandes RTS typiques")
-        elif best['avg_score'] >= 5:
-            print(f"👍 {best['name']} est UTILISABLE avec validation")
-            print("   → Fonctionne pour les commandes simples")
-        else:
-            print(f"⚠️  {best['name']} nécessite des améliorations")
-            print("   → Considérer prompts plus spécifiques")
-        
-        # Analyse des commandes réussies
-        print(f"\n📈 COMMANDES LES MIEUX GÉRÉES:")
-        
-        for result in results:
-            best_commands = [r for r in result['results'] if r['score'] >= 7]
-            if best_commands:
-                print(f"\n{result['name']}:")
-                for cmd in best_commands[:3]:  # Top 3
-                    print(f"   • {cmd['command']}: {cmd['score']}/10")
-        
-        # Conclusion sur la pertinence du test
-        print(f"\n🔍 PERTINENCE DU TEST:")
-        print("✅ Ce test est BEAUCOUP plus représentatif:")
-        print("   • Commandes réelles d'utilisateurs")
-        print("   • Format de sortie JSON attendu") 
-        print("   • Temps de réponse réaliste")
-        print("   • Pas de prompts artificiels complexes")
-        
-        # Sauvegarder
-        with open("practical_mcp_results.json", "w") as f:
-            json.dump(results, f, indent=2)
-        
-        print(f"\n📄 Résultats sauvegardés: practical_mcp_results.json")
-
-if __name__ == "__main__":
-    main()

prompt_format_investigation.py

@@ -1,156 +0,0 @@
-"""
-Investigation des formats de prompt pour modèles spécialisés MCP
-Test de différents formats pour identifier les problèmes de compatibilité
-"""
-
-import os
-import json
-import re
-
-def test_prompt_formats():
-    """Tester différents formats de prompt pour diagnostiquer les problèmes"""
-    
-    print("🧪 INVESTIGATION DES FORMATS DE PROMPT MCP")
-    print("=" * 60)
-    
-    # Formats de prompt à tester
-    prompt_formats = [
-        {
-            'name': 'Format actuel (simple)',
-            'template': '''You are an AI assistant for an RTS game using MCP (Model Context Protocol).
-
-Available tools:
-- get_game_state()
-- move_units(unit_ids, target_x, target_y)
-- attack_unit(attacker_ids, target_id)
-- build_building(building_type, position_x, position_y)
-
-User command: "{command}"
-
-Respond with JSON only: {{"tool": "tool_name", "args": {{}}}}'''
-        },
-        {
-            'name': 'Format MCP structuré',
-            'template': '''<|im_start|>system
-You are an MCP assistant for an RTS game. Respond with ONLY JSON.
-Available tools: get_game_state, move_units, attack_unit, build_building<|im_end|>
-<|im_start|>user
-{command}<|im_end|>
-<|im_start|>assistant
-'''
-        },
-        {
-            'name': 'Format instruct',
-            'template': '''[INST] You are an MCP assistant. Respond with ONLY JSON format:
-{{"tool": "tool_name", "args": {{}}}}
-
-Command: {command} [/INST]'''
-        },
-        {
-            'name': 'Format code',
-            'template': '''// MCP assistant for RTS game
-// Tools: get_game_state(), move_units(), attack_unit(), build_building()
-// Command: {command}
-// Response:'''
-        },
-        {
-            'name': 'Format minimal',
-            'template': '''Tools: get_game_state, move_units, attack_unit, build_building
-Command: {command}
-JSON:'''
-        }
-    ]
-    
-    test_command = "show game state"
-    
-    print(f"\n📝 COMMANDE DE TEST: '{test_command}'")
-    print("-" * 40)
-    
-    for i, format_info in enumerate(prompt_formats, 1):
-        prompt = format_info['template'].format(command=test_command)
-        
-        print(f"\n{i}. {format_info['name']}:")
-        print(f"   Prompt (début): {prompt[:80]}...")
-        
-        # Analyser la structure du prompt
-        lines = prompt.count('\n') + 1
-        json_instructions = prompt.count('JSON')
-        tool_mentions = prompt.count('get_game_state') + prompt.count('move_units') + \
-                       prompt.count('attack_unit') + prompt.count('build_building')
-        
-        print(f"   📊 Analyse: {lines} lignes, {json_instructions} refs JSON, {tool_mentions} refs outils")
-        
-        # Vérifier la présence d'éléments clés
-        has_json_example = '{"tool":' in prompt
-        has_tool_list = 'Available tools:' in prompt or 'Tools:' in prompt
-        has_system_prompt = '<|im_start|>system' in prompt or '[INST]' in prompt
-        
-        print(f"   ✅ Éléments: JSON exemple={has_json_example}, Liste outils={has_tool_list}, Système={has_system_prompt}")
-
-def analyze_model_specific_requirements():
-    """Analyser les besoins spécifiques des modèles MCP"""
-    
-    print(f"\n🔍 BESOINS SPÉCIFIQUES DES MODÈLES MCP")
-    print("=" * 60)
-    
-    model_requirements = {
-        'MCP-Instruct-v1': {
-            'expected_format': 'Format conversation structuré avec tokens MCP',
-            'common_issues': ['Besoin de tokens spécifiques', 'Format de prompt rigide'],
-            'recommendation': 'Utiliser <|im_start|>system/user/assistant format'
-        },
-        'MCPR L-3B-Exa': {
-            'expected_format': 'Format instruct avec contexte MCP', 
-            'common_issues': ['Quantisation agressive', 'Besoin de contexte étendu'],
-            'recommendation': 'Augmenter n_ctx à 4096+ et utiliser Q8_0'
-        },
-        'Gemma-3n-E2B-it': {
-            'expected_format': 'Format technique structuré',
-            'common_issues': ['Quantisation extrême IQ2_XXS', 'Format incompatible'],
-            'recommendation': 'Utiliser version Q8_0 et format code-oriented'
-        }
-    }
-    
-    for model_name, info in model_requirements.items():
-        print(f"\n🧩 {model_name}:")
-        print(f"   📋 Format attendu: {info['expected_format']}")
-        print(f"   ⚠️  Problèmes courants: {', '.join(info['common_issues'])}")
-        print(f"   💡 Recommandation: {info['recommendation']}")
-
-def main():
-    """Investigation principale"""
-    
-    print("🔬 INVESTIGATION DES PROBLÈMES DE FORMAT MCP")
-    print("=" * 70)
-    
-    test_prompt_formats()
-    analyze_model_specific_requirements()
-    
-    print(f"\n🎯 CONCLUSIONS ET SOLUTIONS:")
-    print("=" * 70)
-    
-    print("\n1. PROBLÈMES IDENTIFIÉS:")
-    print("   • Format de prompt incompatible avec les modèles spécialisés")
-    print("   • Quantisation trop agressive (Q2_K, IQ2_XXS)")
-    print("   • Manque de tokens/spécificités MCP dans les prompts")
-    print("   • Contexte insuffisant pour modèles plus grands")
-    
-    print("\n2. SOLUTIONS RECOMMANDÉES:")
-    print("   • Télécharger versions Q8_0 des modèles problématiques")
-    print("   • Utiliser formats structurés avec tokens MCP")
-    print("   • Augmenter n_ctx à 4096 pour modèles 3B+")
-    print("   • Ajouter exemples JSON complets dans les prompts")
-    
-    print(f"\n3. FORMATS DE PROMPT OPTIMAUX:")
-    print("   • MCP-Instruct-v1: <|im_start|>system/user/assistant avec tokens")
-    print("   • MCPR L-3B-Exa: Format instruct avec contexte étendu") 
-    print("   • Gemma-3n-E2B-it: Format code-oriented structuré")
-    
-    print(f"\n🚀 PROCHAINES ÉTAPES:")
-    print("   1. Télécharger mcp-instruct-v1.Q8_0.gguf")
-    print("   2. Télécharger mcprl-3b-exa.Q8_0.gguf")
-    print("   3. Télécharger google_gemma-3n-E2B-it-Q8_0.gguf")
-    print("   4. Tester avec formats de prompts optimisés")
-
-if __name__ == "__main__":
-    main()

quick_model_comparison.json

@@ -1,43 +0,0 @@
-{
-  "results": [
-    {
-      "name": "Qwen2.5-0.5B",
-      "score": 7,
-      "time": 4.335223197937012,
-      "response": ".\n\nOutils: get_game_state(), move_units(unit_ids, target_x, target_y)\n\nCommande: \"Montre-moi l'état du jeu\"\n\nRéponds avec JSON: {{\"tool\": \"nom_outil\", \"args\": {{}}}}. \n\nNote: \"nom_outil\" est le nom de l'outil utilisé pour la commande, \"nom_outil\" est le nom du tool utilisé pour la commande. \"args\" est un dictionnaire contenant"
-    },
-    {
-      "name": "Qwen3-0.6B",
-      "score": 7,
-      "time": 8.028195858001709,
-      "response": "Exemple: {\"tool\": \"get_game_state\", \"args\": {\"unit_ids\": \"1,2,3\", \"target_x\": \"4\", \"target_y\": \"5\"}}\n\nExemple 2: {\"tool\": \"move_units\", \"args\": {\"unit_ids\": \"1,2,3\", \"target_x\": \"4\", \"target_y\": \"5\"}}\n\nExemple 3: {\"tool\": \"get_game_state\", \"args\": {\"unit"
-    },
-    {
-      "name": "Gemma-3-1B",
-      "score": 7,
-      "time": 8.582704782485962,
-      "response": "```json\n{{\"tool\": \"get_game_state\", \"args\": {\"map\": \"map_name\"}}}}\n```\n\n```json\n{{\"tool\": \"get_game_state\", \"args\": {\"map\": \"map_name\"}}}}\n```\n\n```json\n{{\"tool\": \"get_game_state\", \"args\": {\"map\": \"map_name\"}}}}\n```\n\n```json\n{{\"tool\": \"get_game_"
-    }
-  ],
-  "ranking": [
-    {
-      "name": "Qwen2.5-0.5B",
-      "score": 7,
-      "time": 4.335223197937012,
-      "response": ".\n\nOutils: get_game_state(), move_units(unit_ids, target_x, target_y)\n\nCommande: \"Montre-moi l'état du jeu\"\n\nRéponds avec JSON: {{\"tool\": \"nom_outil\", \"args\": {{}}}}. \n\nNote: \"nom_outil\" est le nom de l'outil utilisé pour la commande, \"nom_outil\" est le nom du tool utilisé pour la commande. \"args\" est un dictionnaire contenant"
-    },
-    {
-      "name": "Qwen3-0.6B",
-      "score": 7,
-      "time": 8.028195858001709,
-      "response": "Exemple: {\"tool\": \"get_game_state\", \"args\": {\"unit_ids\": \"1,2,3\", \"target_x\": \"4\", \"target_y\": \"5\"}}\n\nExemple 2: {\"tool\": \"move_units\", \"args\": {\"unit_ids\": \"1,2,3\", \"target_x\": \"4\", \"target_y\": \"5\"}}\n\nExemple 3: {\"tool\": \"get_game_state\", \"args\": {\"unit"
-    },
-    {
-      "name": "Gemma-3-1B",
-      "score": 7,
-      "time": 8.582704782485962,
-      "response": "```json\n{{\"tool\": \"get_game_state\", \"args\": {\"map\": \"map_name\"}}}}\n```\n\n```json\n{{\"tool\": \"get_game_state\", \"args\": {\"map\": \"map_name\"}}}}\n```\n\n```json\n{{\"tool\": \"get_game_state\", \"args\": {\"map\": \"map_name\"}}}}\n```\n\n```json\n{{\"tool\": \"get_game_"
-    }
-  ],
-  "best_model": "Qwen2.5-0.5B"
-}

quick_model_comparison.py

@@ -1,176 +0,0 @@
-"""
-Test rapide de comparaison des modèles pour les tâches MCP
-Focus sur les tests les plus importants
-"""
-
-import sys
-import os
-import json
-import time
-
-# Ajouter le chemin pour les imports
-sys.path.append(os.path.dirname(os.path.abspath(__file__)))
-
-def test_model(model_path, model_name):
-    """Test rapide d'un modèle"""
-    
-    try:
-        from llama_cpp import Llama
-        
-        print(f"🔄 Test de {model_name}...")
-        
-        # Initialiser avec des paramètres plus rapides
-        llm = Llama(
-            model_path=model_path,
-            n_ctx=1024,
-            n_threads=1,
-            verbose=False
-        )
-        
-        # Test simple de compréhension MCP
-        prompt = """Tu es un assistant IA pour un jeu RTS via MCP.
-
-Outils: get_game_state(), move_units(unit_ids, target_x, target_y)
-
-Commande: "Montre-moi l'état du jeu"
-
-Réponds avec JSON: {{"tool": "nom_outil", "args": {{}}}}"""
-        
-        start_time = time.time()
-        
-        response = llm(
-            prompt,
-            max_tokens=100,
-            temperature=0.1,
-            stop=["</s>", "<|im_end|>"]
-        )
-        
-        response_time = time.time() - start_time
-        response_text = response['choices'][0]['text'].strip()
-        
-        # Analyser la réponse
-        score = 0
-        
-        # Vérifier JSON
-        try:
-            json.loads(response_text)
-            score += 3
-        except:
-            pass
-        
-        # Vérifier outil correct
-        if "get_game_state" in response_text:
-            score += 4
-        
-        # Vérifier structure
-        if "tool" in response_text:
-            score += 2
-        
-        # Vérifier cohérence
-        if "game" in response_text.lower():
-            score += 1
-        
-        score = min(score, 10)
-        
-        print(f"✅ {model_name}: {score}/10 | Temps: {response_time:.2f}s")
-        print(f"   Réponse: {response_text[:100]}...")
-        
-        return {
-            'name': model_name,
-            'score': score,
-            'time': response_time,
-            'response': response_text
-        }
-        
-    except Exception as e:
-        print(f"❌ {model_name}: Erreur - {e}")
-        return {
-            'name': model_name,
-            'score': 0,
-            'time': 0,
-            'error': str(e)
-        }
-
-def main():
-    """Test rapide comparatif"""
-    
-    print("🚀 TEST RAPIDE COMPARATIF MCP")
-    print("=" * 50)
-    
-    # Modèles à tester
-    models = [
-        {
-            'name': 'Qwen2.5-0.5B',
-            'path': 'qwen2.5-0.5b-instruct-q4_0.gguf'
-        },
-        {
-            'name': 'Qwen3-0.6B',
-            'path': 'Qwen3-0.6B-Q8_0.gguf'
-        },
-        {
-            'name': 'Gemma-3-1B',
-            'path': 'google_gemma-3-1b-it-qat-Q4_0.gguf'
-        }
-    ]
-    
-    results = []
-    
-    for model in models:
-        if os.path.exists(model['path']):
-            result = test_model(model['path'], model['name'])
-            results.append(result)
-        else:
-            print(f"❌ Fichier non trouvé: {model['path']}")
-    
-    # Résultats
-    print("\n" + "=" * 50)
-    print("📊 RÉSULTATS COMPARATIFS")
-    print("=" * 50)
-    
-    # Classement
-    sorted_results = sorted(results, key=lambda x: x['score'], reverse=True)
-    
-    print(f"\n🏆 CLASSEMENT:")
-    for i, result in enumerate(sorted_results, 1):
-        if 'error' not in result:
-            print(f"   {i}. {result['name']}: {result['score']}/10 ({result['time']:.2f}s)")
-        else:
-            print(f"   {i}. {result['name']}: ÉCHEC")
-    
-    # Analyse
-    successful_results = [r for r in results if 'error' not in r and r['score'] > 0]
-    
-    if successful_results:
-        best_model = successful_results[0]
-        
-        print(f"\n🎯 MEILLEUR MODÈLE: {best_model['name']}")
-        print(f"   Score: {best_model['score']}/10")
-        print(f"   Temps: {best_model['time']:.2f}s")
-        
-        # Recommandations
-        if best_model['score'] >= 7:
-            print(f"\n✅ RECOMMANDATION: {best_model['name']} est EXCELLENT pour MCP")
-        elif best_model['score'] >= 5:
-            print(f"\n👍 RECOMMANDATION: {best_model['name']} est BON pour MCP")
-        else:
-            print(f"\n⚠️  RECOMMANDATION: {best_model['name']} est LIMITÉ pour MCP")
-        
-        # Performance vs taille
-        print(f"\n⚖️  PERFORMANCE:")
-        for result in successful_results:
-            efficiency = result['score'] / result['time'] if result['time'] > 0 else 0
-            file_size = os.path.getsize([m['path'] for m in models if m['name'] == result['name']][0]) / (1024*1024)
-            print(f"   {result['name']}: {efficiency:.2f} score/s | {file_size:.0f} MB")
-    
-    # Sauvegarder
-    with open("quick_model_comparison.json", "w", encoding="utf-8") as f:
-        json.dump({
-            'results': results,
-            'ranking': sorted_results,
-            'best_model': successful_results[0]['name'] if successful_results else None
-        }, f, indent=2, ensure_ascii=False)
-    
-    print(f"\n📄 Résultats sauvegardés dans: quick_model_comparison.json")
-
-if __name__ == "__main__":
-    main()

qwen_mcp_capability_assessment.py

@@ -1,367 +0,0 @@
-"""
-Évaluation complète des capacités MCP de Qwen2.5 0.5B
-Teste la compréhension des outils MCP, la traduction d'instructions et la planification
-"""
-
-import json
-import time
-import sys
-import os
-
-# Ajouter le chemin pour les imports
-sys.path.append(os.path.dirname(os.path.abspath(__file__)))
-
-def test_model_availability():
-    """Vérifier si le modèle est disponible et fonctionnel"""
-    print("🔍 Vérification de la disponibilité du modèle...")
-    
-    # Vérifier si le fichier modèle existe
-    model_path = "qwen2.5-0.5b-instruct-q4_0.gguf"
-    if os.path.exists(model_path):
-        print(f"✅ Modèle trouvé: {model_path}")
-        file_size = os.path.getsize(model_path) / (1024 * 1024)  # MB
-        print(f"📏 Taille du modèle: {file_size:.1f} MB")
-        return True
-    else:
-        print(f"❌ Modèle non trouvé: {model_path}")
-        print("   Le modèle doit être téléchargé pour les tests")
-        return False
-
-def test_basic_mcp_comprehension():
-    """Test de compréhension MCP de base"""
-    
-    print("\n🧪 TEST 1: Compréhension des outils MCP")
-    print("-" * 50)
-    
-    tests = [
-        {
-            "name": "Outils simples",
-            "prompt": """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils disponibles:
-- get_game_state(): Obtenir l'état du jeu
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Montre-moi l'état du jeu"
-
-Réponds avec un objet JSON contenant l'appel d'outil à exécuter.
-""",
-            "expected_tool": "get_game_state"
-        },
-        {
-            "name": "Extraction de paramètres",
-            "prompt": """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace mes tanks vers la position 150, 75"
-
-Réponds avec un objet JSON contenant l'appel d'outil avec les paramètres extraits.
-""",
-            "expected_tool": "move_units",
-            "expected_params": ["unit_ids", "target_x", "target_y"]
-        }
-    ]
-    
-    return tests
-
-def test_complex_planning():
-    """Test de planification complexe"""
-    
-    print("\n🧪 TEST 2: Planification et stratégie")
-    print("-" * 50)
-    
-    tests = [
-        {
-            "name": "Stratégie multi-étapes",
-            "prompt": """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils disponibles:
-- get_game_state(): Obtenir l'état du jeu
-- move_units(): Déplacer des unités
-- attack_unit(): Attaquer une unité ennemie
-- build_building(): Construire un bâtiment
-
-Commande utilisateur: "Construis une base près du minerai et défends-la"
-
-Décompose cette stratégie en une séquence d'appels d'outils MCP.
-Réponds avec un tableau JSON d'actions.
-""",
-            "expected_sequence": True
-        },
-        {
-            "name": "Analyse tactique",
-            "prompt": """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils disponibles:
-- get_game_state(): Obtenir l'état du jeu
-- get_ai_analysis(language): Obtenir une analyse tactique
-
-Commande utilisateur: "Analyse le champ de bataille et suggère une stratégie"
-
-Quels outils MCP utiliserais-tu et dans quel ordre?
-""",
-            "expected_analysis": True
-        }
-    ]
-    
-    return tests
-
-def test_error_handling():
-    """Test de gestion des erreurs et ambiguïtés"""
-    
-    print("\n🧪 TEST 3: Gestion des erreurs")
-    print("-" * 50)
-    
-    tests = [
-        {
-            "name": "Instructions ambiguës",
-            "prompt": """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace mes unités vers la base ennemie"
-
-Comment gérerais-tu cette instruction ambiguë?
-""",
-            "expected_handling": "clarification"
-        },
-        {
-            "name": "Paramètres manquants",
-            "prompt": """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils disponibles:
-- attack_unit(attacker_ids, target_id): Attaquer une unité ennemie
-
-Commande utilisateur: "Attaque l'ennemi"
-
-Que faire si les paramètres nécessaires ne sont pas fournis?
-""",
-            "expected_handling": "parameter_request"
-        }
-    ]
-    
-    return tests
-
-def run_llm_test(prompt, test_name):
-    """Exécuter un test avec le modèle LLM"""
-    
-    try:
-        from ai_analysis import query_llm
-        
-        print(f"\n📋 Test: {test_name}")
-        print(f"Prompt: {prompt[:100]}...")
-        
-        start_time = time.time()
-        
-        response = query_llm(
-            prompt=prompt,
-            max_tokens=300,
-            temperature=0.1,
-            system_message="Tu es un assistant IA spécialisé dans les jeux RTS et le protocole MCP."
-        )
-        
-        response_time = time.time() - start_time
-        
-        print(f"⏱️  Temps de réponse: {response_time:.2f}s")
-        print(f"📝 Réponse: {response[:200]}...")
-        
-        return {
-            "success": True,
-            "response": response,
-            "response_time": response_time,
-            "error": None
-        }
-        
-    except Exception as e:
-        print(f"❌ Erreur: {e}")
-        return {
-            "success": False,
-            "response": None,
-            "response_time": 0,
-            "error": str(e)
-        }
-
-def analyze_response(response, test_config):
-    """Analyser la réponse du modèle"""
-    
-    analysis = {
-        "score": 0,
-        "strengths": [],
-        "weaknesses": [],
-        "details": {}
-    }
-    
-    if not response:
-        analysis["weaknesses"].append("Réponse vide")
-        return analysis
-    
-    # Vérifier la structure JSON
-    try:
-        json.loads(response)
-        analysis["score"] += 3
-        analysis["strengths"].append("JSON valide")
-        analysis["details"]["json_valid"] = True
-    except:
-        analysis["details"]["json_valid"] = False
-    
-    # Vérifier la présence d'outils MCP
-    mcp_tools = ["get_game_state", "move_units", "attack_unit", "build_building", "get_ai_analysis"]
-    tools_found = [tool for tool in mcp_tools if tool in response]
-    
-    if tools_found:
-        analysis["score"] += 2
-        analysis["strengths"].append(f"Outils MCP identifiés: {', '.join(tools_found)}")
-    
-    # Vérifier la cohérence sémantique
-    if "game" in response.lower() or "rts" in response.lower() or "mcp" in response.lower():
-        analysis["score"] += 1
-        analysis["strengths"].append("Contexte RTS/MCP compris")
-    
-    # Vérifier les paramètres
-    if "unit_ids" in response or "target_x" in response or "target_y" in response:
-        analysis["score"] += 1
-        analysis["strengths"].append("Paramètres identifiés")
-    
-    analysis["score"] = min(analysis["score"], 10)
-    
-    return analysis
-
-def generate_assessment_report(results):
-    """Générer un rapport d'évaluation complet"""
-    
-    print("\n" + "="*70)
-    print("📊 RAPPORT D'ÉVALUATION DES CAPACITÉS MCP")
-    print("="*70)
-    
-    total_score = 0
-    total_tests = 0
-    
-    for category, category_results in results.items():
-        print(f"\n🔍 {category.upper()}:")
-        
-        category_score = 0
-        category_count = 0
-        
-        for test_name, result in category_results.items():
-            if result["success"]:
-                analysis = result["analysis"]
-                score = analysis["score"]
-                category_score += score
-                category_count += 1
-                
-                print(f"   {test_name}: {score}/10")
-                if analysis["strengths"]:
-                    print(f"     ✅ Points forts: {', '.join(analysis['strengths'][:2])}")
-                if analysis["weaknesses"]:
-                    print(f"     ⚠️  Points faibles: {', '.join(analysis['weaknesses'][:2])}")
-            else:
-                print(f"   {test_name}: ❌ ÉCHEC - {result['error']}")
-        
-        if category_count > 0:
-            avg_score = category_score / category_count
-            total_score += avg_score
-            total_tests += 1
-            print(f"   📊 Score moyen: {avg_score:.1f}/10")
-    
-    # Score global
-    if total_tests > 0:
-        overall_score = total_score / total_tests
-        
-        print(f"\n🎯 SCORE GLOBAL: {overall_score:.1f}/10")
-        
-        # Recommandations basées sur le score
-        print("\n💡 RECOMMANDATIONS:")
-        
-        if overall_score >= 8:
-            print("✅ EXCELLENT - Qwen2.5 0.5B est très capable pour les tâches MCP")
-            print("   • Utilisez-le avec confiance pour la traduction MCP")
-            print("   • Implémentez une validation légère pour la sécurité")
-        elif overall_score >= 6:
-            print("👍 BON - Qwen2.5 0.5B est capable avec quelques limitations")
-            print("   • Utilisez-le pour les commandes simples")
-            print("   • Ajoutez une validation robuste")
-            print("   • Prévoyez des mécanismes de secours")
-        elif overall_score >= 4:
-            print("⚠️  MODÉRÉ - Limitations significatives")
-            print("   • Utilisez-le uniquement pour des traductions très simples")
-            print("   • Implémentez une validation stricte")
-            print("   • Envisagez un modèle plus grand pour les tâches complexes")
-        else:
-            print("❌ FAIBLE - Pas adapté aux tâches MCP")
-            print("   • Envisagez un modèle plus grand (1.5B+)")
-            print("   • Utilisez des règles fixes pour les commandes simples")
-
-def main():
-    """Fonction principale"""
-    
-    print("🔍 ÉVALUATION DES CAPACITÉS MCP DE QWEN2.5 0.5B")
-    print("=" * 70)
-    
-    # Vérifier la disponibilité du modèle
-    if not test_model_availability():
-        print("\n❌ Le modèle n'est pas disponible. Téléchargez-le d'abord.")
-        return
-    
-    # Définir les tests
-    basic_tests = test_basic_mcp_comprehension()
-    planning_tests = test_complex_planning()
-    error_tests = test_error_handling()
-    
-    # Exécuter les tests
-    results = {}
-    
-    # Tests de base
-    print("\n🚀 Exécution des tests de capacité MCP...")
-    
-    basic_results = {}
-    for test in basic_tests:
-        result = run_llm_test(test["prompt"], test["name"])
-        if result["success"]:
-            analysis = analyze_response(result["response"], test)
-            result["analysis"] = analysis
-        basic_results[test["name"]] = result
-    
-    results["compréhension_de_base"] = basic_results
-    
-    # Tests de planification
-    planning_results = {}
-    for test in planning_tests:
-        result = run_llm_test(test["prompt"], test["name"])
-        if result["success"]:
-            analysis = analyze_response(result["response"], test)
-            result["analysis"] = analysis
-        planning_results[test["name"]] = result
-    
-    results["planification_complexe"] = planning_results
-    
-    # Tests de gestion d'erreurs
-    error_results = {}
-    for test in error_tests:
-        result = run_llm_test(test["prompt"], test["name"])
-        if result["success"]:
-            analysis = analyze_response(result["response"], test)
-            result["analysis"] = analysis
-        error_results[test["name"]] = result
-    
-    results["gestion_des_erreurs"] = error_results
-    
-    # Générer le rapport
-    generate_assessment_report(results)
-    
-    # Sauvegarder les résultats détaillés
-    with open("qwen_mcp_capability_results.json", "w", encoding="utf-8") as f:
-        json.dump(results, f, indent=2, ensure_ascii=False)
-    
-    print(f"\n📄 Résultats détaillés sauvegardés dans: qwen_mcp_capability_results.json")
-
-if __name__ == "__main__":
-    main()

qwen_mcp_evaluation.py

@@ -1,279 +0,0 @@
-"""
-Évaluation des capacités MCP de Qwen2.5 0.5B
-Test maintenant que le modèle fonctionne correctement
-"""
-
-import sys
-import os
-import json
-import time
-
-# Ajouter le chemin pour les imports
-sys.path.append(os.path.dirname(os.path.abspath(__file__)))
-
-def test_qwen_mcp_capabilities():
-    """Test des capacités MCP avec le modèle fonctionnel"""
-    
-    print("🔍 ÉVALUATION DES CAPACITÉS MCP DE QWEN2.5 0.5B")
-    print("=" * 70)
-    
-    # Importer et initialiser le modèle
-    try:
-        from llama_cpp import Llama
-        
-        model_path = "qwen2.5-0.5b-instruct-q4_0.gguf"
-        
-        print("🔄 Initialisation du modèle...")
-        llm = Llama(
-            model_path=model_path,
-            n_ctx=2048,
-            n_threads=2,
-            verbose=False
-        )
-        
-        print("✅ Modèle initialisé avec succès\n")
-        
-    except Exception as e:
-        print(f"❌ Erreur d'initialisation: {e}")
-        return
-    
-    # Tests MCP
-    tests = [
-        {
-            "name": "Compréhension MCP de base",
-            "prompt": """Tu es un assistant IA qui contrôle un jeu RTS via MCP (Model Context Protocol).
-
-Outils MCP disponibles:
-- get_game_state(): Obtenir l'état actuel du jeu
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Montre-moi l'état du jeu"
-
-Réponds avec un objet JSON contenant l'appel d'outil MCP à exécuter.
-""",
-            "expected_tool": "get_game_state"
-        },
-        {
-            "name": "Traduction d'action simple",
-            "prompt": """Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace mon infanterie vers la position 150, 200"
-
-Réponds avec un objet JSON contenant l'appel d'outil avec les paramètres extraits.
-""",
-            "expected_tool": "move_units"
-        },
-        {
-            "name": "Extraction de paramètres complexes",
-            "prompt": """Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace les tanks 1, 3 et 7 vers les coordonnées 120, 80"
-
-Extrais les paramètres et réponds avec un objet JSON d'appel d'outil.
-""",
-            "expected_tool": "move_units"
-        },
-        {
-            "name": "Planification stratégique",
-            "prompt": """Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- get_game_state(): Obtenir l'état du jeu
-- move_units(): Déplacer des unités
-- attack_unit(): Attaquer l'ennemi
-- build_building(): Construire un bâtiment
-
-Commande utilisateur: "Construis une base près du minerai et défends-la"
-
-Décompose cette stratégie en une séquence d'actions MCP.
-""",
-            "expected_complex": True
-        },
-        {
-            "name": "Gestion d'ambiguïté",
-            "prompt": """Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-
-Outils MCP disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace mes unités vers la base ennemie"
-
-Comment gérerais-tu cette instruction ambiguë?
-""",
-            "expected_clarification": True
-        }
-    ]
-    
-    results = []
-    
-    for i, test in enumerate(tests, 1):
-        print(f"🧪 TEST {i}: {test['name']}")
-        print("-" * 50)
-        
-        try:
-            start_time = time.time()
-            
-            # Utiliser le chat template pour de meilleurs résultats
-            messages = [
-                {"role": "system", "content": "Tu es un assistant IA spécialisé dans les jeux RTS et le protocole MCP. Réponds avec des objets JSON valides."},
-                {"role": "user", "content": test['prompt']}
-            ]
-            
-            # Utiliser la méthode standard pour llama-cpp-python
-            response = llm.create_chat_completion(
-                messages=messages,
-                max_tokens=300,
-                temperature=0.1
-            )
-            
-            response_time = time.time() - start_time
-            
-            # Extraire le texte de la réponse correctement
-            if response and 'choices' in response and len(response['choices']) > 0:
-                response_text = response['choices'][0]['message']['content']
-            else:
-                # Fallback: utiliser la méthode simple
-                simple_response = llm(test['prompt'], max_tokens=300, temperature=0.1)
-                response_text = simple_response['choices'][0]['text'] if 'choices' in simple_response else str(simple_response)
-            
-            print(f"⏱️  Temps de réponse: {response_time:.2f}s")
-            print(f"📝 Réponse: {response_text[:300]}...")
-            
-            # Analyser la réponse
-            score = analyze_mcp_response(response_text, test)
-            
-            results.append({
-                'test': test['name'],
-                'response': response_text,
-                'response_time': response_time,
-                'score': score,
-                'success': score >= 5
-            })
-            
-            print(f"📊 Score: {score}/10")
-            
-        except Exception as e:
-            print(f"❌ Erreur: {e}")
-            results.append({
-                'test': test['name'],
-                'error': str(e),
-                'score': 0,
-                'success': False
-            })
-        
-        print()
-    
-    # Générer le rapport final
-    generate_evaluation_report(results)
-
-def analyze_mcp_response(response, test_config):
-    """Analyser la réponse MCP et la noter"""
-    
-    if not response or response.strip() == "":
-        return 0
-    
-    score = 0
-    
-    # Vérifier la structure JSON
-    try:
-        json_data = json.loads(response)
-        score += 3  # JSON valide
-        
-        # Vérifier la présence d'outils MCP
-        if 'tool' in json_data:
-            score += 2
-        if 'args' in json_data:
-            score += 1
-            
-    except json.JSONDecodeError:
-        # Vérifier les patterns dans le texte
-        mcp_tools = ["get_game_state", "move_units", "attack_unit", "build_building"]
-        tools_found = [tool for tool in mcp_tools if tool in response]
-        
-        if tools_found:
-            score += 2
-        if any(param in response for param in ['unit_ids', 'target_x', 'target_y']):
-            score += 1
-    
-    # Vérifier la cohérence sémantique
-    if 'game' in response.lower() or 'rts' in response.lower():
-        score += 1
-    
-    # Vérifier la pertinence par rapport au test
-    if 'expected_tool' in test_config:
-        if test_config['expected_tool'] in response:
-            score += 2
-    elif test_config.get('expected_complex'):
-        if any(word in response.lower() for word in ['sequence', 'steps', 'build', 'defend']):
-            score += 2
-    elif test_config.get('expected_clarification'):
-        if any(word in response.lower() for word in ['clarify', 'coordinates', 'which', 'ambiguous']):
-            score += 2
-    
-    return min(score, 10)
-
-def generate_evaluation_report(results):
-    """Générer un rapport d'évaluation complet"""
-    
-    print("=" * 70)
-    print("📊 RAPPORT D'ÉVALUATION MCP")
-    print("=" * 70)
-    
-    successful_tests = [r for r in results if r.get('success', False)]
-    total_score = sum(r.get('score', 0) for r in results)
-    avg_score = total_score / len(results) if results else 0
-    
-    print(f"\n📈 STATISTIQUES:")
-    print(f"   Tests réussis: {len(successful_tests)}/{len(results)}")
-    print(f"   Score moyen: {avg_score:.1f}/10")
-    
-    print(f"\n🔍 DÉTAILS PAR TEST:")
-    for result in results:
-        status = "✅" if result.get('success', False) else "❌"
-        print(f"   {status} {result['test']}: {result.get('score', 0)}/10")
-        if 'response_time' in result:
-            print(f"     ⏱️  Temps: {result['response_time']:.2f}s")
-    
-    print(f"\n🎯 ÉVALUATION FINALE:")
-    
-    if avg_score >= 8:
-        print("💪 EXCELLENT - Qwen2.5 0.5B est très capable pour les tâches MCP")
-        recommendation = "UTILISER AVEC CONFIANCE"
-    elif avg_score >= 6:
-        print("👍 BON - Qwen2.5 0.5B est capable avec quelques limitations")
-        recommendation = "UTILISER AVEC VALIDATION"
-    elif avg_score >= 4:
-        print("⚠️  MODÉRÉ - Limitations significatives")
-        recommendation = "UTILISER POUR TÂCHES SIMPLES"
-    else:
-        print("❌ FAIBLE - Pas adapté aux tâches MCP")
-        recommendation = "CONSIDÉRER AUTRE SOLUTION"
-    
-    print(f"💡 RECOMMANDATION: {recommendation}")
-    
-    print(f"\n🚀 PROCHAINES ÉTAPES:")
-    if avg_score >= 6:
-        print("1. ✅ Implémenter la traduction MCP dans le jeu")
-        print("2. 🎯 Commencer par les commandes simples")
-        print("3. 📊 Ajouter des métriques de performance")
-        print("4. 🔍 Tester avec des vrais joueurs")
-    else:
-        print("1. 🔧 Améliorer les prompts et exemples")
-        print("2. 📚 Entraîner avec des données MCP spécifiques")
-        print("3. 🔄 Considérer un modèle plus grand")
-        print("4. 🛠️ Implémenter des règles de secours")
-    
-    # Sauvegarder les résultats
-    with open("qwen_mcp_evaluation_results.json", "w", encoding="utf-8") as f:
-        json.dump(results, f, indent=2, ensure_ascii=False)
-    
-    print(f"\n📄 Résultats détaillés sauvegardés dans: qwen_mcp_evaluation_results.json")
-
-if __name__ == "__main__":
-    test_qwen_mcp_capabilities()

qwen_mcp_evaluation_results.json

@@ -1,37 +0,0 @@
-[
-  {
-    "test": "Compréhension MCP de base",
-    "response": "```json\n{\n  \"call\": \"get_game_state\"\n}\n```",
-    "response_time": 0.9023318290710449,
-    "score": 5,
-    "success": true
-  },
-  {
-    "test": "Traduction d'action simple",
-    "response": "```json\n{\n  \"move_units\": {\n    \"unit_ids\": [\"infanterie\"],\n    \"target_x\": 150,\n    \"target_y\": 200\n  }\n}\n```",
-    "response_time": 1.97584867477417,
-    "score": 5,
-    "success": true
-  },
-  {
-    "test": "Extraction de paramètres complexes",
-    "response": "```json\n{\n  \"move_units\": {\n    \"units\": [\n      \"tank_1\",\n      \"tank_3\",\n      \"tank_7\"\n    ],\n    \"target_x\": 120,\n    \"target_y\": 80\n  }\n}\n```",
-    "response_time": 2.287006378173828,
-    "score": 5,
-    "success": true
-  },
-  {
-    "test": "Planification stratégique",
-    "response": "```json\n{\n  \"get_game_state\": {},\n  \"move_units\": {\n    \"units\": [\n      {\n        \"name\": \"Base\",\n        \"position\": {\n          \"x\": 10,\n          \"y\": 10\n        }\n      },\n      {\n        \"name\": \"Minerai\",\n        \"position\": {\n          \"x\": 10,\n          \"y\": 10\n        }\n      }\n    ]\n  },\n  \"attack_unit\": {\n    \"units\": [\n      {\n        \"name\": \"Minerai\",\n        \"position\": {\n          \"x\": 10,\n          \"y\": 10\n        }\n      }\n    ]\n  }\n}\n```",
-    "response_time": 5.687268257141113,
-    "score": 3,
-    "success": false
-  },
-  {
-    "test": "Gestion d'ambiguïté",
-    "response": "Voici comment je gérerais cette instruction ambiguë :\n\n```json\n{\n  \"move_units\": {\n    \"unit_ids\": [\"unit1_id\", \"unit2_id\", \"unit3_id\"],\n    \"target_x\": 0,\n    \"target_y\": 0\n  }\n}\n```\n\nDans ce cas, je vais déplacer les unités vers la base ennemie.",
-    "response_time": 3.0712451934814453,
-    "score": 3,
-    "success": false
-  }
-]

realistic_mcp_evaluation.json

@@ -1,445 +0,0 @@
-{
-  "evaluation_type": "realistic_mcp_game_test",
-  "game_state_sample": {
-    "player_id": 1,
-    "resources": {
-      "credits": 2500,
-      "power": 150
-    },
-    "units": [
-      {
-        "id": 1,
-        "type": "infantry",
-        "x": 100,
-        "y": 100,
-        "health": 100
-      },
-      {
-        "id": 2,
-        "type": "infantry",
-        "x": 110,
-        "y": 105,
-        "health": 100
-      },
-      {
-        "id": 3,
-        "type": "tank",
-        "x": 120,
-        "y": 110,
-        "health": 150
-      },
-      {
-        "id": 4,
-        "type": "harvester",
-        "x": 200,
-        "y": 200,
-        "health": 200
-      }
-    ],
-    "buildings": [
-      {
-        "id": 1,
-        "type": "hq",
-        "x": 50,
-        "y": 50,
-        "health": 500
-      },
-      {
-        "id": 2,
-        "type": "power_plant",
-        "x": 80,
-        "y": 80,
-        "health": 300
-      },
-      {
-        "id": 3,
-        "type": "barracks",
-        "x": 120,
-        "y": 60,
-        "health": 250
-      }
-    ],
-    "enemy_units": [
-      {
-        "id": 101,
-        "type": "infantry",
-        "x": 300,
-        "y": 150,
-        "health": 100
-      },
-      {
-        "id": 102,
-        "type": "tank",
-        "x": 320,
-        "y": 160,
-        "health": 150
-      }
-    ],
-    "map": {
-      "width": 96,
-      "height": 72,
-      "ore_fields": [
-        {
-          "x": 250,
-          "y": 200
-        },
-        {
-          "x": 400,
-          "y": 300
-        }
-      ]
-    }
-  },
-  "results": [
-    {
-      "name": "Qwen2.5-0.5B",
-      "avg_score": 3.0,
-      "avg_time": 8.120952129364014,
-      "results": [
-        {
-          "scenario": "Défense immédiate",
-          "difficulty": "facile",
-          "score": 3,
-          "time": 11.3031907081604,
-          "response": "Action: attack_unit\n- Arguments: [320, 160]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: build_building\n- Arguments: [power_plant, 80, 80, 1]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Collecte de ressources",
-          "difficulty": "moyen",
-          "score": 0,
-          "time": 8.980303764343262,
-          "response": "Démarre une construction de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de",
-          "expected_tool": "move_units"
-        },
-        {
-          "scenario": "Expansion stratégique",
-          "difficulty": "moyen",
-          "score": 7,
-          "time": 9.734914541244507,
-          "response": "ACTION: build_building(building_type, position_x, position_y, player_id)\n\nVeuillez fournir un objet JSON représentant la construction de la caserne. \n\nJSON: {\"building_type\": \"barracks\", \"position_x\": 250, \"position_y\": 200, \"player_id\": 101} \n\nVeuillez noter que la caserne doit être construite avant que les récolteurs ne soient en danger. \n\nVeuillez noter que la caserne doit être construite dans le champ de minerai. \n\nVeuillez noter que la caserne doit être construite avant que les récolteurs ne soient en danger. \n\nVeuillez noter que la caserne doit être construite dans le champ de minerai. \n\nVeuillez noter que la caserne doit être construite avant que les récolteurs ne soient en danger. \n\nVeuillez noter que la caserne",
-          "expected_tool": "build_building"
-        },
-        {
-          "scenario": "Attaque coordonnée",
-          "difficulty": "difficile",
-          "score": 0,
-          "time": 9.359946966171265,
-          "response": "- Attaque les unités ennemies de la position (x, y) = (300, 150) avec une portée de 100.\n- Attaque les unités ennemies de la position (x, y) = (400, 300) avec une portée de 150.\n- Attaque les unités ennemies de la position (x, y) = (300, 150) avec une portée de 100.\n- Attaque les unités ennemies de la position (x, y) = (400, 300) avec une portée de 150.\n- Attaque les unités ennemies de la position (x, y) = (300, 150) avec une portée de 100.\n- Attaque les unités ennemies",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Reconnaissance",
-          "difficulty": "facile",
-          "score": 2,
-          "time": 0.18431353569030762,
-          "response": "",
-          "expected_tool": "get_game_state"
-        },
-        {
-          "scenario": "Gestion de crise",
-          "difficulty": "difficile",
-          "score": 6,
-          "time": 9.16304326057434,
-          "response": "Les unités doivent exister (vérifier les IDs) et exécuter la fonction move_units.\n\nAction: move_units\n{\n  \"unit_ids\": [1, 2, 3, 4],\n  \"target_x\": 50,\n  \"target_y\": 50\n}\n\nRÈGLES DU JEU:\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)",
-          "expected_tool": "move_units"
-        }
-      ],
-      "easy_avg": 2.5,
-      "medium_avg": 3.5,
-      "hard_avg": 3.0
-    },
-    {
-      "name": "Qwen3-0.6B",
-      "avg_score": 2.6666666666666665,
-      "avg_time": 16.65606681505839,
-      "results": [
-        {
-          "scenario": "Défense immédiate",
-          "difficulty": "facile",
-          "score": 3,
-          "time": 23.307448387145996,
-          "response": "Pour répondre, formattez le JSON comme suit:\n{\n  \"action\": \"attack_unit\",\n  \"unit_id\": \"101\",\n  \"target_id\": \"102\",\n  \"attack_range\": \"100\"\n}\n\n{\n  \"action\": \"attack_unit\",\n  \"unit_id\": \"101\",\n  \"target_id\": \"102\",\n  \"attack_range\": \"150\"\n}\n\n{\n  \"action\": \"attack_unit\",\n  \"unit_id\": \"101\",\n  \"target_id\": \"102\",\n  \"attack_range\": \"200\"\n}\n\n{\n  \"action\": \"attack_unit\",\n  \"unit_id\": \"101\",\n  \"target_id\": \"102\",\n  \"attack_range\": \"50\"\n}\nAnswer:\n\n{\n  \"action\": \"attack_unit\",\n  \"unit_id\": \"101\",\n  \"target_id\": \"1",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Collecte de ressources",
-          "difficulty": "moyen",
-          "score": 3,
-          "time": 10.506604194641113,
-          "response": "(100 crédits)\n- { \"action\": \"move_units\", \"units\": [1,4], \"target_x\": 200, \"target_y\": 200 } \n\nLa réponse doit être inutilement en français, with no markdown, and in the correct JSON format.\n\nExample:\n- { \"action\": \"move_units\", \"units\": [1,4], \"target_x\": 100, \"target_y\": 100 }\n\nAnswer:",
-          "expected_tool": "move_units"
-        },
-        {
-          "scenario": "Expansion stratégique",
-          "difficulty": "moyen",
-          "score": 5,
-          "time": 7.1594953536987305,
-          "response": "Je dois construire une caserne de type \"hq\" et avoir une ressource de 500.\n\nPourquoi est-ce important de construire une caserne de type \"hq\" ?\n\nAnswer:\n{\n  \"action\": \"build_building('hq', 250, 200, 1)\"\n}",
-          "expected_tool": "build_building"
-        },
-        {
-          "scenario": "Attaque coordonnée",
-          "difficulty": "difficile",
-          "score": 2,
-          "time": 19.33790349960327,
-          "response": "Pour chaque unité ennemie, determine if it is in range of my units. If it is, execute an attack. If not, do nothing.\n\nIf there are no attacks, then execute the build_building action.\n\nIf there are attacks, then execute the attack_unit action.\n\nIf there are no attacks, then execute the move_units action.\n\nIf there are no attacks, then execute the get_ai_analysis action.\n\nIf there are no attacks, then execute the get_game_state action.\n\nIf there are no attacks, then execute the move_units action.\n\nIf there are no attacks, then execute the get_ai_analysis action.\n\nIf there are no attacks, then execute the get_game_state action.\n\nIf there are no attacks, then execute the move_units action.\n\nIf there are no attacks, then execute the get_ai_analysis action.\n\nIf there are no attacks, then execute the get_game_state action.\n\nIf there are no attacks, then execute the move_units action.\n\nIf there are no attacks, then",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Reconnaissance",
-          "difficulty": "facile",
-          "score": 3,
-          "time": 19.707163333892822,
-          "response": "(Output must be a JSON object)\n{\n  \"game_state\": {\n    \"player_id\": 1,\n    \"resources\": {\n      \"credits\": 2500,\n      \"power\": 150\n    },\n    \"units\": [\n      {\n        \"id\": 1,\n        \"type\": \"infantry\",\n        \"x\": 100,\n        \"y\": 100,\n        \"health\": 100\n      },\n      {\n        \"id\": 2,\n        \"type\": \"infantry\",\n        \"x\": 110,\n        \"y\": 105,\n        \"health\": 100\n      },\n      {\n        \"id\": 3,\n        \"type\": \"tank\",\n        \"x\": 120,\n        \"y\": 110,\n        \"health\": 150\n      },\n      {\n        \"id\": 4,",
-          "expected_tool": "get_game_state"
-        },
-        {
-          "scenario": "Gestion de crise",
-          "difficulty": "difficile",
-          "score": 0,
-          "time": 19.917786121368408,
-          "response": "Les unités sont déplacées vers (100, 100) et (120, 100) et (150, 100). Les unités ennemies sont déplacées vers (300, 150) et (320, 160). Les unités ennemies sont attaquées par le joueur.\n\nLes unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par le joueur.\n\nLes unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par le joueur.\n\nLes unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par",
-          "expected_tool": "move_units"
-        }
-      ],
-      "easy_avg": 3.0,
-      "medium_avg": 4.0,
-      "hard_avg": 1.0
-    },
-    {
-      "name": "Gemma-3-270M",
-      "avg_score": 1.1666666666666667,
-      "avg_time": 7.372450391451518,
-      "results": [
-        {
-          "scenario": "Défense immédiate",
-          "difficulty": "facile",
-          "score": 3,
-          "time": 11.643838167190552,
-          "response": "- get_game_state(): Obtenir l'état actuel du jeu\n- move_units(unit_ids, target_x, target_y): Déplacer des unités\n- attack_unit(attacker_ids, target_id): Attaquer une unité ennemie\n- build_building(building_type, position_x, position_y, player_id): Construire un bâtiment\n- get_ai_analysis(language): Obtenir une analyse tactique\n\nJe suis prêt à répondre à vos questions.\nMerci!\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Collecte de ressources",
-          "difficulty": "moyen",
-          "score": 0,
-          "time": 0.38573145866394043,
-          "response": "Merci!",
-          "expected_tool": "move_units"
-        },
-        {
-          "scenario": "Expansion stratégique",
-          "difficulty": "moyen",
-          "score": 0,
-          "time": 10.55238938331604,
-          "response": "- Quel est le nom de la caserne ?\n- Quelle est la position de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n-",
-          "expected_tool": "build_building"
-        },
-        {
-          "scenario": "Attaque coordonnée",
-          "difficulty": "difficile",
-          "score": 0,
-          "time": 10.54906940460205,
-          "response": "- Défense : Attaque contre les unités ennemies.\n- Destruction : Destruction des unités ennemies.\n-  (Optional)  Utilisation de la force de feu.\n-  (Optional)  Utilisation de la magie.\n-  (Optional)  Utilisation de la technologie.\n-  (Optional)  Utilisation de la base de données.\n-  (Optional)  Utilisation de la base de données de données.\n-  (Optional)  Utilisation de la base de données de données de données.\n-  (Optional)  Utilisation de la base de données de données de données.\n-  (Optional)  Utilisation de la base de données de données de données.\n-  (Optional)  Utilisation de la base de données de données de données.\n-  (Optional)  Utilisation de la base de données de données de données.\n-  (Optional)  Utilisation",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Reconnaissance",
-          "difficulty": "facile",
-          "score": 2,
-          "time": 0.4084289073944092,
-          "response": "Merci!",
-          "expected_tool": "get_game_state"
-        },
-        {
-          "scenario": "Gestion de crise",
-          "difficulty": "difficile",
-          "score": 2,
-          "time": 10.695245027542114,
-          "response": "-  `get_game_state()` : Obtient l'état actuel du jeu.\n- `move_units(unit_ids, target_x, target_y)` : Déplace les unités dans la position spécifiée.\n- `attack_unit(attacker_ids, target_id)` : Attaque une unité ennemie.\n- `build_building(building_type, position_x, position_y, player_id)` : Construis un bâtiment.\n- `get_ai_analysis(language)` : Analyse tactique.\n- `move_units(unit_ids, target_x, target_y)` : Déplace les unités dans la position spécifiée.\n- `attack_unit(attacker_ids, target_id)` : Attaque une unité ennemie.\n- `build_building(building_type, position_x, position_y, player_id)` : Construis un",
-          "expected_tool": "move_units"
-        }
-      ],
-      "easy_avg": 2.5,
-      "medium_avg": 0.0,
-      "hard_avg": 1.0
-    }
-  ],
-  "successful_models": [
-    {
-      "name": "Qwen2.5-0.5B",
-      "avg_score": 3.0,
-      "avg_time": 8.120952129364014,
-      "results": [
-        {
-          "scenario": "Défense immédiate",
-          "difficulty": "facile",
-          "score": 3,
-          "time": 11.3031907081604,
-          "response": "Action: attack_unit\n- Arguments: [320, 160]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: build_building\n- Arguments: [power_plant, 80, 80, 1]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1, 100, 100]\n\nAction: move_units\n- Arguments: [1",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Collecte de ressources",
-          "difficulty": "moyen",
-          "score": 0,
-          "time": 8.980303764343262,
-          "response": "Démarre une construction de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de 1000 pièces de",
-          "expected_tool": "move_units"
-        },
-        {
-          "scenario": "Expansion stratégique",
-          "difficulty": "moyen",
-          "score": 7,
-          "time": 9.734914541244507,
-          "response": "ACTION: build_building(building_type, position_x, position_y, player_id)\n\nVeuillez fournir un objet JSON représentant la construction de la caserne. \n\nJSON: {\"building_type\": \"barracks\", \"position_x\": 250, \"position_y\": 200, \"player_id\": 101} \n\nVeuillez noter que la caserne doit être construite avant que les récolteurs ne soient en danger. \n\nVeuillez noter que la caserne doit être construite dans le champ de minerai. \n\nVeuillez noter que la caserne doit être construite avant que les récolteurs ne soient en danger. \n\nVeuillez noter que la caserne doit être construite dans le champ de minerai. \n\nVeuillez noter que la caserne doit être construite avant que les récolteurs ne soient en danger. \n\nVeuillez noter que la caserne",
-          "expected_tool": "build_building"
-        },
-        {
-          "scenario": "Attaque coordonnée",
-          "difficulty": "difficile",
-          "score": 0,
-          "time": 9.359946966171265,
-          "response": "- Attaque les unités ennemies de la position (x, y) = (300, 150) avec une portée de 100.\n- Attaque les unités ennemies de la position (x, y) = (400, 300) avec une portée de 150.\n- Attaque les unités ennemies de la position (x, y) = (300, 150) avec une portée de 100.\n- Attaque les unités ennemies de la position (x, y) = (400, 300) avec une portée de 150.\n- Attaque les unités ennemies de la position (x, y) = (300, 150) avec une portée de 100.\n- Attaque les unités ennemies",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Reconnaissance",
-          "difficulty": "facile",
-          "score": 2,
-          "time": 0.18431353569030762,
-          "response": "",
-          "expected_tool": "get_game_state"
-        },
-        {
-          "scenario": "Gestion de crise",
-          "difficulty": "difficile",
-          "score": 6,
-          "time": 9.16304326057434,
-          "response": "Les unités doivent exister (vérifier les IDs) et exécuter la fonction move_units.\n\nAction: move_units\n{\n  \"unit_ids\": [1, 2, 3, 4],\n  \"target_x\": 50,\n  \"target_y\": 50\n}\n\nRÈGLES DU JEU:\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)\n- Les unités doivent exister (vérifier les IDs)",
-          "expected_tool": "move_units"
-        }
-      ],
-      "easy_avg": 2.5,
-      "medium_avg": 3.5,
-      "hard_avg": 3.0
-    },
-    {
-      "name": "Qwen3-0.6B",
-      "avg_score": 2.6666666666666665,
-      "avg_time": 16.65606681505839,
-      "results": [
-        {
-          "scenario": "Défense immédiate",
-          "difficulty": "facile",
-          "score": 3,
-          "time": 23.307448387145996,
-          "response": "Pour répondre, formattez le JSON comme suit:\n{\n  \"action\": \"attack_unit\",\n  \"unit_id\": \"101\",\n  \"target_id\": \"102\",\n  \"attack_range\": \"100\"\n}\n\n{\n  \"action\": \"attack_unit\",\n  \"unit_id\": \"101\",\n  \"target_id\": \"102\",\n  \"attack_range\": \"150\"\n}\n\n{\n  \"action\": \"attack_unit\",\n  \"unit_id\": \"101\",\n  \"target_id\": \"102\",\n  \"attack_range\": \"200\"\n}\n\n{\n  \"action\": \"attack_unit\",\n  \"unit_id\": \"101\",\n  \"target_id\": \"102\",\n  \"attack_range\": \"50\"\n}\nAnswer:\n\n{\n  \"action\": \"attack_unit\",\n  \"unit_id\": \"101\",\n  \"target_id\": \"1",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Collecte de ressources",
-          "difficulty": "moyen",
-          "score": 3,
-          "time": 10.506604194641113,
-          "response": "(100 crédits)\n- { \"action\": \"move_units\", \"units\": [1,4], \"target_x\": 200, \"target_y\": 200 } \n\nLa réponse doit être inutilement en français, with no markdown, and in the correct JSON format.\n\nExample:\n- { \"action\": \"move_units\", \"units\": [1,4], \"target_x\": 100, \"target_y\": 100 }\n\nAnswer:",
-          "expected_tool": "move_units"
-        },
-        {
-          "scenario": "Expansion stratégique",
-          "difficulty": "moyen",
-          "score": 5,
-          "time": 7.1594953536987305,
-          "response": "Je dois construire une caserne de type \"hq\" et avoir une ressource de 500.\n\nPourquoi est-ce important de construire une caserne de type \"hq\" ?\n\nAnswer:\n{\n  \"action\": \"build_building('hq', 250, 200, 1)\"\n}",
-          "expected_tool": "build_building"
-        },
-        {
-          "scenario": "Attaque coordonnée",
-          "difficulty": "difficile",
-          "score": 2,
-          "time": 19.33790349960327,
-          "response": "Pour chaque unité ennemie, determine if it is in range of my units. If it is, execute an attack. If not, do nothing.\n\nIf there are no attacks, then execute the build_building action.\n\nIf there are attacks, then execute the attack_unit action.\n\nIf there are no attacks, then execute the move_units action.\n\nIf there are no attacks, then execute the get_ai_analysis action.\n\nIf there are no attacks, then execute the get_game_state action.\n\nIf there are no attacks, then execute the move_units action.\n\nIf there are no attacks, then execute the get_ai_analysis action.\n\nIf there are no attacks, then execute the get_game_state action.\n\nIf there are no attacks, then execute the move_units action.\n\nIf there are no attacks, then execute the get_ai_analysis action.\n\nIf there are no attacks, then execute the get_game_state action.\n\nIf there are no attacks, then execute the move_units action.\n\nIf there are no attacks, then",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Reconnaissance",
-          "difficulty": "facile",
-          "score": 3,
-          "time": 19.707163333892822,
-          "response": "(Output must be a JSON object)\n{\n  \"game_state\": {\n    \"player_id\": 1,\n    \"resources\": {\n      \"credits\": 2500,\n      \"power\": 150\n    },\n    \"units\": [\n      {\n        \"id\": 1,\n        \"type\": \"infantry\",\n        \"x\": 100,\n        \"y\": 100,\n        \"health\": 100\n      },\n      {\n        \"id\": 2,\n        \"type\": \"infantry\",\n        \"x\": 110,\n        \"y\": 105,\n        \"health\": 100\n      },\n      {\n        \"id\": 3,\n        \"type\": \"tank\",\n        \"x\": 120,\n        \"y\": 110,\n        \"health\": 150\n      },\n      {\n        \"id\": 4,",
-          "expected_tool": "get_game_state"
-        },
-        {
-          "scenario": "Gestion de crise",
-          "difficulty": "difficile",
-          "score": 0,
-          "time": 19.917786121368408,
-          "response": "Les unités sont déplacées vers (100, 100) et (120, 100) et (150, 100). Les unités ennemies sont déplacées vers (300, 150) et (320, 160). Les unités ennemies sont attaquées par le joueur.\n\nLes unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par le joueur.\n\nLes unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par le joueur.\n\nLes unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par le joueur. Les unités ennemies sont attaquées par",
-          "expected_tool": "move_units"
-        }
-      ],
-      "easy_avg": 3.0,
-      "medium_avg": 4.0,
-      "hard_avg": 1.0
-    },
-    {
-      "name": "Gemma-3-270M",
-      "avg_score": 1.1666666666666667,
-      "avg_time": 7.372450391451518,
-      "results": [
-        {
-          "scenario": "Défense immédiate",
-          "difficulty": "facile",
-          "score": 3,
-          "time": 11.643838167190552,
-          "response": "- get_game_state(): Obtenir l'état actuel du jeu\n- move_units(unit_ids, target_x, target_y): Déplacer des unités\n- attack_unit(attacker_ids, target_id): Attaquer une unité ennemie\n- build_building(building_type, position_x, position_y, player_id): Construire un bâtiment\n- get_ai_analysis(language): Obtenir une analyse tactique\n\nJe suis prêt à répondre à vos questions.\nMerci!\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-\n-",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Collecte de ressources",
-          "difficulty": "moyen",
-          "score": 0,
-          "time": 0.38573145866394043,
-          "response": "Merci!",
-          "expected_tool": "move_units"
-        },
-        {
-          "scenario": "Expansion stratégique",
-          "difficulty": "moyen",
-          "score": 0,
-          "time": 10.55238938331604,
-          "response": "- Quel est le nom de la caserne ?\n- Quelle est la position de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n- Quelle est la taille de la caserne ?\n- Quelle est la couleur de la caserne ?\n-",
-          "expected_tool": "build_building"
-        },
-        {
-          "scenario": "Attaque coordonnée",
-          "difficulty": "difficile",
-          "score": 0,
-          "time": 10.54906940460205,
-          "response": "- Défense : Attaque contre les unités ennemies.\n- Destruction : Destruction des unités ennemies.\n-  (Optional)  Utilisation de la force de feu.\n-  (Optional)  Utilisation de la magie.\n-  (Optional)  Utilisation de la technologie.\n-  (Optional)  Utilisation de la base de données.\n-  (Optional)  Utilisation de la base de données de données.\n-  (Optional)  Utilisation de la base de données de données de données.\n-  (Optional)  Utilisation de la base de données de données de données.\n-  (Optional)  Utilisation de la base de données de données de données.\n-  (Optional)  Utilisation de la base de données de données de données.\n-  (Optional)  Utilisation de la base de données de données de données.\n-  (Optional)  Utilisation",
-          "expected_tool": "attack_unit"
-        },
-        {
-          "scenario": "Reconnaissance",
-          "difficulty": "facile",
-          "score": 2,
-          "time": 0.4084289073944092,
-          "response": "Merci!",
-          "expected_tool": "get_game_state"
-        },
-        {
-          "scenario": "Gestion de crise",
-          "difficulty": "difficile",
-          "score": 2,
-          "time": 10.695245027542114,
-          "response": "-  `get_game_state()` : Obtient l'état actuel du jeu.\n- `move_units(unit_ids, target_x, target_y)` : Déplace les unités dans la position spécifiée.\n- `attack_unit(attacker_ids, target_id)` : Attaque une unité ennemie.\n- `build_building(building_type, position_x, position_y, player_id)` : Construis un bâtiment.\n- `get_ai_analysis(language)` : Analyse tactique.\n- `move_units(unit_ids, target_x, target_y)` : Déplace les unités dans la position spécifiée.\n- `attack_unit(attacker_ids, target_id)` : Attaque une unité ennemie.\n- `build_building(building_type, position_x, position_y, player_id)` : Construis un",
-          "expected_tool": "move_units"
-        }
-      ],
-      "easy_avg": 2.5,
-      "medium_avg": 0.0,
-      "hard_avg": 1.0
-    }
-  ]
-}

realistic_mcp_evaluation.py

@@ -1,387 +0,0 @@
-"""
-Test réaliste des capacités MCP pour un jeu RTS
-Simule véritablement l'usage avec contexte de jeu, états, et scénarios réels
-"""
-
-import sys
-import os
-import json
-import time
-import random
-
-# Ajouter le chemin pour les imports
-sys.path.append(os.path.dirname(os.path.abspath(__file__)))
-
-# État de jeu simulé
-GAME_STATE = {
-    "player_id": 1,
-    "resources": {"credits": 2500, "power": 150},
-    "units": [
-        {"id": 1, "type": "infantry", "x": 100, "y": 100, "health": 100},
-        {"id": 2, "type": "infantry", "x": 110, "y": 105, "health": 100},
-        {"id": 3, "type": "tank", "x": 120, "y": 110, "health": 150},
-        {"id": 4, "type": "harvester", "x": 200, "y": 200, "health": 200}
-    ],
-    "buildings": [
-        {"id": 1, "type": "hq", "x": 50, "y": 50, "health": 500},
-        {"id": 2, "type": "power_plant", "x": 80, "y": 80, "health": 300},
-        {"id": 3, "type": "barracks", "x": 120, "y": 60, "health": 250}
-    ],
-    "enemy_units": [
-        {"id": 101, "type": "infantry", "x": 300, "y": 150, "health": 100},
-        {"id": 102, "type": "tank", "x": 320, "y": 160, "health": 150}
-    ],
-    "map": {
-        "width": 96,
-        "height": 72,
-        "ore_fields": [{"x": 250, "y": 200}, {"x": 400, "y": 300}]
-    }
-}
-
-def create_realistic_prompt(scenario, game_state):
-    """Crée un prompt réaliste avec contexte de jeu"""
-    
-    base_context = f"""
-Tu es un assistant IA qui contrôle un jeu RTS via MCP (Model Context Protocol).
-
-ÉTAT ACTUEL DU JEU:
-{json.dumps(game_state, indent=2)}
-
-OUTILS MCP DISPONIBLES:
-- get_game_state(): Obtenir l'état actuel du jeu
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-- attack_unit(attacker_ids, target_id): Attaquer une unité ennemie
-- build_building(building_type, position_x, position_y, player_id): Construire un bâtiment
-- get_ai_analysis(language): Obtenir une analyse tactique
-
-RÈGLES IMPORTANTES:
-- Les coordonnées doivent être valides (0-95 pour x, 0-71 pour y)
-- Les unités doivent exister (vérifier les IDs)
-- Les bâtiments nécessitent des ressources suffisantes
-- Les attaques nécessitent une portée valide
-
-Réponds UNIQUEMENT avec un objet JSON contenant l'action MCP à exécuter.
-"""
-    
-    return base_context + "\n\n" + scenario
-
-def test_model_realistic(model_path, model_name):
-    """Test réaliste d'un modèle avec scénarios de jeu"""
-    
-    try:
-        from llama_cpp import Llama
-        
-        print(f"🎮 Test réaliste de {model_name}...")
-        
-        # Initialiser le modèle
-        llm = Llama(
-            model_path=model_path,
-            n_ctx=2048,  # Plus grand pour le contexte
-            n_threads=1,
-            verbose=False
-        )
-        
-        # Scénarios réels de jeu
-        scenarios = [
-            {
-                "name": "Défense immédiate",
-                "scenario": "Il y a un tank ennemi à (320, 160) qui menace ma base. Attaque-le avec mes unités disponibles!",
-                "expected_tool": "attack_unit",
-                "difficulty": "facile"
-            },
-            {
-                "name": "Collecte de ressources",
-                "scenario": "Mes crédits sont bas (2500). Envoie le récolteur vers le champ de minerai le plus proche.",
-                "expected_tool": "move_units",
-                "difficulty": "moyen"
-            },
-            {
-                "name": "Expansion stratégique",
-                "scenario": "Je veux construire une caserne près du champ de minerai à (250, 200) pour défendre mes récolteurs.",
-                "expected_tool": "build_building",
-                "difficulty": "moyen"
-            },
-            {
-                "name": "Attaque coordonnée",
-                "scenario": "Prépare une attaque sur les positions ennemies. Utilise toutes mes unités militaires disponibles.",
-                "expected_tool": "attack_unit",
-                "difficulty": "difficile"
-            },
-            {
-                "name": "Reconnaissance",
-                "scenario": "Montre-moi l'état complet du jeu pour analyser la situation tactique.",
-                "expected_tool": "get_game_state",
-                "difficulty": "facile"
-            },
-            {
-                "name": "Gestion de crise",
-                "scenario": "Mon QG est attaqué! Déplace toutes les unités disponibles pour défendre la position (50, 50).",
-                "expected_tool": "move_units",
-                "difficulty": "difficile"
-            }
-        ]
-        
-        results = []
-        total_score = 0
-        total_time = 0
-        
-        for scenario in scenarios:
-            print(f"\n📋 Scénario: {scenario['name']} ({scenario['difficulty']})")
-            
-            # Créer le prompt réaliste
-            prompt = create_realistic_prompt(scenario['scenario'], GAME_STATE)
-            
-            start_time = time.time()
-            
-            # Tester le modèle
-            response = llm(
-                prompt,
-                max_tokens=200,
-                temperature=0.1,
-                stop=["</s>", "<|im_end|>", "```"]
-            )
-            
-            response_time = time.time() - start_time
-            response_text = response['choices'][0]['text'].strip()
-            
-            # Évaluer la réponse de manière approfondie
-            score = evaluate_realistic_response(response_text, scenario, GAME_STATE)
-            
-            total_score += score
-            total_time += response_time
-            
-            print(f"   ⏱️  Temps: {response_time:.2f}s")
-            print(f"   📊 Score: {score}/10")
-            print(f"   📝 Réponse: {response_text[:100]}...")
-            
-            results.append({
-                'scenario': scenario['name'],
-                'difficulty': scenario['difficulty'],
-                'score': score,
-                'time': response_time,
-                'response': response_text,
-                'expected_tool': scenario['expected_tool']
-            })
-        
-        avg_score = total_score / len(scenarios)
-        avg_time = total_time / len(scenarios)
-        
-        print(f"\n📈 Résultats pour {model_name}:")
-        print(f"   Score moyen: {avg_score:.1f}/10")
-        print(f"   Temps moyen: {avg_time:.2f}s")
-        
-        # Analyse par difficulté
-        easy_scores = [r['score'] for r in results if r['difficulty'] == 'facile']
-        medium_scores = [r['score'] for r in results if r['difficulty'] == 'moyen']
-        hard_scores = [r['score'] for r in results if r['difficulty'] == 'difficile']
-        
-        print(f"   Scénarios faciles: {sum(easy_scores)/len(easy_scores):.1f}/10" if easy_scores else "   Scénarios faciles: N/A")
-        print(f"   Scénarios moyens: {sum(medium_scores)/len(medium_scores):.1f}/10" if medium_scores else "   Scénarios moyens: N/A")
-        print(f"   Scénarios difficiles: {sum(hard_scores)/len(hard_scores):.1f}/10" if hard_scores else "   Scénaires difficiles: N/A")
-        
-        return {
-            'name': model_name,
-            'avg_score': avg_score,
-            'avg_time': avg_time,
-            'results': results,
-            'easy_avg': sum(easy_scores)/len(easy_scores) if easy_scores else 0,
-            'medium_avg': sum(medium_scores)/len(medium_scores) if medium_scores else 0,
-            'hard_avg': sum(hard_scores)/len(hard_scores) if hard_scores else 0
-        }
-        
-    except Exception as e:
-        print(f"❌ Erreur avec {model_name}: {e}")
-        return {
-            'name': model_name,
-            'avg_score': 0,
-            'avg_time': 0,
-            'error': str(e)
-        }
-
-def evaluate_realistic_response(response, scenario, game_state):
-    """Évaluation approfondie de la réponse MCP"""
-    
-    score = 0
-    
-    # 1. Format JSON valide (3 points)
-    try:
-        json_response = json.loads(response)
-        score += 3
-    except:
-        # Essayer d'extraire JSON du texte
-        import re
-        json_match = re.search(r'\{.*\}', response, re.DOTALL)
-        if json_match:
-            try:
-                json_response = json.loads(json_match.group())
-                score += 2  # JSON partiellement valide
-            except:
-                json_response = {}
-        else:
-            json_response = {}
-    
-    # 2. Outil correct (3 points)
-    expected_tool = scenario['expected_tool']
-    if 'tool' in json_response and json_response['tool'] == expected_tool:
-        score += 3
-    elif expected_tool in response:
-        score += 2  # Outil mentionné mais pas dans le bon format
-    elif any(tool in response for tool in ['get_game_state', 'move_units', 'attack_unit', 'build_building']):
-        score += 1  # Un outil MCP est mentionné
-    
-    # 3. Paramètres valides (2 points)
-    if expected_tool == 'attack_unit':
-        if 'attacker_ids' in json_response and 'target_id' in json_response:
-            # Vérifier si les IDs existent
-            attackers = json_response['attacker_ids']
-            target = json_response['target_id']
-            
-            valid_attackers = any(unit['id'] in attackers if isinstance(attackers, list) else unit['id'] == attackers for unit in game_state['units'])
-            valid_target = any(unit['id'] == target for unit in game_state['enemy_units'])
-            
-            if valid_attackers and valid_target:
-                score += 2
-            elif valid_attackers or valid_target:
-                score += 1
-                
-    elif expected_tool == 'move_units':
-        if 'unit_ids' in json_response and 'target_x' in json_response and 'target_y' in json_response:
-            # Vérifier coordonnées valides
-            x, y = json_response['target_x'], json_response['target_y']
-            if 0 <= x <= 95 and 0 <= y <= 71:
-                score += 2
-                
-    elif expected_tool == 'build_building':
-        if 'building_type' in json_response and 'position_x' in json_response and 'position_y' in json_response:
-            score += 2
-            
-    elif expected_tool == 'get_game_state':
-        if not json_response or len(json_response) == 0:
-            score += 2  # get_game_state n'a pas besoin de paramètres
-    
-    # 4. Cohérence tactique (2 points)
-    if scenario['difficulty'] == 'difficile':
-        # Pour les scénarios difficiles, vérifier la pertinence tactique
-        if 'attack' in scenario['scenario'].lower() and 'attack' in response.lower():
-            score += 1
-        if 'defend' in scenario['scenario'].lower() and ('defend' in response.lower() or 'move' in response.lower()):
-            score += 1
-    else:
-        # Pour les scénarios plus simples
-        if any(word in response.lower() for word in ['game', 'state', 'move', 'attack', 'build']):
-            score += 1
-    
-    return min(score, 10)
-
-def run_realistic_evaluation():
-    """Exécute l'évaluation réaliste complète"""
-    
-    print("🎮 ÉVALUATION RÉALISTE MCP POUR JEU RTS")
-    print("=" * 70)
-    print("Test avec contexte de jeu, scénarios réels et validation tactique")
-    print("=" * 70)
-    
-    # Modèles à tester
-    models = [
-        {
-            'name': 'Qwen2.5-0.5B',
-            'path': 'qwen2.5-0.5b-instruct-q4_0.gguf'
-        },
-        {
-            'name': 'Qwen3-0.6B',
-            'path': 'Qwen3-0.6B-Q8_0.gguf'
-        },
-        {
-            'name': 'Gemma-3-270M',
-            'path': 'gemma-3-270m-it-qat-Q8_0.gguf'
-        }
-    ]
-    
-    results = []
-    
-    for model in models:
-        if os.path.exists(model['path']):
-            result = test_model_realistic(model['path'], model['name'])
-            results.append(result)
-            print("\n" + "="*50)
-        else:
-            print(f"❌ Modèle non trouvé: {model['path']}")
-    
-    # Analyse comparative réaliste
-    print("\n" + "="*70)
-    print("📊 ANALYSE COMPARATIVE RÉALISTE")
-    print("="*70)
-    
-    successful_results = [r for r in results if 'error' not in r and r['avg_score'] > 0]
-    
-    if successful_results:
-        # Classement par performance globale
-        sorted_by_performance = sorted(successful_results, key=lambda x: x['avg_score'], reverse=True)
-        
-        print(f"\n🏆 CLASSEMENT PAR PERFORMANCE RÉELLE:")
-        for i, result in enumerate(sorted_by_performance, 1):
-            print(f"   {i}. {result['name']}: {result['avg_score']:.1f}/10 | {result['avg_time']:.2f}s")
-        
-        # Analyse par difficulté
-        print(f"\n📈 PERFORMANCE PAR DIFFICULTÉ:")
-        
-        difficulties = ['facile', 'moyen', 'difficile']
-        for diff in difficulties:
-            print(f"\n🔸 Scénarios {diff}s:")
-            for result in successful_results:
-                avg_key = f"{diff}_avg"
-                if hasattr(result, avg_key):
-                    score = getattr(result, avg_key)
-                    print(f"   {result['name']}: {score:.1f}/10")
-        
-        # Recommandations basées sur l'usage réel
-        best_overall = sorted_by_performance[0]
-        
-        print(f"\n🎯 RECOMMANDATIONS POUR VOTRE JEU RTS:")
-        
-        if best_overall['avg_score'] >= 7:
-            print(f"✅ {best_overall['name']} est EXCELLENT pour la production")
-            print(f"   • Gère bien les scénarios complexes")
-            print(f"   • Réponses tactiques cohérentes")
-        elif best_overall['avg_score'] >= 5:
-            print(f"👍 {best_overall['name']} est BON pour la production")
-            print(f"   • Fonctionne bien pour les commandes de base")
-            print(f"   • Nécessite peut-être une validation supplémentaire")
-        else:
-            print(f"⚠️  {best_overall['name']} nécessite des améliorations")
-            print(f"   • Considérer des prompts plus structurés")
-            print(f"   • Ajouter des règles de validation")
-        
-        # Analyse spécifique aux cas d'usage
-        print(f"\n🎮 ANALYSE SPÉCIFIQUE AU JEU:")
-        
-        for result in successful_results:
-            print(f"\n🔹 {result['name']}:")
-            
-            # Analyser les résultats par scénario
-            scenario_scores = {}
-            for scenario_result in result['results']:
-                scenario_name = scenario_result['scenario']
-                if scenario_name not in scenario_scores:
-                    scenario_scores[scenario_name] = []
-                scenario_scores[scenario_name].append(scenario_result['score'])
-            
-            for scenario, scores in scenario_scores.items():
-                avg_score = sum(scores) / len(scores)
-                print(f"   {scenario}: {avg_score:.1f}/10")
-    
-    # Sauvegarder les résultats réalistes
-    realistic_results = {
-        'evaluation_type': 'realistic_mcp_game_test',
-        'game_state_sample': GAME_STATE,
-        'results': results,
-        'successful_models': successful_results
-    }
-    
-    with open("realistic_mcp_evaluation.json", "w", encoding="utf-8") as f:
-        json.dump(realistic_results, f, indent=2, ensure_ascii=False)
-    
-    print(f"\n📄 Résultats réalistes sauvegardés dans: realistic_mcp_evaluation.json")
-
-if __name__ == "__main__":
-    run_realistic_evaluation()

simple_model_diagnostic.py

@@ -1,94 +0,0 @@
-"""
-Diagnostic simple des modèles MCP problématiques
-Test basique pour identifier les problèmes techniques
-"""
-
-import os
-import sys
-
-def check_model_files():
-    """Vérifier l'intégrité des fichiers modèles"""
-    
-    print("🔍 VÉRIFICATION DES FICHIERS MODÈLES")
-    print("=" * 50)
-    
-    models = [
-        ('mcp-instruct-v1.Q4_K_M.gguf', 'MCP-Instruct-v1'),
-        ('mcprl-3b-exa.Q2_K.gguf', 'MCPR L-3B-Exa'),
-        ('gemma-3n-E2B-it-UD-IQ2_XXS.gguf', 'Gemma-3n-E2B-it'),
-        ('qwen2.5-coder-0.5b-instruct-q4_0.gguf', 'Qwen2.5-Coder-0.5B')
-    ]
-    
-    for filename, model_name in models:
-        if os.path.exists(filename):
-            size_mb = os.path.getsize(filename) / (1024 * 1024)
-            print(f"✅ {model_name}: {size_mb:.1f} MB")
-        else:
-            print(f"❌ {model_name}: Fichier non trouvé")
-
-def test_basic_llm():
-    """Test basique de chargement LLM"""
-    
-    print("\n🧪 TEST DE CHARGEMENT LLAMA-CPP")
-    print("=" * 50)
-    
-    try:
-        from llama_cpp import Llama
-        print("✅ llama-cpp-python importé avec succès")
-        
-        # Test avec un modèle qui fonctionne
-        test_model = 'qwen2.5-coder-0.5b-instruct-q4_0.gguf'
-        if os.path.exists(test_model):
-            try:
-                llm = Llama(model_path=test_model, n_ctx=512, verbose=False)
-                print("✅ Modèle de test chargé avec succès")
-                
-                # Test de génération simple
-                response = llm("Hello", max_tokens=5, temperature=0.1)
-                print(f"✅ Génération test: {response}")
-                
-            except Exception as e:
-                print(f"❌ Erreur chargement modèle: {e}")
-        else:
-            print("❌ Modèle de test non trouvé")
-            
-    except ImportError:
-        print("❌ llama-cpp-python non installé")
-    except Exception as e:
-        print(f"❌ Erreur import: {e}")
-
-def analyze_quantization_issues():
-    """Analyser les problèmes de quantisation"""
-    
-    print("\n📊 ANALYSE DES PROBLÈMES DE QUANTISATION")
-    print("=" * 50)
-    
-    quantization_issues = {
-        'Q2_K': "Quantisation très agressive - peut causer corruption",
-        'IQ2_XXS': "Quantisation extrême - perte significative de capacité", 
-        'Q4_K_M': "Quantisation moyenne - généralement fiable",
-        'Q8_0': "Quantisation légère - meilleure qualité"
-    }
-    
-    for quant, description in quantization_issues.items():
-        print(f"{quant}: {description}")
-
-def main():
-    """Diagnostic principal"""
-    
-    print("🔧 DIAGNOSTIC TECHNIQUE DES MODÈLES MCP")
-    print("=" * 60)
-    
-    check_model_files()
-    test_basic_llm()
-    analyze_quantization_issues()
-    
-    print("\n🎯 CONCLUSIONS:")
-    print("-" * 30)
-    print("1. MCP-Instruct-v1: Problème technique (llama_decode=-1)")
-    print("2. MCPR L-3B-Exa: Quantisation Q2_K trop agressive") 
-    print("3. Gemma-3n-E2B-it: Quantisation IQ2_XXS extrême")
-    print("4. Solution: Télécharger versions Q8_0")
-
-if __name__ == "__main__":
-    main()

simple_qwen_test.py

@@ -1,137 +0,0 @@
-"""
-Test simple pour évaluer les capacités MCP de Qwen2.5 0.5B
-"""
-
-import sys
-import os
-
-# Ajouter le chemin pour importer les modules locaux
-sys.path.append(os.path.dirname(os.path.abspath(__file__)))
-
-def test_basic_capability():
-    """Test de capacité MCP de base"""
-    
-    print("=== Test de capacité MCP de base ===")
-    
-    # Test 1: Compréhension des outils MCP
-    prompt1 = """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-Outils disponibles:
-- get_game_state(): Obtenir l'état du jeu
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Montre-moi l'état du jeu"
-
-Réponds avec un objet JSON contenant l'appel d'outil à exécuter.
-"""
-    
-    print("Test 1 - Compréhension des outils:")
-    print(f"Prompt: {prompt1[:100]}...")
-    
-    # Test 2: Traduction d'action simple
-    prompt2 = """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-Outils disponibles:
-- move_units(unit_ids, target_x, target_y): Déplacer des unités
-
-Commande utilisateur: "Déplace mon infanterie vers la position 100, 200"
-
-Réponds avec un objet JSON contenant l'appel d'outil à exécuter.
-"""
-    
-    print("\nTest 2 - Traduction d'action:")
-    print(f"Prompt: {prompt2[:100]}...")
-    
-    # Test 3: Planification complexe
-    prompt3 = """
-Tu es un assistant IA qui contrôle un jeu RTS via MCP.
-Outils disponibles:
-- get_game_state(): Obtenir l'état du jeu
-- move_units(): Déplacer des unités
-- attack_unit(): Attaquer une unité ennemie
-- build_building(): Construire un bâtiment
-
-Commande utilisateur: "Je veux construire une base près du champ de minerai et la défendre avec des tourelles"
-
-Décompose cette commande en une séquence d'appels d'outils MCP.
-"""
-    
-    print("\nTest 3 - Planification complexe:")
-    print(f"Prompt: {prompt3[:100]}...")
-    
-    return {
-        "test1": {"prompt": prompt1, "expected": "get_game_state"},
-        "test2": {"prompt": prompt2, "expected": "move_units avec paramètres"},
-        "test3": {"prompt": prompt3, "expected": "séquence d'actions"}
-    }
-
-def test_with_ai_model():
-    """Test avec le modèle AI réel"""
-    
-    try:
-        from ai_analysis import query_llm
-        
-        print("\n=== Test avec le modèle Qwen2.5 0.5B ===")
-        
-        # Test simple
-        test_prompt = """
-Tu es un assistant IA. Réponds simplement avec "TEST_RÉUSSI" si tu comprends cette instruction.
-"""
-        
-        print("Test de connexion au modèle...")
-        
-        response = query_llm(
-            prompt=test_prompt,
-            max_tokens=50,
-            temperature=0.1
-        )
-        
-        print(f"Réponse du modèle: {response}")
-        
-        if "TEST_RÉUSSI" in response:
-            print("✅ Modèle fonctionne correctement")
-            return True
-        else:
-            print("❌ Problème avec le modèle")
-            return False
-            
-    except Exception as e:
-        print(f"❌ Erreur lors du test du modèle: {e}")
-        return False
-
-def main():
-    """Fonction principale"""
-    
-    print("🔍 Évaluation des capacités MCP de Qwen2.5 0.5B")
-    print("=" * 60)
-    
-    # Test de base
-    tests = test_basic_capability()
-    
-    # Test avec le modèle
-    model_working = test_with_ai_model()
-    
-    # Résumé
-    print("\n" + "=" * 60)
-    print("📊 RÉSUMÉ DE L'ÉVALUATION")
-    print("=" * 60)
-    
-    print("\n1. CAPACITÉS MCP DE BASE:")
-    print("   ✅ Compréhension des outils MCP: Test défini")
-    print("   ✅ Traduction d'actions simples: Test défini")
-    print("   ✅ Planification complexe: Test défini")
-    
-    print(f"\n2. ÉTAT DU MODÈLE: {'✅ OPÉRATIONNEL' if model_working else '❌ PROBLÈME'}")
-    
-    print("\n3. RECOMMANDATIONS:")
-    if model_working:
-        print("   • Le modèle peut être utilisé pour la traduction MCP")
-        print("   • Commencez par des commandes simples")
-        print("   • Ajoutez une validation pour les actions complexes")
-    else:
-        print("   • Vérifiez que le modèle est téléchargé")
-        print("   • Assurez-vous que llama-cpp-python est installé")
-        print("   • Testez avec des commandes plus simples d'abord")
-
-if __name__ == "__main__":
-    main()

test_improved_mcp_formats.py

@@ -1,229 +0,0 @@
-"""
-Test amélioré avec formats de prompts optimisés pour modèles spécialisés MCP
-"""
-
-import sys
-import os
-import json
-import time
-from llama_cpp import Llama
-
-def test_mcp_formats(model_path, model_name):
-    """Tester différents formats de prompt pour les modèles MCP"""
-    
-    print(f"\n🧪 TEST AVEC FORMATS OPTIMISÉS: {model_name}")
-    print("=" * 60)
-    
-    if not os.path.exists(model_path):
-        print(f"❌ Fichier modèle non trouvé: {model_path}")
-        return
-    
-    try:
-        # Initialiser avec différents paramètres pour modèles MCP
-        llm = Llama(
-            model_path=model_path,
-            n_ctx=2048,  # Plus grand contexte pour modèles MCP
-            n_threads=1,
-            verbose=False,
-            n_gpu_layers=0,
-            temperature=0.3,  # Température légèrement plus élevée
-            top_p=0.9
-        )
-        
-        test_command = "show game state"
-        
-        # Formats de prompt optimisés basés sur les standards MCP
-        prompt_formats = [
-            {
-                'name': 'Format MCP structuré complet',
-                'template': '''[INST] You are an MCP (Model Context Protocol) assistant for an RTS game. Your task is to respond ONLY with JSON format.
-
-Available tools:
-- get_game_state(): Returns current game state information
-- move_units(unit_ids, target_x, target_y): Move specified units to coordinates
-- attack_unit(attacker_ids, target_id): Attack target unit with attackers
-- build_building(building_type, position_x, position_y): Construct building at position
-
-User command: "{command}"
-
-Respond with ONLY JSON in this format:
-{{"tool": "tool_name", "args": {{<parameters>}}}}
-
-Do not include any explanations, comments, or additional text. Only JSON. [/INST]'''
-            },
-            {
-                'name': 'Format MCP instruct simple',
-                'template': '''<|system|>
-You are an MCP assistant for an RTS game. Respond with ONLY JSON.
-Available tools: get_game_state, move_units, attack_unit, build_building
-</|system|>
-<|user|>
-{command}
-</|user|>
-<|assistant|>'''
-            },
-            {
-                'name': 'Format JSON strict',
-                'template': '''Your role: MCP assistant for RTS game
-Tools: get_game_state(), move_units(), attack_unit(), build_building()
-Command: {command}
-
-JSON Response:'''
-            },
-            {
-                'name': 'Format conversation MCP',
-                'template': '''User: {command}
-
-Assistant: {{"tool": "get_game_state", "args": {{}}}}'''
-            },
-            {
-                'name': 'Format code-oriented',
-                'template': '''// MCP assistant for RTS game
-// Available functions:
-// - get_game_state()
-// - move_units(unit_ids, target_x, target_y) 
-// - attack_unit(attacker_ids, target_id)
-// - build_building(building_type, position_x, position_y)
-
-// User: {command}
-// Response (JSON only):'''
-            }
-        ]
-        
-        for i, format_info in enumerate(prompt_formats, 1):
-            print(f"\n{i}. {format_info['name']}:")
-            
-            prompt = format_info['template'].format(command=test_command)
-            
-            try:
-                # Essayer avec différents paramètres
-                response = llm(
-                    prompt,
-                    max_tokens=100,
-                    temperature=0.3,
-                    stop=["</s>", "<|im_end|>", "```", "\n\n", "User:"]
-                )
-                
-                # Extraire la réponse selon le format retourné
-                if hasattr(response, 'choices') and len(response.choices) > 0:
-                    response_text = response.choices[0].text.strip()
-                elif isinstance(response, dict) and 'choices' in response:
-                    response_text = response['choices'][0]['text'].strip()
-                else:
-                    response_text = str(response)
-                
-                print(f"   Prompt (début): {prompt[:80]}...")
-                print(f"   Réponse: {response_text[:150]}")
-                
-                # Vérifier JSON
-                try:
-                    parsed = json.loads(response_text)
-                    print(f"   ✅ JSON VALIDE: {parsed}")
-                except json.JSONDecodeError:
-                    # Essayer d'extraire JSON
-                    import re
-                    json_match = re.search(r'\{[^{}]*\{[^{}]*\}[^{}]*\}|\{[^{}]*\}', response_text)
-                    if json_match:
-                        try:
-                            parsed = json.loads(json_match.group())
-                            print(f"   ✅ JSON EXTRACTÉ: {parsed}")
-                        except:
-                            print(f"   ❌ JSON INVALIDE")
-                    else:
-                        print(f"   ❌ PAS DE JSON DÉTECTÉ")
-                        
-            except Exception as e:
-                print(f"   ❌ Erreur: {e}")
-                
-    except Exception as e:
-        print(f"❌ Erreur de chargement: {e}")
-
-def test_model_capabilities(model_path, model_name):
-    """Tester les capacités de base du modèle"""
-    
-    print(f"\n🔍 TEST DE CAPACITÉS: {model_name}")
-    print("=" * 60)
-    
-    if not os.path.exists(model_path):
-        print(f"❌ Fichier modèle non trouvé: {model_path}")
-        return
-    
-    try:
-        llm = Llama(
-            model_path=model_path,
-            n_ctx=2048,
-            n_threads=1,
-            verbose=False,
-            n_gpu_layers=0
-        )
-        
-        # Test de génération de base
-        test_prompts = [
-            "Hello",
-            "What is your name?",
-            "Write JSON",
-            "{\"test\": \"value\"}"
-        ]
-        
-        for prompt in test_prompts:
-            try:
-                response = llm(
-                    prompt,
-                    max_tokens=20,
-                    temperature=0.1
-                )
-                response_text = response['choices'][0]['text'].strip()
-                print(f"   '{prompt}' → '{response_text}'")
-            except Exception as e:
-                print(f"   '{prompt}' → Erreur: {e}")
-                
-    except Exception as e:
-        print(f"❌ Erreur de chargement: {e}")
-
-def main():
-    """Test des modèles problématiques avec formats améliorés"""
-    
-    print("🚀 TEST AMÉLIORÉ DES MODÈLES SPÉCIALISÉS MCP")
-    print("=" * 70)
-    
-    models_to_test = [
-        {
-            'name': 'MCP-Instruct-v1',
-            'path': 'mcp-instruct-v1.Q4_K_M.gguf',
-            'type': 'mcp_specialized'
-        },
-        {
-            'name': 'MCPR L-3B-Exa', 
-            'path': 'mcprl-3b-exa.Q2_K.gguf',
-            'type': 'mcp_specialized'
-        },
-        {
-            'name': 'Gemma-3n-E2B-it',
-            'path': 'gemma-3n-E2B-it-UD-IQ2_XXS.gguf',
-            'type': 'mcp_specialized'
-        },
-        {
-            'name': 'Qwen2.5-Coder-0.5B (référence)',
-            'path': 'qwen2.5-coder-0.5b-instruct-q4_0.gguf',
-            'type': 'code_specialized'
-        }
-    ]
-    
-    for model in models_to_test:
-        if os.path.exists(model['path']):
-            test_model_capabilities(model['path'], model['name'])
-            test_mcp_formats(model['path'], model['name'])
-        else:
-            print(f"\n❌ Modèle non trouvé: {model['path']}")
-    
-    print("\n" + "=" * 70)
-    print("📊 RÉCAPITULATIF DES PROBLÈMES:")
-    print("""
-1. **MCP-Instruct-v1**: Erreur technique (llama_decode=-1) - probablement incompatible
-2. **MCPR L-3B-Exa**: Génération de texte corrompu - mauvaise quantisation ou format
-3. **Gemma-3n-E2B-it**: Réponses vides - format de prompt incompatible
-4. **Modèles code-spécialisés**: Fonctionnent mieux car habitués au JSON
-    """)
-
-if __name__ == "__main__":
-    main()

test_mcp.py

@@ -1,50 +0,0 @@
-"""
-Test script for the MCP server
-"""
-
-import asyncio
-import sys
-import os
-
-# Add the web directory to the path so we can import the mcp_server module
-sys.path.insert(0, os.path.join(os.path.dirname(__file__)))
-
-def test_fastmcp_import():
-    """Test that we can import the FastMCP class."""
-    try:
-        from mcp.server import FastMCP
-        print("✓ Successfully imported FastMCP")
-        return True
-    except Exception as e:
-        print(f"✗ Failed to import FastMCP: {e}")
-        return False
-
-def test_imports():
-    """Test that we can import the MCP server module."""
-    try:
-        # We need to mock the app import since it's not available in this context
-        import app
-        from mcp_server import RTSGameMCP
-        print("✓ Successfully imported RTSGameMCP")
-        return True
-    except Exception as e:
-        print(f"✗ Failed to import RTSGameMCP: {e}")
-        return False
-
-if __name__ == "__main__":
-    print("Testing MCP server imports...")
-    
-    tests = [
-        test_fastmcp_import
-        # test_imports  # Skip this for now since it depends on the app module
-    ]
-    
-    results = []
-    for test in tests:
-        results.append(test())
-    
-    if all(results):
-        print("\n✓ All tests passed!")
-    else:
-        print("\n✗ Some tests failed!")
-        sys.exit(1)

test_q8_models.py

@@ -1,180 +0,0 @@
-#!/usr/bin/env python3
-"""
-Test des modèles Q8_0 pour voir si la meilleure quantisation résout les problèmes
-"""
-
-import sys
-import os
-import time
-import json
-
-def test_q8_models():
-    """Tester les modèles Q8_0 avec différents formats de prompt"""
-    
-    print("🚀 TEST DES MODÈLES Q8_0 - FORMATS AMÉLIORÉS")
-    print("=" * 70)
-    
-    # Modèles Q8_0 téléchargés
-    models = [
-        {
-            'name': 'MCP-Instruct-v1-Q8',
-            'path': 'mcp-instruct-v1.Q8_0.gguf',
-            'format': 'mcp_structured',
-            'n_ctx': 4096
-        },
-        {
-            'name': 'MCPR-L-3B-Exa-Q8', 
-            'path': 'mcprl-3b-exa.Q8_0.gguf',
-            'format': 'instruct',
-            'n_ctx': 4096
-        },
-        {
-            'name': 'Gemma-3n-E2B-it-Q8',
-            'path': 'google_gemma-3n-E2B-it-Q8_0.gguf',
-            'format': 'code',
-            'n_ctx': 4096
-        }
-    ]
-    
-    # Formats de prompts optimisés
-    prompt_templates = {
-        'mcp_structured': '''<|im_start|>system
-You are an MCP assistant for an RTS game. Respond with ONLY JSON format:
-{"tool": "tool_name", "args": {}}
-
-Available tools:
-- get_game_state(): Get current game state
-- move_units(unit_ids, target_x, target_y): Move units to coordinates
-- attack_unit(attacker_ids, target_id): Attack a specific unit
-- build_building(building_type, position_x, position_y): Construct a building
-
-Always respond with valid JSON, no additional text.<|im_end|>
-<|im_start|>user
-{command}<|im_end|>
-<|im_start|>assistant
-''',
-        
-        'instruct': '''[INST] You are an MCP assistant. Respond with ONLY JSON format:
-{"tool": "tool_name", "args": {}}
-
-Available tools:
-- get_game_state(): Get current game state
-- move_units(unit_ids, target_x, target_y): Move units to coordinates
-- attack_unit(attacker_ids, target_id): Attack a specific unit
-- build_building(building_type, position_x, position_y): Construct a building
-
-User command: {command}
-
-Respond with only JSON: [/INST]
-''',
-        
-        'code': '''// MCP assistant for RTS game
-// Tools: get_game_state(), move_units(), attack_unit(), build_building()
-// Command: {command}
-// Output JSON only:
-{"tool": "tool_name", "args": {}}'''
-    }
-    
-    test_command = "show game state"
-    
-    for model in models:
-        print(f"\n🧪 TEST: {model['name']}")
-        print("-" * 50)
-        
-        if not os.path.exists(model['path']):
-            print(f"   ❌ Fichier non trouvé: {model['path']}")
-            continue
-        
-        file_size_mb = os.path.getsize(model['path']) / (1024*1024)
-        print(f"   📏 Taille: {file_size_mb:.1f} MB")
-        print(f"   🔧 Format: {model['format']}")
-        print(f"   🧠 Contexte: {model['n_ctx']} tokens")
-        
-        try:
-            from llama_cpp import Llama
-            
-            llm = Llama(
-                model_path=model['path'],
-                n_ctx=model['n_ctx'],
-                n_threads=1,
-                verbose=False,
-                n_gpu_layers=0
-            )
-            
-            prompt = prompt_templates[model['format']].format(command=test_command)
-            
-            print(f"   📝 Prompt (début): {prompt[:80]}...")
-            
-            start_time = time.time()
-            
-            try:
-                response = llm(
-                    prompt,
-                    max_tokens=100,
-                    temperature=0.3,
-                    stop=["</s>", "<|im_end|>", "```", "\n\n"]
-                )
-                
-                response_time = time.time() - start_time
-                
-                # Gérer la réponse (compatible avec différentes versions de llama-cpp)
-                if isinstance(response, dict) and 'choices' in response:
-                    response_text = response['choices'][0]['text'].strip()
-                elif hasattr(response, '__iter__') and not isinstance(response, str):
-                    # Gérer les réponses stream
-                    response_text = ""
-                    for chunk in response:
-                        if isinstance(chunk, dict) and 'choices' in chunk:
-                            response_text += chunk['choices'][0]['text']
-                else:
-                    response_text = str(response)
-                
-                print(f"   ⏱️  Temps: {response_time:.2f}s")
-                print(f"   📄 Réponse: {response_text[:200]}")
-                
-                # Vérifier JSON
-                try:
-                    parsed = json.loads(response_text)
-                    print(f"   ✅ JSON VALIDE: {json.dumps(parsed, indent=2)}")
-                    
-                    # Évaluer la pertinence
-                    if 'tool' in parsed:
-                        print(f"   🎯 Outil identifié: {parsed['tool']}")
-                    if 'args' in parsed:
-                        print(f"   📋 Arguments: {parsed['args']}")
-                        
-                except json.JSONDecodeError:
-                    print(f"   ❌ JSON INVALIDE")
-                    # Essayer d'extraire JSON
-                    import re
-                    json_match = re.search(r'\{[^{}]*\{[^{}]*\}[^{}]*\}|\{[^{}]*\}', response_text)
-                    if json_match:
-                        try:
-                            extracted_json = json.loads(json_match.group())
-                            print(f"   🔍 JSON extrait: {json.dumps(extracted_json, indent=2)}")
-                        except:
-                            print(f"   ❌ Impossible d'extraire JSON valide")
-                            
-            except Exception as e:
-                print(f"   ❌ Erreur génération: {e}")
-                
-        except Exception as e:
-            print(f"   ❌ Erreur chargement: {e}")
-    
-    print(f"\n{'='*70}")
-    print("🎯 COMPARAISON AVEC ANCIENNES VERSIONS")
-    print("=" * 70)
-    
-    print("\n📊 RÉSULTATS ANCIENNES VERSIONS:")
-    print("   • MCP-Instruct-v1 (Q4_K_M): Erreur technique (llama_decode=-1)")
-    print("   • MCPR L-3B-Exa (Q2_K): Texte corrompu avec caractères spéciaux")
-    print("   • Gemma-3n-E2B-it (IQ2_XXS): Réponses vides, pas de sortie")
-    
-    print("\n📈 ATTENTES POUR Q8_0:")
-    print("   • Meilleure qualité de quantisation")
-    print("   • Moins de corruption de texte")
-    print("   • Réponses plus cohérentes")
-    print("   • JSON valide possible")
-
-if __name__ == "__main__":
-    test_q8_models()

test_qwen_mcp_capabilities.py

@@ -1,326 +0,0 @@
-"""
-Test script to evaluate Qwen2.5 0.5B capabilities with MCP for RTS game
-Tests basic MCP understanding and complex planning capabilities
-"""
-
-import json
-import time
-from typing import Dict, Any, List
-from ai_analysis import query_llm
-
-# Test prompts for different capability levels
-TEST_PROMPTS = {
-    "basic_mcp": [
-        {
-            "name": "Simple Tool Call",
-            "prompt": """
-You are an AI assistant controlling an RTS game through MCP (Model Context Protocol).
-Available tools:
-- get_game_state(): Get current game state
-- move_units(unit_ids, target_x, target_y): Move units to position
-- attack_unit(attacker_ids, target_id): Attack enemy unit
-
-User command: "Show me the current game state"
-
-Respond with a JSON object containing the tool call to execute.
-""",
-            "expected": {"tool": "get_game_state", "args": {}}
-        },
-        {
-            "name": "Basic Action Translation",
-            "prompt": """
-You are an AI assistant controlling an RTS game through MCP.
-Available tools:
-- move_units(unit_ids, target_x, target_y): Move units to position
-- attack_unit(attacker_ids, target_id): Attack enemy unit
-
-User command: "Move my infantry to position 100, 200"
-
-Respond with a JSON object containing the tool call to execute.
-""",
-            "expected": {"tool": "move_units", "args": {"unit_ids": ["infantry"], "target_x": 100, "target_y": 200}}
-        }
-    ],
-    "complex_planning": [
-        {
-            "name": "Multi-step Strategy",
-            "prompt": """
-You are an AI assistant controlling an RTS game through MCP.
-Available tools:
-- get_game_state(): Get current game state
-- move_units(unit_ids, target_x, target_y): Move units
-- attack_unit(attacker_ids, target_id): Attack enemy
-- build_building(building_type, position_x, position_y, player_id): Build building
-
-User command: "I want to build a base near the ore field and defend it with turrets"
-
-Break this down into a sequence of MCP tool calls. Respond with a JSON array of tool calls.
-""",
-            "expected": {"type": "sequence", "steps": ["get_game_state", "build_building", "build_building"]}
-        },
-        {
-            "name": "Strategic Analysis",
-            "prompt": """
-You are an AI assistant controlling an RTS game through MCP.
-Available tools:
-- get_game_state(): Get current game state
-- get_ai_analysis(language): Get tactical analysis
-
-User command: "Analyze the battlefield and suggest the best strategy"
-
-Respond with a JSON object containing the tool calls needed.
-""",
-            "expected": {"type": "analysis", "steps": ["get_game_state", "get_ai_analysis"]}
-        }
-    ],
-    "advanced_mcp": [
-        {
-            "name": "Parameter Extraction",
-            "prompt": """
-You are an AI assistant controlling an RTS game through MCP.
-Available tools:
-- move_units(unit_ids, target_x, target_y): Move units
-
-User command: "Move tanks 1, 3, and 7 to coordinates 150, 75"
-
-Extract the parameters and respond with a JSON tool call.
-""",
-            "expected": {"tool": "move_units", "args": {"unit_ids": [1, 3, 7], "target_x": 150, "target_y": 75}}
-        },
-        {
-            "name": "Error Handling",
-            "prompt": """
-You are an AI assistant controlling an RTS game through MCP.
-Available tools:
-- move_units(unit_ids, target_x, target_y): Move units
-
-User command: "Move my units to the enemy base"
-
-Since you don't know the exact coordinates, how would you handle this?
-Respond with a JSON object showing your approach.
-""",
-            "expected": {"type": "needs_clarification", "message": "Need coordinates for enemy base"}
-        }
-    ]
-}
-
-def test_qwen_capabilities():
-    """Run comprehensive tests on Qwen2.5 0.5B MCP capabilities"""
-    
-    print("=== Testing Qwen2.5 0.5B MCP Capabilities ===\n")
-    
-    results = {}
-    
-    for category, tests in TEST_PROMPTS.items():
-        print(f"\n📊 Testing {category.replace('_', ' ').title()}:")
-        print("-" * 50)
-        
-        category_results = []
-        
-        for test in tests:
-            print(f"\n🧪 Test: {test['name']}")
-            print(f"Prompt: {test['prompt'][:100]}...")
-            
-            try:
-                # Query the LLM
-                start_time = time.time()
-                response = query_llm(
-                    prompt=test['prompt'],
-                    max_tokens=500,
-                    temperature=0.1,
-                    system_message="You are an AI assistant that responds with JSON objects for MCP tool calls."
-                )
-                response_time = time.time() - start_time
-                
-                # Parse and analyze response
-                analysis = analyze_response(test, response, response_time)
-                category_results.append(analysis)
-                
-                print(f"✅ Response time: {response_time:.2f}s")
-                print(f"📝 Response: {response[:200]}...")
-                print(f"📊 Analysis: {analysis['score']}/10")
-                
-            except Exception as e:
-                print(f"❌ Error: {e}")
-                category_results.append({
-                    'test': test['name'],
-                    'error': str(e),
-                    'score': 0
-                })
-        
-        results[category] = category_results
-    
-    # Generate summary report
-    generate_summary_report(results)
-    
-    return results
-
-def analyze_response(test: Dict[str, Any], response: str, response_time: float) -> Dict[str, Any]:
-    """Analyze the LLM response and score its performance"""
-    
-    analysis = {
-        'test': test['name'],
-        'response': response,
-        'response_time': response_time,
-        'score': 0,
-        'strengths': [],
-        'weaknesses': [],
-        'details': {}
-    }
-    
-    # Basic response quality checks
-    if not response or response.strip() == "":
-        analysis['weaknesses'].append("Empty response")
-        return analysis
-    
-    # Check for JSON structure
-    try:
-        # Try to parse as JSON
-        parsed = json.loads(response)
-        analysis['details']['json_valid'] = True
-        analysis['strengths'].append("Valid JSON structure")
-        
-        # Check if it matches expected structure
-        if 'expected' in test:
-            expected = test['expected']
-            if isinstance(expected, dict):
-                if 'tool' in expected and 'tool' in parsed:
-                    if parsed['tool'] == expected['tool']:
-                        analysis['score'] += 4
-                        analysis['strengths'].append("Correct tool selection")
-                    else:
-                        analysis['weaknesses'].append(f"Wrong tool: {parsed.get('tool')} vs {expected['tool']}")
-                
-                # Check arguments
-                if 'args' in expected and 'args' in parsed:
-                    arg_match = compare_arguments(parsed['args'], expected['args'])
-                    analysis['score'] += arg_match * 3
-                    if arg_match > 0.7:
-                        analysis['strengths'].append("Good argument matching")
-                    else:
-                        analysis['weaknesses'].append("Poor argument matching")
-        
-    except json.JSONDecodeError:
-        analysis['details']['json_valid'] = False
-        analysis['weaknesses'].append("Invalid JSON format")
-        
-        # Check for tool-like patterns in text
-        if 'get_game_state' in response:
-            analysis['score'] += 2
-            analysis['strengths'].append("Mentions correct tool")
-        if 'move_units' in response or 'attack_unit' in response:
-            analysis['score'] += 1
-    
-    # Response time scoring
-    if response_time < 5.0:
-        analysis['score'] += 1
-        analysis['strengths'].append("Fast response")
-    elif response_time > 15.0:
-        analysis['weaknesses'].append("Slow response")
-    
-    # Content relevance scoring
-    if any(keyword in response.lower() for keyword in ['game', 'state', 'move', 'attack', 'build']):
-        analysis['score'] += 1
-        analysis['strengths'].append("Relevant content")
-    
-    # Cap for score
-    analysis['score'] = min(analysis['score'], 10)
-    
-    return analysis
-
-def compare_arguments(actual: Dict, expected: Dict) -> float:
-    """Compare argument dictionaries and return match percentage"""
-    if not actual or not expected:
-        return 0.0
-    
-    matches = 0
-    total = len(expected)
-    
-    for key, expected_value in expected.items():
-        if key in actual:
-            actual_value = actual[key]
-            if isinstance(expected_value, list) and isinstance(actual_value, list):
-                # Compare lists
-                if set(expected_value) == set(actual_value):
-                    matches += 1
-            elif expected_value == actual_value:
-                matches += 1
-    
-    return matches / total if total > 0 else 0.0
-
-def generate_summary_report(results: Dict[str, List[Dict]]):
-    """Generate a comprehensive summary report"""
-    
-    print("\n" + "="*60)
-    print("📊 QWEN2.5 0.5B MCP CAPABILITY ASSESSMENT REPORT")
-    print("="*60)
-    
-    overall_scores = []
-    
-    for category, category_results in results.items():
-        if not category_results:
-            continue
-            
-        category_scores = [r.get('score', 0) for r in category_results if 'score' in r]
-        avg_score = sum(category_scores) / len(category_scores) if category_scores else 0
-        overall_scores.append(avg_score)
-        
-        print(f"\n🔍 {category.replace('_', ' ').title()}:")
-        print(f"   Average Score: {avg_score:.1f}/10")
-        
-        for result in category_results:
-            if 'error' in result:
-                print(f"   ❌ {result['test']}: ERROR - {result['error']}")
-            else:
-                print(f"   {'✅' if result['score'] >= 6 else '⚠️'} {result['test']}: {result['score']}/10")
-                if result['strengths']:
-                    print(f"     Strengths: {', '.join(result['strengths'][:2])}")
-                if result['weaknesses']:
-                    print(f"     Weaknesses: {', '.join(result['weaknesses'][:2])}")
-    
-    # Overall assessment
-    if overall_scores:
-        overall_avg = sum(overall_scores) / len(overall_scores)
-        print(f"\n🎯 OVERALL ASSESSMENT: {overall_avg:.1f}/10")
-        
-        if overall_avg >= 8:
-            print("💪 EXCELLENT - Qwen2.5 0.5B is highly capable for MCP tasks")
-        elif overall_avg >= 6:
-            print("👍 GOOD - Qwen2.5 0.5B is capable with some limitations")
-        elif overall_avg >= 4:
-            print("⚠️  MODERATE - Qwen2.5 0.5B has significant limitations")
-        else:
-            print("❌ POOR - Qwen2.5 0.5B is not suitable for MCP tasks")
-        
-        # Recommendations
-        print(f"\n💡 RECOMMENDATIONS:")
-        if overall_avg >= 7:
-            print("- Use Qwen2.5 0.5B for MCP translation with confidence")
-            print("- Implement prompt engineering for complex tasks")
-            print("- Add validation layer for safety")
-        elif overall_avg >= 5:
-            print("- Use Qwen2.5 0.5B for simple MCP tasks")
-            print("- Implement strong validation and fallback mechanisms")
-            print("- Consider using larger models for complex planning")
-        else:
-            print("- Consider upgrading to a larger model (1.5B+)")
-            print("- Use Qwen2.5 0.5B only for very simple translations")
-            print("- Implement extensive error handling")
-
-if __name__ == "__main__":
-    print("Starting Qwen2.5 0.5B MCP capability assessment...")
-    print("This will test the model's ability to translate user commands to MCP tool calls.")
-    print("Make sure the model is downloaded and available at: qwen2.5-0.5b-instruct-q4_0.gguf")
-    
-    try:
-        results = test_qwen_capabilities()
-        
-        # Save detailed results to file
-        with open("/home/luigi/rts/web/qwen_mcp_assessment.json", "w") as f:
-            json.dump(results, f, indent=2)
-        
-        print("\n📄 Detailed results saved to: qwen_mcp_assessment.json")
-        
-    except Exception as e:
-        print(f"❌ Assessment failed: {e}")
-        print("Make sure the AI model is properly downloaded and configured.")