docs/MCP_AI_TRANSLATION_EXAMPLES.md

Using Qwen2.5 0.5B for User Instruction Translation via MCP

Overview

The MCP interface we've implemented is perfectly designed to work with the Qwen2.5 0.5B model you already have for translating natural language user instructions into specific game actions. The model can understand user requests and convert them into precise MCP tool calls.

How It Works

1. User Input: Player gives natural language instruction
2. AI Processing: Qwen2.5 0.5B interprets the instruction
3. MCP Translation: AI converts instruction to specific MCP tool calls
4. Game Execution: MCP tools execute actions in the game

Example Translations

Example 1: Resource Management

User Instruction: "Build a power plant near my HQ" AI Translation Process:

1. Use get_game_state() to locate player HQ
2. Find suitable position near HQ
3. Check if player has enough credits
4. Call build_building("power_plant", x, y, 0)

Example 2: Military Command

User Instruction: "Move my tanks to defend the north base" AI Translation Process:

1. Use get_game_state() to identify player tanks and north base location
2. Calculate appropriate defensive positions
3. Call move_units([tank_ids], target_x, target_y)

Example 3: Tactical Attack

User Instruction: "Attack the enemy barracks with my infantry" AI Translation Process:

1. Use get_game_state() to locate player infantry and enemy barracks
2. Verify infantry are available and barracks exists
3. Call attack_unit([infantry_ids], barracks_id)

Example 4: Economic Strategy

User Instruction: "I need more harvesters to collect ore" AI Translation Process:

1. Use get_game_state() to check current harvesters and refinery status
2. Use get_ai_analysis() to confirm this is a good strategy
3. Call build_unit("harvester", 0, refinery_id) (if refinery exists)
4. Or call build_building("refinery", x, y, 0) then build harvester

Implementation Approach

Direct Integration

Since you already have Qwen2.5 0.5B integrated in ai_analysis.py, you can extend it to:

1. Create a new translation function:

def translate_user_instruction(instruction: str, game_state: dict) -> List[dict]:
    """
    Translate natural language instruction to MCP tool calls
    
    Returns list of tool call dictionaries:
    [
        {"tool": "move_units", "arguments": {...}},
        {"tool": "build_building", "arguments": {...}}
    ]
    """
    # Use the same LLM with a specialized prompt
    prompt = f"""
    You are an RTS game command interpreter. Convert the user instruction 
    into specific game actions using available MCP tools.
    
    Available tools:
    - move_units(unit_ids, target_x, target_y)
    - attack_unit(attacker_ids, target_id)
    - build_building(building_type, position_x, position_y, player_id)
    - build_unit(unit_type, player_id, building_id)
    - get_game_state()
    - get_ai_analysis(language)
    
    Current game state: {json.dumps(game_state, indent=2)}
    
    User instruction: {instruction}
    
    Return ONLY a JSON array of tool calls.
    """
    
    # Process with your existing LLM infrastructure
    # ...

MCP Client Implementation

You could create a specialized MCP client that uses your Qwen2.5 model:

class RTSAIController:
    def __init__(self):
        self.ai_analyzer = get_ai_analyzer()  # Your existing analyzer
    
    async def process_user_command(self, user_input: str):
        # Get current game state
        game_state = await self.get_game_state_via_mcp()
        
        # Use Qwen2.5 to translate instruction to actions
        tool_calls = self.ai_analyzer.translate_user_instruction(
            user_input, game_state
        )
        
        # Execute each tool call via MCP
        results = []
        for call in tool_calls:
            result = await self.execute_mcp_tool(call["tool"], call["arguments"])
            results.append(result)
        
        return results

Advanced Features

Context-Aware Translation

The Qwen2.5 model can make decisions based on:

• Current game state (via get_game_state())
• Tactical analysis (via get_ai_analysis())
• Previous actions and their results
• Player preferences and play style

Multi-Step Planning

Complex instructions can be broken down: User: "Win the game" AI Process:

1. Analyze current situation with get_game_state() and get_ai_analysis()
2. Create multi-step plan:
   • Build more military units if ahead economically
   • Build economic structures if behind
   • Launch nuke if charged and strategically beneficial
   • Coordinate combined arms attacks

Error Handling and Recovery

The AI can handle failures gracefully:

• If a building action fails (insufficient funds), suggest alternatives
• If a movement is blocked, find alternative paths
• If units are lost, adjust strategy accordingly

Benefits of This Approach

1. Consistent AI Personality

Using the same Qwen2.5 model ensures consistent tactical understanding between:

• The in-game AI analysis panel
• User instruction translation
• Strategic decision making

2. Enhanced User Experience

Players can use natural language commands:

• "Defend our base"
• "Attack the enemy's weak spot"
• "Focus on economy for now"
• "I need more power"

3. Learning and Adaptation

The system can learn from:

• Successful command translations
• Player preferences
• Game outcomes

Implementation Recommendations

1. Specialized Prompts

Create prompts that guide the model to produce structured output:

Convert this user instruction to MCP tool calls.
Return ONLY a JSON array with this exact format:
[
  {"tool": "tool_name", "arguments": {"param1": "value1", "param2": "value2"}}
]

Available tools:
- move_units(unit_ids: List[str], target_x: float, target_y: float)
- attack_unit(attacker_ids: List[str], target_id: str)
- build_building(building_type: str, position_x: float, position_y: float, player_id: int)
- etc.

User instruction: "Move my tanks to the north"

2. Validation Layer

Add a validation step to ensure tool calls are safe:

def validate_tool_call(tool_name: str, arguments: dict) -> bool:
    """Validate that tool call is reasonable before execution"""
    # Check that unit IDs exist
    # Check that coordinates are valid
    # Check that player has resources
    # etc.
    pass

3. Feedback Loop

Provide feedback to improve translation quality:

def provide_feedback(tool_call: dict, result: dict, success: bool):
    """Provide feedback to the AI about translation quality"""
    # Log successful/failed translations
    # Adjust future behavior based on results
    pass

Example Integration Code

Here's how you might integrate this with your existing system:

# In ai_analysis.py, add a new method:
class AIAnalyzer:
    # ... existing methods ...
    
    def translate_command_to_actions(self, user_command: str, game_state: dict) -> List[dict]:
        """Translate natural language command to MCP actions"""
        
        # Create specialized prompt
        prompt = self._create_translation_prompt(user_command, game_state)
        
        # Use existing LLM infrastructure
        response = self._query_llm(prompt)
        
        # Parse response into tool calls
        try:
            tool_calls = json.loads(response)
            return tool_calls
        except json.JSONDecodeError:
            # Handle parsing errors
            return []
    
    def _create_translation_prompt(self, command: str, game_state: dict) -> str:
        """Create prompt for command translation"""
        return f"""
You are an RTS game command interpreter. Convert the user's natural language 
instruction into specific MCP tool calls that can be executed in the game.

Current game situation:
{json.dumps(game_state, indent=2)}

User instruction: "{command}"

Available MCP tools:
1. get_game_state() - Get current game state
2. get_ai_analysis(language) - Get tactical analysis
3. move_units(unit_ids, target_x, target_y) - Move units
4. attack_unit(attacker_ids, target_id) - Attack enemy unit
5. build_building(building_type, position_x, position_y, player_id) - Build structure
6. build_unit(unit_type, player_id, building_id) - Produce unit
7. send_game_command(command_type, **kwargs) - Send any command

Return ONLY a JSON array of tool calls in this format:
[
  {{"tool": "move_units", "arguments": {{"unit_ids": ["unit1"], "target_x": 100, "target_y": 200}}}}
]

Important guidelines:
- Always verify that units/buildings exist before targeting them
- Check that player has sufficient resources for construction
- Consider tactical positioning (don't move into water or blocked areas)
- Be specific about unit selection
- Use appropriate building/unit types

JSON array of tool calls:
"""

# Usage example:
async def handle_voice_command(user_speech: str):
    """Handle voice command from user"""
    # Get current game state via MCP
    game_state = await mcp_client.call_tool("get_game_state", {})
    
    # Translate to actions
    tool_calls = ai_analyzer.translate_command_to_actions(user_speech, game_state)
    
    # Execute actions
    results = []
    for call in tool_calls:
        result = await mcp_client.call_tool(call["tool"], call["arguments"])
        results.append(result)
    
    return results

Conclusion

The MCP interface is perfectly suited for use with your Qwen2.5 0.5B model to translate natural language user instructions into game actions. The combination provides:

1. Natural Interaction: Users can speak naturally to control the game
2. Intelligent Translation: The AI understands both the instruction and game context
3. Safe Execution: All actions go through the existing game validation systems
4. Consistent Experience: Uses the same AI that powers the tactical analysis panel

This creates a powerful voice/command interface for your RTS game that feels truly intelligent and responsive to user needs.