CS588 Programming Assignment #2

Overview

• Due: 2026.04.03 11:59PM
• Objective: Understanding agentic AI through observing an LLM-based agent
• In this PA, you will learn about the ReAct framework (Thought → Action → Observation) by running an LLM-based agent in ALFWorld, a text-based household environment.
• You will observe how the agent reasons and acts to complete household tasks, then analyze its trajectories to understand its strengths and limitations.

Related Materials

• PA2.zip
• ReAct (ICLR'23): Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models"
• ALFWorld (ICLR'21): Shridhar et al., "ALFWorld: Aligning Text and Embodied Environments for Interactive Learning"

Developing Environment

• Python 3.10
• ALFWorld
• Google Generative AI SDK (Gemini API, free tier)

Code Instruction

1. Create the environment

conda create -n alfworld_agent python=3.10 -y
conda activate alfworld_agent
pip install -r requirements.txt

2. Download ALFWorld data (~5 GB, only needs to run once)

alfworld-download

3. Get a Gemini API key (free)

• Go to https://aistudio.google.com/apikey and create a free API key.
• Set it as an environment variable:

export GEMINI_API_KEY="your-key-here"

4. Run the agent

• Before running, change STUDENT_ID at the top of src/main.py to your own student ID.

python src/main.py

• This runs 15 episodes and saves all trajectories to eval_results/. The full run takes approximately 60–90 minutes due to API rate limits. The script includes automatic rate-limit handling.

Background

ALFWorld is a text-based environment where an agent must complete household tasks such as "put a clean mug on the shelf." The agent receives:

• Text observation: description of what it sees (e.g., "On the desk 1, you see a mug 1, a pen 2.")
• Admissible commands: list of valid actions (e.g., "go to shelf 1", "take mug 1 from desk 1")

The provided code implements a ReAct agent (Yao et al., ICLR 2023) that uses a Thought → Action → Observation loop with the Gemma 3 27B model via the Gemini API.

Requirements

1. Trajectory Analysis (10 pts)

Open the result JSON file in eval_results/. Each episode contains a trajectory array where each entry has thought, action, and observation.

• Q1. Report the overall success rate (how many of the 15 episodes succeeded).

• Q2. Pick two failed episodes. For each, describe:

  • What was the task?
  • What did the agent do wrong? (e.g., stuck in a loop, picked the wrong object, skipped a step, repeated invalid actions)
  • At which step did the agent first go off track?

• Q3. Pick two successful episode. Describe how the agent's reasoning (in the "thought" field) helped it follow the plan correctly.

2. Discussion (10 pts)

• Analyze the experimental results and discuss failure patterns across episodes.
• Discuss how the 30-step limit affects the agent.
• Discuss what happens when the agent encounters "Nothing happens" (invalid actions).
• Propose at least one concrete idea to improve the ReAct agent's performance.

Submission

• Submit your report document in PDF to KLMS. (Your report should contain the requirements and be written within 2 pages.)
• Also submit the eval_results/ folder with your result JSON file.
• Do NOT include your Gemini API key anywhere in the submission.

Policies

• Everyone must turn in their own assignment. You can collaborate with others, but any work that you turn in should be your own.
• Since results depend on the LLM's behavior, your trajectories will differ from others — this is expected.