Found while running FLE 0.4.3 as an evaluation environment. Every item below was reproduced against a
live headless server, not read off the source. Happy to send a PR for any or all of these.
These matter more than ordinary doc bugs because SystemPromptGenerator.generate() concatenates the
per-tool agent.md manuals into the system prompt every agent receives, so a broken example is
something the model is actively taught before it writes its first line.
1. extract_item raises UnboundLocalError when source is a Position
fle/env/tools/agent/extract_item/client.py:24-33
source_name is initialised but x/y are not, and they are only assigned inside the two
isinstance branches. Any source that is neither a Position nor an Entity reaches
self.execute(..., x, y, ...) with x unbound:
UnboundLocalError: cannot access local variable 'x' where it is not associated with a value
Reproduced with extract_item(Prototype.Coal, "not-an-entity", 5). The docstring advertises both
Position and Entity sources, so the natural fix is an else raising the same
ValueError/AssertionError the neighbouring tools raise — rotate_entity and pickup_entity
already do this well ("The first argument must be an Entity object").
This is the only item here that is a code bug rather than a docs bug.
2. get_entities/agent.md — the "Efficient Searching" example cannot work
fle/env/tools/agent/get_entities/agent.md:64-82, under ## Best Practices → 1. Efficient Searching.
Three independent problems in one block:
- line 69, 77:
get_entities(prototype=resource_type, ...) — prototype is not a parameter.
The signature is get_entities(entities, position, radius), so this is
TypeError: GetEntities.__call__() got an unexpected keyword argument 'prototype'.
- lines 75-76:
player.position.x — player is not bound in the agent's namespace, so this is
a NameError.
- the premise:
get_entities reports placed entities, not resources, so this function returns
nothing even with both fixed. Measured: get_entities({Prototype.IronOre}, position=ore, radius=40)
returns 0 while standing on a 6,240,000-tile iron patch.
Since the section is headed "Efficient Searching", the net effect is to teach an expanding-radius
scan loop as the recommended way to find a resource, when nearest() answers it in one call and
get_resource_patch() gives the extent. Suggested replacement:
# `get_entities` reports entities that have been PLACED. It does not find ore, coal, stone or
# water. Use `nearest` for where a resource is, and `get_resource_patch` for how much is there:
ore_position = nearest(Resource.IronOre)
patch = get_resource_patch(Resource.IronOre, ore_position, radius=20)
# `get_entities` is for reading back what you built:
furnaces = get_entities({Prototype.StoneFurnace}, position=ore_position, radius=20)
3. get_entities/agent.md — the ## Parameters block names three parameters that do not exist, and omits the one that does
fle/env/tools/agent/get_entities/agent.md:22-26
- `prototype`: The Prototype of entities to find (optional)
- `name`: Exact entity name to search for (optional)
- `type`: Entity type category to search for (optional)
None of prototype, name or type is a parameter; the real signature is
get_entities(entities, position, radius), and entities is documented nowhere. Verified:
get_entities(name='iron-chest') and get_entities(type='container') both raise TypeError.
The same prototype= spelling appears in seven runnable examples in this file — lines 9, 15, 40,
45, 69, 77 and 88 — including the two in ## Basic Usage, which is the first code an agent reads for
this tool.
4. insert_item/agent.md — documents a parameter named item that does not exist
fle/env/tools/agent/insert_item/agent.md:15 — - `item`: Prototype of the item to insert.
The real signature is insert_item(entity, target, quantity); insert_item(item=Prototype.Coal, ...)
raises TypeError.
5. move_to/agent.md — two examples raise
- line 42:
place_entity(Prototypw.WoodenChest, ...) — typo for Prototype, so AttributeError.
- line 33:
move_to(nearest(IronOre)) — IronOre is a member of Resource and nothing binds it
bare; should be move_to(nearest(Resource.IronOre)). NameError as written.
6. get_entities docstring: a parameter that does not exist, and a documented argument that raises
fle/env/tools/agent/get_entities/client.py:26-28
:param player_only: — not a parameter of __call__, whose signature is
(entities, position, radius).
:param position: says it "Can be a Position object or "player" for player's position".
Passing "player" raises: Exception: Error in GetEntities (entities=all, position=player, ...).
The manual also states "Maximum radius is limited to 50 tiles for performance reasons", while the
real default is radius=1000 and a call with radius=200 is served normally.
7. get_resource_patch: documented default radius is 10, actual is 30
get_resource_patch/agent.md states (default: 10) in both the Basic Usage comment and the
Parameters list. inspect.signature gives radius: int = 30.
8. nearest_buildable: :return: promises None, the tool raises
nearest_buildable/client.py — ":return: BoundingBox of the nearest buildable area or None if no
such area exists." Asking for a drill 900,000 tiles from any ore raises
Exception: No viable place to put Prototype.BurnerMiningDrill near the centre position ....
An agent following the docstring writes if area is None: and never reaches it.
9. get_prototype_recipe never populates Recipe.category or Recipe.energy — and the category is the one field that says which machine makes an item
get_prototype_recipe/client.py:53 builds Recipe(name=name, ingredients=ingredients, products=products)
and stops, so the model's declared category (entities.py:499) and energy (:498) are always
their defaults. Measured on every recipe: category=None, energy=0. The manual meanwhile promises
"Additional metadata like crafting time and category".
This one is more than a docs mismatch. The crafting category is the only attribute that tells an agent
which machine a recipe needs — smelting (iron-plate, stone-brick, steel-plate: a furnace, burning
coal) versus crafting (iron-gear-wheel, stone-wall, electronic-circuit: an assembling machine, hence
electricity). With it unpopulated, the environment offers no way to learn whether a target needs
power at all. In our traces, agents built steam engines for steel-plate, which needs none.
The data is one call away: game.forces.player.recipes[name].category / .energy returns
smelting|16 for steel-plate and crafting|0.5 for iron-gear-wheel, in both Factorio 1.1 and 2.0
(game.recipe_prototypes is 1.1-only). We've worked around it on our side by filling both fields
from that call; a two-line change in client.py would fix it at the source.
Checked against main at the time of writing: this file is identical to the 0.4.3 release.
10. place_entity_next_to: the "Smart Placement Feedback" section reads an attribute that is never set
place_entity_next_to/agent.md — the section tells the agent to read
inserter._placement_feedback. hasattr(entity, "_placement_feedback") is False on a returned
entity, so the block is unreachable by construction.
11. connect_entities: NameError in the pipe example
connect_entities/agent.md — the "Pipe Connections" example assigns water_pipes and then prints
{pipes}. Also worth a look: that example is headed "Pipe Connections" but passes
{Prototype.TransportBelt, Prototype.UndergroundBelt} to connect an offshore pump to a boiler.
12. Three manuals call tools through a game object that is not bound
fle/env/tools/agent/place_entity/agent.md:102 — game.get_entity(...)
fle/env/tools/agent/nearest_buildable/agent.md:116 — game.get_entity(...)
fle/env/tools/agent/sleep/agent.md:25 — game.sleep(10)
game is bound by nothing, including FLE's own namespace, so each is a NameError. The tools are
available under their bare names, so dropping the prefix is the fix.
13. Minor: several tools raise errors that do not identify the problem
Not a bug report so much as a consistency note, since six tools already do this well. Probing each
agent tool with a type-violating first argument:
| tool |
error an agent sees |
get_entity, can_place_entity |
AssertionError with an empty message |
nearest, get_entities, place_entity_next_to, set_entity_recipe, get_connection_amount |
AttributeError: 'str' object has no attribute 'value' / 'position' |
move_to |
AttributeError: 'str' object has no attribute 'x' |
get_resource_patch |
Could not get i at x=0.0 y=0.0: i (the string is being iterated) |
sleep |
accepted silently — sleep("ten") does not raise; sleep(-5) and sleep(0) also return True |
move_to(laying=...) |
UnboundLocalError: cannot access local variable 'response' |
inspect_inventory(all_players=...) |
accepted silently for a non-bool |
craft_item(quantity="zzz") |
raw Lua: attempt to compare number with string |
harvest_resource(quantity="zzz") |
Could not harvest. Nothing within reach to harvest — a type error reported as a fact about the world |
Compare inspect_inventory, which is exactly right: "The first argument must be an Entity or
Position object, you passed in a <class 'str'> object." An agent that gets that message fixes its
call; an agent that gets 'str' object has no attribute 'value' cannot tell which argument of which
call was wrong.
Versions: factorio-learning-environment 0.4.3 (the latest release, and every file cited here is identical on main), Python 3.14, headless Factorio via
fle cluster start. Reproduction for every item above is a single call against a live instance; I can
supply a script if useful.
🤖 Generated with Claude Code
https://claude.ai/code/session_01XtSKreMvmyd1GGef3t4uTu
Found while running FLE 0.4.3 as an evaluation environment. Every item below was reproduced against a
live headless server, not read off the source. Happy to send a PR for any or all of these.
These matter more than ordinary doc bugs because
SystemPromptGenerator.generate()concatenates theper-tool
agent.mdmanuals into the system prompt every agent receives, so a broken example issomething the model is actively taught before it writes its first line.
1.
extract_itemraisesUnboundLocalErrorwhensourceis aPositionfle/env/tools/agent/extract_item/client.py:24-33source_nameis initialised butx/yare not, and they are only assigned inside the twoisinstancebranches. Anysourcethat is neither aPositionnor anEntityreachesself.execute(..., x, y, ...)withxunbound:Reproduced with
extract_item(Prototype.Coal, "not-an-entity", 5). The docstring advertises bothPositionandEntitysources, so the natural fix is anelseraising the sameValueError/AssertionErrorthe neighbouring tools raise —rotate_entityandpickup_entityalready do this well ("The first argument must be an Entity object").
This is the only item here that is a code bug rather than a docs bug.
2.
get_entities/agent.md— the "Efficient Searching" example cannot workfle/env/tools/agent/get_entities/agent.md:64-82, under## Best Practices → 1. Efficient Searching.Three independent problems in one block:
get_entities(prototype=resource_type, ...)—prototypeis not a parameter.The signature is
get_entities(entities, position, radius), so this isTypeError: GetEntities.__call__() got an unexpected keyword argument 'prototype'.player.position.x—playeris not bound in the agent's namespace, so this isa
NameError.get_entitiesreports placed entities, not resources, so this function returnsnothing even with both fixed. Measured:
get_entities({Prototype.IronOre}, position=ore, radius=40)returns
0while standing on a 6,240,000-tile iron patch.Since the section is headed "Efficient Searching", the net effect is to teach an expanding-radius
scan loop as the recommended way to find a resource, when
nearest()answers it in one call andget_resource_patch()gives the extent. Suggested replacement:3.
get_entities/agent.md— the## Parametersblock names three parameters that do not exist, and omits the one that doesfle/env/tools/agent/get_entities/agent.md:22-26None of
prototype,nameortypeis a parameter; the real signature isget_entities(entities, position, radius), andentitiesis documented nowhere. Verified:get_entities(name='iron-chest')andget_entities(type='container')both raiseTypeError.The same
prototype=spelling appears in seven runnable examples in this file — lines 9, 15, 40,45, 69, 77 and 88 — including the two in
## Basic Usage, which is the first code an agent reads forthis tool.
4.
insert_item/agent.md— documents a parameter nameditemthat does not existfle/env/tools/agent/insert_item/agent.md:15—- `item`: Prototype of the item to insert.The real signature is
insert_item(entity, target, quantity);insert_item(item=Prototype.Coal, ...)raises
TypeError.5.
move_to/agent.md— two examples raiseplace_entity(Prototypw.WoodenChest, ...)— typo forPrototype, soAttributeError.move_to(nearest(IronOre))—IronOreis a member ofResourceand nothing binds itbare; should be
move_to(nearest(Resource.IronOre)).NameErroras written.6.
get_entitiesdocstring: a parameter that does not exist, and a documented argument that raisesfle/env/tools/agent/get_entities/client.py:26-28:param player_only:— not a parameter of__call__, whose signature is(entities, position, radius).:param position:says it "Can be a Position object or"player"for player's position".Passing
"player"raises:Exception: Error in GetEntities (entities=all, position=player, ...).The manual also states "Maximum radius is limited to 50 tiles for performance reasons", while the
real default is
radius=1000and a call withradius=200is served normally.7.
get_resource_patch: documented default radius is 10, actual is 30get_resource_patch/agent.mdstates(default: 10)in both the Basic Usage comment and theParameters list.
inspect.signaturegivesradius: int = 30.8.
nearest_buildable::return:promisesNone, the tool raisesnearest_buildable/client.py— ":return: BoundingBox of the nearest buildable area or None if nosuch area exists." Asking for a drill 900,000 tiles from any ore raises
Exception: No viable place to put Prototype.BurnerMiningDrill near the centre position ....An agent following the docstring writes
if area is None:and never reaches it.9.
get_prototype_recipenever populatesRecipe.categoryorRecipe.energy— and the category is the one field that says which machine makes an itemget_prototype_recipe/client.py:53buildsRecipe(name=name, ingredients=ingredients, products=products)and stops, so the model's declared
category(entities.py:499) andenergy(:498) are alwaystheir defaults. Measured on every recipe:
category=None,energy=0. The manual meanwhile promises"Additional metadata like crafting time and category".
This one is more than a docs mismatch. The crafting category is the only attribute that tells an agent
which machine a recipe needs —
smelting(iron-plate, stone-brick, steel-plate: a furnace, burningcoal) versus
crafting(iron-gear-wheel, stone-wall, electronic-circuit: an assembling machine, henceelectricity). With it unpopulated, the environment offers no way to learn whether a target needs
power at all. In our traces, agents built steam engines for
steel-plate, which needs none.The data is one call away:
game.forces.player.recipes[name].category/.energyreturnssmelting|16for steel-plate andcrafting|0.5for iron-gear-wheel, in both Factorio 1.1 and 2.0(
game.recipe_prototypesis 1.1-only). We've worked around it on our side by filling both fieldsfrom that call; a two-line change in
client.pywould fix it at the source.Checked against
mainat the time of writing: this file is identical to the 0.4.3 release.10.
place_entity_next_to: the "Smart Placement Feedback" section reads an attribute that is never setplace_entity_next_to/agent.md— the section tells the agent to readinserter._placement_feedback.hasattr(entity, "_placement_feedback")isFalseon a returnedentity, so the block is unreachable by construction.
11.
connect_entities:NameErrorin the pipe exampleconnect_entities/agent.md— the "Pipe Connections" example assignswater_pipesand then prints{pipes}. Also worth a look: that example is headed "Pipe Connections" but passes{Prototype.TransportBelt, Prototype.UndergroundBelt}to connect an offshore pump to a boiler.12. Three manuals call tools through a
gameobject that is not boundfle/env/tools/agent/place_entity/agent.md:102—game.get_entity(...)fle/env/tools/agent/nearest_buildable/agent.md:116—game.get_entity(...)fle/env/tools/agent/sleep/agent.md:25—game.sleep(10)gameis bound by nothing, including FLE's own namespace, so each is aNameError. The tools areavailable under their bare names, so dropping the prefix is the fix.
13. Minor: several tools raise errors that do not identify the problem
Not a bug report so much as a consistency note, since six tools already do this well. Probing each
agent tool with a type-violating first argument:
get_entity,can_place_entityAssertionErrorwith an empty messagenearest,get_entities,place_entity_next_to,set_entity_recipe,get_connection_amountAttributeError: 'str' object has no attribute 'value'/'position'move_toAttributeError: 'str' object has no attribute 'x'get_resource_patchCould not get i at x=0.0 y=0.0: i(the string is being iterated)sleepsleep("ten")does not raise;sleep(-5)andsleep(0)also returnTruemove_to(laying=...)UnboundLocalError: cannot access local variable 'response'inspect_inventory(all_players=...)craft_item(quantity="zzz")attempt to compare number with stringharvest_resource(quantity="zzz")Could not harvest. Nothing within reach to harvest— a type error reported as a fact about the worldCompare
inspect_inventory, which is exactly right: "The first argument must be an Entity orPosition object, you passed in a
<class 'str'>object." An agent that gets that message fixes itscall; an agent that gets
'str' object has no attribute 'value'cannot tell which argument of whichcall was wrong.
Versions:
factorio-learning-environment0.4.3 (the latest release, and every file cited here is identical onmain), Python 3.14, headless Factorio viafle cluster start. Reproduction for every item above is a single call against a live instance; I cansupply a script if useful.
🤖 Generated with Claude Code
https://claude.ai/code/session_01XtSKreMvmyd1GGef3t4uTu