Support for Large File Chunking

Batch operations support large file chunking for scripts that exceed typical AI context limits. This advanced feature enables Vex to create massive script files (1000+ lines) by sending them in manageable chunks that GameCatalyst assembles, validates, and writes as a single file.

The chunking problem:

AI models have context limits (typically 8K-32K tokens, where 1 token ≈ 4 characters). A large script file (2000+ lines, ~100K characters) might exceed this limit, making it impossible to generate in one response. Even if the AI could generate it, transmitting it in one message might fail due to network limits or message size restrictions.

Without chunking, Vex would be limited to creating relatively small scripts (300-500 lines max). This is fine for most use cases, but occasionally you need large files: procedurally generated code, comprehensive manager classes, or complex state machines.

The chunking solution:

GameCatalyst supports create_chunked action where Vex sends the file in chunks (typically 200-300 lines each), and GameCatalyst assembles them in order, validates completeness, and writes the final file.

How chunking works:

1. Vex calls unity.batch_operation action="begin"
   Recompilation freezes as usual
2. Vex calls unity.batch_operation action="execute" files=[{action: "create_chunked", path: "Assets/Scripts/LargeFile.cs", chunks: ["chunk1", "chunk2", "chunk3"], validate_complete: true}]
   GameCatalyst receives the chunked file creation request
3. GameCatalyst assembles chunks in order
   Chunk 1 + Chunk 2 + Chunk 3 = Complete file content
4. If validate_complete=true: GameCatalyst checks for syntax completeness
   Validates matching braces, complete statements, proper file ending
5. File is created
   The assembled content is written to disk as a single file
6. Recompilation triggers
   Unity compiles the complete file

Chunk validation:

When validate_complete=true, GameCatalyst performs syntax validation to ensure the assembled file is complete:

• All opening braces have closing braces — Counts { and } to ensure they match
• No truncated statements — Checks that the last line isn’t cut off mid-statement
• File ends properly — Ensures the file ends with a closing brace for the namespace or class
• No chunk boundaries mid-string — Verifies string literals aren’t split across chunks

If validation fails, the file is not created and an error is returned with details about what failed. Vex can then regenerate the problematic chunk and retry.

When to use chunking:

Chunking is appropriate for:

• Large MonoBehaviour scripts (500+ lines) — Complex player controllers, AI systems, or manager classes
• Complex systems with many methods — State machines with dozens of states, procedural generation systems
• Generated code from templates — Code generated from data files, configuration, or external tools
• Porting code from other projects — When migrating large scripts from other engines or frameworks

Chunking best practices:

• Keep chunks at logical boundaries (end of methods, end of classes) to make validation easier
• Use validate_complete=true to catch assembly errors early
• If a chunk fails validation, regenerate just that chunk instead of the entire file
• Consider breaking very large files (2000+ lines) into multiple smaller files instead of using chunking

Performance considerations:

Chunked files take slightly longer to create (assembly + validation overhead), but the difference is negligible (milliseconds). The real benefit is enabling file creation that would otherwise be impossible due to AI context limits.