Skip to content

Eve: Simulation templates app/UI handoff

Historical integration/qualification notes: source-specific origin exports and shared delivery sidecars described below are retired. For current authoring and delivery, use compact metadata and origin packaging.

Current AUTO/USER tokens, metadata defaults, migration status and app dependencies: template token author guide. Each analysis uses one canonical notebook, shared source adapters and the generic auto_user_tokens binder; app integration remains separate.

Implement a shared app flow for generating simulation notebooks from true templates. The full spec owns catalog/publication, representation/PDK contracts, readiness gates, roadmap and notebook audit. Development notes retain test evidence, solver limitations and temporary compatibility code. Current notebooks are drafts; this reference generator is not production app orchestration.

The settled app behavior is recorded in GDSFactory+'s specs/simulation/notebook-launcher.md for issue 5167. This handoff follows that contract. In particular, notebooks follow saved original design files; existing reference-binder captures and Config loading still need the live-Prepare changes below. Passing binder tests does not establish that those changes or the app creation coordinator are implemented.

The app runtime uses a standalone Python worker under the owning project's environment interpreter. Extend its Rust-authoritative operation registry and generated Python contract; do not add handlers to the removed embedded runtime. Normal worker startup can install missing runtime dependencies, so notebook Resolve/Preflight needs a nonprovisioning boundary to preserve the no-install creation policy. Read-only Worker Health alone cannot prevent a later startup from repairing the environment. Kernel selection follows the source project, including when Save to is outside that project. Keep indexed partial aliases and project PDK cell overrides intact, and reuse current SDK materialization for repeated GSCH Prepare. Conversion warnings are not evidence of physical/LVS readiness. Template-registry API-key access remains separate from private-package session tokens; structured catalog/release endpoints are still proposed.

The app now has the internal WorkerManager::call_existing_project boundary: fresh project workers retain each job's PDK manifest, inspect the existing venv under the preparation lock, never provision/retry through repair, and preserve Python error diagnostics. Its standalone notebook_inspect_python operation executes the exact alias with raw JSON arguments and reports full bounds, ports and retained PDK identities; active-PDK changes fail. This is not a complete runtime Resolve/Preflight endpoint. The internal app runtime service now retains saved source/configuration, completed-index fingerprints and the exact indexed owning PDK/cell/model manifest, revalidating before/after inspection and staged parameterized export. Export writes embedded-metadata GDS from the exact alias and JSON arguments into exclusively owned staging; it has no public export RPC or publication endpoint. Windows notebook worker descendants are contained and output ownership survives caller cancellation through process termination. Conservative indexed-source checks are not physical/material qualification. Physical/parameter/port qualification and notebook creation/kernel orchestration remain required; catalog drafts stay unavailable.

The app's internal typed physical-preflight context now retains that private runtime and revalidates before/after derivation and again on demand. It records source-port selection/exclusion provenance and explicit project sim.x/legacy sim.wls wavelength provenance, with validated deliberate overrides. It never uses GUI fallback ranges or sample counts. Port display observations supply no excitation/material/units proof. Required physical providers (units, layers, stack, materials, cladding, port conventions or models) remain typed unavailable with corrective guidance; even a context with no such requirements is not Ready. There is no public preflight RPC or qualified generation adapter. Existing sidecar route requires strings are not the shared typed requirements contract; an adapter must declare source-port binding policy, accepted types/capabilities, wavelength use and reconstructable physical providers before it can be enabled. Legacy reference-loader wavelength/cladding defaults and short-name/snapshot behavior are not used by this app service. No authored origins or MEOW files are changed by this boundary.

Internal app adapter contracts now pin the exact template ID and metadata/asset content hash as well as adapter identity, binding ID, source route and shared typed requirements. Retained preflight rechecks current server requirements, reacquires the original catalog projection and exact release, then revalidates saved source/runtime authority. Catalog support declarations do not register a runtime implementation. Changed content cannot pass solely by retaining the same adapter name: retained bytes and hashes must still agree.

The app also validates one explicit stack-backed native provider declaration family: PDK identity/activation, micron/DBU conventions, stack/material source references and digests, exact cladding role/key/material and physical-port plus excitation evidence declarations. This is coherence checking, without importing references, verifying source digests or qualifying material bands, modes or geometry. Layer-map/model providers are not represented by that family. No provider loader, physical worker qualifier or generation adapter is enabled. A supported reviewed backend and authoritative evidence from the selected project remain required; current sample verification and native material-name fallbacks cannot supply it.

The zapFDTD Python/GDS/GSCH origins retain the selected module as pdk_provider and construct fdtd.Simulation(pdk=pdk_provider). Passing only its PDK object would discard module-owned MATERIAL_CARDS supported by the explicit declaration family and pinned backend. PDK activation and native attached-card precedence remain intact; this provider handoff fix does not qualify materials or the backend. Meep's get_stack() uses a separate conversion/material path.

Notebook creation reads only the PDK values required by the selected origin. There is no additional worker audit of stack/card ownership, native units or thickness, material-card membership, or provider/parent source hashes. Native physical validation remains in manual notebook preparation. Ordinary saved-source, project/runtime and exact template-release freshness still protect binding.

The app's simulationNotebook.catalog / catalogRelease operations have an explicit server-configured local development provider for these authored sidecars. It reports drafts, unsupported adapters and content hashes without running notebooks or claiming production qualification. Production registry integration remains separate; the legacy filename list/download is preserved.

Current drafts are Meep → Generic 3D Template and zapFDTD → Generic 3D Template; MEOW, then Femwell, are next. Reuse the launcher, source selection, generation and Config initialization across engines. Read each template's metadata for accepted routes and its native settings/results contract; do not hard-code Meep's capabilities into the app. zapFDTD details are in its implementation notes; its gsim/cloud engine ID is fdtd.

zapFDTD cloud execution is currently unqualified: two real diagnostic jobs crashed with exit 139; Eve subsequently identified CRLF inputs as the cause. The development notes record a temporary LF upload workaround; patched execution still needs verification here. The local writer/parser also disagree on required grid bounds; see its implementation notes. Do not present field/mode images or a successful solve as available until the solver integration has passed that gate.

1. Entry points and source choice

Add sidebar Simulate in Notebook; rename existing Simulate to Simulate in GUI without changing its behavior. Dispatch a shared launcher into template selection, with the clicked design selected. Sidebar/wizard owners must agree on the launcher; Eve owns menu/launcher UI, not automatically every wizard/runtime task.

Meep and zapFDTD are both in the initial app scope, each with Python, GSCH layout and direct-GDS inputs. GSCH schematic is a declared route with shared scaffolding; incompatible geometry templates stay visible with a reason and cannot advance. Additional compatible templates plug in through metadata and app-owned binding/adapter contracts rather than sidebar engine branches.

Catalog/capabilities, source/context resolution, preflight and generation are general Rust runtime operations with serializable source descriptors, structured choices/errors, progress/cancellation and revision checks. Python operations use the worker protocol. Sidebar, future editor top bar and other UI sections call the same operations. The extension owns Quick Picks, notebook opening and kernel/setup-cell actions through one shared coordinator.

Sidebar row Default Alternative
Python PCell From Python (.py) From GDS (.gds)
GSCH From Layout Editor From Schematic Editor

Filter eligible templates by the selected route and display each incompatible template's reason. Meep 3D and zapFDTD currently declare input_routes.gsch_schematic.accepted=false because they require physical layout; future circuit templates may accept schematic inputs. Keep internal editor names out of notebook/menu text. The sidebar action opens sequential native VS Code Quick Picks at the top of the window, starting with source route, then engine and eligible analysis/template. Resolve PDK, physical/material providers and paths automatically from the clicked design and its owning project/indexed PDK data; ask a targeted question only when there are genuinely ambiguous valid choices. Do not guess physical defaults from filenames. Use one shared launcher, not per-engine submenus.

Eventually add the same action near the .py/.gds eyeball and the .gsch top-right GUI simulation button. Assess placement/feasibility. Sidebar and editor must share one workflow, resolving the clicked item or active editor. Preserve selected cell and subcells; collect selection only when ambiguous. Top-bar work may follow the initial sidebar release. Both entry points follow the full spec's freshness, independent-copy and no-automatic-execution rules.

2. Files and cells to generate

Keep the PDK import and PDK.activate() uncommented in every Meep/zapFDTD origin and generated notebook. Origins retain the active placeholder import from <INSERT_AUTO_PDK_MODULE_HERE> import PDK; generation substitutes the validated module before execution. Never comment either line out during future edits.

Use engine-first selection, followed by the eligible generic template and future presets. Each origin notebook has a paired .py and capability metadata. Current origin notebooks:

Engine / choice Origin notebook in simulation_templates/ Template ID
Meep / Generic 3D Template meep_3d.ipynb meep_3d
zapFDTD / Generic 3D Template Three separate origins: zapfdtd_template_py.ipynb, zapfdtd_template_gsch_layouteditor.ipynb, zapfdtd_template_gds.ipynb zapfdtd_3d

The Meep display title is Generic 3D Template, declared in meep_3d_template.metadata.json; the builder and binder propagate it into notebook metadata and generated headings. Keep the stable meep_3d ID and filenames. The compact native review uses Simulator & Template: Meep, Generic 3D Template; other review labels use colons, including Simulation target:, Source port: and Save to:. Simulation target shows the file plus selected factory/cell; its edit action changes only the representation of the same clicked design, not the design identity. The representation picker uses matching two-line rows with paths beginning at the owning project folder, using the source filesystem's separators. Python includes its explicit factory call; exported GDS omits a redundant cell suffix. Direct multicell GDS retains cell identity when needed. Keep the review enabled by default, with editable rows and no tagline beneath the selected Review Completed, Create Notebook action.

Current generated filenames and geometry handling:

Route Meep filename zapFDTD filename Required loading placement
Python meep_3dtemplate_py.ipynb zapfdtd_template_py.ipynb Factory import and component call together in Settings; refresh on Prepare.
Layout Editor meep_3dtemplate_gsch_layouteditor.ipynb zapfdtd_template_gsch_layouteditor.ipynb Saved GSCH → rebuilt GDS → selected cell import.
Direct GDS meep_3dtemplate_gds.ipynb zapfdtd_template_gds.ipynb Selected cell import in Settings, retaining ports; reread on Prepare.
Schematic Editor No runnable notebook No runnable notebook Show incompatibility; the Meep _gsch_schematiceditor preview is explanatory only.

Use matching paired-source suffixes and source-specific titles. Layout: one collapsed Config setup cell → Settings → Prepare/preview → Run → Results. Config is notebook cell 2 (code-cell ordinal 0), collapsed but accessible. It holds paths, ordinary imports and PDK activation. Python component-producing imports/calls and GDS loading move together to Settings. GSCH keeps its authored build, warnings, LVS, import and component-assignment chain together in Config, made reusable. Settings is the first visible code. These loading moves are specified work, not a claim that current origins already implement them.

For zapFDTD, each route uses its separately authored October 14 origin, including native sim.add_monitors() and Results. Do not collapse these into one combined origin or replace authored setup/settings cells. Bind only declared placeholders, retaining comments, cell order and metadata. All five user settings are explicit slots: source_port, wavelength_um, wavelength_span_um, num_wavelengths, cell_size_nm.

Settings and result methods are engine-specific. Meep exposes wavelength/span, frequency count, one source port and resolution (pixels/µm); wavelengths are in µm. Keep native meep.Simulation() setup/validation, get_stack(), configuration write and interactive preview readable; results call plot_interactive() / show_diagnostics(). zapFDTD uses native fdtd.Simulation(pdk=pdk_provider), grid spacing in nm, PDK cladding material and its own preview/results API; see its implementation notes. Future engines define their own controls and units. The current Meep/zapFDTD Run cell is:

# Config binds results_path to this notebook's selected bundle.
# Results are downloaded into results_path/sim-data-<job_name>/.
# None would instead use the kernel's current working directory.
result = sim.run(parent_dir=results_path)

No execution toggle or cell tags. Only Config is collapsed; later code, outputs and execution timing remain visible. Mark its Jupyter input/output hidden, and apply the editor's actual hidden state explicitly: VS Code may not honor that metadata alone. Bind by checked code-cell order; preserve metadata and clear counts/ outputs on generation. Use .venv kernelspec name/display, language python.

Creation hook: after writing a newly generated notebook, choose its project Python kernel, set cwd to the notebook directory, and execute only code-cell ordinal 0. Wait for completion before treating the notebook as ready. Never execute Settings, preparation, Run or Results automatically. Do this once per creation, not on every open/focus or for raw downloads/existing notebooks. Surface setup errors and its LVS report; do not continue with stale geometry. The notebook metadata declares creation_setup={code_cell:0, execute_on_creation:true}; the app implements the hook, not Jupyter itself. Missing kernel/dependencies leave the created/open notebook Setup pending; no automatic installs. Setup errors are recovered by manually rerunning Config, with no separate Retry setup action.

Prepare follows the latest saved original design through compact shared support. Python refreshes local implementation/helpers/caches while retaining explicit notebook arguments. Direct GDS rereads the original file; Python-derived GDS first refreshes/re-exports the exact upstream factory/arguments. GSCH defaults to the full shared rebuild/LVS chain on every Prepare, with one shared switch to manual Config reload if too slow. Run uses prepared state without freshness guards, automatic refresh or extra confirmation.

3. Concrete app-supplied values

Value Resolution / destination
Source route / design / cell Clicked item or active editor and user choice; preserve exported cell identity/hierarchy.
gsch_path Absolute saved GSCH path, passed as nyancir_path.
gds_path Original direct GDS or a generated bundle preparation/export artifact; a captured copy cannot become the authoritative live input.
dschematic_path, lvs_path Intermediate DSchematic and LVS LYRDB destinations inside inputs/.
source_path, project_path Original selected file and project root.
prepare_path Local preparation directory inside the notebook bundle.
results_path Absolute selected bundle path; default project simulations/<design>/<template>-NNN/. An edited Save to means exactly that folder, including existing/outside-project folders. Run passes it as parent_dir; gsim adds sim-data-<job_name>/.
PDK module / qualified name Resolve through full spec's project/indexed activation contract; import module and <module>.PDK build argument.
Python module/factory/settings Exact indexed clicked factory and current card arguments; import/call go together in Settings, retaining explicit notebook arguments on refresh.
Ports/source/monitors Template-declared required/optional/unused and optical/electrical eligibility; missing required direct-GDS ports block with an explanation, without a new port editor.
Heading/filename/bundle/cwd Template/route-specific names; default numbered destination or exact user-selected folder, kernel cwd at notebook, owning project/PDK stays tied to source.
Physical/solver overrides Declared stack/materials/source/domain/numerical settings from setup.
Initial wavelength/span Resolve project pyproject.toml simulation sweep through the same settings path as SAX. Convert min/max to center/full span per the full spec; write editable literals in Settings.
zapFDTD cladding layer Ask the solver/PDK integration to return the resolved layer-stack key (cladding_layer_name), material and provenance. Bind that key and layers = PDK.layer_stack.layers once in Config; Settings uses layers[cladding_layer_name].material. Do not have the sidebar guess arbitrary layer names.

Create inputs directories. Quote paths/data correctly; prefer /, including Windows absolute paths. Origin notebook files have no user paths; review copies contain concrete test bindings. Omit unused None assignments from Python/GDS Config cells; GDS imports still require populated gds_path and cell_name. Keep GSCH rebuild/LVS paths on the Layout Editor route. design.ports=[] preserves embedded ports in the current Meep/zapFDTD binders; a nonempty list currently replaces them. The live-source implementation must not replay captured ports.clear()/add_port(...) definitions over fresh GSCH ports. Explicit user-authored overrides remain editable Python. Overrides specify name, center, width, orientation, layer and optical type. Meep's get_stack() uses activated PDK: PH18DA providers are in ph18da.layers, Intel 0.4.0 in intel.tech, registered on PDK.layer_stack. Intel boolean-derived layer expressions need geometry qualification. Factory imports remain live project references; shared automatic refresh on Prepare must handle local helpers and component caches, rather than relying on a repeated import.

Origin notebooks expose named placeholders such as input_path = "<INSERT_INPUT_PATH_HERE>", with corresponding project, preparation, results, PDK-module and applicable material/layer bindings. Generation replaces declared slots with concrete editable values, not global text replacement over arbitrary cells. No required unresolved placeholder reaches setup execution. New bundles default under project simulations/; reference binders accept an explicit destination and bind results to that destination.

Reference binders scripts/bind_meep_template.py and scripts/bind_zapfdtd_template.py read app-generation simulation.json against their template schemas, stage inputs and refuse existing destinations. Generated notebooks do not need JSON or runtime assets. The Meep context includes validated structure, template/engine, PDK, provenance, design, source and monitors; optional stack/materials, port vertical overrides, frequencies and native domain/ solver controls. Provenance revision is not an automatic freshness check. Production coordinator must perform the full spec's validation and publication. Its destination policy differs from these prototypes: existing folders are valid, without sibling redirection, forced suffixing or an empty-folder requirement. Exact notebook-file collision handling is the remaining narrow publication decision; future sim-data-<job_name>/ results are separate and require no naming redesign. Use zapFDTD's documented native contract for its fields; new engines supply their own schemas rather than copying Meep-only controls.

The reference binders read project_root/pyproject.toml during creation; production should reuse the app's resolved settings.sim.x value (including sim.wls normalization). wavelength_overrides carries deliberate wizard center/span edits; old example values in source must not supersede project defaults. Keep num_freqs or num_wavelengths independent of the GUI sweep's num. Config reruns refresh geometry, not overwrite the user's edited Settings; create a new notebook or edit the wavelength values to apply later project-default changes.

4. GSCH build and direct-GDS behavior

The app checks source route/selection and resolves saved edits before creation. Full build/LVS runs in the hidden setup. Never accept an existing build/gds artifact as fresh. Keep physical-placement qualification via existing SDK/app semantics; schematic-coordinate fallback is not layout. Hierarchical dialects remain unqualified.

GSCH setup calls build_nyancir_gds with gsch_path, gds_path, pdk_name and dschematic_path (all four SDK arguments required). Reject returned warnings. Then use await asyncio.to_thread(check_lvs, ...), write lvs_path, load it with KLayout ReportDatabase, and stop if num_items() is nonzero. Only then import geometry. No ET parser, hidden full-LVS inside the build function, or custom comparison logic. Prepare invokes the same reusable chain by default; manual mode uses an explicit Config rerun. Opening an existing notebook does not execute it. All concrete paths occur as Config variables, not repeated literals.

Direct GDS imports preserve embedded ports. Use the clicked valid cell, otherwise the sole top cell, otherwise a cell Quick Pick; never infer it from a filename. Missing required ports stop creation with guidance to existing project/PDK mechanisms, without sample ports or a new port-definition editor. Standalone GDS has no reference schematic for LVS. Import error text: “This GDS file is corrupt or unreadable. Export it again or select another GDS file.” Material/stack errors identify the unresolved property.

5. Implementation checks and remaining tasks

Apply the full spec's release gates; the app-specific checks are:

  1. Missing/stale project GDS still rebuilds from valid GSCH; changed saved source reaches a fresh bundle. Invalid placement/connections/LVS prevents successful initialization.
  2. Repeating generation preserves previous notebooks/results. Explicit execution of the hidden setup rebuilds its own input; reopening does not auto-run it.
  3. Only setup auto-executes once on creation; paths/PDK/geometry resolve, settings remain first visible, and setup failures are clearly reported. Kernel/cwd are correct.
  4. Sidebar and eventual top bar reach equivalent setup with the correct selected cell.

Latest recorded shared Meep/zapFDTD/project checks: 88 passed (October 15), not proof of end-to-end app integration. Supported source routes and valid/broken rebuild fixtures were verified on October 12 (see each engine's notes). Finish shared worker/registry/wizard orchestration and the creation hook. Current Meep/zapFDTD templates monitor all exported optical ports; no editable monitor selector. The revised hidden/LVS setup was also verified: supported routes pass and both broken fixtures stop before import. Editor auto-execution/hiding remains app work. Solver convergence/mode diagnostics remain SDK/solver follow-up, recorded in development notes, not sidebar implementation.

If Config fails, an earlier component may remain in the kernel. Ordinary recovery is to fix and manually rerun Config. Do not add a Run guard or separate retry UI.

Source-port preference order is explicit choice, eligible remembered choice for the exact canonical project/design, project preferred choice, sole eligible port, otherwise a picker. Remember is opt-in, staged until publication; cancel writes no new preference. Change/Forget remain accessible even with review disabled. No o1 or alphabetical guessing. A removed explicit notebook port raises the ordinary Prepare error. Only multiple valid PDK candidates cause a PDK picker, with: warning: multiple PDKs in the same project file may result in unpredictable behavior. Missing PDK/physical data is an actionable error, not a transcription prompt or filename-based guess.

Shared template edits: use C:/Users/eve/Documents/Codex/worktrees/simulation-template-spec for these route-specific origins. Other worktrees do not synchronize edits. Coordinate edits to the same files; preserve the declared <INSERT_..._HERE> slots. The zap origins require gsim PR #371 (9c9975b or later); the app must not reinsert the retired notebook-local monitor helper.