## Foundation
Small shared mechanics: Python API behavior, failure telemetry, cell
parsing, validation, chapter navigation, and dynamic cell classification
for notebook context.
This notebook is the shared basement of the project. It keeps the small
private helpers that every public tool relies on: Python API behavior,
safe error reporting, cell selection, cell parsing, semantic cell
classification, and chapter addressing.
Most functions here are intentionally private. The rest of the project
can stay user-facing because this notebook absorbs the messy details of
notebooks as data structures.
Read it as a tour of the shapes nbskill needs to recognize. A notebook
is not just text: it has cells, headings, generated Python files,
visible outputs, and tiny bits of metadata. The foundation layer gives
all later notebooks one vocabulary for those shapes, so the public tools
can talk about notebooks without making users think in raw JSON.
### Production contract
This notebook owns the shared invariants for every other tool.
Production behavior is: path helpers resolve notebooks from project-root
and notebook-dir execution, cell-block parsing creates valid notebook
cells, metadata stamping records semantic classes and export hashes, and
validation reports missing or stale metadata before edits proceed.
The helpers here deliberately stay small because every higher-level tool
depends on them. For example,
[`parse_cells`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#parse_cells)
turns friendly cell-block text into notebook cells, the semantic
classifiers let readers say “show me tests” or “show me exported code”
without parsing raw notebook JSON.
``` python
from contextlib import redirect_stdout
from io import StringIO
import os
from fastcore.test import test_eq
from tempfile import TemporaryDirectory
```
------------------------------------------------------------------------
source
### numbered_notebook_path
``` python
def numbered_notebook_path(
nbs_path, name, index:NoneType=None
):
```
*Return a flat numbered notebook path for a simple module name.*
### Demo scratch files
Examples and tests in later notebooks need small notebooks they can
safely mutate. Scratch helpers create disposable artifacts in the system
temporary directory and remove them after use, so notebook test runners
never create repository data folders.
[`tool_notebook`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#tool_notebook)
builds the shared reader, executor, reviewer, and example shape in code.
------------------------------------------------------------------------
source
### remove_demo_path
``` python
def remove_demo_path(
path
):
```
[`demo_path`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#demo_path)
names a scratch artifact in the system temporary directory by default
and optionally removes any previous copy. It only creates the parent
folder; helpers that write content decide what belongs at the path. An
explicit `base` remains available when an example needs a chosen
location.
Use
[`demo_path_context`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#demo_path_context)
when a test needs an arbitrary scratch file or directory and should
clean it up after the block. That keeps cleanup policy in one helper
instead of repeating `try`/`finally` in every test.
------------------------------------------------------------------------
source
### demo_path_context
``` python
def demo_path_context(
name, base:NoneType=None, reset:bool=True
):
```
*Yield a demo path and remove it when the block exits.*
------------------------------------------------------------------------
source
### demo_path
``` python
def demo_path(
name, base:NoneType=None, reset:bool=True
):
```
*Return a scratch path in the system temporary directory or `base`,
optionally resetting it.*
------------------------------------------------------------------------
source
### path_candidates
``` python
def path_candidates(
path
):
```
*Return cwd-relative path candidates for root and notebook-dir
execution.*
``` python
_path_candidate_rows = path_candidates("nbs/00_foundation.ipynb")
assert _path_candidate_rows[0] == Path("nbs/00_foundation.ipynb")
if Path.cwd().name == "nbs":
assert _path_candidate_rows[-1] == Path("00_foundation.ipynb")
else:
assert _path_candidate_rows == [Path("nbs/00_foundation.ipynb")]
assert path_candidates(Path("/tmp/example.ipynb")) == [Path("/tmp/example.ipynb")]
print("path_candidates handles root and notebook-dir paths")
```
path_candidates handles root and notebook-dir paths
Notebook helpers should fail gently when a path is missing, unreadable,
or shaped like ordinary JSON instead of a Jupyter notebook.
[`is_valid_ipynb`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#is_valid_ipynb)
keeps that contract small and dependency-free.
------------------------------------------------------------------------
source
### is_valid_ipynb
``` python
def is_valid_ipynb(
path
):
```
[`write_demo_notebook`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#write_demo_notebook)
owns the whole lifetime of a scratch notebook: create it, yield its path
to the example, then remove it even if the example fails.
------------------------------------------------------------------------
source
### write_demo_notebook
``` python
def write_demo_notebook(
name, cells:NoneType=None, base:NoneType=None, reset:bool=True
):
```
[`tool_notebook`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#tool_notebook)
keeps the shared reader, executor, reviewer, and test shape in code.
Each test writes a fresh notebook and removes it afterwards.
------------------------------------------------------------------------
source
### tool_notebook
``` python
def tool_notebook():
```
*Return the canonical notebook used by tool tests and examples.*
[`write_tool_notebook`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#write_tool_notebook)
gives each example and test an isolated copy of the canonical tool
notebook.
------------------------------------------------------------------------
source
### write_tool_notebook
``` python
def write_tool_notebook(
name:str='tool.ipynb', base:NoneType=None
):
```
*Write the canonical tool notebook for one example or test.*
[`write_demo_file`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#write_demo_file)
keeps temporary conversion inputs out of the repository.
------------------------------------------------------------------------
source
### write_demo_file
``` python
def write_demo_file(
name, source, base:NoneType=None
):
```
*Write one disposable text file for an example or test.*
Here is the tiny notebook fixture in the wild. It gives examples a real
`.ipynb` to read, then vanishes after the context exits, which keeps the
repo tidy while still letting the rendered notebook show concrete
output.
``` python
from fastcore.nbio import read_nb
with write_demo_notebook("00_foundation_tour.ipynb") as path:
nb = read_nb(path)
print(path.name)
print([cell.cell_type for cell in nb.cells])
print(str(nb.cells[0].source).splitlines()[0])
print(path.exists())
```
00_foundation_tour.ipynb
['markdown', 'code', 'code']
## Demo notebook
False
``` python
with write_demo_notebook("00_foundation_bad.ipynb") as bad_path:
bad_path.write_text("{}", encoding="utf-8")
assert not is_valid_ipynb(bad_path)
assert not is_valid_ipynb(bad_path.with_suffix(".missing.ipynb"))
```
### Python API contracts and diagnostics
The public functions in later notebooks are direct Python APIs. These
helpers keep return values and validation errors predictable while
recording tool starts or failures in a small local failure map for
debugging repeated friction.
------------------------------------------------------------------------
source
### api_return
``` python
def api_return(
value:NoneType=None
):
```
*Return a Python API value unchanged.*
------------------------------------------------------------------------
source
### api_error
``` python
def api_error(
msg
):
```
*Raise a Python API validation error.*
------------------------------------------------------------------------
source
### failure_map_path
``` python
def failure_map_path():
```
------------------------------------------------------------------------
source
### empty_failure_map
``` python
def empty_failure_map():
```
------------------------------------------------------------------------
source
### load_failure_map
``` python
def load_failure_map(
path
):
```
------------------------------------------------------------------------
source
### git_root
``` python
def git_root(
path:str='.'
):
```
*Return the containing git repository root, if one exists.*
------------------------------------------------------------------------
source
### git_run
``` python
def git_run(
root, *args
):
```
*Run a git command from `root` and return stdout, or empty text on
failure.*
------------------------------------------------------------------------
source
### git_tracked_paths
``` python
def git_tracked_paths(
root
):
```
*Return git-tracked paths relative to `root`.*
------------------------------------------------------------------------
source
### git_status_paths
``` python
def git_status_paths(
root
):
```
*Return paths with working-tree changes relative to `root`.*
------------------------------------------------------------------------
source
### git_diff_stats
``` python
def git_diff_stats(
root, ref:str='HEAD'
):
```
*Return added/deleted line stats for working-tree changes from `ref`.*
------------------------------------------------------------------------
source
### load_nbskill_config
``` python
def load_nbskill_config(
path:str='.'
):
```
*Load .nbskill.py config from project root, merged over defaults.*
------------------------------------------------------------------------
source
### notebook_paths
``` python
def notebook_paths(
path:str='nbs'
):
```
*Return visible notebook paths for a file, directory, glob, or nbdev
project path.*
[`notebook_paths`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#notebook_paths)
keeps an explicitly selected notebook reachable inside a hidden
workspace folder such as `.codex`. It checks the filename in this case;
directory discovery still filters hidden paths. A temporary hidden
folder demonstrates the explicit-file contract.
``` python
visible_notebooks = notebook_paths("nbs/00_foundation.ipynb")
visible_notebooks
```
------------------------------------------------------------------------
source
### source_without_directives
``` python
def source_without_directives(
source
):
```
*Return source with nbdev cell directives removed.*
------------------------------------------------------------------------
source
### source_hash
``` python
def source_hash(
source, length:int=12
):
```
*Return a SHA-256 source hash, shortened unless `length` is `None`.*
------------------------------------------------------------------------
source
### cap_text
``` python
def cap_text(
text, limit:int=2000, max_output_chars:NoneType=None
):
```
*Cap text, returning a string or review-style metadata when
`max_output_chars` is used.*
------------------------------------------------------------------------
source
### file_line_count
``` python
def file_line_count(
path
):
```
*Return the number of text lines in `path`, or zero when it cannot be
read.*
------------------------------------------------------------------------
source
### notebook_result
``` python
def notebook_result(
data, max_display_chars:int=6000
):
```
*Keep complete structured data while bounding the interpreter display.*
------------------------------------------------------------------------
source
### NotebookResult
``` python
def NotebookResult(
*args, **kwargs
):
```
*Dictionary result with one compact interpreter display.*
## Generated source ownership
nbdev can place a module docstring before its autogenerated-file notice.
Source ownership therefore finds the notice in the exported text instead
of assuming that it is in the first three physical lines. This keeps
generated modules read-only for agents.
------------------------------------------------------------------------
source
### generated_owner
``` python
def generated_owner(
path, notebooks:NoneType=None
):
```
*Return the source notebook for an nbdev-generated Python file, when
known.*
------------------------------------------------------------------------
source
### install_nbdev_pre_commit_hooks
``` python
def install_nbdev_pre_commit_hooks(
path:str='.', run_nbdev_install_hooks:bool=True
):
```
*Install nbdev-clean and nbdev-test pre-commit hooks in a git-backed
nbdev project.*
A project can need a small idempotent bootstrap before using nbskill.
Once a workspace root is known,
[`bootstrap_nbskill_project`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#bootstrap_nbskill_project)
can copy the shared agent instructions, make notebook CI serial, and
preserve nbskill metadata through nbdev cleanups without clobbering
local `AGENTS.md` edits.
------------------------------------------------------------------------
source
### bootstrap_nbskill_project
``` python
def bootstrap_nbskill_project(
path:str='.', overwrite_agents:bool=False, pause:str='1'
):
```
*Install nbskill project guardrails once a workspace root is known.*
### Parsing user-facing selectors
Selectors are the boundary between human shorthand and list operations.
A caller might ask for `3`, `2:5`, `-1`, or `None`; these helpers
normalize that text before any read or write tool touches notebook
cells. Keeping the grammar here makes higher-level tools predictable and
keeps cell-index mistakes from spreading.
------------------------------------------------------------------------
source
### parse_literal
``` python
def parse_literal(
value
):
```
------------------------------------------------------------------------
source
### none_if_string
``` python
def none_if_string(
value
):
```
### Turning text into notebook cells
Write tools accept plain text because that is what agents and humans can
produce quickly. This section turns that text into real cells: split on
block separators, honor `%%markdown`/`%%code` markers, and split
exported code into symbol-sized chunks when needed. It is the small
parser that makes notebook edits feel like editing a document instead of
assembling JSON.
------------------------------------------------------------------------
source
### symbol_short_name
``` python
def symbol_short_name(
symbol
):
```
*Return the final dotted component of a symbol name.*
------------------------------------------------------------------------
source
### call_name
``` python
def call_name(
node
):
```
*Return dotted name text for an AST name, attribute, or call
expression.*
------------------------------------------------------------------------
source
### short_call_name
``` python
def short_call_name(
node, default:NoneType=None
):
```
*Return the final dotted component of a call/name node, or `default`.*
------------------------------------------------------------------------
source
### is_definition_node
``` python
def is_definition_node(
node
):
```
------------------------------------------------------------------------
source
### node_start_line
``` python
def node_start_line(
node
):
```
------------------------------------------------------------------------
source
### is_export_directive
``` python
def is_export_directive(
line
):
```
------------------------------------------------------------------------
source
### cell_source
``` python
def cell_source(
cell
):
```
------------------------------------------------------------------------
source
### parse_code_cell
``` python
def parse_code_cell(
cell
):
```
*Parse a code cell after removing nbdev directives, or return `None`.*
### Cell metadata and semantic classes
Once cells can be parsed, the tools need names for what they *are
doing*. A code cell with output is an example; a code cell with an
`assert` is a test; a markdown cell with prose is documentation. Those
labels power filters such as `type=tests` and help review tools talk
about notebooks in human terms.
### Domain entities
These tiny models give the project names for the shapes it already
manipulates: cells, chapters, notebooks, and symbols. The classes stay
thin; most behavior is patched onto them so each concept can grow near
the helper that understands it.
### Typed cell vocabularies
[`CellType`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#celltype)
names the three notebook cell kinds,
[`SemanticType`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#semantictype)
names the reader-facing role a cell plays, and
[`Directive`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#directive)
names the nbdev directives an editor can stamp onto a cell. Enums keep
the rest of the project from passing bare strings around, and let
context drop directive lines because the type already carries that
meaning.
------------------------------------------------------------------------
source
### apply_directives
``` python
def apply_directives(
source, directives
):
```
*Prepend any missing `#| ...` directive lines to `source`.*
------------------------------------------------------------------------
source
### directive_lines
``` python
def directive_lines(
directives
):
```
*Return `#| ...` lines for
[`Directive`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#directive)s
given as members, names, or a comma string.*
------------------------------------------------------------------------
source
### Directive
``` python
def Directive(
*args, **kwds
):
```
*nbdev cell directives an editor can stamp onto a cell.*
------------------------------------------------------------------------
source
### SemanticType
``` python
def SemanticType(
*args, **kwds
):
```
*The reader-facing role of a cell, used by context and review tools.*
------------------------------------------------------------------------
source
### CellType
``` python
def CellType(
*args, **kwds
):
```
*The notebook cell kinds nbskill understands.*
------------------------------------------------------------------------
source
### NotebookSymbol
``` python
def NotebookSymbol(
name:str, path:object=None, cell_id:str='', cell_idx:object=None, module:str='', kind:str='',
data:object=
)->None:
```
*A named notebook symbol plus optional placement and graph data.*
------------------------------------------------------------------------
source
### Notebook
``` python
def Notebook(
nb:object, path:object=None
)->None:
```
*A notebook plus the path it was read from.*
------------------------------------------------------------------------
source
### Chapter
``` python
def Chapter(
title:str, start:int, end:int, cells:object=
)->None:
```
*A contiguous heading-owned span inside a notebook.*
------------------------------------------------------------------------
source
### Cell
``` python
def Cell(
cell:object, idx:object=None, path:object=None
)->None:
```
*A focused view of one notebook cell plus its optional location.*
[`xml_escape`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#xml_escape)
and
[`xml_attrs`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#xml_attrs)
keep model-to-XML snippets boring and predictable. They are
intentionally small because the XML is only for readable LLM context,
not a general document writer.
------------------------------------------------------------------------
source
### xml_attrs
``` python
def xml_attrs(
**kwargs
):
```
*Render non-empty keyword values as XML attributes.*
------------------------------------------------------------------------
source
### xml_escape
``` python
def xml_escape(
text
):
```
*Escape text for simple XML-shaped context snippets.*
``` python
{
"escaped": xml_escape("| "),
"attrs": xml_attrs(kind="cell", title=""),
}
```
{'escaped': '<cell>', 'attrs': 'kind="cell" title="<demo>"'}
------------------------------------------------------------------------
### Cell.from_source
``` python
def from_source(
source, default_type:str='code', idx:NoneType=None, path:NoneType=None
):
```
*Build a cell model from editable source text.*
------------------------------------------------------------------------
source
### Cell.cell_type
``` python
def cell_type():
```
*Return the cell type as a
[`CellType`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#celltype).*
------------------------------------------------------------------------
source
### Cell.id
``` python
def id():
```
*Return the notebook cell id.*
------------------------------------------------------------------------
### Cell.source
``` python
def source():
```
*Return the cell source as one string.*
------------------------------------------------------------------------
### Cell.metadata
``` python
def metadata():
```
*Return mutable cell metadata.*
------------------------------------------------------------------------
### Cell.code
``` python
def code():
```
*Return source without nbdev directives.*
------------------------------------------------------------------------
### Cell.directive
``` python
def directive(
name:NoneType=None
):
```
*Return the first directive, or the value for one directive name.*
------------------------------------------------------------------------
### Cell.directives
``` python
def directives():
```
*Return nbdev directive lines without the leading marker.*
------------------------------------------------------------------------
### Cell.heading_title
``` python
def heading_title(
levels:tuple=(2,)
):
```
*Return the first markdown heading title whose level is allowed.*
------------------------------------------------------------------------
### Cell.class_names
``` python
def class_names():
```
*Return semantic class names for the cell.*
------------------------------------------------------------------------
### Cell.prefix
``` python
def prefix(
show_ids:bool=False
):
```
*Return the compact rendered cell prefix used in context output.*
------------------------------------------------------------------------
source
### Cell.to_xml
``` python
def to_xml(
include_output:bool=False
):
```
*Return an XML-shaped cell view for LLM context; directives become the
semantic type.*
------------------------------------------------------------------------
source
### Cell.to_record
``` python
def to_record():
```
*Return a small structured record for this cell.*
------------------------------------------------------------------------
### Chapter.all
``` python
def all(
cells, levels:tuple=(2,), fallback:NoneType=None
):
```
*Return all chapter models opened by markdown headings.*
------------------------------------------------------------------------
### Chapter.from_span
``` python
def from_span(
span, cells:NoneType=None
):
```
*Build a chapter model from a span dictionary.*
------------------------------------------------------------------------
### Chapter.body
``` python
def body():
```
*Return cells inside this chapter span.*
------------------------------------------------------------------------
### Chapter.to_span
``` python
def to_span():
```
*Return the dictionary span shape used by existing APIs.*
------------------------------------------------------------------------
### Chapter.to_record
``` python
def to_record():
```
*Return structured chapter metadata.*
------------------------------------------------------------------------
### Chapter.to_xml
``` python
def to_xml(
include_cells:bool=True
):
```
*Return an XML-shaped chapter view for LLM context.*
------------------------------------------------------------------------
### Notebook.cells
``` python
def cells():
```
*Return notebook cells as cell models.*
------------------------------------------------------------------------
### Notebook.from_path
``` python
def from_path(
path
):
```
*Read a notebook path into a notebook document model.*
------------------------------------------------------------------------
source
### find_cell_by_id
``` python
def find_cell_by_id(
cells, cell_id
):
```
------------------------------------------------------------------------
### Notebook.cell
``` python
def cell(
cell_id
):
```
*Return one cell model by stable id.*
------------------------------------------------------------------------
source
### notebook_hash
``` python
def notebook_hash(
nb
):
```
*Return a stable source hash for a notebook’s ordered cell contents.*
------------------------------------------------------------------------
source
### matches_filter
``` python
def matches_filter(
source, pattern
):
```
------------------------------------------------------------------------
source
### chapter_spans
``` python
def chapter_spans(
cells, levels:tuple=(2,), fallback:NoneType=None
):
```
*Return contiguous notebook cell spans opened by markdown headings.*
------------------------------------------------------------------------
source
### one_chapter
``` python
def one_chapter(
cells, chapter, create:bool=False
):
```
------------------------------------------------------------------------
### Notebook.chapter
``` python
def chapter(
title, create:bool=False
):
```
*Return one chapter model by title or regex.*
------------------------------------------------------------------------
### Notebook.notebook_source_hash
``` python
def notebook_source_hash():
```
*Return the notebook’s ordered source hash.*
------------------------------------------------------------------------
source
### ensure_cell_ids
``` python
def ensure_cell_ids(
nb
):
```
*Add nbdev-style ids to notebook cells missing `id`, mutating and
returning `nb`.*
------------------------------------------------------------------------
source
### validate_code_cells
``` python
def validate_code_cells(
cells
):
```
------------------------------------------------------------------------
source
### exported_py_path
``` python
def exported_py_path(
nb_path, nb:NoneType=None
):
```
*Return the generated Python file path for an nbdev notebook, if it has
one.*
------------------------------------------------------------------------
source
### commit_notebook
``` python
def commit_notebook(
path, trial, before:NoneType=None, affected_cell_ids:NoneType=None, validate_code:bool=True, dry_run:bool=False,
export:bool=True, writer:function=write_nb
):
```
*Validate, stamp, write, export, and rollback one notebook as one
commit.*
------------------------------------------------------------------------
### Notebook.commit
``` python
def commit(
**kwargs
):
```
*Commit this notebook model through the shared atomic writer.*
### Notebook document helpers
[`NotebookDocument.cell`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#notebookdocument.cell)
returns one cell model by id. `chapter` resolves a heading-owned span,
`notebook_source_hash` gives the model’s stable source hash, and
`commit` sends the current model through
[`commit_notebook`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#commit_notebook).
``` python
document = NotebookDocument(new_nb([mk_cell("## Alpha", cell_type="markdown"), mk_cell("answer = 42")]), path="demo.ipynb")
(document.cell(document.nb.cells[1].id).source, document.chapter("Alpha").title, document.notebook_source_hash())
```
The notebook domain model gives callers small, navigable objects:
`Notebook.cell` finds a stable cell id, `Notebook.chapter` returns a
dynamic
[`Chapter`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#chapter),
`Notebook.notebook_source_hash` summarizes ordered source, and
`Notebook.commit` writes through the same atomic path as the editing
tools.
------------------------------------------------------------------------
### Notebook.to_xml
``` python
def to_xml(
levels:tuple=(2,), include_cells:bool=True
):
```
*Return an XML-shaped notebook view for LLM context.*
------------------------------------------------------------------------
### Notebook.chapters
``` python
def chapters(
levels:tuple=(2,), fallback:NoneType=None
):
```
*Return notebook chapters as chapter models.*
------------------------------------------------------------------------
source
### NotebookSymbol.from_record
``` python
def from_record(
record, data:NoneType=None
):
```
*Build a symbol model from a graph or definition record.*
------------------------------------------------------------------------
source
### NotebookSymbol.to_record
``` python
def to_record():
```
*Return a small structured record for this symbol.*
------------------------------------------------------------------------
source
### NotebookSymbol.to_xml
``` python
def to_xml():
```
*Return an XML-shaped symbol view for LLM context.*
------------------------------------------------------------------------
source
### cell_metadata
``` python
def cell_metadata(
cell
):
```
------------------------------------------------------------------------
source
### notebook_metadata
``` python
def notebook_metadata(
nb
):
```
------------------------------------------------------------------------
source
### nbskill_cell_metadata
``` python
def nbskill_cell_metadata(
cell, create:bool=True
):
```
*Return the shared nbskill metadata block for a cell.*
------------------------------------------------------------------------
source
### file_hash
``` python
def file_hash(
path
):
```
``` python
assert source_without_directives("#| export\nx = 1\n#| default_exp demo") == "x = 1"
with write_demo_notebook("foundation_notebook_paths.ipynb") as nb_path:
checkpoint = nb_path.parent / ".ipynb_checkpoints" / "skip.ipynb"
checkpoint.parent.mkdir(parents=True, exist_ok=True)
write_nb(new_nb([mk_cell("assert True")]), checkpoint)
try:
paths = notebook_paths(nb_path.parent)
assert nb_path in paths
assert checkpoint not in paths
finally:
remove_demo_path(checkpoint.parent)
generated = demo_path("generated_owner_sample.py")
try:
generated.write_text(
'\n'.join([
'\"\"\"Generated module documentation.',
'It can span several lines',
'before the nbdev header.',
'\"\"\"',
'',
'# AUTOGENERATED! DO NOT EDIT! File to edit: ../00_foundation.ipynb.',
]) + '\n',
encoding="utf-8",
)
assert generated_owner(generated).name == "00_foundation.ipynb"
assert file_line_count(generated) == 6
finally:
remove_demo_path(generated)
assert cap_text("abcdef", limit=3).startswith("abc")
assert cap_text("abcdef", max_output_chars=3)["truncated"] is True
```
------------------------------------------------------------------------
source
### stamp_export_metadata
``` python
def stamp_export_metadata(
nb, py_path
):
```
### Project-scoped nbdev export
[`run_nbdev_export_from_project`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#run_nbdev_export_from_project)
and
[`export_notebook`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#export_notebook)
deliberately run nbdev export in an isolated process under a
project-scoped lock. This looks heavier than calling `nbdev_export`
directly, but it protects concurrent Python callers working in different
repositories.
The two failures this avoids are subtle. First, nbdev export depends on
project context; changing the process cwd with `os.chdir` is global
state, so concurrent exports from different projects can race. Second, a
caller timeout does not stop a Python worker thread, so a timed-out edit
can keep exporting in the background and consume memory. A child process
is a boundary the caller can terminate, while the lock only serializes
exports for the same project. Different projects get different lock
files and can still make progress independently.
------------------------------------------------------------------------
source
### run_nbdev_export_from_project
``` python
def run_nbdev_export_from_project(
nb_path
):
```
*Run nbdev export in a project-scoped process without changing server
cwd.*
------------------------------------------------------------------------
source
### export_notebook
``` python
def export_notebook(
nb, nb_path, writer:function=write_nb
):
```
*Export an nbdev notebook and stamp export metadata when possible.*
``` python
# Export a real nbdev notebook without changing the current process cwd.
run_nbdev_export_from_project("nbs/00_foundation.ipynb")
```
### Stable ids and notebook commits
[`ensure_cell_ids`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#ensure_cell_ids)
fills in missing notebook cell ids.
[`notebook_hash`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#notebook_hash)
changes when a cell’s id or source changes.
[`commit_notebook`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#commit_notebook)
validates and records a proposed notebook change, or writes and exports
it when `dry_run` is false.
``` python
trial = new_nb([mk_cell("value = 1")])
ensure_cell_ids(trial)
commit_notebook("nbs/data/commit_notebook_example.ipynb", trial, dry_run=True, export=False)["planned_hash"] == notebook_hash(trial)
```
[`notebook_hash`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#notebook_hash)
and
[`commit_notebook`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#commit_notebook)
form the shared commit primitive. Callers build a trial notebook, pass
the original as `before`, and get read-back hashes after the write
succeeds. The commit boundary rejects malformed notebook structure and
duplicate cell ids before it writes. Changed code-cell outputs are
cleared before the notebook is committed.
------------------------------------------------------------------------
source
### parse_one_cell
``` python
def parse_one_cell(
text, default_type:str='code'
):
```
------------------------------------------------------------------------
source
### find_cell_by_text
``` python
def find_cell_by_text(
cells, old_str
):
```
------------------------------------------------------------------------
source
### replace_cell
``` python
def replace_cell(
nb, idx, new_cell
):
```
------------------------------------------------------------------------
source
### clear_outputs
``` python
def clear_outputs(
cell
):
```
------------------------------------------------------------------------
source
### load_cells_text
``` python
def load_cells_text(
cells:str='', cells_file:NoneType=None
):
```
*Return direct Python text or the contents of an explicit text file.*
------------------------------------------------------------------------
source
### parse_cells
``` python
def parse_cells(
cells, default_type:str='code'
):
```
### Naming cells by behavior
The reading layer presents cells by meaning, not only by `code` or
`markdown`. This section detects imports, private helpers, exported
code, tests, examples, docs, section headers, and mixed cells so callers
can filter notebooks at a useful level. A test that produces a useful
output is intentionally both a test and an example; that pairing is not
an unclean mixed cell.
------------------------------------------------------------------------
source
### first_line
``` python
def first_line(
source
):
```
------------------------------------------------------------------------
source
### cell_prefix
``` python
def cell_prefix(
idx, cell, show_ids:bool=False
):
```
------------------------------------------------------------------------
source
### is_exported_code_cell
``` python
def is_exported_code_cell(
cell
):
```
------------------------------------------------------------------------
source
### output_value_text
``` python
def output_value_text(
value
):
```
*Return a notebook output value as plain text.*
------------------------------------------------------------------------
source
### output_text
``` python
def output_text(
output
):
```
*Return a notebook output object as plain text.*
------------------------------------------------------------------------
source
### cell_output_text
``` python
def cell_output_text(
cell
):
```
*Return all visible outputs for `cell` as compact plain text.*
------------------------------------------------------------------------
source
### cell_semantic_warnings
``` python
def cell_semantic_warnings(
cell
):
```
*Return semantic warning codes for one cell.*
------------------------------------------------------------------------
source
### stamp_notebook_metadata
``` python
def stamp_notebook_metadata(
nb, exported_py_path:NoneType=None
):
```
``` python
from fastcore.nbio import read_nb
with write_demo_notebook("commit_notebook_example.ipynb", cells=[mk_cell(chr(10).join(["value = 1", "print(value)"]))]) as commit_path:
before_nb = read_nb(commit_path)
trial_nb = read_nb(commit_path)
print(notebook_hash(before_nb))
trial_nb.cells[0].source = chr(10).join(["value = 2", "print(value)"])
trial_nb.cells[0].outputs = [dict(output_type="stream", name="stdout", text="stale" + chr(10))]
result = commit_notebook(commit_path, trial_nb, before=before_nb, affected_cell_ids=[trial_nb.cells[0].id], export=False)
print(result["changed"], result["affected_cell_ids"])
print(read_nb(commit_path).cells[0].outputs)
```
32c31893da74
True ['18ce0375']
[]
------------------------------------------------------------------------
source
### cell_class_names
``` python
def cell_class_names(
cell
):
```
``` python
with write_demo_notebook("00_foundation_context.ipynb") as path:
assert path.exists()
assert is_valid_ipynb(path)
assert not path.exists()
```
[`parse_cells`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#parse_cells)
is the little translator between a friendly edit format and real
notebook cells. The dashed line means “start a new cell”, while
`%%markdown` and `%%code` make the intended cell type explicit.
``` python
_demo_text = "\n".join(["%%markdown", "## Demo", "---", "%%code", "value = 42"])
_demo_cells = parse_cells(_demo_text)
for idx, cell in enumerate(_demo_cells):
print(cell_prefix(idx, cell))
print(" type:", cell.cell_type)
print(" first line:", first_line(cell_source(cell)))
```
Cell id=25790bac: markdown
type: markdown
first line: ## Demo
Cell id=38f08303: code
type: code
first line: value = 42
------------------------------------------------------------------------
source
### cell_matches_type
``` python
def cell_matches_type(
cell, cell_type
):
```
------------------------------------------------------------------------
source
### Cell.output_text
``` python
def output_text():
```
*Return all visible outputs for this cell as compact plain text.*
------------------------------------------------------------------------
source
### Cell.semantic_type
``` python
def semantic_type():
```
*Return the primary reader-facing semantic type for this cell.*
------------------------------------------------------------------------
source
### Cell.context_record
``` python
def context_record():
```
*Return the context record shape used by read helpers, with directives
stripped.*
### Visible cell output
[`output_value_text`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#output_value_text),
[`output_text`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#output_text),
and
[`cell_output_text`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#cell_output_text)
turn Jupyter output payloads into plain text.
[`Cell.output_text`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#cell.output_text),
[`Cell.semantic_type`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#cell.semantic_type),
and
[`Cell.context_record`](https://MorsCerta-crypto.github.io/nbskill/foundation.html#cell.context_record)
apply the same rules to a cell model used by readers.
``` python
output_cell = mk_cell("print('ok')")
output_cell.outputs = [dict(output_type="stream", name="stdout", text="ok\n")]
output_model = Cell(output_cell, idx=4)
(output_value_text(["o", "k"]), output_text(output_cell.outputs[0]), cell_output_text(output_cell), output_model.output_text(), output_model.semantic_type(), output_model.context_record()["cell_idx"])
```
------------------------------------------------------------------------
source
### with_context
``` python
def with_context(
cells, items, include:bool=False
):
```
### Chapters as editing scopes
Chapters are the notebook-native version of a module section. A heading
cell opens a span, and everything until the next heading belongs to that
span. Write, execute, and read tools use these spans so users can say
“the query language chapter” instead of counting cell indexes by hand.
------------------------------------------------------------------------
source
### heading_title
``` python
def heading_title(
cell, levels:tuple=(2,)
):
```
*Return the first markdown heading title whose level is allowed.*
------------------------------------------------------------------------
source
### chapter_index_set
``` python
def chapter_index_set(
cells, chapter
):
```
``` python
with write_demo_notebook("domain_model_example.ipynb", cells=[mk_cell("## Alpha", cell_type="markdown"), mk_cell("answer = 42")]) as model_path:
model_doc = Notebook.from_path(model_path)
model_cell = model_doc.cell(model_doc.nb.cells[1].id)
model_chapter = model_doc.chapter("Alpha")
print(model_cell.source)
print(model_chapter.to_record())
print(model_doc.notebook_source_hash())
print(model_doc.commit(before=model_doc.nb, export=False)["changed"])
```
answer = 42
{'title': 'Alpha', 'start': 0, 'end': 2, 'cell_count': 2}
a584b5a3979c
False
On a real notebook, chapter spans become a tiny table of contents with
cell ranges. Here `index.ipynb` is being parsed by the foundation
helpers that the reading tools themselves rely on.
``` python
_index_cells = read_nb("index.ipynb").cells
for span in chapter_spans(_index_cells)[:5]: print(f"{span['title']}: cells {span['start']}..{span['end'] - 1}")
print("reading indexes:", sorted(chapter_index_set(_index_cells, "Reading"))[:5])
```
The problem this project solves: cells 2..2
How the notebooks fit together: cells 3..3
Production readiness: cells 4..5
A tiny notebook to work on: cells 6..7
Reading: choose the smallest useful context: cells 8..9
reading indexes: [8, 9]
``` python
custom_map = demo_path("00_foundation_errors.json")
_old_failure_map = os.environ.get("NBSKILL_FAILURE_MAP")
try:
os.environ["NBSKILL_FAILURE_MAP"] = str(custom_map)
assert failure_map_path() == custom_map
finally:
if _old_failure_map is None: os.environ.pop("NBSKILL_FAILURE_MAP", None)
else: os.environ["NBSKILL_FAILURE_MAP"] = _old_failure_map
```
``` python
cells = parse_cells("%%markdown\n## Demo\n---\n%%code\nvalue = 42")
assert len(cells) == 2
assert cells[0].cell_type == "markdown"
assert cells[1].source == "value = 42"
one = parse_one_cell("%%code\nvalue = 99")
assert one.cell_type == "code"
assert one.source == "value = 99"
encoded = "def f():" + chr(92) + "n return 1"
assert load_cells_text("%%code\\nvalue = 1") == "%%code\\nvalue = 1"
assert load_cells_text(encoded) == encoded
```
Here are the labels on toy cells. This is deliberately small, but it is
the same classification that lets the reading and review tools
distinguish examples from tests without asking a model to guess.
``` python
public_cell = mk_cell("#| export\ndef public():\n return 1")
private_cell = mk_cell("#| export\ndef _private():\n return 1")
example_cell = mk_cell("value = 1")
example_cell.outputs = [dict(output_type="stream", name="stdout", text="1\n")]
for label, cell in {"public": public_cell, "private": private_cell, "example": example_cell}.items():
print(f"{label:8} -> {', '.join(cell_class_names(cell))}")
```
The tests below define the complete semantic contract: public and
private exported symbols, multicells, output-bearing tests, examples,
documentation, titles, metadata refresh, and the only allowed mixed role
(`test_code` plus `example_code`).
A semantic cell gets explicit source-and-output roles. Imports
distinguish `exported_import_cell` from `internal_import_cell`; a
visible test is the only allowed `test_code` + `example_code`
combination. `multicell` is reserved for a cell with multiple
differently named top-level functions, public or private. Every other
mixed role is a writing warning.
Metadata stamping is separate from classification: it records the
generated Python artifact and caches the semantic labels that downstream
tools can reuse.
Python API helpers return values directly and raise `ValueError` for
invalid inputs; notebook execution stays independent of process
arguments.
|