Episode 6: A Function as a Node

Run a plain numpy function over a recording as a node, with Apply or the @gp.node decorator, and see why a realtime pipeline refuses it.

A transform that is one line of numpy needs no node class. gp.Apply(fn) is the node, and @gp.node turns the function into a factory for one while leaving the function callable on its own, so it can be tested without a pipeline. Both are batch only.

A recording, and a function

Two seconds of a 10 Hz sine, amplitude 20, written to the temporary folder; and full-wave rectification, once as a plain function and once decorated.

24def write_recording(path: str = RECORDING) -> None:
25    """Write two seconds of a 10 Hz sine, amplitude 20.
26
27    Args:
28        path: File to write.
29    """
30    times = np.arange(SAMPLES) / RATE
31    signal = 20.0 * np.sin(2 * np.pi * 10.0 * times)
32    with open(path, "w", encoding="utf-8", newline="") as handle:
33        handle.write("Time, Ch01\n")
34        for time, value in zip(times, signal):
35            handle.write(f"{time:g}, {value:.10f}\n")
36
37
38def rectify(block: np.ndarray) -> np.ndarray:
39    """Full-wave rectification: the magnitude of every sample."""
40    return np.abs(block)
41
42
43@gp.node
44def rectified(block: np.ndarray) -> np.ndarray:
45    """The same function, decorated: rectified() builds an Apply."""
46    return np.abs(block)
Run it through, two ways

Apply takes any callable: one array of (samples, channels) in, one of the same shape and type out. rectified() is an Apply named after the function.

49def run(node) -> gp.Result:
50    """Read the recording through *node* in one batch run.
51
52    Args:
53        node: The node to put between the reader and the Collector.
54
55    Returns:
56        What the Collector kept.
57    """
58    with gp.Pipeline() as p:
59        reader = gp.CsvReader(file_name=RECORDING, mode="batch")
60        collector = gp.Collector()
61        p.connect(reader, node)
62        p.connect(node, collector)
63        return p.run()
64
65
66if __name__ == "__main__":
67
68    write_recording()
69
70    # 1. Apply takes any callable: one array of (samples, channels) in,
71    #    one of the same shape and dtype out.
72    explicit = run(gp.Apply(rectify, name="rectify"))
73
74    # 2. The decorated form. rectified() is an Apply named after the
75    #    function; rectified.function is the function itself.
76    decorated = run(rectified())
77
78    print(f"Apply(rectify)   : {explicit.data.shape}, "
79          f"min {explicit.data.min():.2f}, max {explicit.data.max():.2f}")
80    print(f"@gp.node         : {decorated.data.shape}, "
81          f"identical: {np.array_equal(explicit.data, decorated.data)}")
Still a function

rectified.function is the function itself, testable with no pipeline anywhere near it.

83    # 3. The function stays testable with no pipeline anywhere near it.
84    probe = np.array([[-3.0], [0.0], [2.0]], dtype=np.float32)
85    print(f"rectified.function({probe[:, 0].tolist()}) = "
86          f"{rectified.function(probe)[:, 0].tolist()}")
Against the clock, refused

A function has no bound on how long it takes, and a realtime frame has a deadline, so a realtime pipeline refuses an Apply at start(), naming the node.

 88    # 4. Against the clock, the same node is refused. A function has no
 89    #    bound on how long it takes, and a realtime frame has a deadline.
 90    print()
 91    print("Against the clock, Apply is refused:")
 92    with gp.Pipeline() as p:
 93        source = gp.Generator(sampling_rate=RATE, channel_count=1)
 94        p.connect(source, gp.Apply(rectify, name="rectify"))
 95        try:
 96            p.start()
 97        except Exception as refusal:
 98            print(f"  {type(refusal).__name__}: {refusal}")
 99        else:
100            p.stop()
101            print("  ...which did not happen; the node was accepted.")
Result

Run the script:

Apply(rectify)   : (500, 1), min ..., max ...
@gp.node         : (500, 1), identical: True
rectified.function([-3.0, 0.0, 2.0]) = [3.0, 0.0, 2.0]

Against the clock, Apply is refused:
  ValueError: ...

Five hundred samples are two seconds at 250 Hz; the minimum is zero, where the sine crosses it, and the maximum just under its amplitude, since no sample falls exactly on a peak.

Summary

In this episode, you learned how to:

  • turn a plain function into a batch node with Apply or @gp.node

  • keep the function testable on its own

  • expect a realtime pipeline to refuse it, and why

That closes Season 7. Proceed to Season 8, where one script runs as two processes and a pipeline travels as a serialized pipeline.

Nodes in this episode: Apply, CsvReader, Collector, Generator

File s7e6_a_function_as_a_node.py – View file on GitHub

  1"""Season 7, Episode 6: a plain function as a batch node, two ways.
  2
  3A transform that is one line of numpy needs no node class. gp.Apply(fn)
  4is the node, and @gp.node turns the function into a factory for one
  5while leaving it callable on its own. Both are batch only: a realtime
  6pipeline refuses them at start(), naming the node.
  7"""
  8
  9import os
 10import tempfile
 11
 12import numpy as np
 13
 14import gpype as gp
 15
 16#: The recording this example writes and then reads back.
 17RECORDING = os.path.join(tempfile.gettempdir(),
 18                         "s7e6_a_function_as_a_node.csv")
 19
 20RATE = 250.0
 21SAMPLES = 500
 22
 23
 24def write_recording(path: str = RECORDING) -> None:
 25    """Write two seconds of a 10 Hz sine, amplitude 20.
 26
 27    Args:
 28        path: File to write.
 29    """
 30    times = np.arange(SAMPLES) / RATE
 31    signal = 20.0 * np.sin(2 * np.pi * 10.0 * times)
 32    with open(path, "w", encoding="utf-8", newline="") as handle:
 33        handle.write("Time, Ch01\n")
 34        for time, value in zip(times, signal):
 35            handle.write(f"{time:g}, {value:.10f}\n")
 36
 37
 38def rectify(block: np.ndarray) -> np.ndarray:
 39    """Full-wave rectification: the magnitude of every sample."""
 40    return np.abs(block)
 41
 42
 43@gp.node
 44def rectified(block: np.ndarray) -> np.ndarray:
 45    """The same function, decorated: rectified() builds an Apply."""
 46    return np.abs(block)
 47
 48
 49def run(node) -> gp.Result:
 50    """Read the recording through *node* in one batch run.
 51
 52    Args:
 53        node: The node to put between the reader and the Collector.
 54
 55    Returns:
 56        What the Collector kept.
 57    """
 58    with gp.Pipeline() as p:
 59        reader = gp.CsvReader(file_name=RECORDING, mode="batch")
 60        collector = gp.Collector()
 61        p.connect(reader, node)
 62        p.connect(node, collector)
 63        return p.run()
 64
 65
 66if __name__ == "__main__":
 67
 68    write_recording()
 69
 70    # 1. Apply takes any callable: one array of (samples, channels) in,
 71    #    one of the same shape and dtype out.
 72    explicit = run(gp.Apply(rectify, name="rectify"))
 73
 74    # 2. The decorated form. rectified() is an Apply named after the
 75    #    function; rectified.function is the function itself.
 76    decorated = run(rectified())
 77
 78    print(f"Apply(rectify)   : {explicit.data.shape}, "
 79          f"min {explicit.data.min():.2f}, max {explicit.data.max():.2f}")
 80    print(f"@gp.node         : {decorated.data.shape}, "
 81          f"identical: {np.array_equal(explicit.data, decorated.data)}")
 82
 83    # 3. The function stays testable with no pipeline anywhere near it.
 84    probe = np.array([[-3.0], [0.0], [2.0]], dtype=np.float32)
 85    print(f"rectified.function({probe[:, 0].tolist()}) = "
 86          f"{rectified.function(probe)[:, 0].tolist()}")
 87
 88    # 4. Against the clock, the same node is refused. A function has no
 89    #    bound on how long it takes, and a realtime frame has a deadline.
 90    print()
 91    print("Against the clock, Apply is refused:")
 92    with gp.Pipeline() as p:
 93        source = gp.Generator(sampling_rate=RATE, channel_count=1)
 94        p.connect(source, gp.Apply(rectify, name="rectify"))
 95        try:
 96            p.start()
 97        except Exception as refusal:
 98            print(f"  {type(refusal).__name__}: {refusal}")
 99        else:
100            p.stop()
101            print("  ...which did not happen; the node was accepted.")