Episode 3: Is the Pipeline Keeping Up?
Ask a running pipeline how close it is to its deadline with get_load(), and read headroom, duty, the busiest node and the worst cycle for what they mean.
A realtime pipeline has a deadline. Every frame must be processed before the next one arrives, and if it is not, the backlog grows until the pipeline is stopped for you. By then it is too late to ask what happened. Pipeline.get_load() answers the question in advance.
Headroom of 1.0 means idle and 0.0 means there is nothing left; duty is the fraction of wall time spent inside nodes; busiest names the node spending the most; and short of headroom lists every node over Pipeline.LOAD_WARNING_DUTY, 0.70, which is the verdict rather than the number: a duty of 0.72 reads as fine until you know what fine was. Per node there is its duty and its worst single cycle in milliseconds.
8def report(pipeline, title):
9 """Print one load reading."""
10 load = pipeline.get_load()
11 print()
12 print("---", title)
13 print(" headroom : %.3f (1.0 = idle, 0.0 = saturated)"
14 % load["headroom"])
15 print(" duty : %.3f" % load["duty"])
16 print(" busiest : %s" % load["busiest"])
17 # The verdict, not just the number: a node listed here spends more
18 # than LOAD_WARNING_DUTY of the budget on its own step()
19 short = load["short_of_headroom"]
20 print(" short of headroom : %s" % (", ".join(short) if short else "-"))
21 for row in sorted(load["nodes"], key=lambda r: -(r["duty"] or 0)):
22 print(" %-24s duty=%.4f worst=%.2f ms"
23 % (row["name"], row["duty"], row["worst_cycle_ms"]))
One source, one filter, one sink, read after a second.
28 # A modest pipeline: one source, one filter, one sink
29 p = gp.Pipeline()
30 source = gp.Generator(sampling_rate=250, channel_count=8)
31 band = gp.Bandpass(f_lo=8, f_hi=12)
32 p.connect(source, band)
33 p.connect(band, gp.Collector())
34 p.start()
35 time.sleep(1.0)
36 report(p, "one filter")
37 p.stop()
38 p.close()
256 channels at 1 kHz through twelve filters. How far the headroom falls depends on the machine, so read which node is busiest and the worst single cycle rather than the absolute number.
40 # A much heavier one. How far the headroom falls depends on the
41 # machine, so read which node is busiest and the worst single cycle
42 # rather than the absolute number
43 p = gp.Pipeline()
44 source = gp.Generator(sampling_rate=1000, channel_count=256)
45 stage = source
46 for _ in range(12):
47 nxt = gp.Bandpass(f_lo=1, f_hi=40)
48 p.connect(stage, nxt)
49 stage = nxt
50 p.connect(stage, gp.Collector())
51 p.start()
52 time.sleep(1.0)
53 report(p, "256 channels at 1 kHz through twelve filters")
54 p.stop()
55 p.close()
Run the script:
--- one filter
headroom : ... (1.0 = idle, 0.0 = saturated)
duty : ...
busiest : ...
short of headroom : -
... duty=... worst=... ms
--- 256 channels at 1 kHz through twelve filters
headroom : ...
...
Two things before you read the numbers. Each node’s figure is its own time, with everything it feeds excluded, so the busiest node is where the work is rather than the one furthest downstream. And a clocked source often ranks near the top, which is not a defect: it wakes once per sample while everything after it wakes once per frame, at 1000 Hz with frames of 16 that is 1000 cycles a second against 62.5, so compare a node against its own history, not against the source.
worst_cycle_ms. One frame at 250 Hz with frames of 4 is 16 ms, so a worst cycle near that is already late even when the average looks comfortable. The warning threshold sits at 0.70 rather than at 1.0 on purpose: a pipeline that is exactly keeping up has no margin left for the next scope repaint or garbage collection, and the warning worth having is the one that arrives before frames are late.
In this episode, you learned how to:
read one load reading: headroom, duty, the busiest node, the verdict
find where the work is, and why the source ranks high
judge a deadline by the worst cycle, not the average
Proceed to Episode 4, where a pipeline that was keeping up fails on one frame, and you find out what it left behind.
Nodes in this episode: Generator, Bandpass, Collector
File s6e3_is_the_pipeline_keeping_up.py – View file on GitHub
1"""Season 6, Episode 3: how close a pipeline is to its deadline."""
2
3import time
4
5import gpype as gp
6
7
8def report(pipeline, title):
9 """Print one load reading."""
10 load = pipeline.get_load()
11 print()
12 print("---", title)
13 print(" headroom : %.3f (1.0 = idle, 0.0 = saturated)"
14 % load["headroom"])
15 print(" duty : %.3f" % load["duty"])
16 print(" busiest : %s" % load["busiest"])
17 # The verdict, not just the number: a node listed here spends more
18 # than LOAD_WARNING_DUTY of the budget on its own step()
19 short = load["short_of_headroom"]
20 print(" short of headroom : %s" % (", ".join(short) if short else "-"))
21 for row in sorted(load["nodes"], key=lambda r: -(r["duty"] or 0)):
22 print(" %-24s duty=%.4f worst=%.2f ms"
23 % (row["name"], row["duty"], row["worst_cycle_ms"]))
24
25
26if __name__ == "__main__":
27
28 # A modest pipeline: one source, one filter, one sink
29 p = gp.Pipeline()
30 source = gp.Generator(sampling_rate=250, channel_count=8)
31 band = gp.Bandpass(f_lo=8, f_hi=12)
32 p.connect(source, band)
33 p.connect(band, gp.Collector())
34 p.start()
35 time.sleep(1.0)
36 report(p, "one filter")
37 p.stop()
38 p.close()
39
40 # A much heavier one. How far the headroom falls depends on the
41 # machine, so read which node is busiest and the worst single cycle
42 # rather than the absolute number
43 p = gp.Pipeline()
44 source = gp.Generator(sampling_rate=1000, channel_count=256)
45 stage = source
46 for _ in range(12):
47 nxt = gp.Bandpass(f_lo=1, f_hi=40)
48 p.connect(stage, nxt)
49 stage = nxt
50 p.connect(stage, gp.Collector())
51 p.start()
52 time.sleep(1.0)
53 report(p, "256 channels at 1 kHz through twelve filters")
54 p.stop()
55 p.close()