Episode 6: Capturing Keystrokes

Turn key presses into an event channel that travels beside the signal, so a mark lands on the sample that was being recorded at that moment.

A key press is the simplest trigger there is: someone marks the moment something happened. Keyboard turns every press into a number, and a Router puts that number’s channel beside the signal’s, so the mark stays with the data from then on.

Before you start
  • Install the devices extra: pip install "gpype[devices]". The keyboard listener comes with it.

  • On macOS, the application that runs Python has to be allowed to read the keyboard, under System Settings > Privacy & Security > Input Monitoring. Quit and reopen it after allowing: the permission is checked when the process starts. The keyboard library may still log that the process “is not trusted”; that warning is about Accessibility, which key capture does not need.

  • On Linux, the listener needs a graphical session. On a machine with no display the keyboard library cannot even be imported, which is why Keyboard only loads it when the pipeline starts.

A signal and a keyboard

A Generator stands in for the amplifier. The keyboard is a second source; the router concatenates the generator’s eight channels with the keyboard’s one, so the key codes land on channel 8, counting from 0.

10    source = gp.Generator(sampling_rate=250, channel_count=8,
11                          signal_frequency=10, signal_amplitude=10,
12                          noise_amplitude=10)
13
14    # Every press becomes one sample carrying the key's code
15    keyboard = gp.Keyboard()
16
17    # The generator's 8 channels, then the keyboard's one: the key codes
18    # land on channel 8, counting from 0
19    router = gp.Router(input_channels=[gp.Router.ALL, gp.Router.ALL])
Markers for the keys

A marker per key, pinned to the code channel; the values are the key codes. The channel itself is hidden from the scope, because a raw code of 40 would run far past a lane scaled to 30. It is declared a trigger channel, so a filter placed after the router leaves it alone.

21    # A marker per key, pinned to that channel; the values are key codes
22    mk = gp.TimeSeriesScope.Markers
23    markers = [mk(color="r", label="up", channel=8, value=38),
24               mk(color="g", label="right", channel=8, value=39),
25               mk(color="c", label="down", channel=8, value=40),
26               mk(color="y", label="left", channel=8, value=37)]
27
28    # The code channel itself is hidden: a raw 40 would overrun the lane
29    scope = gp.TimeSeriesScope(amplitude_limit=30, time_window=10,
30                               markers=markers, hidden_channels=[8])
Result

Run the script and press the arrow keys. A coloured marker appears on the scope at the moment of each press.

S3E6 example screenshot

Figure 31: The generator’s channels; a key press draws a marker across them.

Each press is one sample wide: the code appears on the sample the key went down on, and the channel reads 0 on every other sample, including while the key is still held. A release emits 0 as well, which looks exactly like no key at all, so the stream records when a key was pressed, not how long it stayed down.

What a press looks like in the data

The codes are Windows virtual-key codes, on every platform: the up arrow is 38 on Windows, macOS and Linux alike, so a marker written for 38 fires everywhere and a recording means the same key wherever it is read. Letters take the code of the key, not the character: a and A are both 65. Punctuation is numbered by where a US layout puts it; a key Windows has no code for, such as a media key, emits -1:

up arrow        38
right arrow     39
a               65
media eject     -1

The signal sets the timing, not the keyboard. Keyboard is a sparse source: it only emits when a key is pressed, so it cannot be the clock of a run. The continuous source beside it, the generator here and an amplifier in a real experiment, is, and each press is placed on that source’s grid. The listener hears the whole desktop: every key typed into any window is captured, your editor included.

Summary

In this episode, you learned how to:

  • add key presses as an event channel with Keyboard

  • place markers on a scope by key code

  • hide a code channel and keep it a trigger

That closes Season 3. Proceed to Season 4, where data leaves the pipeline: files, LSL and UDP.

Nodes in this episode: Generator, Keyboard, Router, TimeSeriesScope

File s3e6_keyboard.py – View file on GitHub

 1"""Season 3, Episode 6: key presses as an event channel beside the signal."""
 2
 3import gpype as gp
 4
 5if __name__ == "__main__":
 6
 7    app = gp.MainApp()
 8    p = gp.Pipeline()
 9
10    source = gp.Generator(sampling_rate=250, channel_count=8,
11                          signal_frequency=10, signal_amplitude=10,
12                          noise_amplitude=10)
13
14    # Every press becomes one sample carrying the key's code
15    keyboard = gp.Keyboard()
16
17    # The generator's 8 channels, then the keyboard's one: the key codes
18    # land on channel 8, counting from 0
19    router = gp.Router(input_channels=[gp.Router.ALL, gp.Router.ALL])
20
21    # A marker per key, pinned to that channel; the values are key codes
22    mk = gp.TimeSeriesScope.Markers
23    markers = [mk(color="r", label="up", channel=8, value=38),
24               mk(color="g", label="right", channel=8, value=39),
25               mk(color="c", label="down", channel=8, value=40),
26               mk(color="y", label="left", channel=8, value=37)]
27
28    # The code channel itself is hidden: a raw 40 would overrun the lane
29    scope = gp.TimeSeriesScope(amplitude_limit=30, time_window=10,
30                               markers=markers, hidden_channels=[8])
31
32    p.connect(source, router["in1"])
33    p.connect(keyboard, router["in2"])
34    p.connect(router, scope)
35    app.add_widget(scope)
36
37    p.start()
38    app.run()
39    p.stop()
40    p.close()