Analog Stick Decoding

The analog stick on the N64 standard controller uses a pair of optical encoding disks to determine its relative position. This works similar to linear encoders found in printers, except that instead of being a strip of tape/film, the N64 uses a rotating disk for each axis. Each disk has small holes along the circumference of the disk. These holes allow light to pass through to an optical sensor on the opposite side.

When the stick is moved, there are two signals per axis produced, which are slightly offset from each other. One of the two signals is used as the interrupt signal (XA and YA for the x-axis and y-axis respectively), and the other (XB and YB) is compared with the state of the first in order to determine which direction (positive or negative) that analog stick is moving for that axis.

If the stick is moving in the positive direction at a constant speed, the two signals will look similar to this:

If the stick is moving in the negative direction, they will look similar to this:

To determine the direction along a particular axis and how far the stick has moved, the microcontroller will listen for any changing edge of that axis' interrupt signal (e.g. XA for the x-axis). A changing edge means any time the signal goes from LOW to HIGH, or HIGH to LOW. When an edge change is detected, the state of XA is compared against XB. If XA == XB, then the stick has moved +1 units. If XA != XB, then the stick has moved -1 units.

Although not tested/quantified, it may be possible to extrapolate the speed at which the user is moving the stick. The signals produced vary in length between each edge change. The time between edge changes may be able to be used to indicate speed. While speed isn't needed by the console, this could still offer some additional data in a custom controller or interface.

While the analog stick reports relative movement whenever it moves, the N64 console requires the current position, not relative. In order to provide that, the microcontroller keeps track of these movements using an 8bit register for each axis. The register will be incremented for each positive movement, and decremented for each negative movement. The initial values for each register is 0x00. Just as the original CNT-NUS microchip assumes the stick starts in the center/neutral position, so does the microcontroller.

A user on the r/gifs subreddit posted this animation which may help visualize the inner workings of the analog stick. The analog stick has a 6-pin connector:

VCC is 3.3V which can be supplied by the microcontroller and in turn the N64 console.
GND is common ground.
XA and XB are the signal pins for the x-axis.
YA and YB are for the y-axis.