Apple has recently obtained a technical patent for the Apple Pencil, as revealed by a recent entry in the database of the United States Patent and Trademark Office (USPTO). This enhancement is not only applicable to the iPhone but extends its functionality into the arenas of augmented reality (AR), virtual reality (VR), and mixed reality (MR) applications.

According to the patent details, the latest iteration of the Apple Pencil is outfitted with an optical sensor which enables interaction on surfaces that lack touch capabilities.

The optical sensor array includes a light flow sensor, a laser speckle flow sensor, among others, that calculate the motion of the input device by detecting spatial-temporal changes in brightness, such as alterations in brightness between frames.

The laser speckle flow sensor is capable of detecting speckle patterns produced by coherent illumination in the imaging setup, to estimate the motion of the input device.

Optical sensors are designed to detect various characteristics of the input device including its position, orientation, and/or motion.

In certain examples, the laser speckle flow sensor might feature an emitter (for example, a Vertical Cavity Surface Emitting Laser (VCSEL)) to project light onto a surface, and a detector (like an image sensor) to capture images of the environment to detect light reflected or shone from the input device onto the surface.

Additionally, information from one or multiple auxiliary sensors could be gathered, which might include measurements such as the force exerted by the input device, force at the pen tip, inertial measurements, magnetic measurements, and/or computer vision measurements.

Below, Apple’s patent image 1A displays an example of a mobile phone (iPhone) featuring a touch screen #124, where inputs can be received from an input device, such as an active stylus, through both touch-sensitive surfaces (like the touch screen 124) and non-touch-sensitive surfaces. The image provided by IT之家 is as follows:

In certain scenarios, the use of an input device equipped with an optical sensor system could be employed to provide inputs for augmented reality, virtual reality, or mixed reality applications.