JP2016162331A - Information processing device - Google Patents

Abstract

Kind Code: A1 Various operations can be performed using image reading means for reading a user's finger as a two-dimensional image.
An information processing apparatus includes a display unit that displays information on a display surface, an image reading unit that is provided on the same surface as the display surface and that reads a user's finger as a two-dimensional image, and the image reading unit. It has a movement detecting means for detecting the moving direction of the finger using the read image, and a processing means for performing processing according to the moving direction detected by the movement detecting means.
[Selection] Figure 1

Description

  The present invention relates to an information processing apparatus having reading means for reading an image of a user's finger on the same surface as a display surface.

  A technique for performing fingerprint authentication for user authentication in an information processing apparatus such as a smartphone or a tablet terminal is known. Patent Documents 1 and 2 describe techniques for detecting lateral movement of a finger using a one-dimensional fingerprint sensor.

JP 2007-189395 A International Publication No. WO2009 / 147755

  In the techniques described in Patent Documents 1 and 2, it is only possible to detect movement in one direction of a finger, and various operations cannot be performed using a fingerprint sensor.

  In contrast, the present invention provides a technique that enables various operations using an image reading unit that reads a user's finger as a two-dimensional image.

The present invention provides a display means for displaying information on a display surface, an image reading means provided on the same surface as the display surface, for reading a user's finger as a two-dimensional image, and an image read by the image reading means. The information processing apparatus includes: a movement detection unit that detects a movement direction of the finger; and a processing unit that performs a process according to the movement direction detected by the movement detection unit.
According to this information processing apparatus, various operations can be performed using the image reading means.

In the processing means, different processing may be assigned depending on whether the moving direction is the first direction or the second direction.
According to this information processing apparatus, it is possible to perform different processing when the moving direction of the finger on the image reading unit is the first direction and the second direction.

The information processing apparatus includes: an authentication unit that performs fingerprint authentication of a user using an image of the user's finger read by the image reading unit; and a moving speed of the finger indicated by the image read by the image reading unit And a switching unit that switches the output destination of the image to one of the movement detection unit and the authentication unit.
According to this information processing apparatus, it is possible to automatically switch between fingerprint authentication and processing according to the direction of finger movement.

The information processing apparatus includes direction detection means for detecting the direction of the own apparatus, and setting means for changing the association between the movement direction and the processing in the processing means in accordance with the direction detected by the direction detection means. May be.
According to this information processing apparatus, it is possible to dynamically change the correspondence between the finger moving direction on the image reading unit and the processing to be executed according to the orientation of the own device.

The image reading unit intermittently reads an image at a first time interval, and the information processing apparatus includes a determination unit that determines whether the image read by the image reading unit includes the user's finger, When the determination unit determines that the user's finger is included in the image, the image reading unit may read the image at a second time interval shorter than the first time interval.
According to this information processing apparatus, it is possible to reduce the processing load as compared with the case where images are always read at the second time interval.

The movement detecting unit detects that the user's finger swipes on the reading unit when the finger is moved from the state included in the image to the outside of the image in the plurality of images. And when the said swipe is detected, the said process means may switch the information displayed on the said display means according to the direction of the said swipe.
According to this information processing apparatus, information displayed on the display unit can be switched by swiping the image reading unit with a finger.

The figure which shows the function structure of the display system 1 which concerns on one Embodiment. The figure which illustrates the hardware constitutions of the portable terminal. The figure which illustrates the external appearance of the portable terminal. The figure which illustrates the hardware constitutions of the display apparatus 20. 7 is a flowchart showing the operation of the mobile terminal 10. 3 is a diagram illustrating an image stored in a RAM 102. FIG.

1. Configuration FIG. 1 is a diagram illustrating a functional configuration of a display system 1 according to an embodiment. The display system 1 includes a mobile terminal 10 and a display device 20. The display device 20 is a device that displays an image. The mobile terminal 10 communicates with the display device 20. In this example, the mobile terminal 10 functions as a device that inputs an instruction to the display device 20, that is, a so-called remote controller.

  The portable terminal 10 is an information processing apparatus having a display unit 11, an image reading unit 12, a movement detection unit 13, a processing unit 14, an authentication unit 15, a switching unit 16, a direction detection unit 17, a setting unit 18, and a determination unit 19. is there.

  The display means 11 displays information on the display surface. The image reading unit 12 is provided on the same surface as the display surface, and reads the user's finger as a two-dimensional image. The movement detection unit 13 detects the moving direction of the finger using the image read by the image reading unit 12. The processing unit 14 performs processing according to the moving direction detected by the movement detecting unit 13. In the processing means 14, different processing is assigned to each of the movement direction in the first direction and the second direction. In this example, the processing executed by the processing unit 14 includes transmission of an instruction to the display device 20.

  The authentication unit 15 performs fingerprint authentication of the user using the image of the user's finger read by the image reading unit 12. The switching unit 16 switches the output destination of the image to either the movement detection unit 13 or the authentication unit 15 according to the moving speed of the finger indicated by the image read by the image reading unit 12.

  The direction detection means 17 detects the direction of its own device. The direction here is, for example, a direction with respect to the direction of gravity (vertically downward). The setting unit 18 changes the association between the movement direction and the processing in the processing unit 14 in accordance with the direction detected by the direction detection unit 17.

  In this example, the image reading unit 12 reads images intermittently at the first time interval. The determination unit 19 determines whether the image read by the image reading unit 12 includes a user's finger. When it is determined that the user's finger is included in the image, the image reading unit 12 intermittently reads the image at a second time interval shorter than the first time interval.

  The display device 20 includes a receiving unit 21 and a display unit 22. The receiving unit 21 receives an instruction from the mobile terminal 10. The display means 22 displays an image according to the received instruction.

  FIG. 2 is a diagram illustrating a hardware configuration of the mobile terminal 10. The mobile terminal 10 is, for example, a smartphone or a tablet terminal. The mobile terminal 10 includes a CPU 100, a ROM 101, a RAM 102, a storage 103, a communication IF 104, a touch screen 105, a home button 106, and an acceleration sensor 107.

  The CPU 100 is a device that performs various calculations according to a program. The ROM 101 is a nonvolatile storage device that stores various data and programs. The RAM 102 is a volatile storage device that functions as a work area when the CPU 100 executes a program. The storage 103 is a non-volatile storage device that stores application programs and data used in the application programs. In this example, the storage 103 stores an operating system (OS) for operating the mobile terminal 10 and a program for causing the mobile terminal 10 to function as a remote controller of the display device 20.

  The communication IF 104 is an interface for communicating with other devices such as the display device 20. The communication IF 104 communicates with other devices in accordance with a predetermined communication standard such as WiFi (registered trademark) or Bluetooth (registered trademark).

  The touch screen 105 includes a display device and a touch sensor (both not shown). The display device is a device that displays an image on a display surface, such as a liquid crystal display or an organic EL display. The touch sensor is a device that is provided on the display surface and detects a position touched by an indicator (for example, a user's finger), for example, a capacitive sensor.

  The home button 106 is a button for the user to input an instruction to the mobile terminal 10, and is provided on the same surface as the display surface of the touch screen 105. The home button 106 includes a press sensor 1061 and a fingerprint sensor 1062. The press sensor 1061 is a sensor that detects that the button is pressed by the user's finger. The fingerprint sensor 1062 is a sensor that reads a fingerprint from a user's finger as a two-dimensional image.

  The acceleration sensor 107 is a sensor for measuring the tilt of the mobile terminal 10.

  The touch screen 105 is an example of the display unit 11. The fingerprint sensor 1062 controlled by the CPU 100 is an example of the image reading unit 12. The CPU 100 is an example of the movement detection unit 13, the processing unit 14, the authentication unit 15, the switching unit 16, the setting unit 18, and the determination unit 19. The acceleration sensor 107 is an example of the direction detection unit 17.

  FIG. 3 is a diagram illustrating an appearance of the mobile terminal 10. The mobile terminal 10 has a touch screen 105 on one surface of the housing and a home button 106 at the lower part thereof. The user can operate the home button 106 while viewing the image displayed on the touch screen 105.

  FIG. 4 is a diagram illustrating a hardware configuration of the display device 20. In this example, the display device 20 is a device that projects an image, that is, a projector. The display device 20 includes a central processing unit (CPU) 200, a read only memory (ROM) 201, a random access memory (RAM) 202, a storage unit 203, an IF unit 204, an image processing circuit 205, and a projection unit 206.

  The CPU 100 is a control device that controls each unit of the display device 20. The ROM 101 is a non-volatile storage device that stores various programs and data. The RAM 102 is a volatile storage device that stores data, and functions as a work area when the CPU 100 executes processing. The storage unit 203 is a nonvolatile storage device that stores data, programs, and the like, and includes, for example, a flash ROM, an SSD (Solid State Drive), or an HDD (Hard Disk Drive). Note that the storage unit 203 may be included in the ROM 101. Alternatively, the storage unit 203 may be a storage device on a network.

  The IF unit 204 is an interface that mediates exchange of signals or data with an external device (for example, the display device 20). The IF unit 204 is a terminal (for example, a VGA terminal, a USB terminal, a wired LAN interface, an S terminal, an RCA terminal, an HDMI (High-Definition Multimedia Interface: registered trademark) terminal) for exchanging signals or data with an external device. Microphone terminal) and a wireless LAN interface. These terminals may include a video output terminal in addition to the video input terminal.

  The image processing circuit 205 performs predetermined image processing (eg, size change, keystone correction, etc.) on the input video signal (hereinafter referred to as “input video signal”).

  The projection unit 206 projects an image on a projection surface such as a screen or a wall surface in accordance with a video signal subjected to image processing. The projection unit 206 includes a light source, an optical modulator, and an optical system (all not shown). The light source includes a lamp such as a high-pressure mercury lamp, a halogen lamp, or a metal halide lamp, or a solid light source such as an LED (Light Emitting Diode) or a laser diode, and a drive circuit thereof. The optical modulator is a device that modulates light emitted from a light source according to a video signal, and includes, for example, a liquid crystal panel or a DMD (Digital Mirror Device) and a drive circuit thereof. The liquid crystal panel may be either a transmission type or a reflection type. The optical system includes an element that projects light modulated by the light modulator onto a screen, and includes, for example, a mirror, a lens, and a prism. A light source and a light modulator may be provided for each color component.

  The IF unit 204 is an example of the receiving unit 21. The projection unit 206 is an example of the display unit 22.

2. Operation FIG. 5 is a flowchart showing the operation of the mobile terminal 10. The flow in FIG. 5 is started, for example, when the power of the mobile terminal 10 is turned on or when a specific program is started. The flow of FIG. 5 is implemented by the OS of the mobile terminal 10 or an application program.

  Although not described in the flow of FIG. 5, the fingerprint sensor 1062 is originally provided for use in fingerprint authentication. When performing fingerprint authentication, the CPU 100 authenticates whether the fingerprint image read by the fingerprint sensor 1062 is a correct fingerprint (that is, whether it is the person himself) by pattern matching with a fingerprint image registered in advance. In the fingerprint authentication, for example, the mobile terminal 10 operates from a sleep mode (an operation mode that reduces power consumption such as turning off the touch screen 105) to a normal mode (a mode that performs normal operation such as displaying an image on the touch screen 105). This is done when returning to, or when authentication is required to access a specific site.

  The home button 106 is also used as a button for inputting an instruction to the mobile terminal 10. That is, when the user presses the home button 106, information corresponding to “confirmation” or “OK” of the operation is input to the CPU 100.

  In step S100, the CPU 100 activates the fingerprint sensor 1062. While active, the fingerprint sensor 1062 outputs an image signal indicating the read image. The CPU 100 intermittently writes data indicating the image indicated by the image signal in the RAM 102 at the time interval T1. The RAM 102 stores a transition of a predetermined number of images, that is, images in the latest finite period.

  In step S <b> 110, the CPU 100 determines whether the user's finger is reflected in the image read by the fingerprint sensor 1062. When it is determined that the finger is not captured (S110: NO), the CPU 100 waits until it is determined that the finger is captured. When it is determined that the finger is captured (S110: YES), the CPU 100 shifts the process to step S120.

  In step S120, the CPU 100 changes the time interval for writing data indicating the image indicated by the image signal in the RAM 102 to T2. Here, the time interval T2 is shorter than the time interval T1 (T2 <T1). That is, when the mobile terminal 10 reads an image from the fingerprint sensor 1062 at a certain time interval T1, if it is determined that a finger is included in the read image, the mobile terminal 10 reads the image at a shorter time interval T2. In this sense, the time intervals T1 and T2 are hereinafter referred to as image reading intervals.

  In step S130, the CPU 100 determines whether a predetermined operation has been performed. The predetermined operation is, for example, a swipe. Swipe refers to moving the finger from the state where the home button 106 is covered with the finger to the outside of the home button 106. Specifically, the CPU 100 determines whether or not the swipe has been performed using a plurality of images stored in the RAM 102.

  FIG. 6 is a diagram illustrating an image stored in the RAM 102. In this example, four images are stored in the RAM 102. In this example, the image at the left end has the oldest time, and the time becomes newer as it goes to the right. The CPU 100 calculates the ratio of the area occupied by the finger in these images. Since these images are read by the fingerprint sensor 1062, the resolution is high enough to identify the fingerprint. However, fingerprints (patterns) are not used for swipe determination. The CPU 100 specifies a region occupied by a finger (hatched portion in the figure) in the image, and further calculates a ratio of the image to the area of the image. In this example, in order from the leftmost image, the ratio of fingers is 95%, 60%, 30%, and 0%. The CPU 100 decreases the transition of the area ratio from a predetermined condition (for example, the area ratio is equal to or higher than the first threshold value (for example, 80%), and finally decreases to the threshold value (for example, 10%) or lower. It is determined that a swipe has been performed.

  Further, the CPU 100 identifies the moving direction of the finger from the transition of the feature points (for example, the center of gravity) of the area occupied by the finger in the image. In the example of FIG. 6, since the center of gravity of the region occupied by the finger has moved to the right, the CPU 100 determines that the right swipe has been performed. The algorithm for determining the presence and direction of the swipe described here is merely an example, and any algorithm that uses a plurality of images read by the fingerprint sensor 1062 may be used. May be specified.

  Refer to FIG. 5 again. If it is determined that a predetermined operation has been performed (S130: YES), the CPU 100 shifts the processing to step S160. When it is not determined that the predetermined operation has been performed (S130: NO), the CPU 100 shifts the processing to step S140.

  In step S140, the CPU 100 determines whether a condition for changing the image reading interval is satisfied. This condition is, for example, that (i) a threshold time has elapsed since the reading interval was changed from T1 to T2, and (ii) images that do not include a finger are continuously read for a threshold time number or more. It is said that at least one report was satisfied. When it is determined that the change condition is satisfied (S140: YES), the CPU 100 shifts the process to step S150. When it is determined that the change condition is not satisfied (S140: NO), the CPU 100 shifts the processing to step S110.

  In step S150, the CPU 100 changes the image reading interval to T1. When the image reading interval is changed, the CPU 100 shifts the processing to step S110.

  In step S160, the CPU 100 performs processing according to the swipe direction. In this example, a process is assigned for each swipe direction, and the CPU 100 transmits information indicating the assigned process to the display device 20. If the process according to a direction is performed, CPU100 will transfer a process to step S150. The display device 20 performs processing according to information transmitted from the mobile terminal 10.

3. Modifications The present invention is not limited to the above-described embodiments, and various modifications can be made. Hereinafter, some modifications will be described. Two or more of the following modifications may be used in combination.

3-1. Modification 1
The portable terminal 10 is not limited to that used as a remote controller for the display device 20. The mobile terminal 10 may be used alone independently of the display device 20. In this case, the mobile terminal 10 displays an image on the touch screen 105 by the function of the OS or application program. This image is, for example, one image selected from a plurality of images included in a specific folder. At this time, the image displayed on the touch screen 105 is switched according to the swipe on the home button 106 and its direction.

For example, the following processing is assigned according to the direction of swipe.
Left: Go back to the previous image Right: Go to the next image Up: Show the menu Down: Hide the menu

  If swipe detection is implemented in the OS, it is possible to set processing allocation for an operation (swipe direction) for each application program.

3-2. Modification 2
The use of the fingerprint sensor 1062 may be automatically switched from swipe detection to fingerprint authentication according to the detected finger movement speed. For example, in step S140, it is determined whether the moving speed of the finger is equal to or lower than the threshold value. If it is determined that the finger moving speed is equal to or lower than the threshold value, the use of the fingerprint sensor 1062 may be switched to fingerprint authentication. Alternatively, the use of the fingerprint sensor 1062 may be switched to swipe detection when the finger moving speed is faster than the threshold value in the fingerprint authentication mode.

3-3. Modification 3
The CPU 100 may change the assignment of the swipe direction and the process according to the orientation of the mobile terminal 10. Since the swipe direction is determined from a plurality of images read by the fingerprint sensor 1062, the “swipe direction” here is a direction determined based on the arrangement of the sensor elements of the fingerprint sensor 1062. On the other hand, in consideration of the direction of the mobile terminal 10, it may be determined in which direction the swipe is performed with respect to gravity, and processing may be performed according to the result.

3-4. Other Modifications Finger movements detected from a plurality of images read by the fingerprint sensor 1062 are not limited to those illustrated in the embodiment. For example, the CPU 100 may detect that the finger is slid in either direction while the finger is placed on the fingerprint sensor 1062.

  The hardware configuration of the mobile terminal 10 is not limited to that illustrated in FIG. The mobile terminal 10 may have any hardware configuration as long as the required function can be realized. Further, a part of the functional configuration of FIG. 1 may be omitted.

  The display device 20 is not limited to a projector, and may be a direct-view display device such as a liquid crystal display or an organic EL display. Moreover, the portable terminal 10 may be used as a remote controller other than the display device such as a printer.

DESCRIPTION OF SYMBOLS 1 ... Display system, 10 ... Portable terminal, 11 ... Display means, 12 ... Image reading means, 13 ... Movement detection means, 14 ... Processing means, 15 ... Authentication means, 16 ... Switching means, 17 ... Direction detection means, 18 ... Setting means, 19 ... determining means, 20 ... display device, 21 ... receiving means, 22 ... display means, 100 ... CPU, 101 ... ROM, 102 ... RAM, 103 ... storage, 104 ... communication IF, 105 ... touch screen, 106 ... Home button, 107 ... Acceleration sensor, 200 ... CPU, 201 ... ROM, 202 ... RAM, 203 ... Storage section, 204 ... IF section 15 section, 205 ... Image processing circuit, 206 ... Projection unit

Claims (6)

Display means for displaying information on the display surface;
An image reading means provided on the same surface as the display surface, for reading a user's finger as a two-dimensional image;
A movement detecting means for detecting a moving direction of the finger using an image read by the image reading means;
An information processing apparatus comprising: processing means for performing processing according to the movement direction detected by the movement detection means. The information processing apparatus according to claim 1, wherein different processing is assigned to the processing unit depending on whether the moving direction is the first direction or the second direction. Authentication means for performing fingerprint authentication of the user using an image of the user's finger read by the image reading means;
3. A switching unit that switches an output destination of the image to either the movement detection unit or the authentication unit in accordance with a moving speed of the finger indicated by the image read by the image reading unit. The information processing apparatus according to any one of the above. Orientation detection means for detecting the orientation of the device itself;
4. The information processing apparatus according to claim 1, further comprising: a setting unit that changes association between the moving direction and the process in the processing unit in accordance with the direction detected by the direction detection unit. 5. The image reading means intermittently reads an image at a first time interval,
Determining means for determining whether the image of the user is included in the image read by the image reading means;
The image reading unit reads an image at a second time interval shorter than the first time interval when the determination unit determines that the user's finger is included in the image. Information processing apparatus as described in any one of thru | or 4. The movement detecting unit detects that the user's finger swipes on the reading unit when the finger is moved from the state included in the image to the outside of the image in the plurality of images. And
The information according to any one of claims 1 to 5, wherein when the swipe is detected, the processing unit switches information displayed on the display unit in accordance with a direction of the swipe. Processing equipment.

Images