WO2012126426A2

WO2012126426A2 - Method and device for contact-free control by hand gesture

Info

Publication number: WO2012126426A2
Application number: PCT/CN2012/075798
Authority: WO
Inventors: 李英涛
Original assignee: 华为技术有限公司
Priority date: 2012-05-21
Filing date: 2012-05-21
Publication date: 2012-09-27
Also published as: EP2853989A1; WO2012126426A3; EP2631739A4; US8866781B2; EP2631739A2; CN103229127A; US20130307765A1; EP2631739B1

Abstract

The present invention is applicable in the technical field of human computer interaction, and provides a method and device for contact-free control by hand gestures. The method comprises: acquiring a hand gesture from a user; acquiring a control command corresponding to the hand gesture according to pre-stored mapping relationships between hand gestures and control commands; and executing the control command; wherein the hand gesture includes: drawing a circle by an index finger with the remaining four fingers clenched into a fist and with the circle being larger than the contour of the hand; or keeping the five stretched fingers together with the palm moving towards the device; or stretching the thumb, index finger and middle finger out in a natural way with the remaining two fingers kept together and the index finger moving freely; or stretching the thumb, index finger and middle finger out in a natural way with the remaining two fingers kept together and the thumb moving up and down; or stretching the thumb, index finger and middle finger out in a natural way with the remaining two fingers kept together and the middle finger moving up and down.The present invention effectively solves the problems in the prior art that there are too many hand gesture rules, making them difficult to learn and to memorize, and making the operation thereof complicated, thus leading to poor operation smoothness.

Description

Non-contact gesture control method and device

Technical field

The invention belongs to the technical field of human-computer interaction, and in particular relates to a contactless gesture control method and device.

Background technique

The gesture control device is one of the human-computer interaction technologies. Compared with the traditional graphical user interface, the gesture control does not require the user to hold a specific input device, and only needs to control the device or the device through a specific hand motion. Enter specific information.

The existing gesture control is mainly divided into two categories: one is based on the position information of the gesture, that is, the movement of the related elements on the display screen is mapped by the spatial movement of the finger; the other is the gesture information based on the gesture, that is, the formation by the human hand Various complex gestures, corresponding to different control commands to control the device.

A series of operations such as selecting, copying, pasting, moving, deleting, and switching of a specific display object (such as an icon, a box, a scroll bar, etc.) on a pair of display screens in the prior art define corresponding gestures. E.g A gesture to copy a display object: click on the object with one finger when selected; a gesture to paste a display object: Quickly double-click with a finger.

The prior art has the following drawbacks: the operation instructions for the display object are various, and the gesture rules that the user needs to memorize are too many to learn and memorize.

In the prior art, the basic operations of the mouse (clicking the left button, right clicking, cursor shift, etc.) are converted into gesture definitions. Different from the prior art, the prior art 2 does not define a specific operation of the display object, but implements functions of all specific operations by defining a mouse-like operation (for example, by selecting a specific operation in the right-click menu) achieve).

The prior art 2 has the following drawbacks: The gesture actions of the mouse-like operation are quite different. When the user performs a series of operations, it is necessary to frequently switch between various actions. For example, when the user needs to select a display object and then open the right-click menu, the user needs to make a fist and reach out. The movement of the thumb moves the orientation, and then the hand needs to be extended to extend the thumb and forefinger, and then the thumb is moved. This series of motion switching is complicated, which affects the convenience and smoothness of the operation.

technical problem

An object of the embodiments of the present invention is to provide a non-contact gesture control method, which solves the problem that the prior art has too many gesture rules, is difficult to learn and memorize, and has complicated operation and poor operation fluency.

Technical solution

The embodiment of the present invention is implemented as follows. A contactless gesture control method includes:

Get the gesture action of the user;

Obtaining a control instruction corresponding to the gesture action according to a mapping relationship between the pre-stored gesture action and the control instruction;

Executing the control instruction;

The gesture action includes: the index finger is circled, the remaining four fingers are clenched and the circle is larger than the contour of the hand; or the five fingers are close together, the palm moves in the direction of the device; or the thumb, the index finger and the middle finger are naturally extended, and the other two fingers are close together, the index finger is free Move; or the thumb, index finger, middle finger naturally extend, the other two fingers close together, the thumb moves up and down; or the thumb, index finger, middle finger naturally extend, the other two fingers close together, the middle finger moves up and down.

The embodiment of the invention further provides a contactless gesture control device, the device comprising:

a gesture action obtaining unit, configured to acquire a gesture action of the user;

a control instruction acquiring unit, configured to acquire, according to a mapping relationship between the pre-stored gesture action and the control instruction, a control instruction corresponding to the gesture action acquired by the gesture action acquiring unit;

a control instruction execution unit, configured to execute a control instruction acquired by the control instruction acquisition unit;

Beneficial effect

The beneficial effects obtained by the embodiment of the present invention compared with the prior art are as follows: First, the gesture rules are small, and the user is easy to learn and memorize. Compared with the prior art solution 1, the embodiment of the invention greatly simplifies the number of gesture rules, and the user can complete most operations like a mouse by simply remembering three basic gesture actions. Second, the operation is simple and smooth. Compared with the prior art solution 2, when the user implements three basic gestures, the basic hand posture is the same (extending the thumb, the index finger, the middle finger, and the other two fingers together), so that the user does not need to operate continuously. By changing the basic pose, you can smoothly switch between different gestures, and the ease of use and practicability are strong.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments or the description of the prior art will be briefly described below. It is obvious that the drawings in the following description are only the present invention. For some embodiments, other drawings may be obtained from those of ordinary skill in the art in light of the inventive workability.

1 is a flowchart of an implementation of a contactless gesture control method according to an embodiment of the present invention;

2a, 2b, 2c, 2d, 2e, 2f in FIG. 2 is an example of a gesture provided by an embodiment of the present invention;

FIG. 3 is a structural diagram of a non-contact gesture control apparatus according to an embodiment of the present invention.

Embodiments of the invention

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

In order to explain the technical solution described in the present invention, the following description will be made by way of specific embodiments.

FIG. 1 is a flowchart showing an implementation process of a contactless gesture control method according to an embodiment of the present invention. The method is detailed as follows:

In step S101, a gesture action of the user is acquired.

In this embodiment, the gesture action of the user may be acquired in various manners, including but not limited to collecting a gesture image of the user through the image collector, and extracting a gesture action from the gesture image; or, by reflective gesture tracking The user's gestures are acquired by an infrared attitude tracking or an ultrasonic attitude tracker.

Wherein, the gesture action includes but is not limited to: the index finger is circled, the remaining four fingers are clenched and the circle is larger than the outline of the hand; or the five fingers are close together, the palm moves in the direction of the device; or the thumb, the index finger, the middle finger naturally extend, and the rest The two fingers are close together, and the index finger moves freely; or the thumb, index finger, and middle finger naturally extend, the other two fingers close together, and the thumb moves up and down; or the thumb, index finger, and middle finger naturally extend, and the other two fingers close together, and the middle finger moves up and down.

In step S102, a control instruction corresponding to the gesture action is acquired according to a mapping relationship between the pre-stored gesture action and the control command.

In this embodiment, before acquiring the control instruction corresponding to the gesture action, the method further includes creating and storing a mapping relationship between the gesture action and the control instruction. Preferably, the gesture action includes a right hand gesture action and a left hand gesture action.

The mapping relationship between the creating and storing the gesture action and the control instruction specifically includes:

Acquiring N gesture actions of the user, the gesture motion including the motion track of the hand and the hand at the beginning of the motion track (when the hand enters the preset target area) and the end (the hand leaves the preset a hand gesture when the target area, or when the hand stays in a certain area of the target area, exceeds a preset time value;

Selecting N control commands from pre-edited control commands and establishing a one-to-one correspondence with the gesture actions;

The correspondence is stored. Wherein, the N is greater than or equal to 1.

In this embodiment, the mapping relationship between the right-hand gesture action and the control instruction includes but is not limited to at least one of the following:

When the gesture action is for the index finger to circle the remaining four fingers and the circle is larger than the contour of the hand, the corresponding control command is “determine” or “negative”. Preferably, when the “OK” and “Negative” options are displayed on the display screen, the gesture action is that the index finger is clockwise and the remaining four fingers are clenched and the circle is larger than the contour of the hand, and the corresponding control instruction is “determined” (as shown in the figure). 2a); gesture action is the index finger counterclockwise circled the remaining four fingers clenched fists and the circle is larger than the outline of the hand, the corresponding control command is "negative" (as shown in Figure 2b);

When the gesture action is five fingers and the palm moves to the direction of the device, the corresponding control command is “return”, for example, returning to the previous page or the previous menu (as shown in FIG. 2c);

When the gesture action is that the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the index finger moves freely, the corresponding control command is “the movement of the mouse track along the moving finger”. (as shown in Figure 2d);

When the gesture action is the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the thumb moves up and down, the corresponding control command is “left mouse click” (as shown in FIG. 2 e );

When the gesture action is naturally extended for the thumb, forefinger and middle finger, and the other two fingers are close together, and the middle finger moves up and down, the corresponding control command is “right mouse click” (as shown in Fig. 2f).

The mapping relationship between the left-hand gesture action and the control instruction includes but is not limited to at least one of the following:

When the gesture action is for the index finger to circle the remaining four fingers and the circle is larger than the outline of the hand, the corresponding control instruction is “determine” or “negative”; preferably, when the gesture is for the index finger, the remaining four fingers are fisted and circled. When the contour of the hand is larger than the contour of the hand, the corresponding control command is “determined”; when the gesture action is that the index finger is clockwise and the remaining four fingers are clenched and the circle is larger than the contour of the hand, the corresponding control command is “negative”;

When the gesture action is five fingers and the palm moves toward the direction of the device, the corresponding control command is “return”;

When the gesture action is that the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the index finger moves freely, the corresponding control command is “the movement of the mouse track along the moving finger”;

When the gesture action is the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the thumb moves up and down, the corresponding control command is “right mouse click”;

When the gesture action is naturally extended for the thumb, forefinger and middle finger, and the other two fingers are close together, and the middle finger moves up and down, the corresponding control command is “left mouse click”.

In step S103, the control instruction is executed.

In this embodiment, the system reads the control instructions and executes to perform the corresponding functions. For example, the system reads the "return" instruction and executes the function of returning to the previous page or the previous menu.

It should be noted that the gesture action of the embodiment of the present invention defaults to the gesture action of the right hand.

The beneficial effects obtained by the embodiment of the present invention compared with the prior art are as follows: First, the gesture rules are small, and the user is easy to learn and memorize. Compared with the prior art solution 1, the embodiment of the invention greatly simplifies the number of gesture rules, and the user can complete most operations like a mouse by simply remembering three basic gesture actions. Second, the operation is simple and smooth. Compared with the prior art solution 2, when the user implements three basic gestures, the basic hand posture is the same (extending the thumb, the index finger, the middle finger, and the other two fingers together), so that the user does not need to operate continuously. Transform the basic pose and switch between different gestures smoothly. In addition, the definition of the gesture action is consistent with the user's habit. For example, the left button defined in the embodiment of the present invention is (if the right hand) the thumb moves, and the right button is the (right hand) middle finger movement, and the intuitive left and right mapping is consistent with the user's habit. It is convenient for users to learn and remember, and it is easy to use and practical.

As another embodiment of the present invention, in order to satisfy the use requirements of different users (customized with right-handed or left-handed users), the ease of use and practicability of the present invention are further enhanced, the method is after step S101, before step S102, Also includes:

Detecting whether the gesture action is a right-hand gesture gesture or a left-hand gesture gesture, and when the gesture gesture is a right-hand gesture gesture, acquiring a control instruction corresponding to the right-hand gesture gesture according to a mapping relationship between the right-hand gesture gesture and the control instruction When the described action is a left-hand gesture, the control command corresponding to the left-hand gesture is acquired according to the mapping relationship between the left-hand gesture and the control command.

The detecting whether the gesture action is a right-hand gesture motion or a left-hand gesture motion is specifically: setting a left target region and a right target region in advance (the center point or the nose position of the two eyes of the face may be defined as a boundary line, and the left portion is a left target region) The right part is the right target area. The gesture is detected whether it is in the left target area or the right target area. If it is in the left target area, it is a left-hand gesture, otherwise it is a right-hand gesture.

FIG. 3 shows a composition of a contactless gesture control apparatus according to Embodiment 2 of the present invention. For convenience of description, only parts related to the embodiment of the present invention are shown.

The contactless gesture control device can be operated on various electronic devices (including mobile phones, tablet mobile phones, desktop computers, tablet computers, televisions, refrigerators, washing machines, air conditioners, digital cameras, surveillance cameras, medical electronic instruments, etc.). A software unit, a hardware unit, or a combination of hardware and software.

The non-contact gesture control device 3 includes a gesture action acquisition unit 31, a control instruction acquisition unit 32, and a control instruction execution unit 33. Among them, the specific functions of each unit are as follows:

The gesture action acquiring unit 31 is configured to acquire a gesture action of the user; preferably, the gesture action acquisition unit 31 is configured to: collect a gesture image of the user through the image collector, and extract a gesture action from the gesture image; or The user's gestures are acquired by a reflective attitude tracker, an infrared attitude tracking or an ultrasonic attitude tracker.

The control instruction acquiring unit 32 is configured to acquire, according to the mapping relationship between the pre-stored gesture action and the control instruction, a control instruction corresponding to the gesture action acquired by the gesture action acquiring unit 31;

a control instruction execution unit 33, configured to execute the control instruction acquired by the control instruction acquisition unit 32;

Further, the device 3 further includes:

The mapping relationship creating unit 34 is configured to create and store a mapping relationship between the gesture action and the control instruction before the control instruction acquiring unit 32 acquires the control instruction corresponding to the gesture action acquired by the gesture action acquiring unit 31, where the gesture Actions include right hand gestures and left hand gestures;

The detecting unit 35 is configured to detect whether the gesture action is a right-hand gesture or a left-hand gesture before the control instruction acquisition unit 32 acquires the control instruction corresponding to the gesture action after the gesture action acquisition unit 31 acquires the gesture action. action.

When the gesture action is for the index finger to circle the remaining four fingers and the circle is larger than the outline of the hand, the corresponding control command is “determine” or “negative”; preferably, when the gesture action is the index finger, the remaining four fingers are fisted clockwise and When the circle is larger than the outline of the hand, the corresponding control command is “determined”; when the gesture action is the index finger counterclockwise circled and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “negative”;

When the gesture action is the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the thumb moves up and down, the corresponding control command is “left mouse click”;

When the gesture action is naturally extended for the thumb, forefinger and middle finger, and the other two fingers are close together, and the middle finger moves up and down, the corresponding control command is “right mouse click”.

When the gesture action is for the index finger to circle the remaining four fingers and the circle is larger than the outline of the hand, the corresponding control instruction is “determine” or “negative”; preferably, when the gesture is for the index finger, the remaining four fingers are punched counterclockwise and When the circle is larger than the outline of the hand, the corresponding control command is “OK”; when the gesture action is that the index finger is clockwise and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “negative”;

The non-contact gesture control apparatus provided in this embodiment may be used in the foregoing related non-contact gesture control method. For details, refer to the related description of the first embodiment of the non-contact gesture control method, and details are not described herein again.

In summary, the beneficial effects obtained by the embodiment of the present invention compared with the prior art are as follows: First, the gesture rules are small, and the user is easy to learn and memorize. Compared with the prior art solution 1, the embodiment of the invention greatly simplifies the number of gesture rules, and the user can complete most operations like a mouse by simply remembering three basic gesture actions. Second, the operation is simple and smooth. Compared with the prior art solution 2, when the user implements three basic gestures, the basic hand posture is the same (extending the thumb, the index finger, the middle finger, and the other two fingers together), so that the user does not need to operate continuously. Transform the basic pose and switch between different gestures smoothly. Moreover, the definition of the gesture action is consistent with the user's habit. For example, the left button defined in the embodiment of the present invention is (if the right hand) the thumb moves, and the right button is (if the right hand) the middle finger moves, and the intuitive left and right mapping is consistent with the user's habit. It is convenient for users to learn and remember, and it is easy to use and practical. In addition, when creating and storing the mapping relationship between the gesture action and the control instruction, the embodiment of the present invention creates and stores a mapping relationship between the right-hand gesture action and the left-hand gesture action and the control instruction, and satisfies the usage requirements of different users (habitually using the right hand or the left hand) The user) further enhances the ease of use and utility of the present invention.

The data caching method in the multi-node system provided by the embodiment of the present invention may be completed in whole or in part by hardware related to program instructions. For example, it can be done by computer running. The program can be stored in a readable storage medium such as a random access memory, a magnetic disk, an optical disk, or the like.

Claims

A contactless gesture control method, characterized in that the method comprises:

Get the gesture action of the user;

Obtaining a control instruction corresponding to the gesture action according to a mapping relationship between the pre-stored gesture action and the control instruction;

Executing the control instruction;

The gesture action includes: the index finger is circled, the remaining four fingers are clenched and the circle is larger than the contour of the hand; or the five fingers are close together, the palm moves in the direction of the device; or the thumb, the index finger and the middle finger are naturally extended, and the other two fingers are close together, the index finger is free Move; or the thumb, index finger, middle finger naturally extend, the other two fingers close together, the thumb moves up and down; or the thumb, index finger, middle finger naturally extend, the other two fingers close together, the middle finger moves up and down.
The method according to claim 1, wherein before the obtaining the control instruction corresponding to the gesture action, the method further comprises:

A mapping relationship between the gesture action and the control instruction is created and stored, the gesture action including a right hand gesture action and a left hand gesture action.
The method according to claim 2, wherein after the step of acquiring the gesture of the user, according to the mapping relationship between the pre-stored gesture action and the control instruction, before the step of acquiring the control instruction corresponding to the gesture action, include:

The gesture action is detected as a right hand gesture or a left hand gesture.
The method according to claim 2 or 3, wherein the mapping relationship between the right-hand gesture action and the control instruction comprises at least one of the following:

When the gesture action is that the index finger is circled and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “determined” or “negative”;

When the gesture action is five fingers and the palm moves toward the direction of the device, the corresponding control command is “return”;

When the gesture action is that the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the index finger moves freely, the corresponding control command is “the movement of the mouse track along the moving finger”;

When the gesture action is the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the thumb moves up and down, the corresponding control command is “left mouse click”;

When the gesture action is naturally extended for the thumb, forefinger and middle finger, and the other two fingers are close together, and the middle finger moves up and down, the corresponding control command is “right mouse click”.
The method according to claim 4, wherein when the gesture action is that the index finger is circled and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is "determined" or "negative" includes:

When the gesture action is that the index finger is clockwise and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “determined”;

When the gesture action is that the index finger is counterclockwise and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “negative”.
The method according to claim 2 or 3, wherein the mapping relationship between the left-hand gesture action and the control instruction comprises at least one of the following:

When the gesture action is that the index finger is circled and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “determined” or “negative”;

When the gesture action is five fingers and the palm moves toward the direction of the device, the corresponding control command is “return”;

When the gesture action is that the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the index finger moves freely, the corresponding control command is “the movement of the mouse track along the moving finger”;

When the gesture action is the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the thumb moves up and down, the corresponding control command is “right mouse click”;

When the gesture action is naturally extended for the thumb, forefinger and middle finger, and the other two fingers are close together, and the middle finger moves up and down, the corresponding control command is “left mouse click”.
The method according to claim 6, wherein when the gesture action is that the index finger is circled and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is "determined" or "negative" includes:

When the gesture action is that the index finger is counterclockwise and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control instruction is “determined”;

When the gesture action is that the index finger is clockwise and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “negative”.
The method according to any one of claims 1 to 7, wherein the obtaining a gesture action of the user comprises:

Acquiring a gesture image of the user through the image collector, and extracting a gesture action from the gesture image;

Alternatively, the gesture gesture of the user is obtained by a reflective attitude tracker, an infrared attitude tracking or an ultrasonic attitude tracker.
A contactless gesture control device, characterized in that the device comprises:

a gesture action obtaining unit, configured to acquire a gesture action of the user;

a control instruction acquiring unit, configured to acquire, according to a mapping relationship between the pre-stored gesture action and the control instruction, a control instruction corresponding to the gesture action acquired by the gesture action acquiring unit;

a control instruction execution unit, configured to execute a control instruction acquired by the control instruction acquisition unit;

The gesture action includes: the index finger is circled, the remaining four fingers are clenched and the circle is larger than the contour of the hand; or the five fingers are close together, the palm moves in the direction of the device; or the thumb, the index finger and the middle finger are naturally extended, and the other two fingers are close together, the index finger is free Move; or the thumb, index finger, middle finger naturally extend, the other two fingers close together, the thumb moves up and down; or the thumb, index finger, middle finger naturally extend, the other two fingers close together, the middle finger moves up and down.
The device of claim 9 wherein said device further comprises:

a mapping relationship creating unit, configured to create and store a mapping relationship between the gesture action and the control instruction before the control instruction acquiring unit acquires the control instruction corresponding to the gesture action acquired by the gesture action acquiring unit, where the gesture action includes the right hand Gesture action and left hand gesture action;

The detecting unit is configured to detect whether the gesture action is a right-hand gesture gesture or a left-hand gesture motion before the control instruction acquisition unit acquires the control instruction corresponding to the gesture action, after the gesture action acquisition unit acquires the gesture motion.
The apparatus according to claim 10, wherein the mapping relationship between the right-hand gesture action and the control instruction comprises at least one of the following:

When the gesture action is that the index finger is circled and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “determined” or “negative”;

When the gesture action is five fingers and the palm moves toward the direction of the device, the corresponding control command is “return”;

When the gesture action is that the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the index finger moves freely, the corresponding control command is “the movement of the mouse track along the moving finger”;

When the gesture action is the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the thumb moves up and down, the corresponding control command is “left mouse click”;

When the gesture action is naturally extended for the thumb, forefinger and middle finger, and the other two fingers are close together, and the middle finger moves up and down, the corresponding control command is “right mouse click”.
The device according to claim 11, wherein when the gesture action is that the index finger is circled and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “determined” or “negative” includes:

When the gesture action is that the index finger is clockwise and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “determined”;

When the gesture action is that the index finger is counterclockwise and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “negative”.
The apparatus according to claim 10, wherein the mapping relationship between the left-hand gesture action and the control instruction comprises at least one of the following:

When the gesture action is that the index finger is circled and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “determined” or “negative”;

When the gesture action is five fingers and the palm moves toward the direction of the device, the corresponding control command is “return”;

When the gesture action is that the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the index finger moves freely, the corresponding control command is “the movement of the mouse track along the moving finger”;

When the gesture action is the thumb, the index finger and the middle finger naturally extend, and the other two fingers are close together, and the thumb moves up and down, the corresponding control command is “right mouse click”;

When the gesture action is naturally extended for the thumb, forefinger and middle finger, and the other two fingers are close together, and the middle finger moves up and down, the corresponding control command is “left mouse click”.
The device according to claim 13, wherein when the gesture action is that the index finger is circled and the remaining four fingers are clenched and the circle is larger than the contour of the hand, the corresponding control command is “determined” or “negative” includes:

When the gesture action is that the index finger is counterclockwise and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control instruction is “determined”;

When the gesture action is that the index finger is clockwise and the remaining four fingers are clenched and the circle is larger than the outline of the hand, the corresponding control command is “negative”.
The device according to any one of claims 9 to 14, wherein the gesture action acquisition unit is configured to: collect a gesture image of the user through the image collector, and extract a gesture action from the gesture image; or The user's gestures are acquired by a reflective attitude tracker, an infrared attitude tracking or an ultrasonic attitude tracker.