KR102299158B1 - Trusted predictive analytic execution middleware - Google Patents

Abstract

The computing system includes technologies that provide reliable predictive analytics services. The computing system provides a common description language for predictive analytics services and uses cryptographic techniques and digital rights management techniques to secure input data and/or portions of predictive analytics services.

Description

TRUSTED PREDICTIVE ANALYTIC EXECUTION MIDDLEWARE

The art of predictive analytics typically involves the application of computational techniques such as machine learning and data mining to large data sets. Previously unknown patterns, such as similarities, differences, and relationships between different elements of a data set, can be discovered by these computational techniques. Predictive algorithms, such as statistical methods, may be used to identify trends and predict future outcomes based on patterns found in a data set. Many predictive analytics techniques are model-based. For example, a mathematical model can be constructed that expresses relationships between elements in a data set and conclusions regarding those data elements. The model can be “trained” using a “trainer” data set for which relationships are already known.

Predictive analytics may be provided as a service running on a computer network. Face detection (eg, the ability to identify a human face in a digital photograph, using a computer) is an application of predictive analytics that can be served over a network to many different types of computing devices and applications. . In automated face detection, the trainer data set may include a very large number of sample photos of human faces combined with descriptive tags indicating features represented in the photos. The trainer data is used to train the model, for example to establish probabilities that certain combinations of features shown in the photos represent certain types of people (eg, male, female, young, old). The model and trainer data can then be used to classify new input photos (eg, photos not already in the existing data set).

The concepts described herein are illustrated by way of example and not limitation in the accompanying drawings. For simplicity and clarity of illustration, elements illustrated in the figures are not necessarily drawn to scale. Where deemed appropriate, reference labels have been repeated among the figures to indicate corresponding or analogous elements.
1 is a simplified block diagram of at least one embodiment of a computing system including trust prediction analysis middleware as disclosed herein;
2A and 2B are simplified architectural diagrams of the trust prediction analysis middleware of FIG. 1 ;
3A-3D are simplified environmental diagrams of the trust prediction analysis middleware of FIG. 1 ;
4 is a simplified flow diagram of at least one embodiment of a method for providing predictive confidence analyzes as disclosed herein, which may be executed by one or more components of the computing system of FIG. 1 ;
5 is a simplified flow diagram of at least one embodiment of a method for generating a trusted version of a predictive analytics service as disclosed herein, which may be executed by one or more components of the computing system of FIG. 1 ;

Although the concepts of the present disclosure are susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and will be described in detail herein. It is to be understood, however, that there is no intention to limit the concepts of the present disclosure to the specific forms disclosed, on the contrary, the intention is to cover all modifications, equivalents, and alternatives in accordance with this disclosure and the appended claims. should be

References herein to "one embodiment," "an embodiment," "an exemplary embodiment," etc., are made in that the described embodiment may include a particular feature, structure, or characteristic, but not all embodiments may include such particular feature, Indicate structures, or properties, that may or may not necessarily include them. Moreover, such phrases are not necessarily referring to the same embodiment. Moreover, when a particular feature, structure, or characteristic is described in connection with one embodiment, it is common practice in the art to achieve that feature, structure, or characteristic in connection with other embodiments, whether explicitly described or not. It is submitted as being within the knowledge of the technician of Additionally, items included in the list in the form of “at least one of A, B, and C” include (A); (B); (C); (A and B); (B and C); (A and C); or (A, B, and C). Similarly, items listed in the form of "at least one of A, B, or C" include (A); (B); (C); (A and B); (B and C); (A and C); or (A, B, and C).

The disclosed embodiments may in some cases be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented with instructions carried by and stored on a transitory or non-transitory machine-readable (eg, computer-readable) storage medium that can be read and executed by one or more processors. can A machine-readable storage medium is any storage device, mechanism, or other physical structure (eg, volatile or non-volatile memory, media disk, or other media device) that stores or transmits information in a form readable by a machine. can be materialized.

In the drawings, some structure or method features may be shown in specific arrangements and/or arrangements. However, it should be understood that such specific arrangements and/or arrangements may not be required. Rather, in some embodiments, such features may be arranged in a different manner and/or order than shown in the illustrative figures. Additionally, the inclusion of a structural or method feature in a particular figure is not intended to imply that such feature may be required in all embodiments and, in some embodiments, may not be included or may be combined with other features. .

Predictive analytics services may introduce vulnerabilities that elevate user privacy and/or intellectual property management issues. For example, when a digital photo of an end user is uploaded to a network, the facial recognition service may have access to personal information associated with the photo. At the same time, proprietary information regarding the model and/or training data used by the facial recognition service may be exposed to the client device. In addition, many current predictive analytics products are installed as monolithic vertical software stacks in which predictive models are tightly coupled with training data. This may cause the user to need to install many different vertical software stacks to solve different types of predictive analytics problems, or even solve the same problem in different ways. The situation is exacerbated by the proliferation of many different types of networked computing devices (eg, smartphones, tablets, wearable devices, laptops, desktops, etc.), each of which is a different version of a model description. may require a different version of the model that is optimized for a particular platform.

Referring now to FIG. 1 , one embodiment of a computing system 100 includes trust predictive analytics middleware or “trust middleware” 166 . Trust predictive analytics middleware 166 includes model description language 160 and trust predictive analytics middleware service or “middleware service” 164 . Illustratively, the components of predictive trust analysis middleware 166 are embodied in predictive trust analytics middleware computing device 150 ; Portions of the trust prediction analysis middleware 166 may reside on one or more other computing devices, as described in more detail below. In the computing system 100 , a user computing device 110 (eg, a mobile or wearable computing device) sometimes executes a user-level application 118 . A user-level application 118 may, from time to time, request access to a predictive analytics service or “detector” 194 . An example predictive analytics service (eg, “detector”) 194 is embodied in the predictive analytics provider computing device 180 . The user computing device 110 , the trusted predictive analytics middleware computing device 150 , and the predictive analytics provider computing device 180 are communicatively coupled by one or more networks 170 .

In an example scenario, the user level application 118 may be a camera application, a photo uploading service, or a front end to a social media service such as Facebook or Pinterest. The user computing device 110 is configured with a trusted execution subsystem 120 . Trusted execution subsystem 120 may be embodied, for example, as a hardware- or software-implemented Trusted Platform Module (TPM) or using an ARM's trustzone. When a user-level application 118 requests access to a predictive analytics service (eg, a “detector”) 194 , the trusted execution subsystem 120 launches the trusted predictive analytics middleware 166 in the trusted execution environment. . Trust predictive analytics middleware 166 by executing in a trusted execution environment and instantiating executable trust predictive analytics services (eg, “detectors”) 194 that are based on model descriptions generated using model description language 160 . ) provides a common trusted execution environment in which predictive analytics services (eg, “detectors”) 194 can run and sensitive data can be segregated. Exemplary predictive analytics services (eg, “detectors”) 194 provide data analytics services, such as machine learning-based data analytics services or “big data” analytics services. Some example implementations of a predictive analytics service (eg, “detector”) 194 are shown in FIGS. 3A-3D . As described in more detail below, trust predictive analytics middleware 166 protects sensitive user data from accidental exposure or misuse by predictive analytics services (eg, “detectors”) 194 and/or predictive analytics It uses cryptographic techniques and digital rights management techniques to protect the intellectual property rights associated with the service (eg, “detector”) 194 . Additionally, via model description language 160 , trust predictive analytics middleware 166 can range from smartphones, tablets, laptops and personal computers to wearable devices such as smart glasses, smart watches. , to virtual instances running “in the cloud”, and/or to provide a common interface to predictive analytics services across different device architectures to others.

Using model description language 160 , trust middleware 166 avoids the need to have multiple different predictive analytics services (eg, “detectors”) 194 running on the middleware platform. Instead, middleware 166 replaces different predictive analytics services (eg, “detectors”) 194 with different instances of predictive analytics service (eg, “detectors”) 194, Each of the instances is created from an execution primitive whose operation is supported by trust middleware 166 . Trust middleware 166 uses model description 162 to decrypt and instantiate predictive analytics service (eg, “detector”) 194 . Model description 162 is generated by middleware 166 using model description language 160 . The digital rights management functionality of trust middleware 166 enforces license agreement conditions and restrictions regarding the use of predictive analytics service (eg, “detector”) 194 . User data (eg, user content 124 ) is accessed by middleware primitives (eg, predictive analytics services or “detectors” 194 , instantiated using model description 162 ) within a trusted execution environment. is protected, so user data is not exposed directly to predictive analytics services (eg, “detectors”) 194 .

Referring now in more detail to FIG. 1 , user computing device 110 may be embodied as any type of electronic device that performs the functions described herein. For example, without limitation, user computing device 110 may be a smartphone, tablet computer, wearable computing device, laptop computer, notebook computer, mobile computing device, cellular phone, handset, messaging, configured to perform the functions described herein. It may be embodied in a device, a vehicle telematics device, a server computer, a workstation, a distributed computing system, a multiprocessor system, a consumer electronic device, and/or any other computing device. 1 , example user computing device 110 includes at least one processor 112 .

User computing device 110 also includes memory 114 , input/output subsystem 116 , data storage device 122 , camera 132 , one or more sensors 134 , and a user interface (UI). ) subsystem 136 , and communication subsystem 140 . User computing device 110 may, in other embodiments, include other or additional components, such as those commonly found in mobile and/or stationary computers. Additionally, in some embodiments, one or more of the example components may be included in, or otherwise form part of, another component. Each of the components of the user computing device 110 may be embodied in software, firmware, hardware, or a combination of software, firmware, and/or hardware.

Processor 112 may be embodied as any type of processor capable of performing the functions described herein. For example, processor 112 may be embodied as a multi-core processor or other multi-CPU processor or processing/control circuitry. Memory 114 of user computing device 110 may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. During operation, memory 114 may store operating systems, applications, programs, libraries, and drivers, as well as various data and software used during operation of user computing device 110 .

Memory 114 is communicatively coupled to processor 112 , for example, via I/O subsystem 116 . I/O subsystem 116 is embodied in circuitry and/or components to facilitate input/output operations with processor 112 , memory 114 , and other components of user computing device 110 . can be For example, I/O subsystem 116 may include memory controller hubs, input/output control hubs, firmware devices, communication links (ie, point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.) and/or other components and subsystems, or otherwise include them. In some embodiments, I/O subsystem 116 forms part of a system-on-a-chip (SoC) and includes processor 112 , memory ( 114), and/or along with other components, may be included on a single integrated circuit chip.

The data storage device 122 is a memory that is, for example, combinations of memory devices and circuits, memory cards, hard disk drives, solid state drives, flash memory or other read-only memory, read-only memory, and random access memory. devices, or any type of physical device or devices configured for short-term or long-term storage of data, such as other data storage devices. User content 124 (eg, digital content such as photos, videos, music files, and documents) and detector models 190 are stored in data storage device 122 . Portions of user content 124 and/or detector models 190 may be copied to memory 114 from time to time during operation of computing device 110 , for example, for faster processing.

Camera 132 may be any type of camera capable of performing the functions described herein, for example, capable of capturing still and/or video images using camera hardware, software, or a combination of hardware and software. can be materialized. The sensor(s) 134 may be embodied as any suitable type of sensor capable of performing the functions described herein, including one or more of motion sensors, proximity sensors, position sensors, and eye tracking devices. can be

The user interface subsystem 136 provides a number of additional features to facilitate user interaction with the user computing device 110 , including physical or virtual control buttons or keys, a microphone, a speaker, a display device, and/or others. may include devices. For example, a display device may be any type of display capable of displaying digital information such as a liquid crystal display (LCD), a light emitting diode (LED), a plasma display, a cathode ray tube. tube) (CRT), or other types of display devices. In some embodiments, the display device may be coupled to a touch screen or other human computer interface device to allow user interaction with the user computing device 110 . The user interface subsystem 136 may also be configured to detect, capture, and process various other forms of human interactions involving the user computing device 110 , motion sensors, proximity sensors, and eye tracking. may include other devices such as devices.

The user computing device 110 further includes a communication subsystem 140 , which includes any communication circuitry, device, Or it may be embodied as a set of them. Communication subsystem 140 may include any one or more communication technologies (eg, wireless, optical, or wired communications) and associated protocols (eg, Ethernet, Bluetooth®, Wi-Fi) to achieve such communication. ®, WiMAX, 3G/LTE, etc.). Communication subsystem 140 may be embodied as a network adapter, including a wireless network adapter.

Example user computing device 110 also includes a number of computer program components, such as a user level application 118 described below, a trusted execution subsystem 120 , and one or more detector models 190 . . User level application 118 may be any computer application (eg, software, firmware, hardware , or a combination thereof). Some examples of user level applications 118 include word processing programs, document viewers/readers, web browsers, electronic mail programs, messaging services, social media services, content sharing services, computer games, camera and video applications, and the like. Although not specifically shown, user computing device 110 includes a privileged system component that facilitates communication between hardware components and user level applications 118 of user computing device 110 . Portions of privileged system component 142 may be embodied in any operating system capable of performing the functions described herein, such as a version of Windows from Microsoft Corporation, Android from Google Inc, and/or others. Alternatively or additionally, some of the privileged system component 142 may be embodied in any type of virtual machine monitor (eg, a Type I or Type II hypervisor) capable of performing the functions described herein. can be

Trust predictive analytics middleware computing device 150 and predictive analytics provider computing device 180 may each be embodied as any type of electronic device that performs the functions described herein. For example, computing devices 150 , 180 may include, but are not limited to, server computers, workstations, distributed computing systems, multiprocessor systems, consumer electronic devices, smartphones, tablet computers, etc. configured to perform the functions described herein; It may be embodied in a wearable computing device, a laptop computer, a notebook computer, a mobile computing device, a cellular phone, a handset, a messaging device, a vehicular telematics device, and/or any other computing device. 1 , example computing devices 150 , 180 include at least one processor 152 , 182 , memory 154 , an I/O subsystem 156 , and data storage devices 158 , 188 . ), and UI subsystems 168 and 196, respectively. Components of computing devices 150 , 180 having the same or similar name as components of user computing device 110 may be embodied in a similar manner to those components described above; Accordingly, the description is not repeated here. As noted above, the example trust predictive analytics middleware computing device 150 includes a trust predictive analytics middleware that includes a model description language 160 and a trust predictive analytics middleware subsystem 164 stored on a data storage device 158 . (166) is specified. Data storage device 158 also stores model descriptions 162 . In operation, portions of model description language 160 and/or model descriptions 162 may be copied to memory 154 for faster processing, for example. The example predictive analytics provider computing device 180 includes one or more detector models 190 and one or more model trainers (eg, training data sets) 192 stored in a data storage device 188 . , specify a predictive analytics service (eg, “detector”) 194 . In operation, portions of detector model(s) 190 and/or model trainer(s) 192 may be copied to memory 184 for faster processing, for example. Portions of any of the data and/or computer program components of computing system 100 , such as user level applications 118 , trusted execution subsystem 120 , user content 124 , trust predictive analytics middleware ( 166), and a predictive analytics service (eg, a “detector”) 194 may reside on computing devices other than those shown in FIG. 1 . Moreover, in some embodiments, such components (eg, user level application 118 , trusted execution subsystem 120 , user content 124 , trusted predictive analytics middleware 166 , and predictive analytics service) All of (eg, “detectors”) 194) are in a single computing device (eg, computing device 110 , trust predictive analytics middleware computing device 150 , or predictive analytics provider computing device 180 ). may reside on the

As noted above, trust middleware 166 includes model description language 160 and trust prediction analysis middleware subsystem 164 . The model description language 160 includes model specifications and parameter information. Model description language 160 is interpreted using components of middleware subsystem 164 , as described in more detail below. Referring now to FIGS. 2A and 2B , one embodiment of example architectures for portions of the trust prediction analysis middleware subsystem 164 is shown. In FIG. 2A , a high level architecture 200 is shown. In a high level abstraction, architecture 200 includes an interface layer 210 and an execution layer 216 . Within the interface layer 210 , the architecture 200 includes an application programming interface (API) service 212 and a management interface 214 . API service 212 provides an interface through which user level applications 118 (eg, A1 , A2 , A3 , A4 in FIGS. 3A-3D ) can issue predictive analytics service requests to trust middleware 166 . to provide. Management interface 214 provides an interface through which operating systems or basic system services can accomplish tasks such as provisioning, auditing, and upgrading components of trust middleware 166 . Within the execution layer 216 , the exemplary components implement, at a high level, a model execution engine 218 , a trust management subsystem 220 , and a digital rights management (DRM) subsystem 222 . include The model execution layer 216 is responsible for the entire life cycle of a predictive analytics (PA) task, which is responsible for performing activities, e.g., model descriptions using the model description assembler model 252 ( Creating a PA task based on 162 , launching PA tasks, and scheduling different PA tasks by the detector scheduler module 254 , the operation described in the model description 162 , is subjected to a wide range of types of predictive analysis. Resource management and mapping to a set of primitives subtracted from tasks, such as performing convolution of the input, populating the input into a polynomial, etc. (eg, those tasks represented by the PA execution primitive 256 ). and computing (eg, via the resource management module 258 , memory, etc.).

A highly optimized PA execution primitive 256 is implemented on top of one or more platform-specific execution backends 260 (execution backends 260) in a custom processor (e.g., Intel's Xeon processor) with a platform-specific instruction set ( For example, a specific implementation based on the Intel AVX instruction set, an implementation based on a specific graphics processing unit (GPU), or a platform specific acceleration technology (eg, a floating point gate array). (FPGA))). Trust management subsystem 220 is responsible for monitoring the implementation of framework 200 in a trusted manner. The trust management subsystem 220 can prevent modification of the framework 200 to ensure that the functioning of the framework functions as intended and cannot be modified by malfunctioning processes running on the same system. have. Trust management subsystem 220 may protect access to privacy data and prevent access to sensitive data, such as decrypted modeling descriptions, the code page of a PA operation in memory, and the like.

The DRM subsystem 222 is responsible for managing the digital rights access and protection of the owner of the model 190 . The exemplary DRM subsystem 222 may serve two roles. The first role is to manage sensitive and cryptographic operations as defined by the DRM protocol. The second role is to control the framework 200 to grant access to the content of the model 190 only in a manner authorized by the model owners, eg at fixed times, with a fixed size of input data, etc. . The DRM subsystem 222 operates to ensure that the framework 200 protects the digital rights of the model owner in the model 190 . Each of the components 218 , 220 , 222 may be embodied in computer hardware, firmware, software, or a combination thereof.

Referring to FIG. 2B , a more detailed illustration 230 of the components of one embodiment of the execution layer 216 is shown. Exemplary execution layer 230 includes model description assembler module 252 , detector scheduler module 254 , predictive analytics execution primitives 256 , resource management module 258 , platform specific execution backend 260 , trusted execution monitor module 262 , and a DRM manager module 264 . Each of the components 252 , 254 , 256 , 258 , 260 , 262 , 264 may be embodied in computer hardware, firmware, software, or a combination thereof.

The model description assembler module 252 receives as input a detector model 190 (eg, M1, M2, M3, or M4, described below) of a predictive analytics service (eg, a “detector”) 194 . and interpret the detector model 190 using the model description language 160 , thereby generating a model description 162 of the detector model 190 in the model description language 160 . In some embodiments, the generation of the model description 162 (eg, the conversion of the detector 194 to the model description 162 ) is performed as a one-time event, which is performed on the predictive analytics provider computing device 180 . can be Once the model description 162 is generated (eg, on the provider computing device 180 ), the detector 194 is configured (eg, by the trust prediction analysis middleware subsystem 164 ) on the trusted PA middleware computing device. 150 or other computing devices. As a result of model analysis, assembler module 252 obtains information about the structure of detector model 190 and associated parameters. For this information, the assembler module 252 provides a predictive analytics service (e.g., a “detector”) based on the model description 162 and using the predictive analytics execution primitive 256 supplied by the middleware 166 . Create an executable instance of (194). To do this, the assembler module 252 maps the nodes of the model structure to the predictive analysis execution primitives 256 (eg, by code logic) and maps the execution variables to model parameters embedded in the model description 162 . map to Assembler module 252 may apply one or more optimization algorithms (eg, to remove any redundant operations). In this way, assembler module 252 creates a "trusted" version of predictive analytics service (e.g., "detector") 194 that has common execution primitives with other predictive analytics services and uses common description language 160 . to designate the model description 162 .

The model description 162 provides a minimal set of information that can be used to rebuild an executable detector, assuming the detector runs on middleware. The model description 162 includes model structures, model parameters, and model meta information about the model. References herein to “model structure” may refer to, among other things, a graph or tree structure commonly used to represent analytic models, as well as nodes and network structures (eg, an arrangement of nodes and edges). can As used herein, a “node” may refer to, among other things, a primitive connection. The connections and network structure determine the configuration rules, which establish the flow of control through the model network. Some examples of model structures include acyclic graphs, probabilistic graphical models, Bayesian networks, multi-layer network structures, and/or others. Model parameters provide coefficients, such as primitive coefficients, connection coefficients, and network coefficients. Meta information provides information about the detector model (eg, “annotations”). For example, the meta information may indicate the kind of problem or application for which the detector is suitable, the inputs required by the detector, and information about how the detector is trained. Meta information can typically be released publicly, for example by a directory service. The meta information is used by a search engine so that the user-level application 118 finds appropriate detectors for particular tasks (eg, detectors whose meta-information matches business requirements specified by the user-level application 118 ). to execute queries. Some examples of model meta information include the size of the trained data (eg, the size of the training data set) and the input format (eg, formatting requirements for the input data, if necessary).

Once the assembler module 252 generates an optimized executable of the predictive analytics service (eg, “detector”) 194 , the executable is submitted to the detector scheduler module 254 . The detector scheduler module 254 schedules and executes the detectors instantiated by the assembler module 252 , and interacts with the resource management module 258 , the trusted execution monitor module 262 , and the DRM manager module 264 . Scheduler module 254 handles data distribution and, where possible, identifies and removes duplicate data copies. The scheduler module 254 manages the life cycle of instantiated detectors and removes resources when the detector completes execution.

Predictive analytics execution primitives 256 are executables that perform common predictive analytics tasks in an efficient manner. As such, the predictive analysis execution primitive 256 may form the basis of many types of detectors. The resource management module 258 manages resources such as data storage and memory allocation. Trusted execution monitor module 262 and DRM manager module 264 expose standard trusted execution and DRM management components to the layers described above (eg, scheduler module 254 and/or management interface 214 ). . Platform specific execution backend 260 enables middleware 166 to interface with platform specific capabilities across many different types of devices and computing platforms.

More specifically, trust management subsystem 220 (FIG. 2A) secures the DRM of the instantiated detector and protects the privacy of the input data. Trust middleware 166 has a native connection with the trusted computing components (eg, virtual machine monitor or VMM, or hypervisor) of the native operating system/privileged system component. In a virtualized architecture, when middleware 166 runs in trusted mode, middleware 166 runs within a privileged system component (eg, a VMM or hypervisor). As a result, sensitive data is stored on the page in a secure trust mode. Data is read from an encrypted data source or a sensitive sensor. When trust middleware 166 is launched by trust launch (eg, by trust execution subsystem 120 ), non-sensitive user content 124 (eg, grabbing by the user's web browser) is (grabbed) publicly available news pages) will be allocated in memory managed by the guest operating system, accessible to trusted middleware 166 . The underlying Trusted Execution mechanism may be embodied as a hybrid solution based, for example, on TXT (Intel's Trusted Execution Technology), such as an Intel Trusted Platform Module (TPM) based mechanism or ARM's TrustZone.

Referring now in more detail to the DRM subsystem 222 ( FIG. 2A ), the DRM subsystem 222 includes the detector models 190 (eg, M1 , M2 , M3 , M4 discussed below) and a model trainer 192 (eg, T1 , T2 , T3 , T4 discussed below). When a user level application 118 (eg, A1 , A2 , A3 , A4 discussed below) wants to launch the middleware 166 , the DRM subsystem 222 instead of simply instantiating a detector instance directly , check the license agreement and/or usage restrictions for the requested middleware service. Middleware 166 is referred to as a DRM agent (eg, DRM manager module 264) to check applicable digital rights, including rights and access. If the license is valid, the middleware 166 decrypts the model description 162 into a trusted memory area established by the trust management subsystem 220 . As should be appreciated, cryptographic services such as encryption and decryption are provided by the trusted execution subsystem 120 and are available when the trusted execution environment is established.

Referring now to FIGS. 3A-3D , example implementations of a predictive analytics service (eg, a “detector”) 194 that may be processed by middleware 166 include environments 300 , 320 , 340 , 350 . (eg, native and/or virtual runtime or “execution” environments), respectively. The various modules shown in environments 300 , 320 , 340 , 350 may be embodied in hardware, firmware, software, or a combination thereof. Each of the implementations of the predictive analytics service (eg, “detector”) 194 is a detector (eg, detectors 310 , 322 , 342 , 352 ), a model (eg, models M1 , M2 , M3, M4)), and model trainers (eg, model trainers T1, T2, T3, T4). A detector (eg, detectors 310 , 322 , 342 , 352 ) is a predictive analytics service that may be requested by a user level application 118 (eg, applications A1 , A2 , A3 , A4 ). . For example, a detector (eg, detector 310 , 322 , 342 , 352 ) may request a service (eg, an API call) from an application (eg, application A1 , A2 , A3 , A4 ). In response, classification, tagging, or regression analysis may be performed on data supplied by an application (eg, application A1 , A2 , A3 , A4 ). Aspects of the functionality of a detector (e.g., detector 310, 322, 342, 352), including accuracy, efficiency, performance, etc., are model (e.g., model M1, M2, M3, M4), It is determined by the specific algorithm(s) used, and the training data set used by the model trainer (eg, model trainers T1, T2, T3, T4). A model (eg, model (M1, M2, M3, M4)) is trained using a model trainer (eg, model trainer (T1, T2, T3, T4)) and its associated data set. In some implementations, a detector (eg, detector 310 , 322 , 342 , 352 ) is configured either in source code ( FIG. 3A ) or as a system level (eg, native) library object ( FIGS. 3B and 3D ). released, and the application (eg, application A1 , A2 , A4 ) is placed on a detector (eg, detector 310 , 322 , 342 , 352 ) on the user platform (eg, user computing device 110 ). )) are integrated. For example, in the implementation of FIG. 3A , M1 may be an analytics service that uses data mining to provide marketing analytics (eg, an LBM service). In FIG. 3B , M2 may be a model generated by a machine learning algorithm (eg, adaptive boost or "Adaboost" algorithm), and the detector library object is a digital signal processor (DSP) may be configured for use with In FIG. 3D , M4 may be a predictive model using a dynamic Bayesian network (DBN) and is configured for use with an application-specific integrated circuit (ASIC). These and many other types of analytics services may benefit from the trust middleware 166 disclosed herein.

In the case of a detector (e.g., detector 310, 322, 342, 352) integrated in the manner shown in FIGS. 3A, 3B, and 3D, without the protections provided by middleware 166, the detector ( For example, the detector 310 , 322 , 342 , 352 ) will have direct access to user data (eg, user data 312 , 324 , 344 , 354 ) and will have direct access to the user platform (eg, user computing). Device 110 may discover potentially proprietary aspects of algorithms and/or tuning parameters used by a detector (eg, detector 310 , 322 , 342 , 352 ). The implementation of Figure 3c is a "software as a service" or "SaaS" approach in which M3 and T3 are provided as web services. Hosting model M3 on a web server prevents access to proprietary aspects of model M3 by application A3 or other components of user computing device 110 . However, in environment 340, user data 344 is uploaded to web services M3, T3, thus potentially raising user privacy and data control issues.

Trust middleware 166 addresses the issues escalated by the conventional implementations shown in Figures 3A-3D as follows. First, rather than the original models M1, M2, M3, M4, the trust middleware 166 generates the corresponding model description 162 using the model description language 160 as described above. In each case, middleware 166 deploys model description 162 within the trusted execution environment of the user computing device (eg, by trusted execution subsystem 120 ). For a common middleware interface (eg, API service 212 and management interface 214 ) provided by trust middleware 166 , the same model description 162 may be used across many different devices. Moreover, rather than providing optimized versions of models (eg, M1, M2, 3, M4) for different hardware configurations, the model trainer (eg, T1, T2, T3, T4) is a common The model description 162 can be used to train the model once, and the final trained model can be used on any platform.

Referring now to FIG. 4 , an example of a method 400 of providing predictive trust analysis services is shown. Portions of method 400 may be executed by hardware, firmware, and/or software of computing system 100 (eg, by trust prediction analysis middleware subsystem 164 ). Method 400 begins in response to a request for a predictive analytics service (eg, an API call made by a user level application). At block 410 , the computing system 100 launches the trusted predictive analytics middleware in the trusted execution environment. The trusted execution environment is established, for example, by the trusted execution subsystem 120 . At block 412 , the computing system 100 obtains input data associated with a request made by a user-level application for predictive analytics services. The input data can be, for example, a photograph, a data set, or any combination of text and/or non-text data. As a result of the establishment of a trusted execution environment in which the trust prediction analysis middleware is executed, input data is stored in a trusted memory area (eg, a memory area separated from other memories by hardware or software). At block 414 , the computing system 100 executes stored model descriptions (eg, model descriptions 162 ) that were previously generated by the trust middleware using, for example, the method illustrated in FIG. 5 described below. ))) to select or determine an appropriate model description. As a result of the generation of the model description by the trusted middleware in the trusted execution environment, the model description is encrypted by the components of the trusted execution environment. At block 416 , the computing system 100 checks the license agreement, usage permissions, and/or digital rights associated with the model description based on the input data selected at block 414 and associated with the user application request. . At block 418, the computing system 100 determines whether the request by the user-level application (accompanying the input data obtained at block 412) is authorized by the applicable license/permissions/digital rights management data. do. If the user request is not authorized by the applicable license/permissions/digital rights management data, it fails at block 420 .

If the computing system 100 determines at block 418 that the user request is granted, the computing system 100 sends the selected model description at block 414 (eg, using components of the trusted execution environment). Decrypt At block 424 , the computing system 100 stores the decrypted model description in a trusted memory area (eg, a hardware or software isolated area of the user computing device 110 ). At block 426 , the computing system 100 instantiates the decrypted model. At block 428 , the computing system 100 executes the decrypted model to process the input data, and at block 430 , the computing system 100 outputs the results of running the model on the input data.

Referring now to FIG. 5 , an example of a method 400 of generating a trusted executable detector from a predictive analytics service (eg, “detector”) 194 is shown. Portions of method 500 may be executed by hardware, firmware, and/or software of computing system 100 (eg, by trust prediction analysis middleware subsystem 164 ). Method 500 begins in response to submission of predictive analytics service or “detector” 194 to trust predictive analytics middleware 166 . At block 510 , the computing system 100 generates or selects a trusted execution primitive (eg, predictive analysis execution primitive 256 ) for use as an execution primitive in connection with the submitted predictive analytics service. At block 512 , the computing system 100 generates a model (eg, detector model 190 ) for the predictive analytics service using a common model description language (eg, model description language 160 ). Converts to a model description (eg, model description 162 ) for the submitted predictive analytics service. Illustratively, this conversion is performed by the PA provider computing device 180 . Once the model description 162 is generated (eg, on the computing device 180 ), the detector 194 is configured (eg, by the trust prediction analysis middleware subsystem 164 ), as discussed above. Trusted PA middleware computing device 150 or other computing device may be deployed.

At block 514 , the computing system 100 generates a model structure (eg, nodes, network, etc.) based on the model description prepared at block 512 . At block 516 , the computing system 100 converts the model structure into an executable using the executable primitive created or selected at block 510 . At block 518 , the computing system 100 applies one or more optimizations as needed to the executable generated at block 516 . At block 520 , the computing system 100 submits the executable to a scheduling module of the trusted middleware (eg, the detector scheduler module 254 ).

examples

Illustrative examples of the techniques disclosed herein are provided below. An embodiment of the techniques may include any one or more, and any combination, of the examples described below.

Example 1 includes a computing system comprising one or more computing devices, the computing system to provide a predictive trust analytics service, the computing system to include a predictive trust analytics middleware subsystem, and the predictive trust analytics middleware subsystem to provide a predictive analytics service. responsive to a user-level application request for: in a trusted execution environment of the computing system, cause the computing system to determine a model description for the predictive analytics model, wherein the model description is generated by the predictive analytics model description language; the model description language describes a plurality of different predictive analytics models using a common language; compare data associated with the user level application request to data representing digital rights grants associated with the model description; If the user level application request is granted, based on a comparison of data indicative of digital rights grants associated with the model description and data associated with the user level application request, instantiate an executable associated with the model description.

Example 2 includes the subject matter of Example 1, wherein the predictive analytics model description language includes data representing a predictive analytics model structure, one or more model parameters, and meta information about the predictive analytics model.

Example 3 includes the subject matter of Example 1 or Example 2, wherein the trust predictive analytics middleware subsystem is launched by a user computing device of the computing system in the trusted execution environment.

Example 4 includes the subject matter of Examples 1 or 2, wherein the trusted predictive analytics middleware subsystem obtains input data associated with a user-level application request to the predictive analytics service, and stores the input data in a trusted memory region of the trusted execution environment. save the

Example 5 includes the subject matter of Examples 1 or 2, wherein the predictive trust analysis middleware subsystem is configured to: based on a comparison of data indicative of digital rights grants associated with the model description and data associated with a user-level application request, If the application request is granted, decrypt the model description.

Example 6 includes the subject matter of Example 5, wherein the trust prediction analysis middleware subsystem stores the decrypted model description in a trusted memory area of the trusted execution environment.

Example 7 includes the subject matter of Examples 1 or 2, wherein the trust predictive analytics middleware subsystem generates the executable based on the predictive analytics execution primitives and the model description.

Example 8 includes the subject matter of Examples 1 or 2, wherein the trust predictive analytics middleware subsystem includes a digital rights management (DRM) subsystem to verify digital rights associated with the predictive analytics service.

Example 9 includes the subject matter of Examples 1 or 2, wherein the trust predictive analytics middleware subsystem interprets the model description using the model description language, and generates a model structure for the predictive analytics service based on the model description; It includes a model description assembler module to transform the model structure into an executable based on the predictive analysis execution primitives.

Example 10 includes a method of providing a trusted predictive analytics service, the method comprising: describing, by a computing system, a plurality of different predictive analytics models using a common model description language; and in response to the user-level application request for the predictive analytics service, determining, in a trusted execution environment of the computing system, a model description for the predictive analytics model, the model description being generated by the predictive analytics model description language; comparing data associated with the user level application request to data representing digital rights grants associated with the model description; and instantiating an executable associated with the model description if the user level application request is granted based on a comparison of the data indicative of digital rights grants associated with the model description and the data associated with the user level application request.

Example 11 includes the subject matter of Example 10, comprising launching a trust predictive analytics middleware subsystem in a trusted execution environment.

Example 12 includes the subject matter of Example 10, comprising obtaining input data associated with a user-level application request for a predictive analytics service, and storing the input data in a trusted memory area of the trusted execution environment.

Example 13 includes the subject matter of Example 10, wherein, based on a comparison of data representing digital rights grants associated with the model description and data associated with the user level application request, if the user level application request is granted, decrypt the model description including the steps of

Example 14 includes the subject matter of Example 13, comprising storing the decrypted model description in a trusted memory area of the trusted execution environment.

Example 15 includes the subject matter of Example 10, comprising generating the executable based on the predictive analysis execution primitive and the model description.

Example 16 includes the subject matter of Example 10, comprising verifying, by a digital rights management (DRM) subsystem, digital rights associated with the predictive analytics service.

Example 17 includes the subject matter of Example 10, interpreting a model description using a model description language, generating a model structure for a predictive analytics service based on the model description, and based on the predictive analytics execution primitives. converting the model structure into an executable file.

Example 18 includes the subject matter of Example 10, comprising describing predictive analytics models using a description language comprising data representing a predictive analytics model structure, one or more model parameters, and meta information about the predictive analytics model. include

Example 19 includes one or more non-transitory machine-readable storage media comprising a plurality of stored instructions, wherein in response to being executed, cause the computing device to perform the method of any of Examples 10-18. .

Example 20 includes a computing system that provides predictive trust analysis services, the system comprising means for performing the method of any one of claims 10-18.

Claims (25)

A computing system comprising one or more computing devices, the computing system providing a predictive trust analysis service, the computing system comprising:
a user computing device comprising a user-level application; and
a trust predictive analytics middleware computing device, wherein the trust predictive analytics middleware computing device is adapted to receive user level application requests from the user computing device via a network;
including,
The trust predictive analytics middleware computing device includes a trust predictive analytics middleware subsystem, the trust predictive analytics middleware subsystem responsive to a user level application request for predictive analytics service, in the trust execution environment of the computing system, the computing make the system
determine a model description for the predictive analytics model, wherein the model description is generated by a predictive analytics model description language, the predictive analytics model description language describing a plurality of different predictive analytics models using a common language; including data representing the predictive analytics model structure, one or more model parameters, and meta information about the model;
compare data associated with the user level application request to data representing digital rights grants associated with the model description;
If the user level application request is granted by the trust prediction analysis middleware subsystem based on a comparison of data indicative of digital rights grants associated with the model description and data associated with the user level application request, the model description and adapted to instantiate the associated executable
computing system. delete The computing system of claim 1 , wherein the trust prediction analysis middleware subsystem is launched by a user computing device of the computing system in the trusted execution environment. The computing system of claim 1 , wherein the trusted predictive analytics middleware subsystem obtains input data associated with a user-level application request to the predictive analytics service and stores the input data in a trusted memory area of the trusted execution environment. . 2. The method of claim 1, wherein the trust prediction analysis middleware subsystem is configured to: if the user level application request is granted based on a comparison of data indicative of digital rights grants associated with the model description and data associated with the user level application request. , a computing system that decrypts the model description. 6. The computing system of claim 5, wherein the trust prediction analysis middleware subsystem stores the decrypted model description in a trusted memory area of the trusted execution environment. The computing system of claim 1 , wherein the trust predictive analytics middleware subsystem generates the executable based on predictive analytics execution primitives and the model description. The computing system of claim 1 , wherein the trust predictive analytics middleware subsystem comprises a digital rights management (DRM) subsystem for validating digital rights associated with the predictive analytics service. The predictive analytics execution primitive of claim 1, wherein the trust predictive analytics middleware subsystem interprets the model description using the model description language, generates a model structure for the predictive analytics service based on the model description, and and a model description assembler module for converting the model structure into an executable file based on A method of providing a reliable predictive analytics service, comprising: a computing system:
describing a plurality of different predictive analytics models using a common model description language; and
In response to a user-level application request for a predictive analytics service, in the trusted execution environment of the computing system,
determining a model description for the predictive analytics model, wherein the model description is generated by a predictive analytics model description language, the predictive analytics model description language comprising: a predictive analytics model structure for the predictive analytics model; one or more model parameters; and Contains data representing meta information -;
comparing data associated with the user level application request to data representing digital rights grants associated with the model description; and
instantiating an executable associated with the model description if the user level application request is granted based on a comparison of data indicative of digital rights grants associated with the model description and data associated with the user level application request;
How to include. 11. The method of claim 10, comprising launching the trusted predictive analytics middleware subsystem in the trusted execution environment. 11. The method of claim 10, comprising: obtaining input data associated with a user level application request to the predictive analytics service; and storing the input data in a trusted memory area of the trusted execution environment. 11. The method of claim 10, further comprising: if the user level application request is granted based on a comparison of data indicative of digital rights grants associated with the model description and data associated with the user level application request, then decoding the model description How to include. 14. The method of claim 13, comprising storing the decrypted model description in a trusted memory area of the trusted execution environment. 11. The method of claim 10, comprising generating the executable based on predictive analytics execution primitives and the model description. 11. The method of claim 10, comprising verifying, by a digital rights management (DRM) subsystem, digital rights associated with the predictive analytics service. 11. The method of claim 10, further comprising: interpreting the model description using the model description language; generating a model structure for the predictive analytics service based on the model description; and the model based on predictive analytics execution primitives. A method comprising converting the structure into an executable file. delete One or more non-transitory machine-readable storage media having stored thereon a plurality of instructions, the plurality of instructions, in response to being executed, causing a computing system to:
describe a plurality of different predictive analytics models using a common model description language;
In response to a user-level application request for a predictive analytics service, in the trusted execution environment of the computing system,
determine a model description for the predictive analytics model, wherein the model description is generated by a predictive analytics model description language, the predictive analytics model description language comprising: a predictive analytics model structure, one or more model parameters, and a meta for the predictive analytics model; contains data representing information;
compare data associated with the user level application request to data representing digital rights grants associated with the model description;
instantiate an executable associated with the model description if the user level application request is granted based on a comparison of data indicative of digital rights grants associated with the model description and data associated with the user level application request; One or more non-transitory machine-readable storage media. 20. The one or more non-transitory machine-readable storage media of claim 19, wherein the instructions cause the computing system to launch a trust prediction analysis middleware subsystem in the trusted execution environment. 20. The method of claim 19, wherein the instructions cause the computing system to obtain input data associated with a user level application request to the predictive analytics service and store the input data in a trusted memory area of the trusted execution environment. One or more non-transitory machine-readable storage media. 20. The method of claim 19, wherein the instructions cause the computing system to, based on a comparison of data indicative of digital rights grants associated with the model description and data associated with the user-level application request, if the user-level application request is granted. , decrypt the model description and store the decrypted model description in a trusted memory area of the trusted execution environment. 20. The one or more non-transitory machine-readable storage media of claim 19, wherein the instructions cause the computing system to generate the executable based on a predictive analysis execution primitive and the model description. 20. The predictive analytics execution primitive of claim 19, wherein the instructions cause the computing system to interpret the model description using the model description language, generate a model structure for the predictive analytics service based on the model description, and one or more non-transitory machine-readable storage media to convert the model structure into an executable file based on delete

Images