1*4d7e907cSAndroid Build Coastguard Worker/* 2*4d7e907cSAndroid Build Coastguard Worker * Copyright 2016 The Android Open Source Project 3*4d7e907cSAndroid Build Coastguard Worker * 4*4d7e907cSAndroid Build Coastguard Worker * Licensed under the Apache License, Version 2.0 (the "License"); 5*4d7e907cSAndroid Build Coastguard Worker * you may not use this file except in compliance with the License. 6*4d7e907cSAndroid Build Coastguard Worker * You may obtain a copy of the License at 7*4d7e907cSAndroid Build Coastguard Worker * 8*4d7e907cSAndroid Build Coastguard Worker * http://www.apache.org/licenses/LICENSE-2.0 9*4d7e907cSAndroid Build Coastguard Worker * 10*4d7e907cSAndroid Build Coastguard Worker * Unless required by applicable law or agreed to in writing, software 11*4d7e907cSAndroid Build Coastguard Worker * distributed under the License is distributed on an "AS IS" BASIS, 12*4d7e907cSAndroid Build Coastguard Worker * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13*4d7e907cSAndroid Build Coastguard Worker * See the License for the specific language governing permissions and 14*4d7e907cSAndroid Build Coastguard Worker * limitations under the License. 15*4d7e907cSAndroid Build Coastguard Worker */ 16*4d7e907cSAndroid Build Coastguard Worker 17*4d7e907cSAndroid Build Coastguard Workerpackage [email protected]; 18*4d7e907cSAndroid Build Coastguard Worker 19*4d7e907cSAndroid Build Coastguard Workerimport [email protected]; 20*4d7e907cSAndroid Build Coastguard Worker 21*4d7e907cSAndroid Build Coastguard Workerimport ISoundTriggerHwCallback; 22*4d7e907cSAndroid Build Coastguard Worker 23*4d7e907cSAndroid Build Coastguard Workerinterface ISoundTriggerHw { 24*4d7e907cSAndroid Build Coastguard Worker 25*4d7e907cSAndroid Build Coastguard Worker /** 26*4d7e907cSAndroid Build Coastguard Worker * Sound trigger implementation descriptor read by the framework via 27*4d7e907cSAndroid Build Coastguard Worker * getProperties(). Used by SoundTrigger service to report to applications 28*4d7e907cSAndroid Build Coastguard Worker * and manage concurrency and policy. 29*4d7e907cSAndroid Build Coastguard Worker */ 30*4d7e907cSAndroid Build Coastguard Worker struct Properties { 31*4d7e907cSAndroid Build Coastguard Worker /** Implementor name */ 32*4d7e907cSAndroid Build Coastguard Worker string implementor; 33*4d7e907cSAndroid Build Coastguard Worker /** Implementation description */ 34*4d7e907cSAndroid Build Coastguard Worker string description; 35*4d7e907cSAndroid Build Coastguard Worker /** Implementation version */ 36*4d7e907cSAndroid Build Coastguard Worker uint32_t version; 37*4d7e907cSAndroid Build Coastguard Worker /** 38*4d7e907cSAndroid Build Coastguard Worker * Unique implementation ID. The UUID must change with each version of 39*4d7e907cSAndroid Build Coastguard Worker the engine implementation */ 40*4d7e907cSAndroid Build Coastguard Worker Uuid uuid; 41*4d7e907cSAndroid Build Coastguard Worker /** Maximum number of concurrent sound models loaded */ 42*4d7e907cSAndroid Build Coastguard Worker uint32_t maxSoundModels; 43*4d7e907cSAndroid Build Coastguard Worker /** Maximum number of key phrases */ 44*4d7e907cSAndroid Build Coastguard Worker uint32_t maxKeyPhrases; 45*4d7e907cSAndroid Build Coastguard Worker /** Maximum number of concurrent users detected */ 46*4d7e907cSAndroid Build Coastguard Worker uint32_t maxUsers; 47*4d7e907cSAndroid Build Coastguard Worker /** All supported modes. e.g RecognitionMode.VOICE_TRIGGER */ 48*4d7e907cSAndroid Build Coastguard Worker uint32_t recognitionModes; 49*4d7e907cSAndroid Build Coastguard Worker /** Supports seamless transition from detection to capture */ 50*4d7e907cSAndroid Build Coastguard Worker bool captureTransition; 51*4d7e907cSAndroid Build Coastguard Worker /** Maximum buffering capacity in ms if captureTransition is true */ 52*4d7e907cSAndroid Build Coastguard Worker uint32_t maxBufferMs; 53*4d7e907cSAndroid Build Coastguard Worker /** Supports capture by other use cases while detection is active */ 54*4d7e907cSAndroid Build Coastguard Worker bool concurrentCapture; 55*4d7e907cSAndroid Build Coastguard Worker /** Returns the trigger capture in event */ 56*4d7e907cSAndroid Build Coastguard Worker bool triggerInEvent; 57*4d7e907cSAndroid Build Coastguard Worker /** 58*4d7e907cSAndroid Build Coastguard Worker * Rated power consumption when detection is active with TDB 59*4d7e907cSAndroid Build Coastguard Worker * silence/sound/speech ratio */ 60*4d7e907cSAndroid Build Coastguard Worker uint32_t powerConsumptionMw; 61*4d7e907cSAndroid Build Coastguard Worker }; 62*4d7e907cSAndroid Build Coastguard Worker 63*4d7e907cSAndroid Build Coastguard Worker 64*4d7e907cSAndroid Build Coastguard Worker /** 65*4d7e907cSAndroid Build Coastguard Worker * Base sound model descriptor. This struct is the header of a larger block 66*4d7e907cSAndroid Build Coastguard Worker * passed to loadSoundModel() and contains the binary data of the 67*4d7e907cSAndroid Build Coastguard Worker * sound model. 68*4d7e907cSAndroid Build Coastguard Worker */ 69*4d7e907cSAndroid Build Coastguard Worker struct SoundModel { 70*4d7e907cSAndroid Build Coastguard Worker /** Model type. e.g. SoundModelType.KEYPHRASE */ 71*4d7e907cSAndroid Build Coastguard Worker SoundModelType type; 72*4d7e907cSAndroid Build Coastguard Worker /** Unique sound model ID. */ 73*4d7e907cSAndroid Build Coastguard Worker Uuid uuid; 74*4d7e907cSAndroid Build Coastguard Worker /** 75*4d7e907cSAndroid Build Coastguard Worker * Unique vendor ID. Identifies the engine the sound model 76*4d7e907cSAndroid Build Coastguard Worker * was build for */ 77*4d7e907cSAndroid Build Coastguard Worker Uuid vendorUuid; 78*4d7e907cSAndroid Build Coastguard Worker /** Opaque data transparent to Android framework */ 79*4d7e907cSAndroid Build Coastguard Worker vec<uint8_t> data; 80*4d7e907cSAndroid Build Coastguard Worker }; 81*4d7e907cSAndroid Build Coastguard Worker 82*4d7e907cSAndroid Build Coastguard Worker /** Key phrase descriptor */ 83*4d7e907cSAndroid Build Coastguard Worker struct Phrase { 84*4d7e907cSAndroid Build Coastguard Worker /** Unique keyphrase ID assigned at enrollment time */ 85*4d7e907cSAndroid Build Coastguard Worker uint32_t id; 86*4d7e907cSAndroid Build Coastguard Worker /** Recognition modes supported by this key phrase */ 87*4d7e907cSAndroid Build Coastguard Worker uint32_t recognitionModes; 88*4d7e907cSAndroid Build Coastguard Worker /** List of users IDs associated with this key phrase */ 89*4d7e907cSAndroid Build Coastguard Worker vec<uint32_t> users; 90*4d7e907cSAndroid Build Coastguard Worker /** Locale - Java Locale style (e.g. en_US) */ 91*4d7e907cSAndroid Build Coastguard Worker string locale; 92*4d7e907cSAndroid Build Coastguard Worker /** Phrase text in UTF-8 format. */ 93*4d7e907cSAndroid Build Coastguard Worker string text; 94*4d7e907cSAndroid Build Coastguard Worker }; 95*4d7e907cSAndroid Build Coastguard Worker 96*4d7e907cSAndroid Build Coastguard Worker /** 97*4d7e907cSAndroid Build Coastguard Worker * Specialized sound model for key phrase detection. 98*4d7e907cSAndroid Build Coastguard Worker * Proprietary representation of key phrases in binary data must match 99*4d7e907cSAndroid Build Coastguard Worker * information indicated by phrases field 100*4d7e907cSAndroid Build Coastguard Worker */ 101*4d7e907cSAndroid Build Coastguard Worker struct PhraseSoundModel { 102*4d7e907cSAndroid Build Coastguard Worker /** Common part of sound model descriptor */ 103*4d7e907cSAndroid Build Coastguard Worker SoundModel common; 104*4d7e907cSAndroid Build Coastguard Worker /** List of descriptors for key phrases supported by this sound model */ 105*4d7e907cSAndroid Build Coastguard Worker vec<Phrase> phrases; 106*4d7e907cSAndroid Build Coastguard Worker }; 107*4d7e907cSAndroid Build Coastguard Worker 108*4d7e907cSAndroid Build Coastguard Worker /** 109*4d7e907cSAndroid Build Coastguard Worker * Configuration for sound trigger capture session passed to 110*4d7e907cSAndroid Build Coastguard Worker * startRecognition() method 111*4d7e907cSAndroid Build Coastguard Worker */ 112*4d7e907cSAndroid Build Coastguard Worker struct RecognitionConfig { 113*4d7e907cSAndroid Build Coastguard Worker /** 114*4d7e907cSAndroid Build Coastguard Worker * IO handle that will be used for capture. N/A if captureRequested 115*4d7e907cSAndroid Build Coastguard Worker * is false */ 116*4d7e907cSAndroid Build Coastguard Worker AudioIoHandle captureHandle; 117*4d7e907cSAndroid Build Coastguard Worker /** Input device requested for detection capture */ 118*4d7e907cSAndroid Build Coastguard Worker AudioDevice captureDevice; 119*4d7e907cSAndroid Build Coastguard Worker /** Capture and buffer audio for this recognition instance */ 120*4d7e907cSAndroid Build Coastguard Worker bool captureRequested; 121*4d7e907cSAndroid Build Coastguard Worker /** Configuration for each key phrase */ 122*4d7e907cSAndroid Build Coastguard Worker vec<PhraseRecognitionExtra> phrases; 123*4d7e907cSAndroid Build Coastguard Worker /** Opaque capture configuration data transparent to the framework */ 124*4d7e907cSAndroid Build Coastguard Worker vec<uint8_t> data; 125*4d7e907cSAndroid Build Coastguard Worker }; 126*4d7e907cSAndroid Build Coastguard Worker 127*4d7e907cSAndroid Build Coastguard Worker 128*4d7e907cSAndroid Build Coastguard Worker /** 129*4d7e907cSAndroid Build Coastguard Worker * Retrieve implementation properties. 130*4d7e907cSAndroid Build Coastguard Worker * @return retval Operation completion status: 0 in case of success, 131*4d7e907cSAndroid Build Coastguard Worker * -ENODEV in case of initialization error. 132*4d7e907cSAndroid Build Coastguard Worker * @return properties A Properties structure containing implementation 133*4d7e907cSAndroid Build Coastguard Worker * description and capabilities. 134*4d7e907cSAndroid Build Coastguard Worker */ 135*4d7e907cSAndroid Build Coastguard Worker getProperties() generates (int32_t retval, Properties properties); 136*4d7e907cSAndroid Build Coastguard Worker 137*4d7e907cSAndroid Build Coastguard Worker /** 138*4d7e907cSAndroid Build Coastguard Worker * Load a sound model. Once loaded, recognition of this model can be 139*4d7e907cSAndroid Build Coastguard Worker * started and stopped. Only one active recognition per model at a time. 140*4d7e907cSAndroid Build Coastguard Worker * The SoundTrigger service must handle concurrent recognition requests by 141*4d7e907cSAndroid Build Coastguard Worker * different users/applications on the same model. 142*4d7e907cSAndroid Build Coastguard Worker * The implementation returns a unique handle used by other functions 143*4d7e907cSAndroid Build Coastguard Worker * (unloadSoundModel(), startRecognition(), etc... 144*4d7e907cSAndroid Build Coastguard Worker * @param soundModel A SoundModel structure describing the sound model to 145*4d7e907cSAndroid Build Coastguard Worker * load. 146*4d7e907cSAndroid Build Coastguard Worker * @param callback The callback interface on which the soundmodelCallback() 147*4d7e907cSAndroid Build Coastguard Worker * method will be called upon completion. 148*4d7e907cSAndroid Build Coastguard Worker * @param cookie The value of the cookie argument passed to the completion 149*4d7e907cSAndroid Build Coastguard Worker * callback. This unique context information is assigned and 150*4d7e907cSAndroid Build Coastguard Worker * used only by the framework. 151*4d7e907cSAndroid Build Coastguard Worker * @return retval Operation completion status: 0 in case of success, 152*4d7e907cSAndroid Build Coastguard Worker * -EINVAL in case of invalid sound model (e.g 0 data size), 153*4d7e907cSAndroid Build Coastguard Worker * -ENOSYS in case of invalid operation (e.g max number of 154*4d7e907cSAndroid Build Coastguard Worker * models exceeded), 155*4d7e907cSAndroid Build Coastguard Worker * -ENOMEM in case of memory allocation failure, 156*4d7e907cSAndroid Build Coastguard Worker * -ENODEV in case of initialization error. 157*4d7e907cSAndroid Build Coastguard Worker * @return modelHandle A unique handle assigned by the HAL for use by the 158*4d7e907cSAndroid Build Coastguard Worker * framework when controlling activity for this sound model. 159*4d7e907cSAndroid Build Coastguard Worker */ 160*4d7e907cSAndroid Build Coastguard Worker loadSoundModel(SoundModel soundModel, 161*4d7e907cSAndroid Build Coastguard Worker ISoundTriggerHwCallback callback, 162*4d7e907cSAndroid Build Coastguard Worker CallbackCookie cookie) 163*4d7e907cSAndroid Build Coastguard Worker generates (int32_t retval, SoundModelHandle modelHandle); 164*4d7e907cSAndroid Build Coastguard Worker 165*4d7e907cSAndroid Build Coastguard Worker /** 166*4d7e907cSAndroid Build Coastguard Worker * Load a key phrase sound model. Once loaded, recognition of this model can 167*4d7e907cSAndroid Build Coastguard Worker * be started and stopped. Only one active recognition per model at a time. 168*4d7e907cSAndroid Build Coastguard Worker * The SoundTrigger service must handle concurrent recognition requests by 169*4d7e907cSAndroid Build Coastguard Worker * different users/applications on the same model. 170*4d7e907cSAndroid Build Coastguard Worker * The implementation returns a unique handle used by other functions 171*4d7e907cSAndroid Build Coastguard Worker * (unloadSoundModel(), startRecognition(), etc... 172*4d7e907cSAndroid Build Coastguard Worker * @param soundModel A PhraseSoundModel structure describing the sound model 173*4d7e907cSAndroid Build Coastguard Worker * to load. 174*4d7e907cSAndroid Build Coastguard Worker * @param callback The callback interface on which the soundmodelCallback() 175*4d7e907cSAndroid Build Coastguard Worker * method will be called upon completion. 176*4d7e907cSAndroid Build Coastguard Worker * @param cookie The value of the cookie argument passed to the completion 177*4d7e907cSAndroid Build Coastguard Worker * callback. This unique context information is assigned and 178*4d7e907cSAndroid Build Coastguard Worker * used only by the framework. 179*4d7e907cSAndroid Build Coastguard Worker * @return retval Operation completion status: 0 in case of success, 180*4d7e907cSAndroid Build Coastguard Worker * -EINVAL in case of invalid sound model (e.g 0 data size), 181*4d7e907cSAndroid Build Coastguard Worker * -ENOSYS in case of invalid operation (e.g max number of 182*4d7e907cSAndroid Build Coastguard Worker * models exceeded), 183*4d7e907cSAndroid Build Coastguard Worker * -ENOMEM in case of memory allocation failure, 184*4d7e907cSAndroid Build Coastguard Worker * -ENODEV in case of initialization error. 185*4d7e907cSAndroid Build Coastguard Worker * @return modelHandle A unique handle assigned by the HAL for use by the 186*4d7e907cSAndroid Build Coastguard Worker * framework when controlling activity for this sound model. 187*4d7e907cSAndroid Build Coastguard Worker */ 188*4d7e907cSAndroid Build Coastguard Worker loadPhraseSoundModel(PhraseSoundModel soundModel, 189*4d7e907cSAndroid Build Coastguard Worker ISoundTriggerHwCallback callback, 190*4d7e907cSAndroid Build Coastguard Worker CallbackCookie cookie) 191*4d7e907cSAndroid Build Coastguard Worker generates (int32_t retval, SoundModelHandle modelHandle); 192*4d7e907cSAndroid Build Coastguard Worker 193*4d7e907cSAndroid Build Coastguard Worker /** 194*4d7e907cSAndroid Build Coastguard Worker * Unload a sound model. A sound model may be unloaded to make room for a 195*4d7e907cSAndroid Build Coastguard Worker * new one to overcome implementation limitations. 196*4d7e907cSAndroid Build Coastguard Worker * @param modelHandle the handle of the sound model to unload 197*4d7e907cSAndroid Build Coastguard Worker * @return retval Operation completion status: 0 in case of success, 198*4d7e907cSAndroid Build Coastguard Worker * -ENOSYS if the model is not loaded, 199*4d7e907cSAndroid Build Coastguard Worker * -ENODEV in case of initialization error. 200*4d7e907cSAndroid Build Coastguard Worker */ 201*4d7e907cSAndroid Build Coastguard Worker unloadSoundModel(SoundModelHandle modelHandle) 202*4d7e907cSAndroid Build Coastguard Worker generates (int32_t retval); 203*4d7e907cSAndroid Build Coastguard Worker 204*4d7e907cSAndroid Build Coastguard Worker /** 205*4d7e907cSAndroid Build Coastguard Worker * Start recognition on a given model. Only one recognition active 206*4d7e907cSAndroid Build Coastguard Worker * at a time per model. Once recognition succeeds of fails, the callback 207*4d7e907cSAndroid Build Coastguard Worker * is called. 208*4d7e907cSAndroid Build Coastguard Worker * @param modelHandle the handle of the sound model to use for recognition 209*4d7e907cSAndroid Build Coastguard Worker * @param config A RecognitionConfig structure containing attributes of the 210*4d7e907cSAndroid Build Coastguard Worker * recognition to perform 211*4d7e907cSAndroid Build Coastguard Worker * @param callback The callback interface on which the recognitionCallback() 212*4d7e907cSAndroid Build Coastguard Worker * method must be called upon recognition. 213*4d7e907cSAndroid Build Coastguard Worker * @param cookie The value of the cookie argument passed to the recognition 214*4d7e907cSAndroid Build Coastguard Worker * callback. This unique context information is assigned and 215*4d7e907cSAndroid Build Coastguard Worker * used only by the framework. 216*4d7e907cSAndroid Build Coastguard Worker * @return retval Operation completion status: 0 in case of success, 217*4d7e907cSAndroid Build Coastguard Worker * -EINVAL in case of invalid recognition attributes, 218*4d7e907cSAndroid Build Coastguard Worker * -ENOSYS in case of invalid model handle, 219*4d7e907cSAndroid Build Coastguard Worker * -ENOMEM in case of memory allocation failure, 220*4d7e907cSAndroid Build Coastguard Worker * -ENODEV in case of initialization error. 221*4d7e907cSAndroid Build Coastguard Worker */ 222*4d7e907cSAndroid Build Coastguard Worker startRecognition(SoundModelHandle modelHandle, 223*4d7e907cSAndroid Build Coastguard Worker RecognitionConfig config, 224*4d7e907cSAndroid Build Coastguard Worker ISoundTriggerHwCallback callback, 225*4d7e907cSAndroid Build Coastguard Worker CallbackCookie cookie) 226*4d7e907cSAndroid Build Coastguard Worker generates (int32_t retval); 227*4d7e907cSAndroid Build Coastguard Worker 228*4d7e907cSAndroid Build Coastguard Worker /** 229*4d7e907cSAndroid Build Coastguard Worker * Stop recognition on a given model. 230*4d7e907cSAndroid Build Coastguard Worker * The implementation must not call the recognition callback when stopped 231*4d7e907cSAndroid Build Coastguard Worker * via this method. 232*4d7e907cSAndroid Build Coastguard Worker * @param modelHandle The handle of the sound model to use for recognition 233*4d7e907cSAndroid Build Coastguard Worker * @return retval Operation completion status: 0 in case of success, 234*4d7e907cSAndroid Build Coastguard Worker * -ENOSYS in case of invalid model handle, 235*4d7e907cSAndroid Build Coastguard Worker * -ENODEV in case of initialization error. 236*4d7e907cSAndroid Build Coastguard Worker */ 237*4d7e907cSAndroid Build Coastguard Worker stopRecognition(SoundModelHandle modelHandle) 238*4d7e907cSAndroid Build Coastguard Worker generates (int32_t retval); 239*4d7e907cSAndroid Build Coastguard Worker 240*4d7e907cSAndroid Build Coastguard Worker /** 241*4d7e907cSAndroid Build Coastguard Worker * Stop recognition on all models. 242*4d7e907cSAndroid Build Coastguard Worker * @return retval Operation completion status: 0 in case of success, 243*4d7e907cSAndroid Build Coastguard Worker * -ENODEV in case of initialization error. 244*4d7e907cSAndroid Build Coastguard Worker */ 245*4d7e907cSAndroid Build Coastguard Worker stopAllRecognitions() 246*4d7e907cSAndroid Build Coastguard Worker generates (int32_t retval); 247*4d7e907cSAndroid Build Coastguard Worker}; 248