A8 Hybrid Audio SDK for Android

How to install the A8 Audio Recognition for ACTV8me Audio Watermarks and Fingerprints in your Android application.

The ACTV8me Audio Recognition Android SDK allows an app to detect the world around it using audio. We utilize a combination of technologies that trigger user experiences and open up new digital interactions.

I. Getting Started

To use ACTV8me's Audio Watermark & Fingerprint Recognition in an Android application, simply include the following libraries in your project.

Libraries to include:

  • audioHybridSearchSoundCode2Lib-release.aar
  • audioRecordLib-release.aar
  • downloadTaskLib-release.aar
  • hybridSearchCommonLib-release.aar

Finally, before installing, please refer to the references page as there are many useful key fields and definitions available.

II. Using the Hybrid Audio SDK in your Project

Using ACTV8me hybrid audio fingerprint recognition and SoundCode watermark decoder in your application.

Step 1 - Import

Please contact us to get the most up-to-date list of dependencies that need to be imported.

Step 2 - Customize

Overwrite default settings for fingerprint recognition and watermark decoding service and audio recording service.

hsSettings = new AudioHybridSearchSoundCodeSettings();

// Settings for fingerprinting library 
hsSettings.dataDirectory = Environment.getExternalStorageDirectory().getPath() + "/HybridSearchSoundCode";
hsSettings.fileName = "testFP.dat";
hsSettings.url = <This value will be provided to you>;

// Settings for SoundCode watermark decoder library
hsSettings.counterLength = 16;
hsSettings.counterIncrement = 1;
hsSettings.delayAdjustment = +0.6f;

// Audio recorder settings (e.g. use default audio source)
arSettings = new AudioRecordServiceSettings();
arSettings.audioSource = AudioSource.DEFAULT;

Step 3 - Create Listener

Create listener and add your own callback processing functions before starting recognition service:

final AudioHybridSearchSoundCodeListener ahslistener = new AudioHybridSearchSoundCodeListener() {
  
  @Override 
		public void onAudioInitFailed() {
			// Do something if audio initialization is failed
			// (usually it means that the mic is occupied by another application)
		}
  
  	@Override
		public void onError(String message) {
		// Internal error (with error message) 
  	}
	
  	@Override
		public void onSearchCompleted(SearchResult result) {
			// Analyze audio fingerprint recognition result       
			// if (result.getExternalId() == -1) no match detected
			// if (result.getExternalId() > 0)  some  track was detected 
			// (track_ID = result.getExternalId())      
			// Offset in milliseconds from track beginning = result.getShift() 
			// result.getAccuracy()  returns  ResultAccuracy.{UNIQUE, SOLVED, 			
			// CONFLICTING, SELF_CONFLICTING} 
			// if  result.isLocalSearch() == true  then the recognition was done 
			// without server interaction (via local database)
		}
  
  	@Override
		public void onNewDatabaseDownloaded() {
		// This method is called then the new local audio database is loaded 		
		// (usually no action needed)
		}
  
  	@Override
		public void onDetectedSCId(long[] result) {
		// We have recognized SoundCode audio watermark
		// Track ID = result[1]
		// Offset from track beginning (in milliseconds)  =  result[3]
	}
 };

Step 4 - Initialize Service

In your activity or service, initialize context for AudioHybridSearchSoundCode service (usually, in onCreate() method).

// Firstly, initialize context via instance() method:
// 1st parameter 	application or activity context
// 2nd parameter 	if true then the debug mode is enabled
AudioHybridSearchSoundCode.instance(this, false);

// Secondly, prepare the service to run
// 1st parameter 	object of AudioHybridSearchSoundCodeSettings class with 
// 			default or non-default settings
// 2nd parameter 	object of AudioHybridSearchSoundCodeListener class
// 3rd parameter 	object of AudioRecordServiceSettings class with
//			default or non-default settings
// 4th parameter	if true, auto start search upon service creation
AudioHybridSearchSoundCode.instance().prepare(hsSettings, ahslistener, arSettings, false);

Step 5 - Start Listener

To start fingerprint recognition and watermark decoding in your application use startSearch() method.

AudioHybridSearchSoundCode.instance().startSearch();
  • Not necessary if 3-rd parameter of prepare() method is set to "true"
    -- Both services (fingerprint recognition and watermark decoding) will be started simultaneously

Step 6 - Stop Listener

To stop fingerprint recognition and watermark decoding in your application use stopSearch() method.

AudioHybridSearchSoundCode.instance().stopSearch();
  • Both services (fingerprint recognition and watermark decoding) will be stopped simultaneously
    -- You may call start and stop method several times

It is also possible to perform one-time recognition via

AudioHybridSearchSoundCode.instance().singleSearch();

Step 7 - Release

When you no longer need fingerprint recognition and watermark decoding release resources via release() method.

AudioHybridSearchSoundCode.release();

III. References

HIGH-LEVEL Hybrid Audio Fingerprinting and SoundCode Watermark Decoder API

High-level interaction with fingerprint recognition and watermark decoding service is implemented via AudioHybridSearchSoundCode class

To start listening to audio and decoding audio watermarks and recognize audio fingerprints, the application should initialize and start the AudioHybridSearchSoundCode service with an object of AudioHybridSearchSoundCodeSettings class, which contains the system parameters for audio fingerprinting and audio watermarking decoders.

An object of AudioRecordServiceSettings class contains audio recording parameters, e.g. audio recording source.

Audio Hybrid Search SoundCode Settings

Key FieldsDefinition
dataDirectoryDirectory where the audio recognition service shall store intermediate files (local fingerprint database and overlapping map). It is a good idea to set this path to something like: Environment.getExternalStorageDirectory().getPath() + "/NameOfYourApplication";
fileNameName of the local audio fingerprint database file (e.g. default: "testFP.dat")
url
urlBase URL for server installation of the audio fingerprint recognition service
counterLengthLength of "counter" field of SoundCode watermark embedder (e.g. 16)
counterInctementCounter increment parameter for SoundCode watermark embedder (e.g. 1)
delayAdjustmentDelay adjustment constant (sec.) takes into account audio buffering and processing delay to provide accurate timing offset measurement with audio watermarking decoder (default value is 0.6)

Please refer to the table below for key fields (parameters) of the AudioHybridSearchSoundCodeSettings class.

Audio Record Service Settings

Please refer to the table below for key fields (parameters) of the AudioRecordServiceSettings class.

audioSource Selects audio recording source.
Permitted values: AudioSource.DEFAULT (recommended), AudioSource.MIC, AudioSource.CAMCORDER, AudioSource.UNPROCESSED.

Ref: Android MediaRecorder.AudioSource class documentation.

There are three major methods to control AudioHybridSearchSoundCode service: startSearch(), stopSearch(), singleSearch(). To receive information from AudioHybridSearchSoundCode service use AudioHybridSearchSoundCodeListener interface. There are 5 callback functions defined in AudioHybridSearchSoundCodeListener interface:
public void onAudioInitFailed()is invoked when audio initialization has failed. Usually, it means that the device microphone is occupied by another application. To avoid confusion make sure that you let the user know that the mic is busy and audio recognition is not going to work.
onError(String message)is invoked when an internal error occurred in fingerprint recognition or watermark decoding service. The function parameter is the error message string - it is a good idea to display it to the user or save to logs.
public void onSearchCompleted(SearchResult result)is invoked after each audio fingerprint recognition attempt (no matter successful or unsuccessful). The result of recognition is returned in the object of class SearchResult. The SearchResult class has the following methods:
int getExternalId()returns the ID of the recognized track (>0) or -1 if no track is recognized
(please note: 0 is reserved value).
int getShift()returns the timing offset from the track beginning (in milliseconds)
ResultAccuracy getAccuracy()returns the overlapping status of the recognized audio fragment; there are four possible values of ResultAccuracy:
ResultAccuracy.UNIQUEreturned when the recognized audio fragment (~5 sec) is absolutely unique and there is no similar fragment in the same track or in other active tracks in the server database.
ResultAccuracy.SOLVEDreturned when the recognized audio fragment (~5 sec) is not unique, so there might exist a similar audio fragment in one of active audio tracks in the server database; however, based on the recognition history for a particular mobile device the recognition service can resolve this ambiguity with very high level of confidence.
ResultAccuracy.SELF_CONFLICTINGreturned when the recognized audio fragment (~5 sec) is not unique, and there might exist a similar audio fragment, but only in the same audio/video track. In other words, the recognized track ID is definitely correct, but the timing offset may be incorrect.
ResultAccuracy.CONFLICTINGreturned when the recognized audio fragment (~5 sec) is not unique, and there might exist a similar audio fragment in one of active audio tracks in the server database. In other words, the recognized track ID and/or the timing offset might be wrong.

UNIQUE and SOLVED results can be safely used in all situations. SELF_CONFLICTING result can be safely used if the application presents the information (widgets, actions) for entire audio/video track (e.g. TV commercial). CONFLICTING results should be used with caution if the application is sensitive to recognition mistakes (e.g. due to the same music track in different episodes of a TV series).

However, in all cases (even SELF_CONFLICTING and CONFLICTING) the recognition service provides the most likely track ID and the timing offset, so in some applications ResultAccuracy can be simply ignored.
public void onNewDatabaseDownloaded()is invoked when the hybrid audio fingerprint recognition service updates local audio fingerprint database (e.g. after several consecutive recognitions of the same track ID). Usually, no action is needed in application level in such a case.
is invoked when SoundCode watermarking signal is detected. The result of SoundCode decoding shall be returned as long array result, where:

result[1] - contains "Content ID" field
result[2] - contains "Counter" field
result[3] - contains "timestamp" (in milliseconds)

Note that onDetectedSCId() is only invoked then some watermarked content is recognized (unlike onSearchCompleted(), which is invoked at every recognition attempt).