The ACTV8me Audio Recognition iOS SDK allows an app to detect the world around it using audio. We utilize a combination of technologies that trigger user experiences and open up new digital interactions.

I. Getting Started

To use ACTV8me's Audio Watermark & Fingerprint Recognition in an iOS application, simply include the following into your project:

1. The library, header and source files for audio fingerprinting:

• Header files from \FPCore\Include*.h
• Native library \FPCore\FPLibU.a
• Objective-C header and source files (TFSearchManager and support classes) from
  \TFMHAL*.m
  \TFMHAL\CoreData*.h

2. The library, header and source files for SoundCode audio watermarking:

• Header files from \SCS\SCSCore\Include*.h
• Native library \SCS\SCSCore\SoundCodeLibU.a
• Objective-C header/source files (SCSSearchManager and support headers) from
  \SCS*.m
  \SCS*.h

II. Using the Hybrid Audio SDK in your Project

The ins and outs of the hybrid audio fingerprint recognition and SoundCode watermark decoder for your app.

We shall use Swift language to illustrate the basic usage example. Using TFSearchManager and SCSSearchManager in Objective-C application is essentially the same with minor syntax modifications.

Step 1 - Headers

In your bridging header file (YourModuleName-Bridging-Header.h), import two headers:

#import "TFSearchManager.h"
#import "SCSSearchManager.h"

Step 2 - Register Observers

In your view loading function, register observers for TFSearchManager and SCSSearchManager notifications (and define your notification processing functions somewhere), e.g.:

NotificationCenter.default.addObserver(self, selector:#selector(searchStateDidChanged), name: NSNotification.Name(rawValue: TF_SEARCH_STATE_NOTIFICATION), object: nil)
NotificationCenter.default.addObserver(self, selector:#selector(searchResultNotification), name: NSNotification.Name(rawValue: TF_SEARCH_RESULT_NOTIFICATION), object: nil)
NotificationCenter.default.addObserver(self, selector:#selector(searchErrorNotification), name: NSNotification.Name(rawValue: TF_SEARCH_ERROR_NOTIFICATION), object: nil)
NotificationCenter.default.addObserver(self, selector:#selector(eventsSCSSearchManger), name: NSNotification.Name(rawValue: SCS_SEARCH_MANAGER_NOTIFICATION), object: nil)

then start audio recording:

TFSearchManager.defaultManager().startRecording()

Example of audio fingerprint recognition result processing function:

func searchResultNotification(notic:NSNotification) {
	if let result: AnyObject = notic.userInfo?[TF_SEARCH_RESULT_KEY] as AnyObject? {
		let tfResult:TFSearchResult = (result as! TFSearchResult)
		// Track detected:
		// tfResult.externalId 
		// tfResult.shift
		// tfResult.reliability 
	}
	else {
		// No track ID detected
	}
}

Example of SoundCode audio watermark detection processing function:

func eventsSCSSearchManger(notic:NSNotification) {
	if let result: AnyObject = notic.userInfo?[SCS_SEARCH_MANAGER_NOTIFICATION_RESULT_KEY] as AnyObject? { 
		// to get track ID: (result as! UInt)  
		// and then check timing offset (it should be ready too)
		if let result: AnyObject = notic.userInfo?[SCS_SEARCH_MANAGER_NOTIFICATION_OFFSET_KEY] as AnyObject?  {
			// get timing offset (in seconds):  (result as! UInt) 
		}
	}
 }

Step 3 - Start Service

To start audio fingerprint recognition service in your application use startSearching() method (TFSearchManager):

TFSearchManager.defaultManager().startSearching()

To start SoundCode audio watermark detection service use startSearchingManager() method (SCSSearchManager):

SCSSearchManager.sharedManager().startSearchingManager()

Step 4 - Stop Service

To stop audio fingerprint recognition service in your application use startSearching() method (TFSearchManager):

TFSearchManager.defaultManager().stopSearching()

To stop SoundCode audio watermark detection service use stopSearchingManager() method (SCSSearchManager):

SCSSearchManager.sharedManager().stopSearchingManager()

It is also possible to perform one-time audio fingerprint recognition via sendSearchRequestWithComplete (TFSearchManager), and one-time audio watermark detection attempt via singleSearchingManager (SCSSearchManager).

• Note: In Swift 3 methods defaultManager() and sharedManager() will be automatically shortened to default() and shared() during Objective-C to Swift import.

Tip: In iOS platform, it is not allowed to vibrate the phone during active audio recording session. This may present a challenge for some SecondScreen applications that rely on audio-based automatic content recognition. A typical SecondScreen app is used while watching TV, therefore to attract the viewer attention it is often preferable to use vibration instead of sound notifications. To deal with this issue TFSearchManager has a helper function delayRecording(), which temporarily suspends audio recording. Therefore, the vibration alert can be implemented as illustrated below:

TFSearchManager.defaultManager().delayRecording(0.2)  
AudioServicesPlayAlertSound(SystemSoundID(kSystemSoundID_Vibrate))

III. High-level Hybrid Audio Fingerprinting and SoundCode Watermark API

API for Fingerprinting Service

High-level interaction with fingerprint recognition service is implemented via TFSearchManager class. It should be used via singleton defaultManager. There are two possible use-cases of TFSearchManager:

1. Continuous mode: In this mode, the application starts audio recognition at the beginning of app session and stops the process at the end of a session using startSearching/stopSearching methods.
2. Manual mode: In this mode, the application only starts audio recording at the beginning of session using startRecording and performs one-time initiation of audio recognition requests during the app session via sendSearchRequestWithComplete method. Audio recording can be stopped by stopRecording method.

TFSearchManager

The TFSearchManager can broadcast information about state changes, recognition results, and errors via notifications. In particular, TFSearchManager can send the following Notifications:

class TFSearchResult:

Class for storing results of recognition; it has the following properties:

enum TFReliabilityCode:

Notes: TFReliabilityCodeUnique and TFReliabilityCodeResolved results can be safely used in all situations. TFReliabilityCodeSelfConfilcting result can be safely used if the application presents the information (widgets, actions) for the entire audio/video track (e.g. TV commercial). TFReliabilityCodeConflicting results should be used with caution if the application is sensitive to recognition mistakes (e.g. due to the same music track in different episodes of a TV series). However, in all cases (even TFReliabilityCodeSelfConfilcting and TFReliabilityCodeConflicting) the recognition service provides the most likely track ID and the timing offset, so in some applications ResultAccuracy can be simply ignored.

API for SoundCode Audio Watermark Detection Service

High-level interaction with SoundCode watermark detection service is implemented via object of SCSSearchManager class. It should be used via singleton sharedManager. Similar to TFSearchManager, there are two possible use-cases of SCSSearchManager:

1. Continuous mode: In this mode, the application starts watermark detection process at the beginning of app session and stops the process at the end of session using startSearchingManager/stopSearchingManager methods.

2. Manual mode: In this mode, the application only starts audio recording at the beginning of session and performs one-time initiation of watermark detection process during the app session using singleSearchingManager method. Note that the audio recording should be started by TFSearchManager via startRecording method.

The SCSSearchManager can broadcast information about state changes, recognition results, and errors to the application via notifications. In particular, SCSSearchManager may send the following Notification:

IV. SDK Configuration

SDK settings and modifying system parameters

Base URL for server installation of the audio fingerprint recognition service is already defined in \TFMHAL\Defines.h (#define VIDEOTAG_API_PATH_SEARCH)

Audio fingerprint request interval can be set via searchInterval property of TFSearchManager class (default is 4.5 sec, minimum 4 sec).

The SoundCode audio watermark decoder parameters can be set via settingsSearchManager method of SCSSearchManager class (in run-time), or alternatively default values can be modified in \SCS\ SCSSettings.h header.