Android AudioRecord class - process live mic audio quickly, set up callback function
JavaAndroidAudiorecordAndroid HardwareJava Problem Overview
I want to record audio from the mic and access it for possible playback in near real-time. I am unsure of how to use the Android AudioRecord class to record some mic audio and quickly access it.
For the AudioRecord class, the official site says 'the app polls the AudioRecord object in time', and 'the size of the buffer being filled determines the time-length of the recording before over-running unread data'. Later it's suggested that a larger buffer should be used when polling less frequently. They never actually show an example in code.
One example I've seen in a book uses the AudioRecord class to continuously read a buffer freshly populated with live mic audio, and then the app writes this data to an SD file. The pseudo-code looks something like -
set up AudioRecord object with buffer size and recording format info
set up a file and an output stream
myAudioRecord.startRecording();
while(isRecording)
{
// myBuffer is being filled with fresh audio
read audio data into myBuffer
send contents of myBuffer to SD file
}
myAudioRecord.stop();
How this code synchronizes its reading with the rate of recording is unclear - is the boolean "isRecording" sequenced on and off properly elsewhere? It seems this code could either read too frequently or too infrequently, depending on how long the reading and writing takes.
The site doc also says the AudioRecord class has a nested class named OnRecordPositionUpdateListener which is defined as an interface. The information suggests that somehow, you specify the period you want for being notified of the progress of the recording, and the name of your event handler, and a call is automatically made to your event handler at the specified frequency. I think the structure, in pseudo-code would be something like -
set target of period update message = myListener
set period to be about every 250 ms
other code
myListener()
{
if(record button was recently tapped)
handle message that another 250 ms of fresh audio is available
ie, read it and send it somewhere
)
I need to find some specific code which allows me to capture and process mic audio with a delay of less than about 500 ms. Android offers another class called MediaRecorder, but it doesn't support streaming, and I may want to stream live mic audio over a Wi-Fi network in near real-time. Where can I find some specific examples?
Java Solutions
Solution 1 - Java
After experimenting lots with the notifications and a bunch of other techniques I settled on this code:
private class AudioIn extends Thread {
private boolean stopped = false;
private AudioIn() {
start();
}
@Override
public void run() {
android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
AudioRecord recorder = null;
short[][] buffers = new short[256][160];
int ix = 0;
try { // ... initialise
int N = AudioRecord.getMinBufferSize(8000,AudioFormat.CHANNEL_IN_MONO,AudioFormat.ENCODING_PCM_16BIT);
recorder = new AudioRecord(AudioSource.MIC,
8000,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT,
N*10);
recorder.startRecording();
// ... loop
while(!stopped) {
short[] buffer = buffers[ix++ % buffers.length];
N = recorder.read(buffer,0,buffer.length);
//process is what you will do with the data...not defined here
process(buffer);
}
} catch(Throwable x) {
Log.w(TAG,"Error reading voice audio",x);
} finally {
close();
}
}
private void close() {
stopped = true;
}
}
So far it's working pretty robustly on the half a dozen Android phones I've tried it on.
Solution 2 - Java
I wonder if you could combine these answers in the following way...
Use setPositionNotificationPeriod(160) before the while loop. This should cause the callback to be called every time 160 frames are read. Instead of calling process(buffer) inside of the thread that's doing the read loop, call process(buffer) from the callback. Use a variable to keep track of the last read buffer so you process the right one. As it is now, you block on the read, then you're not reading while you're processing. I think it might be better to separate those two.
Solution 3 - Java
Here is the code you need to use the OnRecordPositionUpdateListener and Notification Period.
I noticed that in practice it does not send the notification consistently at the same exact time, I want, but it is close enough.
About detectAfterEvery:
The size of detectEvery needs to be large enough to hold just the amount of data you want. So for this example, we have a sample rate of 44100 Hz, that means we want 44100 samples per second. By setting the setPositionNotificationPeriod to be 44100, the code tells Android to callback after it has recorded 44100 samples, which is about every 1 second.
The complete code is here:
final int sampleRate = 44100;
int bufferSize =
AudioRecord.getMinBufferSize(sampleRate,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
AudioFormat.ENCODING_PCM_16BIT);
//aim for 1 second
int detectAfterEvery = (int)((float)sampleRate * 1.0f);
if (detectAfterEvery > bufferSize)
{
Log.w(TAG, "Increasing buffer to hold enough samples " + detectAfterEvery + " was: " + bufferSize);
bufferSize = detectAfterEvery;
}
recorder =
new AudioRecord(AudioSource.MIC, sampleRate,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
AudioFormat.ENCODING_PCM_16BIT, bufferSize);
recorder.setPositionNotificationPeriod(detectAfterEvery);
final short[] audioData = new short[bufferSize];
final int finalBufferSize = bufferSize;
OnRecordPositionUpdateListener positionUpdater = new OnRecordPositionUpdateListener()
{
@Override
public void onPeriodicNotification(AudioRecord recorder)
{
Date d = new Date();
//it should be every 1 second, but it is actually, "about every 1 second"
//like 1073, 919, 1001, 1185, 1204 milliseconds of time.
Log.d(TAG, "periodic notification " + d.toLocaleString() + " mili " + d.getTime());
recorder.read(audioData, 0, finalBufferSize);
//do something amazing with audio data
}
@Override
public void onMarkerReached(AudioRecord recorder)
{
Log.d(TAG, "marker reached");
}
};
recorder.setRecordPositionUpdateListener(positionUpdater);
Log.d(TAG, "start recording, bufferSize: " + bufferSize);
recorder.startRecording();
//remember to still have a read loop otherwise the listener won't trigger
while (continueRecording)
{
recorder.read(audioData, 0, bufferSize);
}
Solution 4 - Java
private int freq =8000;
private AudioRecord audioRecord = null;
private Thread Rthread = null;
private AudioManager audioManager=null;
private AudioTrack audioTrack=null;
byte[] buffer = new byte[freq];
//call this method at start button
protected void Start()
{
loopback();
}
protected void loopback() {
android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
final int bufferSize = AudioRecord.getMinBufferSize(freq,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
AudioFormat.ENCODING_PCM_16BIT);
audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC, freq,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
MediaRecorder.AudioEncoder.AMR_NB, bufferSize);
audioTrack = new AudioTrack(AudioManager.ROUTE_HEADSET, freq,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
MediaRecorder.AudioEncoder.AMR_NB, bufferSize,
AudioTrack.MODE_STREAM);
audioTrack.setPlaybackRate(freq);
final byte[] buffer = new byte[bufferSize];
audioRecord.startRecording();
Log.i(LOG_TAG, "Audio Recording started");
audioTrack.play();
Log.i(LOG_TAG, "Audio Playing started");
Rthread = new Thread(new Runnable() {
public void run() {
while (true) {
try {
audioRecord.read(buffer, 0, bufferSize);
audioTrack.write(buffer, 0, buffer.length);
} catch (Throwable t) {
Log.e("Error", "Read write failed");
t.printStackTrace();
}
}
}
});
Rthread.start();
}
It plays the recorded audio less than 100 ms delay.