ag-aec/0000755000076500000240000000000011300304151011272 5ustar adigeostaffag-aec/draft-aec-03.txt0000644000076500000240000006777611300303512014130 0ustar adigeostaff Draft Andre Adrian Document: draft-aec-03.txt DFS Deutsche Flugsicherung Category: Experimental december 13th, 2004 Expires: ? Voice over Internet Acoustic Echo Cancellation Status of this Memo This document specifies an Acoustic Echo Cancellation implementation for hands-free Voice over Internet telephony and requests discussion and suggestions for improvements. Distribution of this memo is unlimited. Copyright Notice Copyright (C) DFS Deutsche Flugsicherung (2004). All Rights Reserved. You are allowed to use this source code in any open source or closed source software you want. You are allowed to use the algorithms for a hardware solution. You are allowed to modify the source code. You are not allowed to remove the name of the author from this memo or from the source code files. You are not allowed to monopolize the source code or the algorithms behind the source code as your intellectual property. This source code is free of royalty and comes with no warranty. Abstract This document specifies an acoustic echo cancellation (AEC) for voice over IP. Because of the large latency in VoIP communication (tenths to hunderts of milliseconds), AEC is necessary. The presented implementation is based on the well-known Normalized Least Mean Square (NLMS) and Geigel Double talk detector (DTD) algorithms. To improve performance, a pre-whitening filter is used. The presented algorithm is therefore of NLMS-pw family. The NLMS-pw family is known to give good echo cancellation for moderate processing resources. This algorithm is of complexity O(3*L) with L number of taps in the NLMS filter. Table of Contents 1. INTRODUCTION 2. AEC PRINCIPLES 3. AEC algorithms 3.1. Finite Impulse Response (FIR) Highpass Filter 3.2. Geigel Double Talk Detector 3.3. Normalized Least Mean Square - Pre-Whitening Filter 4. References A. The C++ Source Code A.1 aec.h A.2 aec.cpp A.3 aec_test.cpp A.4 Compile source code A.5 Test source code 1. INTRODUCTION A hands-free telephone or full-duplex intercom system has a feedback or echo problem because the output from the loudspeaker feeds into the microphone. Several methods can be used to reduce or eliminate the problem: 1.) Reduce the overall amplification. If the system amplification is less then 1 a feedback dies away. This solution leads to poor volume. 2.) Use Acoustic Echo Suppression. Echo Suppression is realized with speech activated switches. Suppression reduces the full-duplex telephone to half-duplex. The switches can even "switch away" beginnings of words. 3.) Use Acoustic Echo Cancellation. This is realized with an adaptive or learning filter. First the filter learns from given microphone and speaker signals the acoustics. After learning, the filter can calculate an estimated microphone signal from the loudspeaker signal. This estimated mic signal is subtracted from the real mic signal. The difference signal no longer contains the loudspeaker signal - the feedback loop is broken. The Least Means Square algorithm from Widrow and Hoff is known since 1960. Unfortunately the LMS is a slow learner. The learning speed or convergence rate is controlled by a constant value. This value in the LMS can only be optimized for loud signals or for weak signals. Optimizing for loud signals produces slow convergence with weak signals. Optimizing for weak signals gives divergence with loud signals. Divergence can be defined as "the filter does not reduce the echo but does increase the echo" and is very ugly. The Normalized LMS has a constant convergence rate for loud and weak signals, the convergence rate controlling parameter is derived from the signal energy. For white noise signal, where all frequencies have the same energy, the NLMS performs good. But the human speech has more energy in low frequencies then in high frequencies. Therefore, a NLMS gives good echo cancellation for low frequencies and poor echo cancellation for high frequencies. A pre-whitening filter in front of the echo cancellation filter transforms human speech into something more "white noise" like - the energy of high frequency signals is similar to the energy of low frequency signals. The presented algorithm uses the most simple pre-whitening filter possible, a first order or one pole highpass filter with transfer frequency equal to half of the sample frequency (4kHz for the narrowband sample frequency of 8kHz). Because the pre-whitening filter is fixed, the complexity of this NLMS-pw filter is still the same as for the NLMS filter. One important point should be remembered: The AEC in your telephony device helps your telephony partner to hear no echo. Therefore AEC is an altruistic algorithm. 2. AEC PRINCIPLES The core of the acoustic echo cancellation is described in the introduction. Next to the NLMS-pw three more blocks are used: 1.) A highpass filter for the microphone signal. Telephone users are used to a frequency range between 300Hz and 3400Hz. Narrowband VoIP can give 0Hz to 4000Hz. After hearing a VoIP signal with frequencies below 300Hz testers complained about the bad quality. With a 300Hz cut-off filter sound is limited as in telephone. The highpass filter in use is a 13 taps finite impulse response (FIR) filter. FIR filter was used because of its stability. 2.) A double talk detector. The AEC filter should only learn if the signal from the microphone is determined from the loudspeaker signal only. If the local or near-end user is talking, the filter can no longer learn successful. Detection of user talking is done by comparing the volume levels of loudspeaker and microphone. This implementation uses the well-known Geigel DTD. 3.) An Acoustic Echo Suppressor (AES) or Non Linear Processor (NLP). If the Double talk detector (DTD) detects "no talking", the microphone signal gets attenuated by 6dB. This is done to suppress echo artefacts. AEC block diagram. Sin is the microphone signal, Rout and Rin is the loudspeaker signal. Sout is the echo-cancelled microphone signal: +--+ + +---+ Sin -->---|HP|--+------->(+)----+-->|NLP|--->-- Sout +--+ | /|\ | +---+ | -| | \|/ | | +---+ +----+ | |DTD|---->|NLMS|<-+ +---+ +----+ /|\ /|\ | | | | Rout -<---------+---------+-----------------<-- Rin Figure 1.) AEC block diagram 3. AEC algorithms This chapter gives the mathematical background to the source code. This document will not give derivations of the algorithms or proofs. See references for more information. 3.1. Finite Impulse Response (FIR) Highpass Filter Ambient noises are often prominent in the frequency range to 300Hz. Typical examples are fans (2800 Rpm are 46.7Hz) and hard disks (7200 Rpm are 120Hz). Second, the small loudspeakers have often a resonance frequency around 80Hz. This is a non-linearity to the echo cancellation. Third, and maybe most important, the users are used to telephone quality with a 300Hz cut-off. The FIR filter has 13 taps. That gives a group delay of 0.8ms. Because of the stability a FIR filter was used. 3.2. Geigel Double Talk Detector Talk detection can be done with a threshold for the microphone signal only. This approach is very sensitive to the threshold level. A more robust approach is to compare microphone level with loudspeaker level. The threshold in this solution will be a relative one. Because we deal with echo, it is not sufficient to compare only the actual levels, but we have to consider previous levels, too. The Geigel DTD brings these ideas in one simple formula: The last L levels (index 0 for now and index L-1 for L samples ago) from loudspeaker signal are compared to the actual microphone signal. To avoid problems with phase, the absolute values are used. Double talk is declared if: |d| >= c * max(|x[0]|, |x[1]|, .., |x[L-1]|) with |d| is the absolute level of actual microphone signal, c is a threshold value (typical value 0.5 for -6dB or 0.71 for -3dB), |x[0]| is the absolute level of actual loudspeaker signel, |x[L-1]| is the absolute level of loudspeaker signal L samples ago. See references 3, 7, 9. 3.3. Normalized Least Mean Square - Pre-Whitening Filter The NLMS-pw, NLMS and LMS are of the gradient descent-based algorithms family. The good features of gradient-descent based algorithms are simplicity and robustness. First we look at the "echo cancelling" formula, the convolution. This formula is used to subtract the (from the loudspeaker signal) estimated microphone signal from the real microphone signal. e = d - X' * W with e is the linear error signal or echo-cancelled microphone signal, d is the desired signal or the microphone signal with echo, X' is the transpose of the loudspeaker signals vector, W is the adaptive weights vector. With a matching vector W the echo cancellation can be perfect. Unfortunately, learning the vector W has limitations. The loudspeaker is not the only audio source at filter learning. Ambient sounds and noises, system internal amplifier and converter noises and non-linearities of loudspeaker and microphone have a negative impact on learning. Due to the LMS simplicity, all elements of W are updated with the same "mikro * e" term. This simple approach makes the LMS robust and only demanding moderate processing resources, but this "one term fits all" approach prevents "perfect" learning, too. The LMS algorithm has the update formula: W[n+1] = W[n] + 2*mikro*e*X[n] with W[n+1] is the new adaptive weights vector, W[n] is the previous adaptive weights vector, mikro is the step size constant or variable, e is the error signal X[n] is the loudspeaker signals vector. The constant scalar mikro becomes a variable in NLMS. This variable is calculated from the loudspeaker signals vector with: 1 mikro = ------ X' * X with X' is the transpose of the loudspeaker signals vector, X is the loudspeaker signals vector. Note: The vector dot product is a scalar. It is the sum of the element-wise multiplication of both vectors. The constant value 2 in the LMS formula changes into a stability "tuneing" constant. For stable adaptation this constant should be between 0 and 1, this NLMS-pw uses a value of 0.7. The NLMS-pw uses for the weights vector update and the calculation of mikro highpass-filtered values of e and X. The filtered values are used because the NLMS converges best with white noise signals, and human voice is not white noise. The fixed highpass filter approach used in this NLMS-pw does not increase the overall complexity. With ef = highpass(e) Xf = highpass(X) we get our NLMS-pw weights vector update formulas: 0.7 mikro = -------- Xf' * Xf W[n+1] = W[n] + mikro*ef*Xf[n] with ef is the highpass-filtered value of e, Xf is the highpass-filtered value of X, and the other values are as above. Both filters are 1. order FIR with a transfer frequency of 4000Hz. For other pre-whitening algorithms see references 6, 8, 9. For non-LMS echo cancellation algorithms see references 6 and 9. 4. References [1] B. Widrow, M. E. Hoff Jr., "Adaptive switching circuits", Western Electric Show and Convention Record, Part 4, pages 96-104, Aug. 1960 [2] B. Widrow, et al, "Stationary and Nonstationary Learning Characteristics of the LMS Adaptive Filter", Proc. of the IEEE, vol. 64 No. 8, pp. 1151-1162, Aug. 1976 [3] D.L. Duttweiler, "A twelve-channel digital echo canceller", IEEE Trans. Commun., Vol. 26, pp. 647-653, May 1978 [4] B. Widrow, S.D. Stearns, Adaptive Signal Processing, Prentice-Hall, 1985 [5] D. Messerschmitt, D. Hedberg, C. Cole, A. Haoui, P. Winship, "Digital Voice Echo Canceller with a TMS32020", Application report SPRA129, Texas Instruments, 1989 [6] R. Storn, "Echo Cancellation Techniques for Multimedia Applications - a Survey", TR-96-046, International Computer Science Institute, Berkeley, Nov. 1996 [7] J. Nikolic, "Implementing a Line Echo Canceller using the block update and NLMS algorithms on the TMS320C54x DSP", Application report SPRA188, Texas Instruments, Apr. 1997 [8] M. G. Siqueira, "Adaptive Filtering Algorithms in Acoustic Echo Cancellation and Feedback Reduction", Ph.D. thesis, University of California, Los Angeles, 1998 [9] T. Gaensler, S. L. Gay, M. M. Sondhi, J. Benesty, "Double-Talk robust fast converging algorithms for network echo cancellation", IEEE trans. on speech and audio processing, vol. 8, No. 6, Nov. 2000 [10] M. Hutson, "Acoustic Echo Cancellation using Digital Signal Processing", Bachelor of Engineering (Honours) thesis, The School of Information Technology and Electrical Engineering, The University of Queensland, Nov 2003 [11] A. Adrian, "Audio Echo Cancellation", Free Software/Open Source Telephony Summit 2004, German Unix User Group, Geilenkirchen, Germany, Jan. 16-20, 2004 Appendix A. The C++ Source Code /***************** A.1 APPENDIX aec.h *****************/ /* aec.h * * Copyright (C) DFS Deutsche Flugsicherung (2004). All Rights Reserved. * * Acoustic Echo Cancellation NLMS-pw algorithm * * Version 1.3 filter created with www.dsptutor.freeuk.com */ #ifndef _AEC_H /* include only once */ // use double if your CPU does software-emulation of float typedef float REAL; /* dB Values */ const REAL M0dB = 1.0f; const REAL M3dB = 0.71f; const REAL M6dB = 0.50f; const REAL M9dB = 0.35f; const REAL M12dB = 0.25f; const REAL M18dB = 0.125f; const REAL M24dB = 0.063f; /* dB values for 16bit PCM */ /* MxdB_PCM = 32767 * 10 ^(x / 20) */ const REAL M10dB_PCM = 10362.0f; const REAL M20dB_PCM = 3277.0f; const REAL M25dB_PCM = 1843.0f; const REAL M30dB_PCM = 1026.0f; const REAL M35dB_PCM = 583.0f; const REAL M40dB_PCM = 328.0f; const REAL M45dB_PCM = 184.0f; const REAL M50dB_PCM = 104.0f; const REAL M55dB_PCM = 58.0f; const REAL M60dB_PCM = 33.0f; const REAL M65dB_PCM = 18.0f; const REAL M70dB_PCM = 10.0f; const REAL M75dB_PCM = 6.0f; const REAL M80dB_PCM = 3.0f; const REAL M85dB_PCM = 2.0f; const REAL M90dB_PCM = 1.0f; const REAL MAXPCM = 32767.0f; /* Design constants (Change to fine tune the algorithms */ /* The following values are for hardware AEC and studio quality * microphone */ /* maximum NLMS filter length in taps. A longer filter length gives * better Echo Cancellation, but slower convergence speed and * needs more CPU power (Order of NLMS is linear) */ #define NLMS_LEN (80*8) /* convergence speed. Range: >0 to <1 (0.2 to 0.7). Larger values give * more AEC in lower frequencies, but less AEC in higher frequencies. */ const REAL Stepsize = 0.7f; /* minimum energy in xf. Range: M70dB_PCM to M50dB_PCM. Should be equal * to microphone ambient Noise level */ const REAL Min_xf = M75dB_PCM; /* Double Talk Detector Speaker/Microphone Threshold. Range <=1 * Large value (M0dB) is good for Single-Talk Echo cancellation, * small value (M12dB) is good for Doulbe-Talk AEC */ const REAL GeigelThreshold = M6dB; /* Double Talk Detector hangover in taps. Not relevant for Single-Talk * AEC */ const int Thold = 30 * 8; /* for Non Linear Processor. Range >0 to 1. Large value (M0dB) is good * for Double-Talk, small value (M12dB) is good for Single-Talk */ const REAL NLPAttenuation = M12dB; /* Below this line there are no more design constants */ /* Exponential Smoothing or IIR Infinite Impulse Response Filter */ class IIR_HP { REAL x; public: IIR_HP() { x = 0.0f; }; REAL highpass(REAL in) { const REAL a0 = 0.01f; /* controls Transfer Frequency */ /* Highpass = Signal - Lowpass. Lowpass = Exponential Smoothing */ x += a0 * (in - x); return in - x; }; }; /* 13 taps FIR Finite Impulse Response filter * Coefficients calculated with * www.dsptutor.freeuk.com/KaiserFilterDesign/KaiserFilterDesign.html */ class FIR_HP13 { REAL z[14]; public: FIR_HP13() { memset(this, 0, sizeof(FIR_HP13)); }; REAL highpass(REAL in) { const REAL a[14] = { // Kaiser Window FIR Filter, Filter type: High pass // Passband: 300.0 - 4000.0 Hz, Order: 12 // Transition band: 100.0 Hz, Stopband attenuation: 10.0 dB -0.043183226f, -0.046636667f, -0.049576525f, -0.051936015f, -0.053661242f, -0.054712527f, 0.82598513f, -0.054712527f, -0.053661242f, -0.051936015f, -0.049576525f, -0.046636667f, -0.043183226f, 0.0f }; memmove(z+1, z, 13*sizeof(REAL)); z[0] = in; REAL sum0 = 0.0, sum1 = 0.0; int j; for (j = 0; j < 14; j+= 2) { // optimize: partial loop unrolling sum0 += a[j] * z[j]; sum1 += a[j+1] * z[j+1]; } return sum0+sum1; } }; /* Recursive single pole IIR Infinite Impulse response filter * Coefficients calculated with * http://www.dsptutor.freeuk.com/IIRFilterDesign/IIRFiltDes102.html */ class IIR1 { REAL x, y; public: IIR1() { memset(this, 0, sizeof(IIR1)); }; REAL highpass(REAL in) { // Chebyshev IIR filter, Filter type: HP // Passband: 3700 - 4000.0 Hz // Passband ripple: 1.5 dB, Order: 1 const REAL a0 = 0.105831884f; const REAL a1 = -0.105831884; const REAL b1 = 0.78833646f; REAL out = a0 * in + a1 * x + b1 * y; x = in; y = out; return out; } }; /* Recursive two pole IIR Infinite Impulse Response filter * Coefficients calculated with * http://www.dsptutor.freeuk.com/IIRFilterDesign/IIRFiltDes102.html */ class IIR2 { REAL x[2], y[2]; public: IIR2() { memset(this, 0, sizeof(IIR2)); }; REAL highpass(REAL in) { // Butterworth IIR filter, Filter type: HP // Passband: 2000 - 4000.0 Hz, Order: 2 const REAL a[] = { 0.29289323f, -0.58578646f, 0.29289323f }; const REAL b[] = { 1.3007072E-16f, 0.17157288f }; REAL out = a[0] * in + a[1] * x[0] + a[2] * x[1] - b[0] * y[0] - b[1] * y[1]; x[1] = x[0]; x[0] = in; y[1] = y[0]; y[0] = out; return out; } }; // Extention in taps to reduce mem copies #define NLMS_EXT (10*8) // block size in taps to optimize DTD calculation #define DTD_LEN 16 class AEC { // Time domain Filters IIR_HP hp00, hp1; // DC-level remove Highpass) FIR_HP13 hp0; // 300Hz cut-off Highpass IIR1 Fx, Fe; // pre-whitening Highpass for x, e // Geigel DTD (Double Talk Detector) REAL max_max_x; // max(|x[0]|, .. |x[L-1]|) int hangover; // optimize: less calculations for max() REAL max_x[NLMS_LEN / DTD_LEN]; int dtdCnt; int dtdNdx; // NLMS-pw REAL x[NLMS_LEN + NLMS_EXT]; // tap delayed loudspeaker signal REAL xf[NLMS_LEN + NLMS_EXT]; // pre-whitening tap delayed signal REAL w[NLMS_LEN]; // tap weights int j; // optimize: less memory copies int lastupdate; // optimize: iterative dotp(x,x) double dotp_xf_xf; // double to avoid loss of precision double Min_dotp_xf_xf; REAL s0avg; public: AEC(); /* Geigel Double-Talk Detector * * in d: microphone sample (PCM as REALing point value) * in x: loudspeaker sample (PCM as REALing point value) * return: 0 for no talking, 1 for talking */ int dtd(REAL d, REAL x); /* Normalized Least Mean Square Algorithm pre-whitening (NLMS-pw) * The LMS algorithm was developed by Bernard Widrow * book: Widrow/Stearns, Adaptive Signal Processing, Prentice-Hall, 1985 * * in mic: microphone sample (PCM as REALing point value) * in spk: loudspeaker sample (PCM as REALing point value) * in update: 0 for convolve only, 1 for convolve and update * return: echo cancelled microphone sample */ REAL nlms_pw(REAL mic, REAL spk, int update); /* Acoustic Echo Cancellation and Suppression of one sample * in d: microphone signal with echo * in x: loudspeaker signal * return: echo cancelled microphone signal */ int AEC::doAEC(int d, int x); float AEC::getambient() { return s0avg; }; void AEC::setambient(float Min_xf) { dotp_xf_xf = Min_dotp_xf_xf = NLMS_LEN * Min_xf * Min_xf; }; }; #define _AEC_H #endif /***************** A.2 APPENDIX aec.cpp *****************/ /* aec.cpp * * Copyright (C) DFS Deutsche Flugsicherung (2004). All Rights Reserved. * * Acoustic Echo Cancellation NLMS-pw algorithm * * Version 1.3 filter created with www.dsptutor.freeuk.com */ #include #include #include #include #include "aec.h" /* Vector Dot Product */ REAL dotp(REAL a[], REAL b[]) { REAL sum0 = 0.0, sum1 = 0.0; int j; for (j = 0; j < NLMS_LEN; j+= 2) { // optimize: partial loop unrolling sum0 += a[j] * b[j]; sum1 += a[j+1] * b[j+1]; } return sum0+sum1; } AEC::AEC() { max_max_x = 0.0f; hangover = 0; memset(max_x, 0, sizeof(max_x)); dtdCnt = dtdNdx = 0; memset(x, 0, sizeof(x)); memset(xf, 0, sizeof(xf)); memset(w, 0, sizeof(w)); j = NLMS_EXT; lastupdate = 0; s0avg = M80dB_PCM; setambient(Min_xf); } REAL AEC::nlms_pw(REAL mic, REAL spk, int update) { REAL d = mic; // desired signal x[j] = spk; xf[j] = Fx.highpass(spk); // pre-whitening of x // calculate error value // (mic signal - estimated mic signal from spk signal) REAL e = d - dotp(w, x + j); REAL ef = Fe.highpass(e); // pre-whitening of e // optimize: iterative dotp(xf, xf) dotp_xf_xf += (xf[j]*xf[j] - xf[j+NLMS_LEN-1]*xf[j+NLMS_LEN-1]); if (update) { // calculate variable step size REAL mikro_ef = Stepsize * ef / dotp_xf_xf; // update tap weights (filter learning) int i; for (i = 0; i < NLMS_LEN; i += 2) { // optimize: partial loop unrolling w[i] += mikro_ef*xf[i+j]; w[i+1] += mikro_ef*xf[i+j+1]; } } if (--j < 0) { // optimize: decrease number of memory copies j = NLMS_EXT; memmove(x+j+1, x, (NLMS_LEN-1)*sizeof(REAL)); memmove(xf+j+1, xf, (NLMS_LEN-1)*sizeof(REAL)); } return e; } int AEC::dtd(REAL d, REAL x) { // optimized implementation of max(|x[0]|, |x[1]|, .., |x[L-1]|): // calculate max of block (DTD_LEN values) x = fabsf(x); if (x > max_x[dtdNdx]) { max_x[dtdNdx] = x; if (x > max_max_x) { max_max_x = x; } } if (++dtdCnt >= DTD_LEN) { dtdCnt = 0; // calculate max of max max_max_x = 0.0f; for (int i = 0; i < NLMS_LEN/DTD_LEN; ++i) { if (max_x[i] > max_max_x) { max_max_x = max_x[i]; } } // rotate Ndx if (++dtdNdx >= NLMS_LEN/DTD_LEN) dtdNdx = 0; max_x[dtdNdx] = 0.0f; } // The Geigel DTD algorithm with Hangover timer Thold if (fabsf(d) >= GeigelThreshold * max_max_x) { hangover = Thold; } if (hangover) --hangover; return (hangover > 0); } int AEC::doAEC(int d, int x) { REAL s0 = (REAL)d; REAL s1 = (REAL)x; // Mic Highpass Filter - to remove DC s0 = hp00.highpass(s0); // Mic Highpass Filter - telephone users are used to 300Hz cut-off s0 = hp0.highpass(s0); // ambient mic level estimation s0avg += 1e-4f*(fabsf(s0) - s0avg); // Spk Highpass Filter - to remove DC s1 = hp1.highpass(s1); // Double Talk Detector int update = !dtd(s0, s1); // Acoustic Echo Cancellation s0 = nlms_pw(s0, s1, update); // Acoustic Echo Suppression if (update) { // Non Linear Processor (NLP): attenuate low volumes s0 *= NLPAttenuation; } // Saturation if (s0 > MAXPCM) { return (int)MAXPCM; } else if (s0 < -MAXPCM) { return (int)-MAXPCM; } else { return (int)roundf(s0); } } /***************** A.3 APPENDIX aec_test.cpp *****************/ /* aec_test.cpp * * Copyright (C) DFS Deutsche Flugsicherung (2004). All Rights Reserved. * * Test stub for Acoustic Echo Cancellation NLMS-pw algorithm * Author: Andre Adrian, DFS Deutsche Flugsicherung * * * compile c++ -O2 -o aec_test aec_test.cpp aec.cpp -lm * * Version 1.3 set/get ambient in dB */ #include #include #include #include #include "aec.h" #define TAPS (80*8) typedef signed short MONO; typedef struct { signed short l; signed short r; } STEREO; float dB2q(float dB) { /* Dezibel to Ratio */ return powf(10.0f, dB / 20.0f); } float q2dB(float q) { /* Ratio to Dezibel */ return 20.0f * log10f(q); } /* Read a raw audio file (8KHz sample frequency, 16bit PCM, stereo) * from stdin, echo cancel it and write it to stdout */ int main(int argc, char *argv[]) { STEREO inbuf[TAPS], outbuf[TAPS]; fprintf(stderr, "usage: aec_test [ambient in dB] out.raw\n"); AEC aec; if (argc >= 2) { aec.setambient(MAXPCM*dB2q(atof(argv[1]))); } int taps; while (taps = fread(inbuf, sizeof(STEREO), TAPS, stdin)) { int i; for (i = 0; i < taps; ++i) { int s0 = inbuf[i].l; /* left channel microphone */ int s1 = inbuf[i].r; /* right channel speaker */ /* and do NLMS */ s0 = aec.doAEC(s0, s1); /* copy back */ outbuf[i].l = 0; /* left channel silence */ outbuf[i].r = s0; /* right channel echo cancelled mic */ } fwrite(outbuf, sizeof(STEREO), taps, stdout); } float ambient = aec.getambient(); float ambientdB = q2dB(ambient / 32767.0f); fprintf(stderr, "Ambient = %2.0f dB\n", ambientdB); fflush(NULL); return 0; } /***************** A.4 APPENDIX Compile source code *****************/ On a Linux system with GNU C++ compiler enter: g++ aec_test.cpp aec.cpp -o aec_test -lm /***************** A.5 APPENDIX Test source code *****************/ The microphone and loudspeaker signals have to be synchronized on a sample-to-sample basis to make acoustic echo cancellation working. An AC97 conformal on-board soundcard in a Personal Computer can be set in a special stereo mode: The left channnel records microphone signal and the right channel reports loudspeaker signal. To set-up a Linux PC with ALSA sound system, microphone connected to Mic in and loudspeaker connected to right Line out enter: amixer -q set 'Master',0 50% unmute amixer -q set 'PCM',0 80% unmute amixer -q set 'Line',0 0% mute amixer -q set 'CD',0 0% mute amixer -q set 'Mic',0 0% mute amixer -q set 'Video',0 0% mute amixer -q set 'Phone',0 0% mute amixer -q set 'PC Speaker',0 0% mute amixer -q set 'Aux',0 0% mute amixer -q set 'Capture',0 50%,0% amixer -q set 'Mic Boost (+20dB)',0 1 amixer -q cset iface=MIXER,name='Capture Source' 0,5 amixer -q cset iface=MIXER,name='Capture Switch' 1 To test the acoustic echo cancellation we simulate a real telephone conversation in 5 steps: (1) record far-end speaker, (2) perform acoustic echo cancellation (this should change nothing) (3) playback far-end speaker and at the same time record near-end spk. (4) perform acoustic echo cancellation (5) playback near-end speaker (far-end speech should be cancelled) To record 10 seconds of speech into the file b.raw enter: arecord -D plug:hw:0 -c 2 -t raw -f S16_LE -r 8000 -d 10 >b.raw To perform AEC at the far-end enter: ./aec_test b1.raw To playback file b1.raw and simultaneously record b2.raw enter both commands in one go: aplay -D plug:hw:0 -c 2 -t raw -f S16_LE -r 8000 b1.raw & arecord -D plug:hw:0 -c 2 -t raw -f S16_LE -r 8000 -d 10 >b2.raw To perform AEC at the near-end enter: ./aec_test b3.raw To playback the echo-cancelled near-end enter: aplay -D plug:hw:0 -c 2 -t raw -f S16_LE -r 8000 b3.raw ag-aec/LICENSE0000644000076500000240000000007711300303512012303 0ustar adigeostaffAG-AEC is licensed under GNU General Public License version 3. ag-aec/pjsip/0000755000076500000240000000000011300303512012417 5ustar adigeostaffag-aec/pjsip/pjmedia/0000755000076500000240000000000011300303512014030 5ustar adigeostaffag-aec/pjsip/pjmedia/include/0000755000076500000240000000000011300303512015453 5ustar adigeostaffag-aec/pjsip/pjmedia/include/pjmedia/0000755000076500000240000000000011300303512017064 5ustar adigeostaffag-aec/pjsip/pjmedia/include/pjmedia/config.h0000644000076500000240000005023511300303512020507 0ustar adigeostaff/* $Id: config.h 2394 2008-12-23 17:27:53Z bennylp $ */ /* * Copyright (C) 2008-2009 Teluu Inc. (http://www.teluu.com) * Copyright (C) 2003-2008 Benny Prijono * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ #ifndef __PJMEDIA_CONFIG_H__ #define __PJMEDIA_CONFIG_H__ /** * @file pjmedia/config.h Compile time config * @brief Contains some compile time constants. */ #include /** * @defgroup PJMEDIA_BASE Base Types and Configurations */ /** * @defgroup PJMEDIA_CONFIG Compile time configuration * @ingroup PJMEDIA_BASE * @brief Some compile time configuration settings. * @{ */ /* * Include config_auto.h if autoconf is used (PJ_AUTOCONF is set) */ #if defined(PJ_AUTOCONF) # include #endif /* * Types of sound stream backends. */ /** Constant for NULL sound backend. */ #define PJMEDIA_SOUND_NULL_SOUND 0 /** Constant for PortAudio sound backend. */ #define PJMEDIA_SOUND_PORTAUDIO_SOUND 1 /** Constant for Win32 DirectSound sound backend. */ #define PJMEDIA_SOUND_WIN32_DIRECT_SOUND 2 /** Constant for Win32 MME sound backend. */ #define PJMEDIA_SOUND_WIN32_MME_SOUND 3 /** When this is set, pjmedia will not provide any sound device backend. * Application will have to provide its own sound device backend * and link the application with it. */ #define PJMEDIA_SOUND_EXTERNAL 255 /** * Unless specified otherwise, sound device uses PortAudio implementation * by default. */ #ifndef PJMEDIA_SOUND_IMPLEMENTATION # if defined(PJ_WIN32) && PJ_WIN32!=0 /*# define PJMEDIA_SOUND_IMPLEMENTATION PJMEDIA_SOUND_WIN32_DIRECT_SOUND*/ /*# define PJMEDIA_SOUND_IMPLEMENTATION PJMEDIA_SOUND_WIN32_MME_SOUND*/ # define PJMEDIA_SOUND_IMPLEMENTATION PJMEDIA_SOUND_PORTAUDIO_SOUND # else # define PJMEDIA_SOUND_IMPLEMENTATION PJMEDIA_SOUND_PORTAUDIO_SOUND # endif #endif /** * Specify whether we prefer to use DirectSound on Windows. * * Default: 0 */ #ifndef PJMEDIA_PREFER_DIRECT_SOUND # define PJMEDIA_PREFER_DIRECT_SOUND 0 #endif /** * Specify sound device latency default, in milisecond. */ #ifndef PJMEDIA_SND_DEFAULT_REC_LATENCY # define PJMEDIA_SND_DEFAULT_REC_LATENCY 100 #endif #ifndef PJMEDIA_SND_DEFAULT_PLAY_LATENCY # define PJMEDIA_SND_DEFAULT_PLAY_LATENCY 100 #endif /** * Specify whether delay buffer is used for sound device. * When delay buffer is enabled, the sound device callback * will be called one after another evenly. * The delay buffer also performs the best delay calculation * for the sound device, and will try to limit the delay caused * by uneven callback calls to this delay. * * When this setting is enabled, the PJMEDIA_SOUND_BUFFER_COUNT * macro will specify the maximum size of the delay buffer. */ #ifndef PJMEDIA_SOUND_USE_DELAYBUF # define PJMEDIA_SOUND_USE_DELAYBUF 0 #endif /* * Types of WSOLA backend algorithm. */ /** * This denotes implementation of WSOLA using null algorithm. Expansion * will generate zero frames, and compression will just discard some * samples from the input. * * This type of implementation may be used as it requires the least * processing power. */ #define PJMEDIA_WSOLA_IMP_NULL 0 /** * This denotes implementation of WSOLA using fixed or floating point WSOLA * algorithm. This implementation provides the best quality of the result, * at the expense of one frame delay and intensive processing power * requirement. */ #define PJMEDIA_WSOLA_IMP_WSOLA 1 /** * This denotes implementation of WSOLA algorithm with faster waveform * similarity calculation. This implementation provides fair quality of * the result with the main advantage of low processing power requirement. */ #define PJMEDIA_WSOLA_IMP_WSOLA_LITE 2 /** * Specify type of Waveform based Similarity Overlap and Add (WSOLA) backend * implementation to be used. WSOLA is an algorithm to expand and/or compress * audio frames without changing the pitch, and used by the delaybuf and as PLC * backend algorithm. * * Default is PJMEDIA_WSOLA_IMP_WSOLA */ #ifndef PJMEDIA_WSOLA_IMP # define PJMEDIA_WSOLA_IMP PJMEDIA_WSOLA_IMP_WSOLA #endif /** * Specify number of sound buffers. Larger number is better for sound * stability and to accommodate sound devices that are unable to send frames * in timely manner, however it would probably cause more audio delay (and * definitely will take more memory). One individual buffer is normally 10ms * or 20 ms long, depending on ptime settings (samples_per_frame value). * * The setting here currently is used by the conference bridge, the splitter * combiner port, and dsound.c. * * Default: 6 */ #ifndef PJMEDIA_SOUND_BUFFER_COUNT # define PJMEDIA_SOUND_BUFFER_COUNT 6 #endif /** * Specify which A-law/U-law conversion algorithm to use. * By default the conversion algorithm uses A-law/U-law table which gives * the best performance, at the expense of 33 KBytes of static data. * If this option is disabled, a smaller but slower algorithm will be used. */ #ifndef PJMEDIA_HAS_ALAW_ULAW_TABLE # define PJMEDIA_HAS_ALAW_ULAW_TABLE 1 #endif /** * Unless specified otherwise, G711 codec is included by default. */ #ifndef PJMEDIA_HAS_G711_CODEC # define PJMEDIA_HAS_G711_CODEC 1 #endif /* * Warn about obsolete macros. * * PJMEDIA_HAS_SMALL_FILTER has been deprecated in 0.7. */ #if defined(PJMEDIA_HAS_SMALL_FILTER) # ifdef _MSC_VER # pragma message("Warning: PJMEDIA_HAS_SMALL_FILTER macro is deprecated"\ " and has no effect") # else # warning "PJMEDIA_HAS_SMALL_FILTER macro is deprecated and has no effect" # endif #endif /* * Warn about obsolete macros. * * PJMEDIA_HAS_LARGE_FILTER has been deprecated in 0.7. */ #if defined(PJMEDIA_HAS_LARGE_FILTER) # ifdef _MSC_VER # pragma message("Warning: PJMEDIA_HAS_LARGE_FILTER macro is deprecated"\ " and has no effect") # else # warning "PJMEDIA_HAS_LARGE_FILTER macro is deprecated" # endif #endif /* * These macros are obsolete in 0.7.1 so it will trigger compilation error. * Please use PJMEDIA_RESAMPLE_IMP to select the resample implementation * to use. */ #ifdef PJMEDIA_HAS_LIBRESAMPLE # error "PJMEDIA_HAS_LIBRESAMPLE macro is deprecated. Use '#define PJMEDIA_RESAMPLE_IMP PJMEDIA_RESAMPLE_LIBRESAMPLE'" #endif #ifdef PJMEDIA_HAS_SPEEX_RESAMPLE # error "PJMEDIA_HAS_SPEEX_RESAMPLE macro is deprecated. Use '#define PJMEDIA_RESAMPLE_IMP PJMEDIA_RESAMPLE_SPEEX'" #endif /* * Sample rate conversion backends. * Select one of these backends in PJMEDIA_RESAMPLE_IMP. */ #define PJMEDIA_RESAMPLE_NONE 1 /**< No resampling. */ #define PJMEDIA_RESAMPLE_LIBRESAMPLE 2 /**< Sample rate conversion using libresample. */ #define PJMEDIA_RESAMPLE_SPEEX 3 /**< Sample rate conversion using Speex. */ #define PJMEDIA_RESAMPLE_LIBSAMPLERATE 4 /**< Sample rate conversion using libsamplerate (a.k.a Secret Rabbit Code) */ /** * Select which resample implementation to use. Currently pjmedia supports: * - #PJMEDIA_RESAMPLE_LIBRESAMPLE, to use libresample-1.7, this is the default * implementation to be used. * - #PJMEDIA_RESAMPLE_LIBSAMPLERATE, to use libsamplerate implementation * (a.k.a. Secret Rabbit Code). * - #PJMEDIA_RESAMPLE_SPEEX, to use experimental sample rate conversion in * Speex library. * - #PJMEDIA_RESAMPLE_NONE, to disable sample rate conversion. Any calls to * resample function will return error. * * Default is PJMEDIA_RESAMPLE_LIBRESAMPLE */ #ifndef PJMEDIA_RESAMPLE_IMP # define PJMEDIA_RESAMPLE_IMP PJMEDIA_RESAMPLE_LIBRESAMPLE #endif /** * Specify whether libsamplerate, when used, should be linked statically * into the application. This option is only useful for Visual Studio * projects, and when this static linking is enabled */ /** * Default file player/writer buffer size. */ #ifndef PJMEDIA_FILE_PORT_BUFSIZE # define PJMEDIA_FILE_PORT_BUFSIZE 4000 #endif /** * Maximum frame duration (in msec) to be supported. * This (among other thing) will affect the size of buffers to be allocated * for outgoing packets. */ #ifndef PJMEDIA_MAX_FRAME_DURATION_MS # define PJMEDIA_MAX_FRAME_DURATION_MS 200 #endif /** * Max packet size to support. */ #ifndef PJMEDIA_MAX_MTU # define PJMEDIA_MAX_MTU 1500 #endif /** * DTMF/telephone-event duration, in timestamp. */ #ifndef PJMEDIA_DTMF_DURATION # define PJMEDIA_DTMF_DURATION 1600 /* in timestamp */ #endif /** * Number of packets received from different source IP address from the * remote address required to make the stream switch transmission * to the source address. */ #ifndef PJMEDIA_RTP_NAT_PROBATION_CNT # define PJMEDIA_RTP_NAT_PROBATION_CNT 10 #endif /** * Specify whether RTCP should be advertised in SDP. This setting would * affect whether RTCP candidate will be added in SDP when ICE is used. * Application might want to disable RTCP advertisement in SDP to * reduce the message size. * * Default: 1 (yes) */ #ifndef PJMEDIA_ADVERTISE_RTCP # define PJMEDIA_ADVERTISE_RTCP 1 #endif /** * Interval to send RTCP packets, in msec */ #ifndef PJMEDIA_RTCP_INTERVAL # define PJMEDIA_RTCP_INTERVAL 5000 /* msec*/ #endif /** * Tell RTCP to ignore the first N packets when calculating the * jitter statistics. From experimentation, the first few packets * (25 or so) have relatively big jitter, possibly because during * this time, the program is also busy setting up the signaling, * so they make the average jitter big. * * Default: 25. */ #ifndef PJMEDIA_RTCP_IGNORE_FIRST_PACKETS # define PJMEDIA_RTCP_IGNORE_FIRST_PACKETS 25 #endif /** * Specify whether RTCP XR support should be built into PJMEDIA. Disabling * this feature will reduce footprint slightly. Note that even when this * setting is enabled, RTCP XR processing will only be performed in stream * if it is enabled on run-time on per stream basis. See * PJMEDIA_STREAM_ENABLE_XR setting for more info. * * Default: 1 (yes). */ #ifndef PJMEDIA_HAS_RTCP_XR # define PJMEDIA_HAS_RTCP_XR 0 #endif /** * The RTCP XR feature is activated and used by stream if \a enable_rtcp_xr * field of \a pjmedia_stream_info structure is non-zero. This setting * controls the default value of this field. * * Default: 0 (disabled) */ #ifndef PJMEDIA_STREAM_ENABLE_XR # define PJMEDIA_STREAM_ENABLE_XR 0 #endif /** * Specify how long (in miliseconds) the stream should suspend the * silence detector/voice activity detector (VAD) during the initial * period of the session. This feature is useful to open bindings in * all NAT routers between local and remote endpoint since most NATs * do not allow incoming packet to get in before local endpoint sends * outgoing packets. * * Specify zero to disable this feature. * * Default: 600 msec (which gives good probability that some RTP * packets will reach the destination, but without * filling up the jitter buffer on the remote end). */ #ifndef PJMEDIA_STREAM_VAD_SUSPEND_MSEC # define PJMEDIA_STREAM_VAD_SUSPEND_MSEC 600 #endif /** * Specify the maximum duration of silence period in the codec, in msec. * This is useful for example to keep NAT binding open in the firewall * and to prevent server from disconnecting the call because no * RTP packet is received. * * This only applies to codecs that use PJMEDIA's VAD (pretty much * everything including iLBC, except Speex, which has its own DTX * mechanism). * * Use (-1) to disable this feature. * * Default: 5000 ms * */ #ifndef PJMEDIA_CODEC_MAX_SILENCE_PERIOD # define PJMEDIA_CODEC_MAX_SILENCE_PERIOD 5000 #endif /** * Suggested or default threshold to be set for fixed silence detection * or as starting threshold for adaptive silence detection. The threshold * has the range from zero to 0xFFFF. */ #ifndef PJMEDIA_SILENCE_DET_THRESHOLD # define PJMEDIA_SILENCE_DET_THRESHOLD 4 #endif /** * Maximum silence threshold in the silence detector. The silence detector * will not cut the audio transmission if the audio level is above this * level. * * Use 0x10000 (or greater) to disable this feature. * * Default: 0x10000 (disabled) */ #ifndef PJMEDIA_SILENCE_DET_MAX_THRESHOLD # define PJMEDIA_SILENCE_DET_MAX_THRESHOLD 0x10000 #endif /** * Speex Accoustic Echo Cancellation (AEC). * By default is enabled. */ /* #ifndef PJMEDIA_HAS_SPEEX_AEC # define PJMEDIA_HAS_SPEEX_AEC 1 #endif */ /** * Enable the AG AEC algorithm by default */ #ifndef PJMEDIA_HAS_AG_AEC # define PJMEDIA_HAS_AG_AEC 1 #endif /** * This specifies the behavior of the SDP negotiator when responding to an * offer, whether it should rather use the codec preference as set by * remote, or should it rather use the codec preference as specified by * local endpoint. * * For example, suppose incoming call has codec order "8 0 3", while * local codec order is "3 0 8". If remote codec order is preferable, * the selected codec will be 8, while if local codec order is preferable, * the selected codec will be 3. * * If set to non-zero, the negotiator will use the codec order as specified * by remote in the offer. * * Note that this behavior can be changed during run-time by calling * pjmedia_sdp_neg_set_prefer_remote_codec_order(). * * Default is 1 (to maintain backward compatibility) */ #ifndef PJMEDIA_SDP_NEG_PREFER_REMOTE_CODEC_ORDER # define PJMEDIA_SDP_NEG_PREFER_REMOTE_CODEC_ORDER 1 #endif /** * Support for sending and decoding RTCP port in SDP (RFC 3605). * Default is equal to PJMEDIA_ADVERTISE_RTCP setting. */ #ifndef PJMEDIA_HAS_RTCP_IN_SDP # define PJMEDIA_HAS_RTCP_IN_SDP (PJMEDIA_ADVERTISE_RTCP) #endif /** * This macro controls whether pjmedia should include SDP rtpmap * attribute for static payload types. SDP rtpmap for static * payload types are optional, although they are normally included * for interoperability reason. * * Note that there is also a run-time variable to turn this setting * on or off, defined in endpoint.c. To access this variable, use * the following construct * \verbatim extern pj_bool_t pjmedia_add_rtpmap_for_static_pt; // Do not include rtpmap for static payload types (<96) pjmedia_add_rtpmap_for_static_pt = PJ_FALSE; \endverbatim * * Default: 1 (yes) */ #ifndef PJMEDIA_ADD_RTPMAP_FOR_STATIC_PT # define PJMEDIA_ADD_RTPMAP_FOR_STATIC_PT 1 #endif /** * This macro declares the payload type for telephone-event * that is advertised by PJMEDIA for outgoing SDP. If this macro * is set to zero, telephone events would not be advertised nor * supported. * * If this value is changed to other number, please update the * PJMEDIA_RTP_PT_TELEPHONE_EVENTS_STR too. */ #ifndef PJMEDIA_RTP_PT_TELEPHONE_EVENTS # define PJMEDIA_RTP_PT_TELEPHONE_EVENTS 101 #endif /** * Macro to get the string representation of the telephone-event * payload type. */ #ifndef PJMEDIA_RTP_PT_TELEPHONE_EVENTS_STR # define PJMEDIA_RTP_PT_TELEPHONE_EVENTS_STR "101" #endif /** * Maximum tones/digits that can be enqueued in the tone generator. */ #ifndef PJMEDIA_TONEGEN_MAX_DIGITS # define PJMEDIA_TONEGEN_MAX_DIGITS 32 #endif /* * Below specifies the various tone generator backend algorithm. */ /** * The math's sine(), floating point. This has very good precision * but it's the slowest and requires floating point support and * linking with the math library. */ #define PJMEDIA_TONEGEN_SINE 1 /** * Floating point approximation of sine(). This has relatively good * precision and much faster than plain sine(), but it requires floating- * point support and linking with the math library. */ #define PJMEDIA_TONEGEN_FLOATING_POINT 2 /** * Fixed point using sine signal generated by Cordic algorithm. This * algorithm can be tuned to provide balance between precision and * performance by tuning the PJMEDIA_TONEGEN_FIXED_POINT_CORDIC_LOOP * setting, and may be suitable for platforms that lack floating-point * support. */ #define PJMEDIA_TONEGEN_FIXED_POINT_CORDIC 3 /** * Fast fixed point using some approximation to generate sine waves. * The tone generated by this algorithm is not very precise, however * the algorithm is very fast. */ #define PJMEDIA_TONEGEN_FAST_FIXED_POINT 4 /** * Specify the tone generator algorithm to be used. Please see * http://trac.pjsip.org/repos/wiki/Tone_Generator for the performance * analysis results of the various tone generator algorithms. * * Default value: * - PJMEDIA_TONEGEN_FLOATING_POINT when PJ_HAS_FLOATING_POINT is set * - PJMEDIA_TONEGEN_FIXED_POINT_CORDIC when PJ_HAS_FLOATING_POINT is not set */ #ifndef PJMEDIA_TONEGEN_ALG # if defined(PJ_HAS_FLOATING_POINT) && PJ_HAS_FLOATING_POINT # define PJMEDIA_TONEGEN_ALG PJMEDIA_TONEGEN_FLOATING_POINT # else # define PJMEDIA_TONEGEN_ALG PJMEDIA_TONEGEN_FIXED_POINT_CORDIC # endif #endif /** * Specify the number of calculation loops to generate the tone, when * PJMEDIA_TONEGEN_FIXED_POINT_CORDIC algorithm is used. With more calculation * loops, the tone signal gets more precise, but this will add more * processing. * * Valid values are 1 to 28. * * Default value: 10 */ #ifndef PJMEDIA_TONEGEN_FIXED_POINT_CORDIC_LOOP # define PJMEDIA_TONEGEN_FIXED_POINT_CORDIC_LOOP 10 #endif /** * Enable high quality of tone generation, the better quality will cost * more CPU load. This is only applied to floating point enabled machines. * * By default it is enabled when PJ_HAS_FLOATING_POINT is set. * * This macro has been deprecated in version 1.0-rc3. */ #ifdef PJMEDIA_USE_HIGH_QUALITY_TONEGEN # error "The PJMEDIA_USE_HIGH_QUALITY_TONEGEN macro is obsolete" #endif /** * Fade-in duration for the tone, in milliseconds. Set to zero to disable * this feature. * * Default: 1 (msec) */ #ifndef PJMEDIA_TONEGEN_FADE_IN_TIME # define PJMEDIA_TONEGEN_FADE_IN_TIME 1 #endif /** * Fade-out duration for the tone, in milliseconds. Set to zero to disable * this feature. * * Default: 2 (msec) */ #ifndef PJMEDIA_TONEGEN_FADE_OUT_TIME # define PJMEDIA_TONEGEN_FADE_OUT_TIME 2 #endif /** * The default tone generator amplitude (1-32767). * * Default value: 12288 */ #ifndef PJMEDIA_TONEGEN_VOLUME # define PJMEDIA_TONEGEN_VOLUME 12288 #endif /** * Enable support for SRTP media transport. This will require linking * with libsrtp from the third_party directory. * * By default it is enabled. */ #ifndef PJMEDIA_HAS_SRTP # define PJMEDIA_HAS_SRTP 1 #endif /** * Enable support to handle codecs with inconsistent clock rate * between clock rate in SDP/RTP & the clock rate that is actually used. * This happens for example with G.722 and MPEG audio codecs. * See: * - G.722 : RFC 3551 4.5.2 * - MPEG audio : RFC 3551 4.5.13 & RFC 3119 * * Also when this feature is enabled, some handling will be performed * to deal with clock rate incompatibilities of some phones. * * By default it is enabled. */ #ifndef PJMEDIA_HANDLE_G722_MPEG_BUG # define PJMEDIA_HANDLE_G722_MPEG_BUG 1 #endif /** * Transport info (pjmedia_transport_info) contains a socket info and list * of transport specific info, since transports can be chained together * (for example, SRTP transport uses UDP transport as the underlying * transport). This constant specifies maximum number of transport specific * infos that can be held in a transport info. */ #ifndef PJMEDIA_TRANSPORT_SPECIFIC_INFO_MAXCNT # define PJMEDIA_TRANSPORT_SPECIFIC_INFO_MAXCNT 4 #endif /** * Maximum size in bytes of storage buffer of a transport specific info. */ #ifndef PJMEDIA_TRANSPORT_SPECIFIC_INFO_MAXSIZE # define PJMEDIA_TRANSPORT_SPECIFIC_INFO_MAXSIZE (16*sizeof(long)) #endif /** * @} */ #endif /* __PJMEDIA_CONFIG_H__ */ ag-aec/pjsip/pjmedia/include/pjmedia/echo.h0000644000076500000240000001752711300303512020167 0ustar adigeostaff/* $Id: echo.h 2394 2008-12-23 17:27:53Z bennylp $ */ /* * Copyright (C) 2008-2009 Teluu Inc. (http://www.teluu.com) * Copyright (C) 2003-2008 Benny Prijono * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ #ifndef __PJMEDIA_ECHO_H__ #define __PJMEDIA_ECHO_H__ /** * @file echo.h * @brief Echo Cancellation API. */ #include /** * @defgroup PJMEDIA_Echo_Cancel Accoustic Echo Cancellation API * @ingroup PJMEDIA_PORT * @brief Echo Cancellation API. * @{ * * This section describes API to perform echo cancellation to audio signal. * There may be multiple echo canceller implementation in PJMEDIA, ranging * from simple echo suppressor to a full Accoustic Echo Canceller/AEC. By * using this API, application should be able to use which EC backend to * use base on the requirement and capability of the platform. */ PJ_BEGIN_DECL /** * Opaque type for PJMEDIA Echo Canceller state. */ typedef struct pjmedia_echo_state pjmedia_echo_state; /** * Echo cancellation options. */ typedef enum pjmedia_echo_flag { /** * Use any available backend echo canceller algorithm. This is * the default settings. This setting is mutually exclusive with * PJMEDIA_ECHO_SIMPLE and PJMEDIA_ECHO_SPEEX. */ PJMEDIA_ECHO_DEFAULT = 0, /** * Force to use Speex AEC as the backend echo canceller algorithm. * This setting is mutually exclusive with PJMEDIA_ECHO_SIMPLE. */ PJMEDIA_ECHO_SPEEX = 1, /** * If PJMEDIA_ECHO_SIMPLE flag is specified during echo canceller * creation, then a simple echo suppressor will be used instead of * an accoustic echo cancellation. This setting is mutually exclusive * with PJMEDIA_ECHO_SPEEX. */ PJMEDIA_ECHO_SIMPLE = 2, /** * If PJMEDIA_ECHO_AG flag is specified during echo canceller creation, * then the NLMS-PW algorithm will be used as the backend echo * canceller algorithm. * This setting is mutually exclusive with PJMEDIA_ECHO_SPEEX and * PJMEDIA_ECHO_SIMPLE */ PJMEDIA_ECHO_AG = 3, /** * For internal use. */ PJMEDIA_ECHO_ALGO_MASK = 15, /** * If PJMEDIA_ECHO_NO_LOCK flag is specified, no mutex will be created * for the echo canceller, but application will guarantee that echo * canceller will not be called by different threads at the same time. */ PJMEDIA_ECHO_NO_LOCK = 16 } pjmedia_echo_flag; /** * Create the echo canceller. * * @param pool Pool to allocate memory. * @param clock_rate Media clock rate/sampling rate. * @param samples_per_frame Number of samples per frame. * @param tail_ms Tail length, miliseconds. * @param latency_ms Total lacency introduced by playback and * recording device. Set to zero if the latency * is not known. * @param options Options. If PJMEDIA_ECHO_SIMPLE is specified, * then a simple echo suppressor implementation * will be used instead of an accoustic echo * cancellation. * See #pjmedia_echo_flag for other options. * @param p_echo Pointer to receive the Echo Canceller state. * * @return PJ_SUCCESS on success, or the appropriate status. */ PJ_DECL(pj_status_t) pjmedia_echo_create(pj_pool_t *pool, unsigned clock_rate, unsigned samples_per_frame, unsigned tail_ms, unsigned latency_ms, unsigned options, pjmedia_echo_state **p_echo ); /** * Create multi-channel the echo canceller. * * @param pool Pool to allocate memory. * @param clock_rate Media clock rate/sampling rate. * @param channel_count Number of channels. * @param samples_per_frame Number of samples per frame. * @param tail_ms Tail length, miliseconds. * @param latency_ms Total lacency introduced by playback and * recording device. Set to zero if the latency * is not known. * @param options Options. If PJMEDIA_ECHO_SIMPLE is specified, * then a simple echo suppressor implementation * will be used instead of an accoustic echo * cancellation. * See #pjmedia_echo_flag for other options. * @param p_echo Pointer to receive the Echo Canceller state. * * @return PJ_SUCCESS on success, or the appropriate status. */ PJ_DECL(pj_status_t) pjmedia_echo_create2(pj_pool_t *pool, unsigned clock_rate, unsigned channel_count, unsigned samples_per_frame, unsigned tail_ms, unsigned latency_ms, unsigned options, pjmedia_echo_state **p_echo ); /** * Destroy the Echo Canceller. * * @param echo The Echo Canceller. * * @return PJ_SUCCESS on success. */ PJ_DECL(pj_status_t) pjmedia_echo_destroy(pjmedia_echo_state *echo ); /** * Reset the echo canceller. * * @param echo The Echo Canceller. * * @return PJ_SUCCESS on success. */ PJ_DECL(pj_status_t) pjmedia_echo_reset(pjmedia_echo_state *echo ); /** * Let the Echo Canceller know that a frame has been played to the speaker. * The Echo Canceller will keep the frame in its internal buffer, to be used * when cancelling the echo with #pjmedia_echo_capture(). * * @param echo The Echo Canceller. * @param play_frm Sample buffer containing frame to be played * (or has been played) to the playback device. * The frame must contain exactly samples_per_frame * number of samples. * * @return PJ_SUCCESS on success. */ PJ_DECL(pj_status_t) pjmedia_echo_playback(pjmedia_echo_state *echo, pj_int16_t *play_frm ); /** * Let the Echo Canceller know that a frame has been captured from the * microphone. The Echo Canceller will cancel the echo from the captured * signal, using the internal buffer (supplied by #pjmedia_echo_playback()) * as the FES (Far End Speech) reference. * * @param echo The Echo Canceller. * @param rec_frm On input, it contains the input signal (captured * from microphone) which echo is to be removed. * Upon returning this function, this buffer contain * the processed signal with the echo removed. * The frame must contain exactly samples_per_frame * number of samples. * @param options Echo cancellation options, reserved for future use. * Put zero for now. * * @return PJ_SUCCESS on success. */ PJ_DECL(pj_status_t) pjmedia_echo_capture(pjmedia_echo_state *echo, pj_int16_t *rec_frm, unsigned options ); /** * Perform echo cancellation. * * @param echo The Echo Canceller. * @param rec_frm On input, it contains the input signal (captured * from microphone) which echo is to be removed. * Upon returning this function, this buffer contain * the processed signal with the echo removed. * @param play_frm Sample buffer containing frame to be played * (or has been played) to the playback device. * The frame must contain exactly samples_per_frame * number of samples. * @param options Echo cancellation options, reserved for future use. * Put zero for now. * @param reserved Reserved for future use, put NULL for now. * * @return PJ_SUCCESS on success. */ PJ_DECL(pj_status_t) pjmedia_echo_cancel( pjmedia_echo_state *echo, pj_int16_t *rec_frm, const pj_int16_t *play_frm, unsigned options, void *reserved ); PJ_END_DECL /** * @} */ #endif /* __PJMEDIA_ECHO_H__ */ ag-aec/pjsip/pjmedia/src/0000755000076500000240000000000011300303512014617 5ustar adigeostaffag-aec/pjsip/pjmedia/src/pjmedia/0000755000076500000240000000000011300303512016230 5ustar adigeostaffag-aec/pjsip/pjmedia/src/pjmedia/echo_ag.c0000644000076500000240000005200211300303512017760 0ustar adigeostaff/* * Copyright (C) 2009 AG Projects * http://ag-projects.com */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include "echo_internal.h" #define THIS_FILE "echo_ag.c" /* Description: The AG Acoustic Echo Canceler is based on the NLMS-PW AEC coded by Andre Adrian[1] [1] http://www.andreadrian.de/intercom/index.html */ /* dB Values */ const double M0dB = 1.0f; const double M3dB = 0.71f; const double M6dB = 0.50f; const double M9dB = 0.35f; const double M12dB = 0.25f; const double M18dB = 0.125f; const double M24dB = 0.063f; /* dB values for 16bit PCM */ /* MxdB_PCM = 32767 * 10 ^(x / 20) */ const double M10dB_PCM = 10362.0f; const double M20dB_PCM = 3277.0f; const double M25dB_PCM = 1843.0f; const double M30dB_PCM = 1026.0f; const double M35dB_PCM = 583.0f; const double M40dB_PCM = 328.0f; const double M45dB_PCM = 184.0f; const double M50dB_PCM = 104.0f; const double M55dB_PCM = 58.0f; const double M60dB_PCM = 33.0f; const double M65dB_PCM = 18.0f; const double M70dB_PCM = 10.0f; const double M75dB_PCM = 6.0f; const double M80dB_PCM = 3.0f; const double M85dB_PCM = 2.0f; const double M90dB_PCM = 1.0f; const double MAXPCM = 32767.0f; /* Ageing multiplier for LMS memory vector w */ const double LEAKY = 0.9999f; /* Leaky ageing in samples. */ const int AGEING_TIME = 2 * 60 * 8; /* scale factor*/ #define SCALE 10000 /* * Exponential Smoothing or IIR Infinite Impulse Response Filter */ typedef struct ag_filter_iir { double x; double a0; /* controls Transfer Frequency */ } ag_filter_iir; static void ag_filter_iir_init(ag_filter_iir *filter) { filter->x = 0.0f; filter->a0 = 0.01f; /* constant */ } /* Highpass = Signal - Lowpass. Lowpass = Exponential Smoothing */ static double ag_filter_iir_highpass(ag_filter_iir *filter, double in) { filter->x += filter->a0 * (in - filter->x); return in - filter->x; } /* * 35 taps FIR Finite Impulse Response filter * Passband 150Hz to 4kHz for 8kHz sample rate, 300Hz to 8kHz for 16kHz * sample rate. * Coefficients calculated with * www.dsptutor.freeuk.com/KaiserFilterDesign/KaiserFilterDesign.html */ /* FIR Filter Size */ #define FIRF_SIZE 35 /* Kaiser Window FIR Filter, Filter type: High pass * Passband: 150.0 - 4000.0 Hz, Order: 34 * Transition band: 34.0 Hz, Stopband attenuation: 10.0 dB */ const double firf_coefs[FIRF_SIZE + 1] = { -0.016165324, -0.017454365, -0.01871232, -0.019931411, -0.021104068, -0.022222936, -0.02328091, -0.024271343, -0.025187887, -0.02602462, -0.026776174, -0.027437767, -0.028004972, -0.028474221, -0.028842418, -0.029107114, -0.02926664, 0.8524841, -0.02926664, -0.029107114, -0.028842418, -0.028474221, -0.028004972, -0.027437767, -0.026776174, -0.02602462, -0.025187887, -0.024271343, -0.02328091, -0.022222936, -0.021104068, -0.019931411, -0.01871232, -0.017454365, -0.016165324, 0.0 }; typedef struct ag_filter_fir { double z[FIRF_SIZE + 1]; } ag_filter_fir; static void ag_filter_fir_init(ag_filter_fir *filter) { pj_bzero(filter->z, (FIRF_SIZE + 1) * sizeof(filter->z[0])); } static double ag_filter_fir_highpass(ag_filter_fir *filter, double in) { double *z = filter->z; /* alias */ double sum = 0.0; int i; memmove(z + 1, z, FIRF_SIZE * sizeof(double)); z[0] = in; for (i = 0; i < FIRF_SIZE + 1; i++) { sum += firf_coefs[i] * z[i]; } return sum; } /* Recursive single pole IIR Infinite Impulse response High-pass * filter * * Reference: The Scientist and Engineer's Guide to Digital Processing * * output[N] = A0 * input[N] + A1 * input[N-1] + B1 * output[N-1] * * X = exp(-2.0 * pi * Fc) * A0 = (1 + X) / 2 * A1 = -(1 + X) / 2 * B1 = X * Fc = cutoff freq / sample rate */ typedef struct ag_filter_rsiir { double in0, out0; double a0, a1, b1; } ag_filter_rsiir; static void ag_filter_rsiir_init(ag_filter_rsiir *filter, double fc) { filter->in0 = 0.0f; filter->out0 = 0.0f; filter->b1 = expf(-2.0f * M_PI * fc); filter->a0 = (1.0f + filter->b1) / 2.0f; filter->a1 = - filter->a0; } static double ag_filter_rsiir_highpass(ag_filter_rsiir *filter, double in) { double out; out = filter->a0 * in + filter->a1 * filter->in0 + filter->b1 * filter->out0; filter->in0 = in; filter->out0 = out; return out; } /* * The AG echo canceler state */ typedef struct ag_aec_state { unsigned clock_rate; /* Clock rate. */ pj_uint16_t samples_per_frame; /* Frame length in samples */ pj_uint16_t tail_ms; /* Tail length in milliseconds */ pj_uint16_t tail_len; /* Tail length in samples. */ pj_uint16_t tail_ext; /* Tail extenstion for memcopy optimization. */ /* Time domain filters */ ag_filter_iir mic_filter; /* DC-level remove Highpass */ ag_filter_iir spk_filter; ag_filter_fir cutoff; /* 150Hz cut-off Highpass */ ag_filter_rsiir pwhp_spk; /* pre-whitening Highpass for speaker */ ag_filter_rsiir pwhp_err; /* pre-whitening Highpass for error */ double gain; /* Mic signal amplify */ /* Adrian soft decisi DTD (Double Talk Detector) */ double mic_fast, spk_fast; double mic_slow, spk_slow; /* NLMS-pw */ double *spk_samples; /* tap delayed loudspeaker signal */ double *pw_samples; /* pre-whitening tap delayed signal */ double *weights; /* tap weights */ int tail_index; /* optimize: less memory copies */ double vecprod_pws_pws; /* double to avoid loss of precision */ double delta; /* noise floor to stabilize NLMS */ int ageing; pj_bool_t learning; /* is true when learning is active */ pj_bool_t cancelling; /* is true when cancelling is performed */ /* AEC Algorithm tweak constants */ /* minimum energy in xf. Range: M70dB_PCM to M50dB_PCM. Should be * equal to microphone ambient Noise level */ double noise_floor; /* Double Talk Detector Speaker/Microphone Threshold. Range <=1 * Large value (M0dB) is good for Single-Talk Echo cancellation, * small value (M12dB) is good for Doulbe-Talk AEC */ double geigel_thold; /* for Non Linear Processor. Range >0 to 1. Large value (M0dB) is * good for Double-Talk, small value (M12dB) is good for * Single-Talk */ double NLP_attenuation; double avg_spk; } ag_aec_state; /* Util to display the error message for the specified error code */ static int app_perror( const char *sender, const char *title, pj_status_t status) { char errmsg[PJ_ERR_MSG_SIZE]; PJ_UNUSED_ARG(sender); pj_strerror(status, errmsg, sizeof(errmsg)); printf("%s: %s [code=%d]\n", title, errmsg, status); return 1; } /* * Create. */ PJ_DEF(pj_status_t) ag_aec_create(pj_pool_t *pool, unsigned clock_rate, unsigned channel_count, unsigned samples_per_frame, unsigned tail_ms, unsigned options, void **p_state) { ag_aec_state *ec; PJ_UNUSED_ARG(channel_count); PJ_UNUSED_ARG(options); ec = PJ_POOL_ZALLOC_T(pool, struct ag_aec_state); ec->clock_rate = clock_rate; ec->samples_per_frame = (pj_uint16_t)samples_per_frame; ec->tail_ms = (pj_uint16_t)tail_ms; ec->tail_len = (pj_uint16_t)(tail_ms * clock_rate / 1000); /* set an extension for reducing memcopies */ ec->tail_ext = ec->tail_len / 10; /* algorithm tweak constants */ ec->noise_floor = M55dB_PCM; ec->geigel_thold = M6dB; ec->NLP_attenuation = M6dB; /* allocate sample arrays */ ec->spk_samples = (double *) pj_pool_zalloc(pool, (ec->tail_len + ec->tail_ext) * sizeof(double)); ec->pw_samples = (double *) pj_pool_zalloc(pool, (ec->tail_len + ec->tail_ext) * sizeof(double)); ec->weights = (double *) pj_pool_zalloc(pool, ec->tail_len * sizeof(double)); /* memcopy optimizer */ ec->tail_index = ec->tail_ext; ag_aec_reset(ec); PJ_LOG(2, (THIS_FILE, "AG AEC created: tail_len = %d, tail_ext = %d", ec->tail_len, ec->tail_ext)); *p_state = ec; return PJ_SUCCESS; } /* * Destroy. */ PJ_DEF(pj_status_t) ag_aec_destroy(void *state) { PJ_UNUSED_ARG(state); PJ_LOG(2, (THIS_FILE, "AG AEC destroyed")); return PJ_SUCCESS; } static void ag_aec_setambient(ag_aec_state *ec, double min_pws) { /* subtract old delta */ ec->vecprod_pws_pws -= ec->delta; /* compute new delta */ ec->delta = (ec->tail_len - 1) * min_pws * min_pws; /* add new delta */ ec->vecprod_pws_pws += ec->delta; ec->avg_spk = min_pws; } /* * Hard reset */ PJ_DEF(void) ag_aec_reset(void *state) { ag_aec_state *ec = (ag_aec_state*) state; ec->delta = 0.0f; ec->vecprod_pws_pws = 0.0f; ag_aec_setambient(ec, ec->noise_floor); ec->mic_fast = ec->mic_slow = M75dB_PCM; ec->spk_fast = ec->spk_slow = M80dB_PCM; ec->gain = 1.0f; /* init time domain filters */ ag_filter_iir_init(&ec->mic_filter); ag_filter_iir_init(&ec->spk_filter); ag_filter_fir_init(&ec->cutoff); ag_filter_rsiir_init(&ec->pwhp_spk, 2000.0f/ec->clock_rate); ag_filter_rsiir_init(&ec->pwhp_err, 2000.0f/ec->clock_rate); ec->learning = PJ_FALSE; ec->cancelling = PJ_FALSE; PJ_LOG(2, (THIS_FILE, "AG AEC reset ok, vecprod_pws_pws = %f, delta = %f, noise floor = %f", ec->vecprod_pws_pws, ec->delta, ec->noise_floor)); } /* * Soft reset to force the EC to re-learn without having to discard * all rec and playback history. */ PJ_DEF(void) ag_aec_soft_reset(void *state) { PJ_LOG(4,(THIS_FILE, "AG AEC soft reset. Re-learning..")); } /* * Adrian soft decision DTD (Dual Average Near-End to Far-End signal * Ratio DTD) This algorithm uses exponential smoothing with differnt * ageing parameters to get fast and slow near-end and far-end signal * averages. The ratio of NFRs term (mic_fast / spk_fast) / (mic_slow / spk_slow) * is used to compute the stepsize A ratio value of 2.5 is mapped to * stepsize 0, a ratio of 0 is mapped to 1.0 with a limited linear * function. */ /* left point. X is ratio, Y is stepsize */ const double STEPX1 = 1.0; const double STEPY1 = 1.0; /* right point. STEPX2=2.0 is good double talk, 3.0 is good single talk. */ const double STEPX2 = 2; const double STEPY2 = 0; const double ALPHAFAST = 1.0f / 100.0f; const double ALPHASLOW = 1.0f / 20000.0f; #define TALK_LIMIT M60dB_PCM static double ag_aec_varstepsize(ag_aec_state *ec) { double stepsize; double M = (STEPY2 - STEPY1) / (STEPX2 - STEPX1); /* ratio of NFRs */ double ratio = (ec->mic_fast * ec->spk_slow) / (ec->mic_slow * ec->spk_fast); /* begrenzte lineare Kennlinie */ if (ratio < STEPX1) { stepsize = STEPY1; } else if (ratio > STEPX2) { stepsize = STEPY2; } else { /* Punktrichtungsform einer Geraden */ stepsize = M * (ratio - STEPX1) + STEPY1; } return stepsize; } /* Double-Talk Detector * * in mic_sample: microphone sample (PCM as floating point value) in * spk_sample: loudspeaker sample (PCM as floating point value) * return: from 0 for doubletalk to 1.0 for single talk * */ static double ag_aec_dtd(ag_aec_state *ec, double mic_sample, double spk_sample) { double stepsize; /* fast near-end and far-end average */ ec->mic_fast += ALPHAFAST * (fabsf(mic_sample) - ec->mic_fast); ec->spk_fast += ALPHAFAST * (fabsf(spk_sample) - ec->spk_fast); /* slow near-end and far-end average */ ec->mic_slow += ALPHASLOW * (fabsf(mic_sample) - ec->mic_slow); ec->spk_slow += ALPHASLOW * (fabsf(spk_sample) - ec->spk_slow); //PJ_LOG(2, (THIS_FILE, "Spk:%f, Mic: %f, Limit: %f", ec->spk_fast, ec->mic_fast, M70dB_PCM)); ec->learning = PJ_TRUE; ec->cancelling = PJ_TRUE; stepsize = ag_aec_varstepsize(ec); if (ec->spk_fast < TALK_LIMIT || ec->mic_fast < TALK_LIMIT) { ec->learning = PJ_FALSE; ec->cancelling = PJ_TRUE; } /* if ((ec->spk_fast < TALK_LIMIT) && (ec->mic_fast < TALK_LIMIT)) { */ /* //PJ_LOG(2, (THIS_FILE, "Idle line")); */ /* /\* idle line, do nothing *\/ */ /* ec->learning = PJ_FALSE; */ /* ec->cancelling = PJ_FALSE; */ /* pj_bzero(ec->weights, ec->tail_len); */ /* stepsize = ag_aec_varstepsize(ec); */ /* } else if ((ec->spk_fast >= TALK_LIMIT) && (ec->mic_fast < TALK_LIMIT)) { */ /* //PJ_LOG(2, (THIS_FILE, "Far end active, Near end idle")); */ /* //PJ_LOG(2, (THIS_FILE, "Spk:%f, Mic: %f, Limit: %f", ec->spk_fast, ec->mic_fast, limit)); */ /* /\* FarEnd active, NearEnd idle *\/ */ /* ec->learning = PJ_TRUE; */ /* ec->cancelling = PJ_TRUE; */ /* stepsize = ag_aec_varstepsize(ec); */ /* } else if ((ec->spk_fast < TALK_LIMIT) && (ec->mic_fast >= TALK_LIMIT)) { */ /* //PJ_LOG(2, (THIS_FILE, "Far end idle, Near end active")); */ /* ec->learning = PJ_TRUE; */ /* ec->cancelling = PJ_TRUE; */ /* stepsize = ag_aec_varstepsize(ec); */ /* } else { */ /* //PJ_LOG(2, (THIS_FILE, "DTD")); */ /* //return 0.0; */ /* /\* double talk *\/ */ /* ec->learning = PJ_TRUE; */ /* ec->cancelling = PJ_TRUE; */ /* stepsize = ag_aec_varstepsize(ec); */ /* } */ return stepsize; } /* * The spk_fast signal is used to charge the ageing timer to Thold. * When ageing expires (no Spk signal for some time) the vector w is * erased. This is my implementation of Leaky NLMS. */ static void ag_aec_leaky(ag_aec_state *ec) { if (ec->spk_fast >= TALK_LIMIT) { /* vector weight is valid for ageing Thold time */ ec->ageing = AGEING_TIME; } else { if (ec->ageing > 1) { ec->ageing--; } else if (1 == ec->ageing) { ec->ageing--; /* Leaky NLMS is to erase vector w when ageing expires */ pj_bzero(ec->weights, ec->tail_len); } } } /* vector dot product */ static double ag_vec_prod(double *a, double *b, int size) { double sum = 0.0; int i; for (i = 0; i < size; i++) { sum += a[i] * b[i]; } return sum; } /* Normalized Least Mean Square Algorithm pre-whitening (NLMS-pw) The * LMS algorithm was developed by Bernard Widrow book: Haykin, * Adaptive Filter Theory, 4. edition, Prentice Hall, 2002 * * in mic_sample: microphone sample (16bit PCM value) in spk_sample: * loudspeaker sample (16bit PCM value) in stepsize: NLMS adaptation * variable return: echo cancelled microphone sample */ static double ag_aec_nlms_pw(ag_aec_state *ec, double mic_sample, double spk_sample, double stepsize) { int j = ec->tail_index; ec->spk_samples[j] = spk_sample; ec->pw_samples[j] = ag_filter_rsiir_highpass(&ec->pwhp_spk, spk_sample); /* optimize: iterative vec_prod(ec->pw_samples, ec->pw_samples) */ ec->vecprod_pws_pws += (ec->pw_samples[j] * ec->pw_samples[j]); ec->vecprod_pws_pws -= (ec->pw_samples[j + ec->tail_len - 1] * ec->pw_samples[j + ec->tail_len - 1]); /* average speaker signal */ //ec->avg_spk += (ec->pw_samples[j] - ec->pw_samples[j + ec->tail_len - 1]) / (ec->tail_len - 1); /* compute echo cancelled signal (mic signal - estimated mic signal from * spk signal) */ double ec_signal = mic_sample; double pw_ec = ec_signal; if (ec->cancelling == PJ_TRUE) { ec_signal -= ag_vec_prod(ec->weights, ec->spk_samples + j, ec->tail_len); /* pre-whitening of echo cancelled signal */ pw_ec = ag_filter_rsiir_highpass(&ec->pwhp_err, ec_signal); } //PJ_LOG(2, (THIS_FILE, "Mic signal: %f; EC signal: %f; Pw-echo-cancelled: %f; ageing: %d", mic_sample, ec_signal, pw_ec, ec->ageing)); if (ec->learning == PJ_TRUE) { /* calculate variable step size */ //double mikro_ef = ec->avg_spk * stepsize / ec->vecprod_pws_pws; //double vecprod_pws_pws = ag_vec_prod(ec->pw_samples + j, ec->pw_samples + j, ec->tail_len); //PJ_LOG(2, (THIS_FILE, "vecprod_pws_pws: %lf default: %lf", vecprod_pws_pws, ec->vecprod_pws_pws)); double mikro_ef = stepsize * pw_ec / ec->vecprod_pws_pws; /* update tap weights (filter learning) */ //PJ_LOG(2, (THIS_FILE, "filter learning: %f %lf %f %f %f", ec->avg_spk, SCALE * stepsize / ec->vecprod_pws_pws, pw_ec, ec->vecprod_pws_pws, 1 / ec->vecprod_pws_pws)); int i; for (i = 0; i < ec->tail_len; i++) { ec->weights[i] += mikro_ef * ec->pw_samples[i + j]; } } ec->tail_index--; if (ec->tail_index < 0) { /* optimize: decrease number of memory copies */ ec->tail_index = ec->tail_ext; j = ec->tail_ext; memmove(ec->spk_samples + j + 1, ec->spk_samples, (ec->tail_len - 1) * sizeof(double)); memmove(ec->pw_samples + j + 1, ec->pw_samples, (ec->tail_len - 1) * sizeof(double)); } /* saturation */ double sat = ec_signal; if (ec_signal > MAXPCM) { sat = MAXPCM; } else if (ec_signal < -MAXPCM) { sat = -MAXPCM; } return sat; } /* Acoustic Echo Cancellation and Suppression of one sample in mic: * microphone signal with echo in spk: loudspeaker signal return: echo * cancelled microphone signal */ static int ag_aec_compute(ag_aec_state *ec, int mic, int spk) { double mic_sample = (double) mic; double spk_sample = (double) spk; //PJ_LOG(2, (THIS_FILE, "Initial mic signal: %d %f", mic, mic_sample)); /* Mic Highpass Filter - to remove DC */ mic_sample = ag_filter_iir_highpass(&ec->mic_filter, mic_sample); //PJ_LOG(2, (THIS_FILE, "after iir_highpass: %f", mic_sample)); /* Mic Highpass Filter - cut-off below 300Hz */ mic_sample = ag_filter_fir_highpass(&ec->cutoff, mic_sample); //PJ_LOG(2, (THIS_FILE, "after fir_highpass: %f", mic_sample)); /* Amplify, for e.g. Soundcards with -6dB max. volume */ mic_sample *= ec->gain; //PJ_LOG(2, (THIS_FILE, "after amplify: %f", mic_sample)); /* Spk Highpass Filter - to remove DC */ spk_sample = ag_filter_iir_highpass(&ec->spk_filter, spk_sample); //PJ_LOG(2, (THIS_FILE, "spk after iir_highpass %f", spk_sample)); /* Double Talk Detector */ double stepsize = ag_aec_dtd(ec, mic_sample, spk_sample); //PJ_LOG(2, (THIS_FILE, "stepsize= %f", stepsize)); /* Leaky (ageing of vector weights) */ ag_aec_leaky(ec); /* Acoustic Echo Cancellation */ mic_sample = ag_aec_nlms_pw(ec, mic_sample, spk_sample, stepsize); //PJ_LOG(2, (THIS_FILE, "after nlms: %f (int) %d", mic_sample, (int)roundf(mic_sample))); return (int) roundf(mic_sample); } #define SAMPLE_CHARS 32 static void log_frame(ag_aec_state *ec, pj_int16_t *frm) { int i, bytes; char *buf, *pbuf; buf = malloc(SAMPLE_CHARS * ec->samples_per_frame); buf[0] = 0; pbuf = buf; for (i = 0; i < ec->samples_per_frame; i++) { bytes = sprintf(pbuf, "%10d", frm[i]); pbuf += bytes; } PJ_LOG(2, (THIS_FILE, buf)); free(buf); } static void log_samples(ag_aec_state *ec, double *v) { int i, bytes; char *buf, *pbuf; buf = malloc(SAMPLE_CHARS * (ec->tail_len + ec->tail_ext)); buf[0] = 0; pbuf = buf; for (i = ec->tail_index; i < ec->tail_len + ec->tail_index; i++) { bytes = sprintf(pbuf, "%10.2f", v[i]); pbuf += bytes; } PJ_LOG(2, (THIS_FILE, buf)); free(buf); } static void log_weights(ag_aec_state *ec) { int i, bytes; char *buf, *pbuf; buf = malloc(SAMPLE_CHARS * ec->tail_len); buf[0] = 0; pbuf = buf; for (i = 0; i < ec->tail_len; i++) { bytes = sprintf(pbuf, "%10f", ec->weights[i]); pbuf += bytes; } PJ_LOG(2, (THIS_FILE, buf)); free(buf); } /* * Perform echo cancellation. */ PJ_DEF(pj_status_t) ag_aec_cancel_echo(void *state, pj_int16_t *rec_frm, const pj_int16_t *play_frm, unsigned options, void *reserved) { ag_aec_state *ec = (ag_aec_state*) state; int unsigned i; PJ_UNUSED_ARG(options); PJ_UNUSED_ARG(reserved); //PJ_LOG(2, (THIS_FILE, "Before AEC processing...")); //log_frame(ec, rec_frm); /* perform echo cancellation on each sample from the frame */ for (i = 0; i < ec->samples_per_frame; i++) { rec_frm[i] = ag_aec_compute(ec, rec_frm[i], play_frm[i]); } //PJ_LOG(2, (THIS_FILE, "Samples, tail_index = %d :", ec->tail_index)); //log_samples(ec, ec->spk_samples); //PJ_LOG(2, (THIS_FILE, "After AEC processing...")); //log_frame(ec, rec_frm); //PJ_LOG(2, (THIS_FILE, "Weights:")); //log_weights(ec); //PJ_LOG(2, (THIS_FILE, "Samples:")); //log_pwsamples(ec); return PJ_SUCCESS; } ag-aec/pjsip/pjmedia/src/pjmedia/echo_common.c0000644000076500000240000002303111300303512020661 0ustar adigeostaff/* $Id: echo_common.c 2790 2009-06-24 15:26:59Z nanang $ */ /* * Copyright (C) 2008-2009 Teluu Inc. (http://www.teluu.com) * Copyright (C) 2003-2008 Benny Prijono * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ #include #include #include #include #include #include #include #include "echo_internal.h" #define THIS_FILE "echo_common.c" typedef struct ec_operations ec_operations; struct frame { PJ_DECL_LIST_MEMBER(struct frame); short buf[1]; }; struct pjmedia_echo_state { pj_pool_t *pool; char *obj_name; unsigned samples_per_frame; void *state; ec_operations *op; pj_bool_t lat_ready; /* lat_buf has been filled in. */ unsigned lat_target_cnt;/* Target number of frames in lat_buf */ unsigned lat_buf_cnt; /* Actual number of frames in lat_buf */ struct frame lat_buf; /* Frame queue for delayed playback */ struct frame lat_free; /* Free frame list. */ pjmedia_delay_buf *delay_buf; }; struct ec_operations { const char *name; pj_status_t (*ec_create)(pj_pool_t *pool, unsigned clock_rate, unsigned channel_count, unsigned samples_per_frame, unsigned tail_ms, unsigned options, void **p_state ); pj_status_t (*ec_destroy)(void *state ); void (*ec_reset)(void *state ); pj_status_t (*ec_cancel)(void *state, pj_int16_t *rec_frm, const pj_int16_t *play_frm, unsigned options, void *reserved ); }; static struct ec_operations echo_supp_op = { "Echo suppressor", &echo_supp_create, &echo_supp_destroy, &echo_supp_reset, &echo_supp_cancel_echo }; static struct ec_operations ag_aec_op = { "AG AEC", &ag_aec_create, &ag_aec_destroy, &ag_aec_reset, &ag_aec_cancel_echo }; /* * Speex AEC prototypes */ #if defined(PJMEDIA_HAS_SPEEX_AEC) && PJMEDIA_HAS_SPEEX_AEC!=0 static struct ec_operations speex_aec_op = { "AEC", &speex_aec_create, &speex_aec_destroy, &speex_aec_reset, &speex_aec_cancel_echo }; #endif /* * IPP AEC prototypes */ #if defined(PJMEDIA_HAS_INTEL_IPP_AEC) && PJMEDIA_HAS_INTEL_IPP_AEC!=0 static struct ec_operations ipp_aec_op = { "IPP AEC", &ipp_aec_create, &ipp_aec_destroy, &ipp_aec_reset, &ipp_aec_cancel_echo }; #endif /* * Create the echo canceller. */ PJ_DEF(pj_status_t) pjmedia_echo_create( pj_pool_t *pool, unsigned clock_rate, unsigned samples_per_frame, unsigned tail_ms, unsigned latency_ms, unsigned options, pjmedia_echo_state **p_echo ) { return pjmedia_echo_create2(pool, clock_rate, 1, samples_per_frame, tail_ms, latency_ms, options, p_echo); } /* * Create the echo canceller. */ PJ_DEF(pj_status_t) pjmedia_echo_create2(pj_pool_t *pool, unsigned clock_rate, unsigned channel_count, unsigned samples_per_frame, unsigned tail_ms, unsigned latency_ms, unsigned options, pjmedia_echo_state **p_echo ) { unsigned ptime; pjmedia_echo_state *ec; pj_status_t status; /* Create new pool and instantiate and init the EC */ pool = pj_pool_create(pool->factory, "ec%p", 256, 256, NULL); ec = PJ_POOL_ZALLOC_T(pool, struct pjmedia_echo_state); ec->pool = pool; ec->obj_name = pool->obj_name; pj_list_init(&ec->lat_buf); pj_list_init(&ec->lat_free); /* hard-code to use AG AEC */ options = options & ~PJMEDIA_ECHO_ALGO_MASK; /* Select the backend algorithm */ if (0) { /* Dummy */ ; #if defined(PJMEDIA_HAS_AG_AEC) && PJMEDIA_HAS_AG_AEC!=0 } else if ((options & PJMEDIA_ECHO_ALGO_MASK) == PJMEDIA_ECHO_AG || (options & PJMEDIA_ECHO_ALGO_MASK) == PJMEDIA_ECHO_DEFAULT) { ec->op = &ag_aec_op; #endif #if defined(PJMEDIA_HAS_SPEEX_AEC) && PJMEDIA_HAS_SPEEX_AEC!=0 } else if ((options & PJMEDIA_ECHO_ALGO_MASK) == PJMEDIA_ECHO_SPEEX || (options & PJMEDIA_ECHO_ALGO_MASK) == PJMEDIA_ECHO_DEFAULT) { ec->op = &speex_aec_op; #endif #if defined(PJMEDIA_HAS_INTEL_IPP_AEC) && PJMEDIA_HAS_INTEL_IPP_AEC!=0 } else if ((options & PJMEDIA_ECHO_ALGO_MASK) == PJMEDIA_ECHO_IPP || (options & PJMEDIA_ECHO_ALGO_MASK) == PJMEDIA_ECHO_DEFAULT) { ec->op = &ipp_aec_op; #endif } else { ec->op = &echo_supp_op; } PJ_LOG(5,(ec->obj_name, "Creating %s", ec->op->name)); /* Instantiate EC object */ status = (*ec->op->ec_create)(pool, clock_rate, channel_count, samples_per_frame, tail_ms, options, &ec->state); if (status != PJ_SUCCESS) { pj_pool_release(pool); return status; } /* Create latency buffers */ ptime = samples_per_frame * 1000 / clock_rate; if (latency_ms == 0) { /* Give at least one frame delay to simplify programming */ latency_ms = ptime; } ec->lat_target_cnt = latency_ms / ptime; if (ec->lat_target_cnt != 0) { unsigned i; for (i=0; i < ec->lat_target_cnt; ++i) { struct frame *frm; frm = (struct frame*) pj_pool_alloc(pool, (samples_per_frame<<1) + sizeof(struct frame)); pj_list_push_back(&ec->lat_free, frm); } } else { ec->lat_ready = PJ_TRUE; } /* Create delay buffer to compensate drifts */ status = pjmedia_delay_buf_create(ec->pool, ec->obj_name, clock_rate, samples_per_frame, channel_count, (PJMEDIA_SOUND_BUFFER_COUNT+1) * ptime, 0, &ec->delay_buf); if (status != PJ_SUCCESS) { pj_pool_release(pool); return status; } PJ_LOG(4,(ec->obj_name, "%s created, clock_rate=%d, channel=%d, " "samples per frame=%d, tail length=%d ms, " "latency=%d ms", ec->op->name, clock_rate, channel_count, samples_per_frame, tail_ms, latency_ms)); /* Done */ *p_echo = ec; return PJ_SUCCESS; } /* * Destroy the Echo Canceller. */ PJ_DEF(pj_status_t) pjmedia_echo_destroy(pjmedia_echo_state *echo ) { (*echo->op->ec_destroy)(echo->state); if (echo->delay_buf) { pjmedia_delay_buf_destroy(echo->delay_buf); echo->delay_buf = NULL; } pj_pool_release(echo->pool); return PJ_SUCCESS; } /* * Reset the echo canceller. */ PJ_DEF(pj_status_t) pjmedia_echo_reset(pjmedia_echo_state *echo ) { while (!pj_list_empty(&echo->lat_buf)) { struct frame *frm; frm = echo->lat_buf.next; pj_list_erase(frm); pj_list_push_back(&echo->lat_free, frm); } echo->lat_ready = PJ_FALSE; pjmedia_delay_buf_reset(echo->delay_buf); echo->op->ec_reset(echo->state); return PJ_SUCCESS; } /* * Let the Echo Canceller know that a frame has been played to the speaker. */ PJ_DEF(pj_status_t) pjmedia_echo_playback( pjmedia_echo_state *echo, pj_int16_t *play_frm ) { if (!echo->lat_ready) { /* We've not built enough latency in the buffer, so put this frame * in the latency buffer list. */ struct frame *frm; if (pj_list_empty(&echo->lat_free)) { echo->lat_ready = PJ_TRUE; PJ_LOG(5,(echo->obj_name, "Latency bufferring complete")); pjmedia_delay_buf_put(echo->delay_buf, play_frm); return PJ_SUCCESS; } frm = echo->lat_free.prev; pj_list_erase(frm); pjmedia_copy_samples(frm->buf, play_frm, echo->samples_per_frame); pj_list_push_back(&echo->lat_buf, frm); } else { /* Latency buffer is ready (full), so we put this frame in the * delay buffer. */ pjmedia_delay_buf_put(echo->delay_buf, play_frm); } return PJ_SUCCESS; } /* * Let the Echo Canceller knows that a frame has been captured from * the microphone. */ PJ_DEF(pj_status_t) pjmedia_echo_capture( pjmedia_echo_state *echo, pj_int16_t *rec_frm, unsigned options ) { struct frame *oldest_frm; pj_status_t status, rc; if (!echo->lat_ready) { /* Prefetching to fill in the desired latency */ PJ_LOG(5,(echo->obj_name, "Prefetching..")); return PJ_SUCCESS; } /* Retrieve oldest frame from the latency buffer */ oldest_frm = echo->lat_buf.next; pj_list_erase(oldest_frm); /* Cancel echo using this reference frame */ status = pjmedia_echo_cancel(echo, rec_frm, oldest_frm->buf, options, NULL); /* Move one frame from delay buffer to the latency buffer. */ rc = pjmedia_delay_buf_get(echo->delay_buf, oldest_frm->buf); if (rc != PJ_SUCCESS) { /* Ooops.. no frame! */ PJ_LOG(5,(echo->obj_name, "No frame from delay buffer. This will upset EC later")); pjmedia_zero_samples(oldest_frm->buf, echo->samples_per_frame); } pj_list_push_back(&echo->lat_buf, oldest_frm); return status; } /* * Perform echo cancellation. */ PJ_DEF(pj_status_t) pjmedia_echo_cancel( pjmedia_echo_state *echo, pj_int16_t *rec_frm, const pj_int16_t *play_frm, unsigned options, void *reserved ) { return (*echo->op->ec_cancel)( echo->state, rec_frm, play_frm, options, reserved); } ag-aec/pjsip/pjmedia/src/pjmedia/echo_internal.h0000644000076500000240000000577211300303512021226 0ustar adigeostaff/* $Id: echo_internal.h 2394 2008-12-23 17:27:53Z bennylp $ */ /* * Copyright (C) 2008-2009 Teluu Inc. (http://www.teluu.com) * Copyright (C) 2003-2008 Benny Prijono * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ #ifndef __PJMEDIA_ECHO_INTERNAL_H__ #define __PJMEDIA_ECHO_INTERNAL_H__ #include PJ_BEGIN_DECL /* * Simple echo suppressor */ PJ_DECL(pj_status_t) echo_supp_create(pj_pool_t *pool, unsigned clock_rate, unsigned channel_count, unsigned samples_per_frame, unsigned tail_ms, unsigned options, void **p_state ); PJ_DECL(pj_status_t) echo_supp_destroy(void *state); PJ_DECL(void) echo_supp_reset(void *state); PJ_DECL(pj_status_t) echo_supp_cancel_echo(void *state, pj_int16_t *rec_frm, const pj_int16_t *play_frm, unsigned options, void *reserved ); PJ_DECL(pj_status_t) speex_aec_create(pj_pool_t *pool, unsigned clock_rate, unsigned channel_count, unsigned samples_per_frame, unsigned tail_ms, unsigned options, void **p_state ); PJ_DECL(pj_status_t) speex_aec_destroy(void *state ); PJ_DECL(void) speex_aec_reset(void *state ); PJ_DECL(pj_status_t) speex_aec_cancel_echo(void *state, pj_int16_t *rec_frm, const pj_int16_t *play_frm, unsigned options, void *reserved ); PJ_DECL(pj_status_t) ipp_aec_create(pj_pool_t *pool, unsigned clock_rate, unsigned channel_count, unsigned samples_per_frame, unsigned tail_ms, unsigned options, void **p_echo ); PJ_DECL(pj_status_t) ipp_aec_destroy(void *state ); PJ_DECL(void) ipp_aec_reset(void *state ); PJ_DECL(pj_status_t) ipp_aec_cancel_echo(void *state, pj_int16_t *rec_frm, const pj_int16_t *play_frm, unsigned options, void *reserved ); PJ_DECL(pj_status_t) ag_aec_create(pj_pool_t *pool, unsigned clock_rate, unsigned channel_count, unsigned samples_per_frame, unsigned tail_ms, unsigned options, void **p_state ); PJ_DECL(pj_status_t) ag_aec_destroy(void *state); PJ_DECL(void) ag_aec_reset(void *state); PJ_DECL(pj_status_t) ag_aec_cancel_echo(void *state, pj_int16_t *rec_frm, const pj_int16_t *play_frm, unsigned options, void *reserved); PJ_END_DECL #endif ag-aec/pjsip/pjsip-apps/0000755000076500000240000000000011300303512014505 5ustar adigeostaffag-aec/pjsip/pjsip-apps/build/0000755000076500000240000000000011300303512015604 5ustar adigeostaffag-aec/pjsip/pjsip-apps/build/Samples.mak0000644000076500000240000000300711300303512017702 0ustar adigeostaff include ../../build/common.mak ############################################################################### # Gather all flags. # export _CFLAGS := $(PJ_CFLAGS) $(CFLAGS) export _CXXFLAGS:= $(PJ_CXXFLAGS) export _LDFLAGS := $(PJ_LDFLAGS) $(PJ_LDLIBS) $(LDFLAGS) SRCDIR := ../src/samples OBJDIR := ./output/samples-$(TARGET_NAME) BINDIR := ../bin/samples SAMPLES := confsample \ encdec \ latency \ level \ mix \ pjsip-perf \ pcaputil \ playfile \ playsine \ recfile \ resampleplay \ simpleua \ simple_pjsua \ siprtp \ sipstateless \ sndinfo \ sndtest \ stateful_proxy \ stateless_proxy \ stereotest \ streamutil \ strerror \ tonegen \ aectest EXES := $(foreach file, $(SAMPLES), $(BINDIR)/$(file)-$(TARGET_NAME)$(HOST_EXE)) all: $(OBJDIR) $(EXES) $(BINDIR)/%-$(TARGET_NAME)$(HOST_EXE): $(OBJDIR)/%$(OBJEXT) $(PJ_LIB_FILES) $(LD) $(LDOUT)$(subst /,$(HOST_PSEP),$@) \ $(subst /,$(HOST_PSEP),$<) \ $(_LDFLAGS) $(OBJDIR)/%$(OBJEXT): $(SRCDIR)/%.c $(CC) $(_CFLAGS) \ $(CC_OUT)$(subst /,$(HOST_PSEP),$@) \ $(subst /,$(HOST_PSEP),$<) $(OBJDIR): $(subst @@,$(subst /,$(HOST_PSEP),$@),$(HOST_MKDIR)) depend: clean: $(subst @@,$(subst /,$(HOST_PSEP),$(OBJDIR)/*),$(HOST_RMR)) $(subst @@,$(subst /,$(HOST_PSEP),$(OBJDIR)),$(HOST_RMDIR)) $(subst @@,$(EXES),$(HOST_RM)) distclean realclean: clean # $(subst @@,$(subst /,$(HOST_PSEP),$(EXES)) $(subst /,$(HOST_PSEP),$(EXES)),$(HOST_RM)) # $(subst @@,$(DEP_FILE),$(HOST_RM)) ag-aec/pjsip/pjsip-apps/src/0000755000076500000240000000000011300303512015274 5ustar adigeostaffag-aec/pjsip/pjsip-apps/src/samples/0000755000076500000240000000000011300303512016740 5ustar adigeostaffag-aec/pjsip/pjsip-apps/src/samples/aectest.c0000644000076500000240000002013011300303512020530 0ustar adigeostaff/* $Id: aectest.c 2394 2008-12-23 17:27:53Z bennylp $ */ /* * Copyright (C) 2008-2009 Teluu Inc. (http://www.teluu.com) * Copyright (C) 2003-2008 Benny Prijono * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ /** * \page page_pjmedia_samples_aectest_c Samples: AEC Test (aectest.c) * * Play a file to speaker, run AEC, and record the microphone input * to see if echo is coming. * * This file is pjsip-apps/src/samples/aectest.c * * \includelineno aectest.c */ #include #include /* pj_getopt */ #include #define THIS_FILE "aectest.c" #define PTIME 20 #define TAIL_LENGTH 200 static const char *desc = " FILE \n" " \n" " aectest.c \n" " \n" " PURPOSE \n" " \n" " Test the AEC effectiveness. \n" " \n" " USAGE \n" " \n" " aectest [options] \n" " \n" " is the signal played to the speaker. \n" " is the signal captured from the microphone. \n" " is the output file to store the test result \n" "\n" " options:\n" " -d The delay between playback and capture in ms. Default is zero.\n" " -l Set the echo tail length in ms. Default is 200 ms \n" " -r Set repeat count (default=1) \n" " -a Algorithm: 0=default, 1=speex, 2=ag aec 3=echo suppress \n" " -i Interactive \n"; /* * Sample session: * * -d 100 -a 1 ../bin/orig8.wav ../bin/echo8.wav ../bin/result8.wav */ static void app_perror(const char *sender, const char *title, pj_status_t st) { char errmsg[PJ_ERR_MSG_SIZE]; pj_strerror(st, errmsg, sizeof(errmsg)); PJ_LOG(3,(sender, "%s: %s", title, errmsg)); } /* * main() */ int main(int argc, char *argv[]) { pj_caching_pool cp; pjmedia_endpt *med_endpt; pj_pool_t *pool; pjmedia_port *wav_play; pjmedia_port *wav_rec; pjmedia_port *wav_out; pj_status_t status; pjmedia_echo_state *ec; pjmedia_frame play_frame, rec_frame; unsigned opt = 0; unsigned latency_ms = 0; unsigned tail_ms = TAIL_LENGTH; pj_timestamp t0, t1; int i, repeat=1, interactive=0, c; pj_optind = 0; while ((c=pj_getopt(argc, argv, "d:l:a:r:i")) !=-1) { switch (c) { case 'd': latency_ms = atoi(pj_optarg); break; case 'l': tail_ms = atoi(pj_optarg); break; case 'a': { int alg = atoi(pj_optarg); switch (alg) { case 0: opt = 0; case 1: opt = PJMEDIA_ECHO_SPEEX; break; case 2: opt = PJMEDIA_ECHO_AG; break; case 3: opt = PJMEDIA_ECHO_SIMPLE; break; default: puts("Invalid algorithm"); puts(desc); return 1; } } break; case 'r': repeat = atoi(pj_optarg); if (repeat < 1) { puts("Invalid algorithm"); puts(desc); return 1; } break; case 'i': interactive = 1; break; } } if (argc - pj_optind != 3) { puts("Error: missing argument(s)"); puts(desc); return 1; } /* Must init PJLIB first: */ status = pj_init(); PJ_ASSERT_RETURN(status == PJ_SUCCESS, 1); /* Must create a pool factory before we can allocate any memory. */ pj_caching_pool_init(&cp, &pj_pool_factory_default_policy, 0); /* * Initialize media endpoint. * This will implicitly initialize PJMEDIA too. */ status = pjmedia_endpt_create(&cp.factory, NULL, 1, &med_endpt); PJ_ASSERT_RETURN(status == PJ_SUCCESS, 1); /* Create memory pool for our file player */ pool = pj_pool_create( &cp.factory, /* pool factory */ "wav", /* pool name. */ 4000, /* init size */ 4000, /* increment size */ NULL /* callback on error */ ); /* Open wav_play */ status = pjmedia_wav_player_port_create(pool, argv[pj_optind], PTIME, PJMEDIA_FILE_NO_LOOP, 0, &wav_play); if (status != PJ_SUCCESS) { app_perror(THIS_FILE, "Error opening playback WAV file", status); return 1; } /* Open recorded wav */ status = pjmedia_wav_player_port_create(pool, argv[pj_optind+1], PTIME, PJMEDIA_FILE_NO_LOOP, 0, &wav_rec); if (status != PJ_SUCCESS) { app_perror(THIS_FILE, "Error opening recorded WAV file", status); return 1; } /* play and rec WAVs must have the same clock rate */ if (wav_play->info.clock_rate != wav_rec->info.clock_rate) { puts("Error: clock rate mismatch in the WAV files"); return 1; } /* .. and channel count */ if (wav_play->info.channel_count != wav_rec->info.channel_count) { puts("Error: clock rate mismatch in the WAV files"); return 1; } /* Create output wav */ status = pjmedia_wav_writer_port_create(pool, argv[pj_optind+2], wav_play->info.clock_rate, wav_play->info.channel_count, wav_play->info.samples_per_frame, wav_play->info.bits_per_sample, 0, 0, &wav_out); if (status != PJ_SUCCESS) { app_perror(THIS_FILE, "Error opening output WAV file", status); return 1; } /* Create echo canceller */ status = pjmedia_echo_create2(pool, wav_play->info.clock_rate, wav_play->info.channel_count, wav_play->info.samples_per_frame, tail_ms, latency_ms, opt, &ec); if (status != PJ_SUCCESS) { app_perror(THIS_FILE, "Error creating EC", status); return 1; } /* Processing loop */ play_frame.buf = pj_pool_alloc(pool, wav_play->info.samples_per_frame<<1); rec_frame.buf = pj_pool_alloc(pool, wav_play->info.samples_per_frame<<1); pj_get_timestamp(&t0); for (i=0; i < repeat; ++i) { for (;;) { play_frame.size = wav_play->info.samples_per_frame << 1; status = pjmedia_port_get_frame(wav_play, &play_frame); if (status != PJ_SUCCESS) break; status = pjmedia_echo_playback(ec, (short*)play_frame.buf); rec_frame.size = wav_play->info.samples_per_frame << 1; status = pjmedia_port_get_frame(wav_rec, &rec_frame); if (status != PJ_SUCCESS) break; status = pjmedia_echo_capture(ec, (short*)rec_frame.buf, 0); //status = pjmedia_echo_cancel(ec, (short*)rec_frame.buf, // (short*)play_frame.buf, 0, NULL); pjmedia_port_put_frame(wav_out, &rec_frame); } pjmedia_wav_player_port_set_pos(wav_play, 0); pjmedia_wav_player_port_set_pos(wav_rec, 0); } pj_get_timestamp(&t1); i = pjmedia_wav_writer_port_get_pos(wav_out) * 1000 / (wav_out->info.clock_rate * wav_out->info.channel_count); PJ_LOG(3,(THIS_FILE, "Processed %3d.%03ds audio", i / 1000, i % 1000)); PJ_LOG(3,(THIS_FILE, "Completed in %u msec\n", pj_elapsed_msec(&t0, &t1))); /* Destroy file port(s) */ status = pjmedia_port_destroy( wav_play ); PJ_ASSERT_RETURN(status == PJ_SUCCESS, 1); status = pjmedia_port_destroy( wav_rec ); PJ_ASSERT_RETURN(status == PJ_SUCCESS, 1); status = pjmedia_port_destroy( wav_out ); PJ_ASSERT_RETURN(status == PJ_SUCCESS, 1); /* Destroy ec */ pjmedia_echo_destroy(ec); /* Release application pool */ pj_pool_release( pool ); /* Destroy media endpoint. */ pjmedia_endpt_destroy( med_endpt ); /* Destroy pool factory */ pj_caching_pool_destroy( &cp ); /* Shutdown PJLIB */ pj_shutdown(); if (interactive) { char s[10]; puts("ENTER to quit"); fgets(s, sizeof(s), stdin); } /* Done. */ return 0; } ag-aec/python-sipsimple/0000755000076500000240000000000011300303512014616 5ustar adigeostaffag-aec/python-sipsimple/patches/0000755000076500000240000000000011300303512016245 5ustar adigeostaffag-aec/python-sipsimple/patches/pjsip_ag_aec.patch0000644000076500000240000006267411300303512021711 0ustar adigeostaffIndex: pjsip-apps/build/Samples.mak =================================================================== --- pjsip-apps/build/Samples.mak (revision 2924) +++ pjsip-apps/build/Samples.mak (working copy) @@ -35,7 +35,8 @@ stereotest \ streamutil \ strerror \ - tonegen + tonegen \ + aectest EXES := $(foreach file, $(SAMPLES), $(BINDIR)/$(file)-$(TARGET_NAME)$(HOST_EXE)) Index: pjsip-apps/src/samples/aectest.c =================================================================== --- pjsip-apps/src/samples/aectest.c (revision 2924) +++ pjsip-apps/src/samples/aectest.c (working copy) @@ -58,7 +58,7 @@ " -d The delay between playback and capture in ms. Default is zero.\n" " -l Set the echo tail length in ms. Default is 200 ms \n" " -r Set repeat count (default=1) \n" -" -a Algorithm: 0=default, 1=speex, 3=echo suppress \n" +" -a Algorithm: 0=default, 1=speex, 2=ag aec 3=echo suppress \n" " -i Interactive \n"; /* @@ -114,6 +114,9 @@ case 1: opt = PJMEDIA_ECHO_SPEEX; break; + case 2: + opt = PJMEDIA_ECHO_AG; + break; case 3: opt = PJMEDIA_ECHO_SIMPLE; break; Index: pjmedia/build/Makefile =================================================================== --- pjmedia/build/Makefile (revision 2924) +++ pjmedia/build/Makefile (working copy) @@ -63,6 +63,8 @@ wav_player.o wav_playlist.o wav_writer.o wave.o \ wsola.o $(SOUND_OBJS) $(NULLSOUND_OBJS) +export PJMEDIA_OBJS += echo_ag.o + export PJMEDIA_CFLAGS += $(_CFLAGS) Index: pjmedia/include/pjmedia/config.h =================================================================== --- pjmedia/include/pjmedia/config.h (revision 2924) +++ pjmedia/include/pjmedia/config.h (working copy) @@ -451,11 +451,21 @@ * Speex Accoustic Echo Cancellation (AEC). * By default is enabled. */ +/* #ifndef PJMEDIA_HAS_SPEEX_AEC # define PJMEDIA_HAS_SPEEX_AEC 1 #endif +*/ +/** + * Enable the AG AEC algorithm by default + */ +#ifndef PJMEDIA_HAS_AG_AEC +# define PJMEDIA_HAS_AG_AEC 1 +#endif + + /** * This specifies the behavior of the SDP negotiator when responding to an * offer, whether it should rather use the codec preference as set by Index: pjmedia/include/pjmedia/echo.h =================================================================== --- pjmedia/include/pjmedia/echo.h (revision 2924) +++ pjmedia/include/pjmedia/echo.h (working copy) @@ -62,7 +62,7 @@ * the default settings. This setting is mutually exclusive with * PJMEDIA_ECHO_SIMPLE and PJMEDIA_ECHO_SPEEX. */ - PJMEDIA_ECHO_DEFAULT= 0, + PJMEDIA_ECHO_DEFAULT = 0, /** * Force to use Speex AEC as the backend echo canceller algorithm. @@ -79,6 +79,15 @@ PJMEDIA_ECHO_SIMPLE = 2, /** + * If PJMEDIA_ECHO_AG flag is specified during echo canceller creation, + * then the NLMS-PW algorithm will be used as the backend echo + * canceller algorithm. + * This setting is mutually exclusive with PJMEDIA_ECHO_SPEEX and + * PJMEDIA_ECHO_SIMPLE + */ + PJMEDIA_ECHO_AG = 3, + + /** * For internal use. */ PJMEDIA_ECHO_ALGO_MASK = 15, Index: pjmedia/src/pjmedia/echo_internal.h =================================================================== --- pjmedia/src/pjmedia/echo_internal.h (revision 2924) +++ pjmedia/src/pjmedia/echo_internal.h (working copy) @@ -73,6 +73,21 @@ void *reserved ); +PJ_DECL(pj_status_t) ag_aec_create(pj_pool_t *pool, + unsigned clock_rate, + unsigned channel_count, + unsigned samples_per_frame, + unsigned tail_ms, + unsigned options, + void **p_state ); +PJ_DECL(pj_status_t) ag_aec_destroy(void *state); +PJ_DECL(void) ag_aec_reset(void *state); +PJ_DECL(pj_status_t) ag_aec_cancel_echo(void *state, + pj_int16_t *rec_frm, + const pj_int16_t *play_frm, + unsigned options, + void *reserved); + PJ_END_DECL #endif Index: pjmedia/src/pjmedia/echo_ag.c =================================================================== --- pjmedia/src/pjmedia/echo_ag.c (revision 0) +++ pjmedia/src/pjmedia/echo_ag.c (revision 0) @@ -0,0 +1,679 @@ +/* + * Copyright (C) 2009 AG Projects + * http://ag-projects.com + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "echo_internal.h" + +#define THIS_FILE "echo_ag.c" + +/* Description: + + The AG Acoustic Echo Canceler is based on the NLMS-PW AEC coded by + Andre Adrian[1] + + [1] http://www.andreadrian.de/intercom/index.html +*/ + +/* dB Values */ +const double M0dB = 1.0f; +const double M3dB = 0.71f; +const double M6dB = 0.50f; +const double M9dB = 0.35f; +const double M12dB = 0.25f; +const double M18dB = 0.125f; +const double M24dB = 0.063f; + +/* dB values for 16bit PCM */ +/* MxdB_PCM = 32767 * 10 ^(x / 20) */ +const double M10dB_PCM = 10362.0f; +const double M20dB_PCM = 3277.0f; +const double M25dB_PCM = 1843.0f; +const double M30dB_PCM = 1026.0f; +const double M35dB_PCM = 583.0f; +const double M40dB_PCM = 328.0f; +const double M45dB_PCM = 184.0f; +const double M50dB_PCM = 104.0f; +const double M55dB_PCM = 58.0f; +const double M60dB_PCM = 33.0f; +const double M65dB_PCM = 18.0f; +const double M70dB_PCM = 10.0f; +const double M75dB_PCM = 6.0f; +const double M80dB_PCM = 3.0f; +const double M85dB_PCM = 2.0f; +const double M90dB_PCM = 1.0f; + +const double MAXPCM = 32767.0f; + +/* Ageing multiplier for LMS memory vector w */ +const double LEAKY = 0.9999f; + +/* Leaky hangover in taps. */ +const int LEAKY_THOLD = 60 * 8; + +/* scale factor*/ +#define SCALE 10000 + +/* + * Exponential Smoothing or IIR Infinite Impulse Response Filter + */ +typedef struct ag_filter_iir +{ + double x; + double a0; /* controls Transfer Frequency */ +} ag_filter_iir; + +static void ag_filter_iir_init(ag_filter_iir *filter) +{ + filter->x = 0.0f; + filter->a0 = 0.01f; /* constant */ +} + +/* Highpass = Signal - Lowpass. Lowpass = Exponential Smoothing */ +static double ag_filter_iir_highpass(ag_filter_iir *filter, double in) +{ + filter->x += filter->a0 * (in - filter->x); + return in - filter->x; +} + + +/* + * 35 taps FIR Finite Impulse Response filter + * Passband 150Hz to 4kHz for 8kHz sample rate, 300Hz to 8kHz for 16kHz + * sample rate. + * Coefficients calculated with + * www.dsptutor.freeuk.com/KaiserFilterDesign/KaiserFilterDesign.html + */ + +/* FIR Filter Size */ +#define FIRF_SIZE 35 + +/* Kaiser Window FIR Filter, Filter type: High pass + * Passband: 150.0 - 4000.0 Hz, Order: 34 + * Transition band: 34.0 Hz, Stopband attenuation: 10.0 dB + */ +const double firf_coefs[FIRF_SIZE + 1] = { + -0.016165324, -0.017454365, -0.01871232, -0.019931411, + -0.021104068, -0.022222936, -0.02328091, -0.024271343, + -0.025187887, -0.02602462, -0.026776174, -0.027437767, + -0.028004972, -0.028474221, -0.028842418, -0.029107114, + -0.02926664, 0.8524841, -0.02926664, -0.029107114, + -0.028842418, -0.028474221, -0.028004972, -0.027437767, + -0.026776174, -0.02602462, -0.025187887, -0.024271343, + -0.02328091, -0.022222936, -0.021104068, -0.019931411, + -0.01871232, -0.017454365, -0.016165324, 0.0 +}; + +typedef struct ag_filter_fir +{ + double z[FIRF_SIZE + 1]; +} ag_filter_fir; + +static void +ag_filter_fir_init(ag_filter_fir *filter) +{ + pj_bzero(filter->z, (FIRF_SIZE + 1) * sizeof(filter->z[0])); +} + +static double +ag_filter_fir_highpass(ag_filter_fir *filter, double in) +{ + double *z = filter->z; /* alias */ + double sum = 0.0; + int i; + + memmove(z + 1, z, FIRF_SIZE * sizeof(double)); + z[0] = in; + for (i = 0; i < FIRF_SIZE + 1; i++) { + sum += firf_coefs[i] * z[i]; + } + return sum; +} + +/* Recursive single pole IIR Infinite Impulse response High-pass + * filter + * + * Reference: The Scientist and Engineer's Guide to Digital Processing + * + * output[N] = A0 * input[N] + A1 * input[N-1] + B1 * output[N-1] + * + * X = exp(-2.0 * pi * Fc) + * A0 = (1 + X) / 2 + * A1 = -(1 + X) / 2 + * B1 = X + * Fc = cutoff freq / sample rate + */ + +typedef struct ag_filter_rsiir +{ + double in0, out0; + double a0, a1, b1; +} ag_filter_rsiir; + +static void +ag_filter_rsiir_init(ag_filter_rsiir *filter, double fc) +{ + filter->in0 = 0.0f; + filter->out0 = 0.0f; + filter->b1 = expf(-2.0f * M_PI * fc); + filter->a0 = (1.0f + filter->b1) / 2.0f; + filter->a1 = - filter->a0; +} + +static double +ag_filter_rsiir_highpass(ag_filter_rsiir *filter, double in) +{ + double out; + + out = filter->a0 * in + + filter->a1 * filter->in0 + + filter->b1 * filter->out0; + filter->in0 = in; + filter->out0 = out; + return out; +} + +/* + * The AG echo canceler state + */ +typedef struct ag_aec_state +{ + unsigned clock_rate; /* Clock rate. */ + pj_uint16_t samples_per_frame; /* Frame length in samples */ + pj_uint16_t tail_ms; /* Tail length in milliseconds */ + pj_uint16_t tail_len; /* Tail length in samples. */ + pj_uint16_t tail_ext; /* Tail extenstion for memcopy optimization. */ + + /* Time domain filters */ + ag_filter_iir mic_filter; /* DC-level remove Highpass */ + ag_filter_iir spk_filter; + ag_filter_fir cutoff; /* 150Hz cut-off Highpass */ + ag_filter_rsiir pwhp_spk; /* pre-whitening Highpass for speaker */ + ag_filter_rsiir pwhp_err; /* pre-whitening Highpass for error */ + double gain; /* Mic signal amplify */ + + /* Adrian soft decisi DTD (Double Talk Detector) */ + double mic_fast, spk_fast; + double mic_slow, spk_slow; + + /* NLMS-pw */ + double *spk_samples; /* tap delayed loudspeaker signal */ + double *pw_samples; /* pre-whitening tap delayed signal */ + double *weights; /* tap weights */ + int tail_index; /* optimize: less memory copies */ + double vecprod_pws_pws; /* double to avoid loss of precision */ + double delta; /* noise floor to stabilize NLMS */ + + int hangover; + + /* AEC Algorithm tweak constants */ + + /* minimum energy in xf. Range: M70dB_PCM to M50dB_PCM. Should be + * equal to microphone ambient Noise level */ + double noise_floor; + + /* Double Talk Detector Speaker/Microphone Threshold. Range <=1 + * Large value (M0dB) is good for Single-Talk Echo cancellation, + * small value (M12dB) is good for Doulbe-Talk AEC */ + double geigel_thold; + + /* for Non Linear Processor. Range >0 to 1. Large value (M0dB) is + * good for Double-Talk, small value (M12dB) is good for + * Single-Talk */ + double NLP_attenuation; + +} ag_aec_state; + + + +/* + * Create. + */ +PJ_DEF(pj_status_t) +ag_aec_create(pj_pool_t *pool, + unsigned clock_rate, + unsigned channel_count, + unsigned samples_per_frame, + unsigned tail_ms, + unsigned options, + void **p_state) +{ + ag_aec_state *ec; + + PJ_UNUSED_ARG(channel_count); + PJ_UNUSED_ARG(options); + + ec = PJ_POOL_ZALLOC_T(pool, struct ag_aec_state); + ec->clock_rate = clock_rate; + ec->samples_per_frame = (pj_uint16_t)samples_per_frame; + ec->tail_ms = (pj_uint16_t)tail_ms; + ec->tail_len = (pj_uint16_t)(tail_ms * clock_rate / 1000); + /* set an extension for reducing memcopies */ + ec->tail_ext = ec->tail_len / 10; + + /* algorithm tweak constants */ + ec->noise_floor = M55dB_PCM; + ec->geigel_thold = M6dB; + ec->NLP_attenuation = M6dB; + + /* allocate sample arrays */ + ec->spk_samples = (double *) pj_pool_zalloc(pool, (ec->tail_len + ec->tail_ext) * sizeof(double)); + ec->pw_samples = (double *) pj_pool_zalloc(pool, (ec->tail_len + ec->tail_ext) * sizeof(double)); + ec->weights = (double *) pj_pool_zalloc(pool, ec->tail_len * sizeof(double)); + + /* memcopy optimizer */ + ec->tail_index = ec->tail_ext; + ag_aec_reset(ec); + + PJ_LOG(2, (THIS_FILE, "AG AEC created: tail_len = %d, tail_ext = %d", ec->tail_len, ec->tail_ext)); + + *p_state = ec; + return PJ_SUCCESS; +} + + +/* + * Destroy. + */ +PJ_DEF(pj_status_t) +ag_aec_destroy(void *state) +{ + PJ_UNUSED_ARG(state); + + PJ_LOG(2, (THIS_FILE, "AG AEC destroyed")); + return PJ_SUCCESS; +} + + +static void +ag_aec_setambient(ag_aec_state *ec, double min_pws) +{ + /* subtract old delta */ + ec->vecprod_pws_pws -= ec->delta; + /* compute new delta */ + ec->delta = (ec->tail_len - 1) * min_pws * min_pws; + /* add new delta */ + ec->vecprod_pws_pws += ec->delta; +} + +/* + * Hard reset + */ +PJ_DEF(void) +ag_aec_reset(void *state) +{ + ag_aec_state *ec = (ag_aec_state*) state; + + ec->delta = 0.0f; + ec->vecprod_pws_pws = 0.0f; + ag_aec_setambient(ec, ec->noise_floor); + ec->mic_fast = ec->mic_slow = M75dB_PCM; + ec->spk_fast = ec->spk_slow = M80dB_PCM; + ec->gain = 1.0f; + + /* init time domain filters */ + ag_filter_iir_init(&ec->mic_filter); + ag_filter_iir_init(&ec->spk_filter); + ag_filter_fir_init(&ec->cutoff); + ag_filter_rsiir_init(&ec->pwhp_spk, 2000.0f/ec->clock_rate); + ag_filter_rsiir_init(&ec->pwhp_err, 2000.0f/ec->clock_rate); + + PJ_LOG(2, (THIS_FILE, "AG AEC reset ok, vecprod_pws_pws = %f, delta = %f, noise floor = %f", + ec->vecprod_pws_pws, ec->delta, ec->noise_floor)); +} + +/* + * Soft reset to force the EC to re-learn without having to discard + * all rec and playback history. + */ +PJ_DEF(void) +ag_aec_soft_reset(void *state) +{ + PJ_LOG(4,(THIS_FILE, "AG AEC soft reset. Re-learning..")); +} + +/* + * Adrian soft decision DTD (Dual Average Near-End to Far-End signal + * Ratio DTD) This algorithm uses exponential smoothing with differnt + * ageing parameters to get fast and slow near-end and far-end signal + * averages. The ratio of NFRs term (mic_fast / spk_fast) / (mic_slow / spk_slow) + * is used to compute the stepsize A ratio value of 2.5 is mapped to + * stepsize 0, a ratio of 0 is mapped to 1.0 with a limited linear + * function. + */ + +/* left point. X is ratio, Y is stepsize */ +const double STEPX1 = 1.0; +const double STEPY1 = 1.0; +/* right point. STEPX2=2.0 is good double talk, 3.0 is good single talk. */ +const double STEPX2 = 2.5; +const double STEPY2 = 0; +const double ALPHAFAST = 1.0f / 100.0f; +const double ALPHASLOW = 1.0f / 20000.0f; + +/* Double-Talk Detector + * + * in mic_sample: microphone sample (PCM as floating point value) in + * spk_sample: loudspeaker sample (PCM as floating point value) + * return: from 0 for doubletalk to 1.0 for single talk + * + */ +static double +ag_aec_dtd(ag_aec_state *ec, double mic_sample, double spk_sample) +{ + double stepsize; + double ratio; + double M = (STEPY2 - STEPY1) / (STEPX2 - STEPX1); + + /* fast near-end and far-end average */ + ec->mic_fast += ALPHAFAST * (fabsf(mic_sample) - ec->mic_fast); + ec->spk_fast += ALPHAFAST * (fabsf(spk_sample) - ec->spk_fast); + + /* slow near-end and far-end average */ + ec->mic_slow += ALPHASLOW * (fabsf(mic_sample) - ec->mic_slow); + ec->spk_slow += ALPHASLOW * (fabsf(spk_sample) - ec->spk_slow); + + double limit = M50dB_PCM; + //PJ_LOG(2, (THIS_FILE, "Spk:%f, Mic: %f, Limit: %f", ec->spk_fast, ec->mic_fast, M70dB_PCM)); + + if ((ec->spk_fast < limit) && (ec->mic_fast < limit)) { + //PJ_LOG(2, (THIS_FILE, "Idle line")); + } else if ((ec->spk_fast >= limit) && (ec->mic_fast < limit)) { + PJ_LOG(2, (THIS_FILE, "Far end active, Near end idle")); + PJ_LOG(2, (THIS_FILE, "Spk:%f, Mic: %f, Limit: %f", ec->spk_fast, ec->mic_fast, limit)); + } else if ((ec->spk_fast < limit) && (ec->mic_fast >= limit)) { + //PJ_LOG(2, (THIS_FILE, "Far end idle, Near end active")); + } else { + PJ_LOG(2, (THIS_FILE, "DTD")); + return 0.0; + } + + if (ec->spk_fast < limit) { + return 0.0; /* no spk signal */ + } + + if (ec->mic_fast < limit) { + return 0.0; /* no mic signal */ + } + + /* ratio of NFRs */ + ratio = (ec->mic_fast * ec->spk_slow) / (ec->mic_slow * ec->spk_fast); + + /* begrenzte lineare Kennlinie */ + if (ratio < STEPX1) { + stepsize = STEPY1; + } + else if (ratio > STEPX2) { + stepsize = STEPY2; + } + else { + /* Punktrichtungsform einer Geraden */ + stepsize = M * (ratio - STEPX1) + STEPY1; + } + + return stepsize; +} + +/* + * The spk_fast signal is used to charge the hangover timer to Thold. + * When hangover expires (no Spk signal for some time) the vector w is + * erased. This is my implementation of Leaky NLMS. + */ +static void +ag_aec_leaky(ag_aec_state *ec) +{ + if (ec->spk_fast >= M50dB_PCM) { + /* vector weight is valid for hangover Thold time */ + ec->hangover = LEAKY_THOLD; + } + else + { + if (ec->hangover > 1) { + ec->hangover--; + } + else if (1 == ec->hangover) { + ec->hangover--; + /* Leaky NLMS is to erase vector w when hangover expires */ + pj_bzero(ec->weights, ec->tail_len); + } + } +} + +/* vector dot product */ +static double +ag_vec_prod(double *a, double *b, int size) +{ + double sum = 0.0; + int i; + + for (i = 0; i < size; i++) { + sum += a[i] * b[i]; + } + + return sum; +} + +/* Normalized Least Mean Square Algorithm pre-whitening (NLMS-pw) The + * LMS algorithm was developed by Bernard Widrow book: Haykin, + * Adaptive Filter Theory, 4. edition, Prentice Hall, 2002 + * + * in mic_sample: microphone sample (16bit PCM value) in spk_sample: + * loudspeaker sample (16bit PCM value) in stepsize: NLMS adaptation + * variable return: echo cancelled microphone sample + */ +static double +ag_aec_nlms_pw(ag_aec_state *ec, double mic_sample, double spk_sample, double stepsize) +{ + int j = ec->tail_index; + + ec->spk_samples[j] = spk_sample; + ec->pw_samples[j] = ag_filter_rsiir_highpass(&ec->pwhp_spk, spk_sample); + + /* calculate echo cancelled signal (mic signal - estimated mic signal from + * spk signal) + */ + double ec_signal = mic_sample; + if (ec->hangover > 0) { + ec_signal = mic_sample - ag_vec_prod(ec->weights, ec->spk_samples + j, ec->tail_len) / SCALE; + } + /* pre-whitening of echo canceled signal */ + double pw_ec = ec_signal; + pw_ec = ag_filter_rsiir_highpass(&ec->pwhp_err, ec_signal); + + //PJ_LOG(2, (THIS_FILE, "Mic signal: %f; EC signal: %f; Pw-echo-cancelled: %f; hangover: %d", mic_sample, ec_signal, pw_ec, ec->hangover)); + + /* optimize: iterative vec_prod(ec->pw_samples, ec->pw_samples) */ + ec->vecprod_pws_pws += (ec->pw_samples[j] * ec->pw_samples[j]); + ec->vecprod_pws_pws -= (ec->pw_samples[j + ec->tail_len - 1] * ec->pw_samples[j + ec->tail_len - 1]); + + if (stepsize > 0.0) { + /* calculate variable step size */ + double mikro_ef; + mikro_ef = SCALE * stepsize * pw_ec / ec->vecprod_pws_pws; + + /* update tap weights (filter learning) */ + //PJ_LOG(2, (THIS_FILE, "filter learning: %lf %f %f %f", SCALE * stepsize / ec->vecprod_pws_pws, pw_ec, ec->vecprod_pws_pws, 1 / ec->vecprod_pws_pws)); + int i; + for (i = 0; i < ec->tail_len; i++) { + ec->weights[i] += mikro_ef * ec->pw_samples[i + j]; + } + } + + ec->tail_index--; + if (ec->tail_index < 0) { + /* optimize: decrease number of memory copies */ + ec->tail_index = ec->tail_ext; + j = ec->tail_ext; + memmove(ec->spk_samples + j + 1, ec->spk_samples, + (ec->tail_len - 1) * sizeof(double)); + memmove(ec->pw_samples + j + 1, ec->pw_samples, + (ec->tail_len - 1) * sizeof(double)); + } + + /* saturation */ + double sat = ec_signal; + if (ec_signal > MAXPCM) { + sat = MAXPCM; + } + else if (ec_signal < -MAXPCM) { + sat = -MAXPCM; + } + + return sat; +} + +/* Acoustic Echo Cancellation and Suppression of one sample in mic: + * microphone signal with echo in spk: loudspeaker signal return: echo + * cancelled microphone signal + */ +static int +ag_aec_compute(ag_aec_state *ec, int mic, int spk) +{ + double mic_sample = (double) mic; + double spk_sample = (double) spk; + + //PJ_LOG(2, (THIS_FILE, "Initial mic signal: %d %f", mic, mic_sample)); + + /* Mic Highpass Filter - to remove DC */ + mic_sample = ag_filter_iir_highpass(&ec->mic_filter, mic_sample); + + //PJ_LOG(2, (THIS_FILE, "after iir_highpass: %f", mic_sample)); + + /* Mic Highpass Filter - cut-off below 300Hz */ + mic_sample = ag_filter_fir_highpass(&ec->cutoff, mic_sample); + + //PJ_LOG(2, (THIS_FILE, "after fir_highpass: %f", mic_sample)); + + /* Amplify, for e.g. Soundcards with -6dB max. volume */ + mic_sample *= ec->gain; + + //PJ_LOG(2, (THIS_FILE, "after amplify: %f", mic_sample)); + + /* Spk Highpass Filter - to remove DC */ + spk_sample = ag_filter_iir_highpass(&ec->spk_filter, spk_sample); + + //PJ_LOG(2, (THIS_FILE, "spk after iir_highpass %f", spk_sample)); + + /* Double Talk Detector */ + double stepsize = ag_aec_dtd(ec, mic_sample, spk_sample); + //PJ_LOG(2, (THIS_FILE, "stepsize= %f", stepsize)); + + /* Leaky (ageing of vector weights) */ + ag_aec_leaky(ec); + + /* Acoustic Echo Cancellation */ + mic_sample = ag_aec_nlms_pw(ec, mic_sample, spk_sample, stepsize); + + //PJ_LOG(2, (THIS_FILE, "after nlms: %f (int) %d", mic_sample, (int)roundf(mic_sample))); + + return (int) roundf(mic_sample); +} + +#define SAMPLE_CHARS 32 + +static void +log_frame(ag_aec_state *ec, pj_int16_t *frm) +{ + int i, bytes; + char *buf, *pbuf; + + buf = malloc(SAMPLE_CHARS * ec->samples_per_frame); + + buf[0] = 0; + pbuf = buf; + for (i = 0; i < ec->samples_per_frame; i++) { + bytes = sprintf(pbuf, "%10d", frm[i]); + pbuf += bytes; + } + PJ_LOG(2, (THIS_FILE, buf)); + + free(buf); +} + +static void +log_samples(ag_aec_state *ec, double *v) +{ + int i, bytes; + char *buf, *pbuf; + + buf = malloc(SAMPLE_CHARS * (ec->tail_len + ec->tail_ext)); + + buf[0] = 0; + pbuf = buf; + for (i = ec->tail_index; i < ec->tail_len + ec->tail_index; i++) { + bytes = sprintf(pbuf, "%10.2f", v[i]); + pbuf += bytes; + } + PJ_LOG(2, (THIS_FILE, buf)); + + free(buf); +} + + +static void +log_weights(ag_aec_state *ec) +{ + int i, bytes; + char *buf, *pbuf; + + buf = malloc(SAMPLE_CHARS * ec->tail_len); + + buf[0] = 0; + pbuf = buf; + for (i = 0; i < ec->tail_len; i++) { + bytes = sprintf(pbuf, "%10f", ec->weights[i]); + pbuf += bytes; + } + PJ_LOG(2, (THIS_FILE, buf)); + + free(buf); +} + +/* + * Perform echo cancellation. + */ +PJ_DEF(pj_status_t) +ag_aec_cancel_echo(void *state, + pj_int16_t *rec_frm, + const pj_int16_t *play_frm, + unsigned options, + void *reserved) +{ + ag_aec_state *ec = (ag_aec_state*) state; + int unsigned i; + + PJ_UNUSED_ARG(options); + PJ_UNUSED_ARG(reserved); + + //PJ_LOG(2, (THIS_FILE, "Before AEC processing...")); + //log_frame(ec, rec_frm); + + /* perform echo cancellation on each sample from the frame */ + for (i = 0; i < ec->samples_per_frame; i++) { + rec_frm[i] = ag_aec_compute(ec, rec_frm[i], play_frm[i]); + } + + //PJ_LOG(2, (THIS_FILE, "Samples, tail_index = %d :", ec->tail_index)); + //log_samples(ec, ec->spk_samples); + //PJ_LOG(2, (THIS_FILE, "After AEC processing...")); + //log_frame(ec, rec_frm); + + //PJ_LOG(2, (THIS_FILE, "Weights:")); + //log_weights(ec); + //PJ_LOG(2, (THIS_FILE, "Samples:")); + //log_pwsamples(ec); + + return PJ_SUCCESS; +} + Index: pjmedia/src/pjmedia/echo_common.c =================================================================== --- pjmedia/src/pjmedia/echo_common.c (revision 2924) +++ pjmedia/src/pjmedia/echo_common.c (working copy) @@ -85,8 +85,15 @@ &echo_supp_cancel_echo }; +static struct ec_operations ag_aec_op = + { + "AG AEC", + &ag_aec_create, + &ag_aec_destroy, + &ag_aec_reset, + &ag_aec_cancel_echo + }; - /* * Speex AEC prototypes */ @@ -155,10 +162,21 @@ pj_list_init(&ec->lat_buf); pj_list_init(&ec->lat_free); + /* hard-code to use AG AEC */ + options = options & ~PJMEDIA_ECHO_ALGO_MASK; + /* Select the backend algorithm */ if (0) { /* Dummy */ ; + +#if defined(PJMEDIA_HAS_AG_AEC) && PJMEDIA_HAS_AG_AEC!=0 + } else if ((options & PJMEDIA_ECHO_ALGO_MASK) == PJMEDIA_ECHO_AG || + (options & PJMEDIA_ECHO_ALGO_MASK) == PJMEDIA_ECHO_DEFAULT) + { + ec->op = &ag_aec_op; +#endif + #if defined(PJMEDIA_HAS_SPEEX_AEC) && PJMEDIA_HAS_SPEEX_AEC!=0 } else if ((options & PJMEDIA_ECHO_ALGO_MASK) == PJMEDIA_ECHO_SPEEX || (options & PJMEDIA_ECHO_ALGO_MASK) == PJMEDIA_ECHO_DEFAULT) @@ -171,11 +189,11 @@ (options & PJMEDIA_ECHO_ALGO_MASK) == PJMEDIA_ECHO_DEFAULT) { ec->op = &ipp_aec_op; - #endif - } else { - ec->op = &echo_supp_op; + } + else { + ec->op = &echo_supp_op; } PJ_LOG(5,(ec->obj_name, "Creating %s", ec->op->name)); ag-aec/README0000644000076500000240000000254511300303512012160 0ustar adigeostaff AG-AEC ------ Copyright (C) 2009 AG Projects http://ag-projects.com AG-AEC is an Acoustic Echo Cancellation implementation designed to work with PJSIP library from http://pjsip.org as a component of SIP SIMPLE project from http://sipsimpleclient.com AG-AEC is a C implementation of the NLMS-PW Acoustic Echo Cancellation algorithm of Andre Adrian. The original C++ implementation is available at: http://www.andreadrian.de/intercom/. The echo cancellation algorithm is described at: http://www.andreadrian.de/echo_cancel/draft-aec-03.txt The software consists of a patch file that is applied to the python-sipsimple project. Create the patch ---------------- # Download a fresh pjsip 1.0 svn tree: svn co http://svn.pjsip.org/repos/pjproject/branches/1.0 pjsip-1.0 # Copy the pjsip/* files of this repository to pjsip-1.0 svn tree: cp -r pjsip/* pjsip-1.0/ # Add echo_ag.c to the svn repository: svn add pjsip-1.0/pjmedia/src/pjmedia/echo_ag.c # Create the python-sipsimple patch: svn diff pjsip-1.0 > python-sipsimple/patches/pjsip_ag_aec.patch Apply the patch --------------- 1. Copy pjsip_ag_aec.patch obtained above into a fresh copy of python-sipsimple project in the patches/ directory. 2. Add the patch file pjsip_ag_aec.patch to python-sipsimple/setup_pjsip.py 3. Rebuild python-sipsimple PJSIP core: python setup.py build_ext --pjsip-clean-compile