Fourier amplitude sensitivity testing

Fourier amplitude sensitivity testing (FAST) is a variance-based global sensitivity analysis method. The sensitivity value is defined based on conditional variances which indicate the individual or joint effects of the uncertain inputs on the output.

FAST first represents conditional variances via coefficients from the multiple Fourier series expansion of the output function. Then the ergodic theorem is applied to transform the multi-dimensional integral to a one-dimensional integral in evaluation of the Fourier coefficients. A set of incommensurate frequencies is required to perform the transform and most frequencies are irrational. To facilitate computation a set of integer frequencies is selected instead of the irrational frequencies. The integer frequencies are not strictly incommensurate, resulting in an error between the multi-dimensional integral and the transformed one-dimensional integral. However, the integer frequencies can be selected to be incommensurate to any order so that the error can be controlled meeting any precision requirement in theory. Using integer frequencies in the integral transform, the resulted function in the one-dimensional integral is periodic and the integral only needs to evaluate in a single period. Next, since the continuous integral function can be recovered from a set of finite sampling points if the Nyquist–Shannon sampling theorem is satisfied, the one-dimensional integral is evaluated from the summation of function values at the generated sampling points.

FAST is more efficient to calculate sensitivities than other variance-based global sensitivity analysis methods via Monte Carlo integration. However the calculation by FAST is usually limited to sensitivities referring to “main effect” or “total effect”.

History

The FAST method originated in study of coupled chemical reaction systems in 1973^[1][2] and the detailed analysis of the computational error was presented latter in 1975.^[3] Only the first order sensitivity indices referring to “main effect” were calculated in the original method. A FORTRAN computer program capable of analyzing either algebraic or differential equation systems was published in 1982.^[4] In 1990s, the relationship between FAST sensitivity indices and Sobol’s ones calculated from Monte-Carlo simulation was revealed in the general framework of ANOVA-like decomposition ^[5] and an extended FAST method able to calculate sensitivity indices referring to “total effect” was developed.^[6]

Foundation

Variance-based sensitivity

Sensitivity indices of a variance-based method are calculated via ANOVA-like decomposition of the function for analysis. Suppose the function is $Y=f\left({\mathbf {X}}\right)=f\left(X_{1},X_{2},\dots ,X_{n}\right)$ where $0\leq X_{j}\leq 1,j=1,\dots ,n$. The ANOVA-like decomposition is

$f\left(X_{1},X_{2},\ldots ,X_{n}\right)=f_{0}+\sum _{{j=1}}^{n}f_{j}\left(X_{j}\right)+\sum _{{j=1}}^{{n-1}}\sum _{{k=j+1}}^{n}f_{{jk}}\left(X_{j},X_{k}\right)+\cdots +f_{{12\dots n}}$

provided that $f_{0}$ is a constant and the integral of each term in the sums is zero, i.e.

$\int _{0}^{1}f_{{j_{1}j_{2}\dots j_{r}}}\left(X_{{j_{1}}},X_{{j_{2}}},\dots ,X_{{j_{r}}}\right)dX_{{j_{k}}}=0,{\text{ }}1\leq k\leq r.$

The conditional variance which characterizes the contribution of each term to the total variance of $f\left({\mathbf {X}}\right)$ is

$V_{{j_{1}j_{2}\dots j_{r}}}=\int _{0}^{1}\cdots \int _{0}^{1}f_{{j_{1}j_{2}\dots j_{r}}}^{2}\left(X_{{j_{1}}},X_{{j_{2}}},\dots ,X_{{j_{r}}}\right)dX_{{j_{1}}}dX_{{j_{2}}}\dots dX_{{j_{r}}}.$

The total variance is the sum of all conditional variances

$V=\sum _{{j=1}}^{n}V_{j}+\sum _{{j=1}}^{{n-1}}\sum _{{k=j+1}}^{n}V_{{jk}}+\cdots +V_{{12\dots n}}.$

The sensitivity index is defined as the normalized conditional variance as

$S_{{j_{1}j_{2}\dots j_{r}}}={\frac {V_{{j_{1}j_{2}\dots j_{r}}}}{V}}$

especially the first order sensitivity

$S_{j}={\frac {V_{j}}{V}}$

which indicates the main effect of the input $X_{j}$.

Multiple Fourier series

One way to calculate the ANOVA-like decomposition is based on multiple Fourier series. The function $f\left({\mathbf {X}}\right)$ in the unit hyper-cube can be extended to a multiply periodic function and the multiple Fourier series expansion is

$f\left(X_{1},X_{2},\dots ,X_{n}\right)=\sum _{{m_{1}=-\infty }}^{{\infty }}\sum _{{m_{2}=-\infty }}^{{\infty }}\cdots \sum _{{m_{n}=-\infty }}^{{\infty }}C_{{m_{1}m_{2}...m_{n}}}\exp {\bigl [}2\pi i\left(m_{1}X_{1}+m_{2}X_{2}+\cdots +m_{n}X_{n}\right){\bigr ]},{\text{ for integers }}m_{1},m_{2},\dots ,m_{n}$

where the Fourier coefficient is

$C_{{m_{1}m_{2}...m_{n}}}=\int _{0}^{1}\cdots \int _{0}^{1}f\left(X_{1},X_{2},\dots ,X_{n}\right)\exp {\bigl [}-2\pi i\left(m_{1}X_{1}+m_{2}X_{2}+\dots +m_{n}X_{n}\right){\bigr ]}.$

The ANOVA-like decomposition is

${\begin{aligned}f_{0}&=C_{{00\dots 0}}\\f_{j}&=\sum _{{m_{j}\neq 0}}C_{{0\dots m_{j}\dots 0}}\exp {\bigl [}2\pi im_{j}X_{j}{\bigr ]}\\f_{{jk}}&=\sum _{{m_{j}\neq 0}}\sum _{{m_{k}\neq 0}}C_{{0\dots m_{j}\dots m_{k}\dots 0}}\exp {\bigl [}2\pi i\left(m_{j}X_{j}+m_{k}X_{k}\right){\bigr ]}\\f_{{12\dots n}}&=\sum _{{m_{1}\neq 0}}\sum _{{m_{2}\neq 0}}\cdots \sum _{{m_{n}\neq 0}}C_{{m_{1}m_{2}\dots m_{n}}}\exp {\bigl [}2\pi i\left(m_{1}X_{1}+m_{2}X_{2}+\cdots +m_{n}X_{n}\right){\bigr ]}.\end{aligned}}$

The first order conditional variance is

${\begin{aligned}V_{j}&=\int _{0}^{1}f_{j}^{2}\left(X_{j}\right)dX_{j}\\&=\sum _{{m_{j}\neq 0}}\left|C_{{0\dots m_{j}\dots 0}}\right|^{2}\\&=2\sum _{{m_{j}=1}}^{{\infty }}\left(A_{{m_{j}}}^{2}+B_{{m_{j}}}^{2}\right)\end{aligned}}$

where $A_{{m_{j}}}$ and $B_{{m_{j}}}$ are the real and imaginary part of $C_{{0\dots m_{j}\dots 0}}$ respectively

$A_{{m_{j}}}=\int _{0}^{1}\cdots \int _{0}^{1}f\left(X_{1},X_{2},\dots ,X_{n}\right)\cos \left(2\pi m_{j}X_{j}\right)dX_{1}dX_{2}\dots dX_{n}$

$B_{{m_{j}}}=\int _{0}^{1}\cdots \int _{0}^{1}f\left(X_{1},X_{2},\dots ,X_{n}\right)\sin \left(2\pi m_{j}X_{j}\right)dX_{1}dX_{2}\dots dX_{n}$

Ergodic theorem

A multi-dimensional integral is required to evaluate for calculating the Fourier coefficients. One way is to transform the multi-dimensional integral into a one-dimensional integral by expressing every input as a function of a new independent variable $s$ as

$X_{j}\left(s\right)={\frac {1}{2\pi }}\left(\omega _{j}s{\text{ mod }}2\pi \right),j=1,2,\dots ,n$

where $\left\{\omega _{j}\right\}$ is a set of incommensurate frequencies, i.e.

$\sum _{{j=1}}^{n}\gamma _{j}\omega _{j}=0$

for an integer set of $\left\{\gamma _{j}\right\}$ if and only if $\gamma _{j}=0$ for every $j$. Then the Fourier coefficients can be calculated by a one-dimensional integral according to the ergodic theorem ^[7]

$A_{{m_{j}}}=\lim _{{T\to \infty }}{\frac {1}{2T}}\int _{{-T}}^{T}f{\bigl (}X_{1}\left(s\right),X_{2}\left(s\right),\dots ,X_{n}\left(s\right){\bigr )}\cos {\bigl (}2\pi m_{j}X_{j}\left(s\right){\bigr )}ds$

$B_{{m_{j}}}=\lim _{{T\to \infty }}{\frac {1}{2T}}\int _{{-T}}^{T}f{\bigl (}X_{1}\left(s\right),X_{2}\left(s\right),\dots ,X_{n}\left(s\right){\bigr )}\sin {\bigl (}2\pi m_{j}X_{j}\left(s\right){\bigr )}ds$

Implementation

Integer frequencies

At most one of the incommensurate frequencies $\left\{\omega _{j}\right\}$ can be rational with all others being irrational. Since the numerical value of an irrational number cannot be stored exactly in a computer, an approximation of the incommensurate frequencies by all rational numbers is required in implementation. Without loss of any generality the frequencies can be set as integers instead of any rational numbers. A set of integers $\left\{\omega _{j}\right\}$ is approximately incommensurate to the order of $M$ if

$\sum _{{j=1}}^{n}\gamma _{j}\omega _{j}\neq 0$

for

$\sum _{{j=1}}^{n}\left|\gamma _{j}\right|\leq M+1$

where $M$ is an integer. The exact incommensurate condition is an extreme case when $M\to \infty$.

Using the integer frequencies the function in the transformed one-dimensional integral is periodic so only the integration over a period of $2\pi$ is required. The Fourier coefficients can be approximately calculated as

${\begin{aligned}A_{{m_{j}}}&\approx {\frac {1}{2\pi }}\int _{{-\pi }}^{{\pi }}f{\bigl (}X_{1}\left(s\right),X_{2}\left(s\right),\dots ,X_{n}\left(s\right){\bigr )}\cos \left(m_{j}\omega _{j}s\right)ds:={\hat {A}}_{{m_{j}}}\\B_{{m_{j}}}&\approx {\frac {1}{2\pi }}\int _{{-\pi }}^{{\pi }}f{\bigl (}X_{1}\left(s\right),X_{2}\left(s\right),\dots ,X_{n}\left(s\right){\bigr )}\sin \left(m_{j}\omega _{j}s\right)ds:={\hat {B}}_{{m_{j}}}\end{aligned}}$

The approximation of the incommensurate frequencies for a finite $M$ results in a discrepancy error between the true Fourier coefficients $A_{{m_{j}}}$, $B_{{m_{j}}}$ and their estimates ${\hat {A}}_{{m_{j}}}$, ${\hat {B}}_{{m_{j}}}$. The larger the order $M$ is the smaller the error is but the more computational efforts are required to calculate the estimates in the following procedure. In practice $M$ is frequently set to 4 and a table of resulted frequency sets which have up to 50 frequencies is available. (McRae et al., 1982)

Search curve

The transform, $X_{j}\left(s\right)={\frac {1}{2\pi }}\left(\omega _{j}s{\text{ mod }}2\pi \right)$, defines a search curve in the input space. If the frequencies, $\omega _{j},j=1,\dots ,n$, are incommensurate, the search curve can pass through every point in the input space as $s$ varies from 0 to $\infty$ so the multi-dimensional integral over the input space can be accurately transformed to a one-dimensional integral along the search curve. However, if the frequencies are approximately incommensurate integers, the search curve cannot pass through every point in the input space. If fact the search is repeated since the transform function is periodic, with a period of $2\pi$. The one-dimensional integral can be evaluated over a single period instead of the infinite interval for incommensurate frequencies; However, a computational error arises due to the approximation of the incommensuracy.

• Search curve
• 

  The search curve in the case of ω₁=π and ω₂=7. Since the frequencies are incommensurate, the search curve is not repeated and can pass through every point on the square

• 

  The search curve in the case of ω₁=3 and ω₂=7. Since the frequencies are integers, which are approximately incommensurate, the search curve is repeated and cannot pass through every point on the square

• 

  The search curve in the case of ω₁=11 and ω₂=7. Since the frequencies are integers, which are approximately incommensurate, the search curve is repeated and cannot pass through every point on the square

Sampling

The approximated Fourier can be further expressed as

${\hat {A}}_{{m_{j}}}={\begin{cases}0&m_{j}{\text{ odd}}\\{\frac {1}{\pi }}\int _{{-\pi /2}}^{{\pi /2}}f{\bigl (}{\mathbf X}(s){\bigr )}\cos \left(m_{j}\omega _{j}s\right)ds&m_{j}{\text{ even}}\end{cases}}$

and

${\hat {B}}_{{m_{j}}}={\begin{cases}{\frac {1}{\pi }}\int _{{-\pi /2}}^{{\pi /2}}f{\bigl (}{\mathbf X}(s){\bigr )}\sin \left(m_{j}\omega _{j}s\right)ds&m_{j}{\text{ odd}}\\0&m_{j}{\text{ even}}\end{cases}}$

The non-zero integrals can be calculated from sampling points

${\begin{aligned}{\hat {A}}_{{m_{j}}}&={\frac {1}{2q+1}}\sum _{{k=-q}}^{q}f{\bigl (}{\mathbf X}(s_{k}){\bigr )}\cos \left(m_{j}\omega _{j}s_{k}\right),m_{j}{\text{ even}}\\{\hat {B}}_{{m_{j}}}&={\frac {1}{2q+1}}\sum _{{k=-q}}^{q}f{\bigl (}{\mathbf X}(s_{k}){\bigr )}\sin \left(m_{j}\omega _{j}s_{k}\right),m_{j}{\text{ odd }}\end{aligned}}$

where the uniform sampling point in $\left[-\pi /2,\pi /2\right]$ is

$s_{k}={\frac {\pi k}{2q+1}},k=-q,\dots ,-1,0,1,\dots ,q.$

The total number of sampling points is $2q+1$ which should satisfy the Nyquist sampling criterion, i.e.

$2q+1\geq N\omega _{{max}}+1$

where $\omega _{{max}}$ is the largest frequency in $\left\{\omega _{k}\right\}$ and $N$ is the maximum order of the calculated Fourier coefficients.

Partial sum

After calculating the estimated Fourier coefficients, the first order conditional variance can be approximated by

${\begin{aligned}V_{j}&=2\sum _{{m_{j}=1}}^{{\infty }}\left(A_{{m_{j}}}^{2}+B_{{m_{j}}}^{2}\right)\\&\approx 2\sum _{{m_{j}=1}}^{{\infty }}\left({\hat {A}}_{{m_{j}}}^{2}+{\hat {B}}_{{m_{j}}}^{2}\right)\\&\approx 2\sum _{{m_{j}=1}}^{{2}}\left({\hat {A}}_{{m_{j}}}^{2}+{\hat {B}}_{{m_{j}}}^{2}\right)\\&=2\left({\hat {A}}_{{m_{j}=2}}^{2}+{\hat {B}}_{{m_{j}=1}}^{2}\right)\end{aligned}}$

where only the partial sum of the first two terms is calculated and $N=2$ for determining the number of sampling points. Using the partial sum can usually return an adequately good approximation of the total sum since the terms corresponding to the fundamental frequency and low order frequencies usually contribute most to the total sum. Additionally, the Fourier coefficient in the summation are just an estimate of the true value and adding more higher order terms will not help improve the computational accuracy significantly. Since the integer frequencies are not exactly incommensurate there are two integers $m_j$ and $m_{k}$ such that $m_{j}\omega _{j}=m_{k}\omega _{k}.$ Interference between the two frequencies can occur if higher order terms are included in the summation.

Similarly the total variance of $f\left({\mathbf X}\right)$ can be calculated as

$V\approx {\hat {A}}_{0}\left[f^{2}\right]-{\hat {A}}_{0}\left[f\right]^{2}$

where ${\hat {A}}_{0}\left[f^{2}\right]$ denotes the estimated Fourier coefficient of the function of $f^{2}$ inside the bracket and ${\hat {A}}_{0}\left[f\right]^{2}$ is the squared Fourier coefficient of the function $f$. Finally the sensitivity referring to the main effect of an input can be calculated by dividing the conditional variance by the total variance.

References

1. Cukier, R.I., C.M. Fortuin, K.E. Shuler, A.G. Petschek and J.H. Schaibly (1973). Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory. Journal of Chemical Physics, 59, 3873–3878.
2. Schaibly, J.H. and K.E. Shuler (1973). Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. II Applications. Journal of Chemical Physics, 59, 3879–3888.
3. Cukier, R.I., J.H. Schaibly, and K.E. Shuler (1975). Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. III. Analysis of the approximations. Journal of Chemical Physics, 63, 1140–1149.
4. McRae, G.J., J.W. Tilden and J.H. Seinfeld (1982). Global sensitivity analysis—a computational implementation of the Fourier Amplitude Sensitivity Test (FAST). Computers & Chemical Engineering, 6, 15–25.
5. Archer G.E.B., A. Saltelli and I.M. Sobol (1997). Sensitivity measures, ANOVA-like techniques and the use of bootstrap. Journal of Statistical Computation and Simulation, 58, 99–120.
6. Saltelli A., S. Tarantola and K.P.S. Chan (1999). A quantitative model-independent method for global sensitivity analysis of model output. Technometrics, 41, 39–56.
7. Weyl, H. (1938). Mean motion. American Journal of Mathematics, 60, 889–896.