**Keywords:** transfer function

## Transfer Functions^{1}

As we know, in a neuron there is an operation called threshold operation which acts as a switch, when the conditions it needs are reached it returns a state otherwise it will keep another state. A transfer function is a mathematical model for this mechanism. And this short post is a survey of the transfer functions used in neural network design.

## Hard Limit

$$

f(x) = \begin{cases}

0& \text{ if } x<0\\

1& \text{ if } x\geq 0

\end{cases}

$$

f(x) = \begin{cases}

0& \text{ if } x<0\\

1& \text{ if } x\geq 0

\end{cases}

$$

## Symmetrical Hard Limit

$$

f(x) = \begin{cases}

-1& \text{ if } x<0\\

1& \text{ if } x\geq 0

\end{cases}

$$

f(x) = \begin{cases}

-1& \text{ if } x<0\\

1& \text{ if } x\geq 0

\end{cases}

$$

## Linear

$$

f(x)=x

$$

f(x)=x

$$

## Saturating Linear

$$

f(x) = \begin{cases}

0& \text{ if } x<0\\

x&\text{ if } 0\leq x\leq 1\\

1& \text{ if } x> 1

\end{cases}

$$

f(x) = \begin{cases}

0& \text{ if } x<0\\

x&\text{ if } 0\leq x\leq 1\\

1& \text{ if } x> 1

\end{cases}

$$

## Symmetric Saturating Linear

$$

f(x) = \begin{cases}

-1& \text{ if } x<0\\

x&\text{ if } 0\leq x\leq 1\\

1& \text{ if } x> 1

\end{cases}

$$

f(x) = \begin{cases}

-1& \text{ if } x<0\\

x&\text{ if } 0\leq x\leq 1\\

1& \text{ if } x> 1

\end{cases}

$$

## Log-Sigmoid

$$

f(x)=\frac{1}{1+e^{-x}}

$$

f(x)=\frac{1}{1+e^{-x}}

$$

## Hyperbolic Tangent Sigmoid

$$

f(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}

$$

f(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}

$$

## Positive Linear

$$

f(x) = \begin{cases}

0& \text{ if } x<0\\

x& \text{ if } x\geq 0

\end{cases}

$$

f(x) = \begin{cases}

0& \text{ if } x<0\\

x& \text{ if } x\geq 0

\end{cases}

$$

## Competitive

$$

f(x) = \begin{cases}

1& \text{ if neuron with max x} \\

0& \text{ else }

\end{cases}

$$

f(x) = \begin{cases}

1& \text{ if neuron with max x} \\

0& \text{ else }

\end{cases}

$$

## References

1 Demuth, H.B., Beale, M.H., De Jess, O. and Hagan, M.T., 2014. Neural network design. Martin Hagan. ↩