```python
# Erasmus+ ICCT project (2018-1-SI01-KA203-047081)

# Toggle cell visibility

from IPython.display import HTML
tag = HTML('''
Toggle cell visibility <a href="javascript:code_toggle()">here</a>.''')
display(tag)

# Hide the code completely

# from IPython.display import HTML
# tag = HTML('''<style>
# div.input {
#     display:none;
# }
# </style>''')
# display(tag)
```

Toggle cell visibility here.

##  Regulator Design: observer and state feedback

This example demonstrates the design of a regulator: a type of controller composed by an observer and a full-state feedback controller. It can be proven that, thanks to the separation principle, the controller and the observer can be designed independently - closed-loop eigenvalues and observer dynamics can be set separately without affecting each other.

Nonetheless, closed-loop transient performance depends on how fast the observer is with respect to the desired closed-loop dynamics.

This example shows the design of a regulator for the controllable and observable system:

\begin{cases}
\dot{\textbf{x}}=\begin{bmatrix}1&0&3\\0&-4&-1\\0&1&-4\end{bmatrix}\textbf{x}+\begin{bmatrix}0\\0\\1\end{bmatrix}\textbf{u} \\ \\
\textbf{y}=\begin{bmatrix}1&0&0\end{bmatrix}\textbf{x}
\end{cases}

that has the transfer function:

$$
G(s) = C(sI-A)^{-1}B.
$$

### Development of the state feedback
The goal is to place 3 eigenvalues in $-1$ rad/s or faster in order to have a good transient response. A possible solution is: $K = \begin{bmatrix}\frac{8}{15}&-4.4&-4\end{bmatrix}$.

### Development of the observer
Given the eigenvalues of the controlled system, a better (faster and stable) choice for the observer is $\lambda_{1,2,3} = -10$ rad/s. This can be achieved with $L=\begin{bmatrix}23&66&\frac{107}{3}\end{bmatrix}^T$.

### Composition of the regulator

The regulator can be implemented in two ways, as a transfer function:

where:

$$
K(s) = -K(sI-A+LC+BK)^{-1}L\,.
$$

or as an observer with static feedback:

Recall that although the obtained closed dynamics is the same, the transfer function implementation may lead to an unstable controller - the eigenvalues of matrix (matrix $A-BK-LC$) may be unstable even if closed-loop dynamics is stable.

### How to use this notebook?
- Try to change the initial conditions of the estimator (default is $\begin{bmatrix}0.2&0.2&0.2\end{bmatrix}^T$) and the observer's eigenvalues and see how the controlled system behaviour changes.
- Try to change the values in order to achieve settling time for 5% tolerance band of less than 2 s.

**Note:**

- The ideal values refer to the case in which all the states of the system can be measured.
- The **Inverse reference gain** slider denotes the value for which the reference is divided (as you change the static gain of the closed-loop transfer function); in order to reach zero error for the step response its value is equal to the static gain.

```python
%matplotlib inline
import control as control
import numpy
import sympy as sym
from IPython.display import display, Markdown
import ipywidgets as widgets
import matplotlib.pyplot as plt

#print a matrix latex-like
def bmatrix(a):
     """Returns a LaTeX bmatrix - by Damir Arbula (ICCT project)

     :a: numpy array
     :returns: LaTeX bmatrix as a string
     """
     if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
     lines = str(a).replace('[', '').replace(']', '').splitlines()
     rv = [r'\begin{bmatrix}']
     rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
     rv +=  [r'\end{bmatrix}']
     return '\n'.join(rv)

# Display formatted matrix:
def vmatrix(a):
    if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
    lines = str(a).replace('[', '').replace(']', '').splitlines()
    rv = [r'\begin{vmatrix}']
    rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
    rv +=  [r'\end{vmatrix}']
    return '\n'.join(rv)

#matrixWidget is a matrix looking widget built with a VBox of HBox(es) that returns a numPy array as value !
class matrixWidget(widgets.VBox):
    def updateM(self,change):
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.M_[irow,icol] = self.children[irow].children[icol].value
                #print(self.M_[irow,icol])
        self.value = self.M_

    def dummychangecallback(self,change):
        pass

    def __init__(self,n,m):
        self.n = n
        self.m = m
        self.M_ = numpy.matrix(numpy.zeros((self.n,self.m)))
        self.value = self.M_
        widgets.VBox.__init__(self,
                             children = [
                                 widgets.HBox(children =
                                              [widgets.FloatText(value=0.0, layout=widgets.Layout(width='90px')) for i in range(m)]
                                             )
                                 for j in range(n)
                             ])

        #fill in widgets and tell interact to call updateM each time a children changes value
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
                self.children[irow].children[icol].observe(self.updateM, names='value')
        #value = Unicode('example@example.com', help="The email value.").tag(sync=True)
        self.observe(self.updateM, names='value', type= 'All')

    def setM(self, newM):
        #disable callbacks, change values, and reenable
        self.unobserve(self.updateM, names='value', type= 'All')
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].unobserve(self.updateM, names='value')
        self.M_ = newM
        self.value = self.M_
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].observe(self.updateM, names='value')
        self.observe(self.updateM, names='value', type= 'All')

                #self.children[irow].children[icol].observe(self.updateM, names='value')

#overlaod class for state space systems that DO NOT remove "useless" states (what "professor" of automatic control would do this?)
class sss(control.StateSpace):
    def __init__(self,*args):
        #call base class init constructor
        control.StateSpace.__init__(self,*args)
    #disable function below in base class
    def _remove_useless_states(self):
        pass
```

```python
# Preparatory cell

A = numpy.matrix('1 0 3; 0 -4 -1; 0 1 -4')
B = numpy.matrix('0; 0; 1')
C = numpy.matrix('1 0 0')
X0 = numpy.matrix('0.2; 0.2; 0.2')
K = numpy.matrix([8/15,-4.4,-4])
L = numpy.matrix([[23],[66],[107/3]])

Aw = matrixWidget(3,3)
Aw.setM(A)
Bw = matrixWidget(3,1)
Bw.setM(B)
Cw = matrixWidget(1,3)
Cw.setM(C)
X0w = matrixWidget(3,1)
X0w.setM(X0)
Kw = matrixWidget(1,3)
Kw.setM(K)
Lw = matrixWidget(3,1)
Lw.setM(L)

eig1c = matrixWidget(1,1)
eig2c = matrixWidget(2,1)
eig3c = matrixWidget(1,1)
eig1c.setM(numpy.matrix([-1.]))
eig2c.setM(numpy.matrix([[-1.],[0.]]))
eig3c.setM(numpy.matrix([-1.]))

eig1o = matrixWidget(1,1)
eig2o = matrixWidget(2,1)
eig3o = matrixWidget(1,1)
eig1o.setM(numpy.matrix([-10.]))
eig2o.setM(numpy.matrix([[-10.],[0.]]))
eig3o.setM(numpy.matrix([-10.]))
```

```python
# Misc

#create dummy widget
DW = widgets.FloatText(layout=widgets.Layout(width='0px', height='0px'))

#create button widget
START = widgets.Button(
    description='Test',
    disabled=False,
    button_style='', # 'success', 'info', 'warning', 'danger' or ''
    tooltip='Test',
    icon='check'
)

def on_start_button_clicked(b):
    #This is a workaround to have intreactive_output call the callback:
    #   force the value of the dummy widget to change
    if DW.value> 0 :
        DW.value = -1
    else:
        DW.value = 1
    pass
START.on_click(on_start_button_clicked)

# Define type of method
selm = widgets.Dropdown(
    options= ['Set K and L', 'Set the eigenvalues'],
    value= 'Set the eigenvalues',
    description='',
    disabled=False
)

# Define the number of complex eigenvalues
sele = widgets.Dropdown(
    options= ['0 complex eigenvalues', '2 complex eigenvalues'],
    value= '0 complex eigenvalues',
    description='Complex eigenvalues:',
    style = {'description_width': 'initial'},
    disabled=False
)

#define type of ipout
selu = widgets.Dropdown(
    options=['impulse', 'step', 'sinusoid', 'square wave'],
    value='step',
    description='Type of reference:',
    style = {'description_width': 'initial'},
    disabled=False
)
# Define the values of the input
u = widgets.FloatSlider(
    value=1,
    min=0,
    max=20.0,
    step=0.1,
    description='Reference:',
    disabled=False,
    continuous_update=False,
    orientation='horizontal',
    readout=True,
    readout_format='.1f',
)
period = widgets.FloatSlider(
    value=0.5,
    min=0.01,
    max=4,
    step=0.01,
    description='Period: ',
    disabled=False,
    continuous_update=False,
    orientation='horizontal',
    readout=True,
    readout_format='.2f',
)

gain_w = widgets.FloatText(
    value=1.,
    description='',
    disabled=True
)

gain_id_w = widgets.FloatText(
    value=1.,
    description='',
    disabled=True
)

gain_w2 = widgets.FloatText(
    value=1.,
    description='',
    disabled=True
)

gain_id_w2 = widgets.FloatText(
    value=1.,
    description='',
    disabled=True
)
```

```python
# Support functions

def eigen_choice(sele):
    if sele == '0 complex eigenvalues':
        eig1c.children[0].children[0].disabled = False
        eig2c.children[1].children[0].disabled = True
        eig1o.children[0].children[0].disabled = False
        eig2o.children[1].children[0].disabled = True
        eig = 0
    if sele == '2 complex eigenvalues':
        eig1c.children[0].children[0].disabled = True
        eig2c.children[1].children[0].disabled = False
        eig1o.children[0].children[0].disabled = True
        eig2o.children[1].children[0].disabled = False
        eig = 2
    return eig

def method_choice(selm):
    if selm == 'Set K and L':
        method = 1
        sele.disabled = True
    if selm == 'Set the eigenvalues':
        method = 2
        sele.disabled = False
    return method
```

## Implementation as transfer function controller

```python
sols = numpy.linalg.eig(A)

def main_callback(Aw, Bw, X0w, K, L, eig1c, eig2c, eig3c, eig1o, eig2o, eig3o, u, period, selm, sele, selu, DW):
    eige = eigen_choice(sele)
    method = method_choice(selm)

    if method == 1:
        solc = numpy.linalg.eig(A-B*K)
        solo = numpy.linalg.eig(A-L*C)
    if method == 2:
        if eige == 0:
            K = control.acker(A, B, [eig1c[0,0], eig2c[0,0], eig3c[0,0]])
            Kw.setM(K)
            L = control.acker(A.T, C.T, [eig1o[0,0], eig2o[0,0], eig3o[0,0]]).T
            Lw.setM(L)
        if eige == 2:
            K = control.acker(A, B, [eig3c[0,0],
                                     numpy.complex(eig2c[0,0],eig2c[1,0]),
                                     numpy.complex(eig2c[0,0],-eig2c[1,0])])
            Kw.setM(K)
            L = control.acker(A.T, C.T, [eig3o[0,0],
                                         numpy.complex(eig2o[0,0],eig2o[1,0]),
                                         numpy.complex(eig2o[0,0],-eig2o[1,0])]).T
            Lw.setM(L)

    Gs = sss(A,B,C,0)
    Ks = sss(A-B*K-L*C,L,-K,0)
    Fs = control.series(-Ks,Gs)
    sys = control.feedback(Fs)

    Gs_id = sss(A,B,sym.eye(3),sym.zeros(3,1))
    Fs_id = control.series(K,Gs_id)
    A1 = numpy.matrix(Fs_id.A-Fs_id.B*Fs_id.C)
    B1 = numpy.matrix(Fs_id.B*sym.Matrix([[1],[0],[0]]))
    C1 = numpy.matrix(sym.Matrix([1,0,0]).T*Fs_id.C)
    sys_id = sss(A1,B1,C1,0)

    sys_o = sss(A-L*C,numpy.hstack((L,B)),sym.eye(3),sym.zeros(3,2))

    dcgain = control.dcgain(sys)
    t = numpy.linspace(0, 1000, 2)
    t, y = control.step_response(sys_id,t)
    dcgain_id = y[-1]
    gain_w.value = dcgain
    gain_id_w.value = dcgain_id
    if dcgain != 0 and dcgain_id != 0:
        u1 = u/gain_w.value
        u2 = u/gain_id_w.value
    else:
        print('The inverse gain set is 0 and it is changed to 1')
        u1 = u/1
        u2 = u/1

    solc = numpy.linalg.eig(sys.A)
    solo = numpy.linalg.eig(A-L*C-B*K)
    print('The system\'s eigenvalues are:', round(sols[0][0],2),',', round(sols[0][1],2),'and', round(sols[0][2],2))
    print('The controlled closed loop system\'s eigenvalues are:',
          round(solc[0][0],2),',',
          round(solc[0][1],2),',',
          round(solc[0][2],2),',',
          round(solc[0][3],2),',',
          round(solc[0][4],2),'and',
          round(solc[0][5],2))
    print('The controller\'s eigenvalues are:', round(solo[0][0],2),',', round(solo[0][1],2),'and', round(solo[0][2],2))
    print('')
    print('The static gain of the closed loop system (from the reference to the output) is: %.5f' %dcgain)
    print('The static gain of the closed loop ideal system (from the reference to the output) is: %.5f' %dcgain_id)

    X0w1 = numpy.matrix([[X0w[0,0]],[X0w[1,0]],[X0w[2,0]],[0],[0],[0]])
    T = numpy.linspace(0, 12, 1000)

    if selu == 'impulse': #selu
        U = [0 for t in range(0,len(T))]
        U[0] = u
        U1 = [0 for t in range(0,len(T))]
        U1[0] = u1
        U2 = [0 for t in range(0,len(T))]
        U2[0] = u2
        T, yout, xout = control.forced_response(sys,T,U1,X0w1)
        T, yout_id, xout_id = control.forced_response(sys_id,T,U2,[0, 0, 0])
        T, yout_k, xout_k = control.forced_response(Ks,T,yout-U1,X0w)
        T, yout_o, xout_o = control.forced_response(sys_o,T,[yout,yout_k],X0w)
    if selu == 'step':
        U = [u for t in range(0,len(T))]
        U1 = [u1 for t in range(0,len(T))]
        U2 = [u2 for t in range(0,len(T))]
        T, yout, xout = control.forced_response(sys,T,U1,X0w1)
        T, yout_id, xout_id = control.forced_response(sys_id,T,U2,[0, 0, 0])
        T, yout_k, xout_k = control.forced_response(Ks,T,yout-U1,X0w)
        T, yout_o, xout_o = control.forced_response(sys_o,T,[yout,yout_k],X0w)
    if selu == 'sinusoid':
        U = u*numpy.sin(2*numpy.pi/period*T)
        U1 = u1*numpy.sin(2*numpy.pi/period*T)
        U2 = u2*numpy.sin(2*numpy.pi/period*T)
        T, yout, xout = control.forced_response(sys,T,U1,X0w1)
        T, yout_id, xout_id = control.forced_response(sys_id,T,U2,[0, 0, 0])
        T, yout_k, xout_k = control.forced_response(Ks,T,yout-U1,X0w)
        T, yout_o, xout_o = control.forced_response(sys_o,T,[yout,yout_k],X0w)
    if selu == 'square wave':
        U = u*numpy.sign(numpy.sin(2*numpy.pi/period*T))
        U1 = u1*numpy.sign(numpy.sin(2*numpy.pi/period*T))
        U2 = u2*numpy.sign(numpy.sin(2*numpy.pi/period*T))
        T, yout, xout = control.forced_response(sys,T,U1,X0w1)
        T, yout_id, xout_id = control.forced_response(sys_id,T,U2,[0, 0, 0])
        T, yout_k, xout_k = control.forced_response(Ks,T,yout-U1,X0w)
        T, yout_o, xout_o = control.forced_response(sys_o,T,[yout,yout_k],X0w)
    # N.B. i primi 3 stati di xout sono quelli dello stimatore, mentre gli ultimi 3 sono quelli del sistema "reale"

    fig = plt.figure(num='Simulation1', figsize=(16,17))
    mag, phase, omega = control.bode_plot(sys,Plot = False)
    mag = control.mag2db(mag)
    phase = phase*180/numpy.pi
    fig.add_subplot(321)
    plt.title('Bode plot: magnitude')
    plt.semilogx(omega,mag)
    plt.xlabel('$\omega$ [rad/s]')
    plt.ylabel('Mag. [dB]')
    plt.grid(True,which="both")

    fig.add_subplot(323)
    plt.title('Bode plot: phase')
    plt.semilogx(omega,phase)
    plt.xlabel('$\omega$ [rad/s]')
    plt.ylabel('Phase [deg]')
    plt.grid(True,which="both")

    fig.add_subplot(325)
    plt.title('Output response')
    plt.ylabel('Output')
    plt.plot(T,yout,T,yout_id,'g',T,U,'r--')
    plt.xlabel('$t$ [s]')
    plt.legend(['$y$','$y_{ideal}$','Reference'])
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

    fig.add_subplot(322)
    plt.title('First state response')
    plt.ylabel('$x_1$')
    plt.plot(T,xout_o[0],T,xout[3],T,xout_id[0],'g')
    plt.xlabel('$t$ [s]')
    plt.legend(['$x_{1est}$','$x_{1real}$','$x_{1ideal}$'])
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

    fig.add_subplot(324)
    plt.title('Second state response')
    plt.ylabel('$x_2$')
    plt.plot(T,xout_o[1],T,xout[4],T,xout_id[1],'g')
    plt.xlabel('$t$ [s]')
    plt.legend(['$x_{2est}$','$x_{2real}$','$x_{2ideal}$'])
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

    fig.add_subplot(326)
    plt.title('Third state response')
    plt.ylabel('$x_3$')
    plt.plot(T,xout_o[2],T,xout[5],T,xout_id[2],'g')
    plt.xlabel('$t$ [s]')
    plt.legend(['$x_{3est}$','$x_{3real}$','$x_{3ideal}$'])
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

alltogether = widgets.VBox([widgets.HBox([selm,
                                          sele,
                                          selu]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([widgets.Label('K:',border=3), Kw,
                                          widgets.Label(' ',border=3),
                                          widgets.Label(' ',border=3),
                                          widgets.Label('Eigenvalues:',border=3),
                                          eig1c,
                                          eig2c,
                                          eig3c,
                                          widgets.Label(' ',border=3),
                                          widgets.Label(' ',border=3),
                                          widgets.Label('X0 est.:',border=3), X0w]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([widgets.Label('L:',border=3), Lw,
                                          widgets.Label(' ',border=3),
                                          widgets.Label(' ',border=3),
                                          widgets.Label('Eigenvalues:',border=3),
                                          eig1o,
                                          eig2o,
                                          eig3o,
                                          widgets.Label(' ',border=3),
                                          widgets.VBox([widgets.Label('Inverse reference gain:',border=3),
                                                        widgets.Label('Inverse ideal reference gain:',border=3)]),
                                          widgets.VBox([gain_w,gain_id_w])]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([u,
                                          period,
                                          START])])
out = widgets.interactive_output(main_callback, {'Aw':Aw, 'Bw':Bw, 'X0w':X0w, 'K':Kw, 'L':Lw,
                                                 'eig1c':eig1c, 'eig2c':eig2c, 'eig3c':eig3c, 'eig1o':eig1o, 'eig2o':eig2o, 'eig3o':eig3o,
                                                 'u':u, 'period':period, 'selm':selm, 'sele':sele, 'selu':selu, 'DW':DW})
out.layout.height = '1120px'
display(out, alltogether)
```

    Output(layout=Layout(height='1120px'))

    VBox(children=(HBox(children=(Dropdown(index=1, options=('Set K and L', 'Set the eigenvalues'), value='Set the…

## Implementation as observer

```python
sols = numpy.linalg.eig(A)

def main_callback2(Aw, Bw, X0w, K, L, eig1c, eig2c, eig3c, eig1o, eig2o, eig3o, u, period, selm, sele, selu, DW):
    eige = eigen_choice(sele)
    method = method_choice(selm)

    if method == 1:
        solc = numpy.linalg.eig(A-B*K)
        solo = numpy.linalg.eig(A-L*C)
    if method == 2:
        if eige == 0:
            K = control.acker(A, B, [eig1c[0,0], eig2c[0,0], eig3c[0,0]])
            Kw.setM(K)
            L = control.acker(A.T, C.T, [eig1o[0,0], eig2o[0,0], eig3o[0,0]]).T
            Lw.setM(L)
        if eige == 2:
            K = control.acker(A, B, [eig3c[0,0],
                                     numpy.complex(eig2c[0,0],eig2c[1,0]),
                                     numpy.complex(eig2c[0,0],-eig2c[1,0])])
            Kw.setM(K)
            L = control.acker(A.T, C.T, [eig3o[0,0],
                                         numpy.complex(eig2o[0,0],eig2o[1,0]),
                                         numpy.complex(eig2o[0,0],-eig2o[1,0])]).T
            Lw.setM(L)

    Gs = sss(A,B,numpy.vstack((numpy.eye(3),[0,0,0])),[[0],[0],[0],[1]])
    Os = sss(A-L*C,numpy.hstack((L,B)),numpy.vstack((-K,numpy.eye(3))),[[0,0],[0,0],[0,0],[0,0]])
    Gas = control.append(Gs,Os)
    sys = control.connect(Gas,[[2,1],[3,4],[1,5]],[1],[1,2,3,6,7,8])

    Gs_id = sss(A,B,sym.eye(3),sym.zeros(3,1))
    Fs_id = control.series(K,Gs_id)
    A1 = numpy.matrix(Fs_id.A-Fs_id.B*Fs_id.C)
    B1 = numpy.matrix(Fs_id.B*sym.Matrix([[1],[0],[0]]))
    C1 = numpy.matrix(sym.Matrix([1,0,0]).T*Fs_id.C)
    sys_id = sss(A1,B1,C1,0)

    dcgain = control.dcgain(sys[0,0])
    t = numpy.linspace(0, 1000, 2)
    t, y = control.step_response(sys_id,t)
    dcgain_id = y[-1]
    gain_w2.value = dcgain
    gain_id_w2.value = dcgain_id
    if dcgain != 0 and dcgain_id != 0:
        u1 = u/gain_w2.value
        u2 = u/gain_id_w2.value
    else:
        print('The inverse gain setted is 0 and it is changed to 1')
        u1 = u/1
        u2 = u/1

    solc = numpy.linalg.eig(sys.A)
    print('The system\'s eigenvalues are:', round(sols[0][0],2),',', round(sols[0][1],2),'and', round(sols[0][2],2))
    print('The controlled closed loop system\'s eigenvalues are:',
          round(solc[0][0],2),',',
          round(solc[0][1],2),',',
          round(solc[0][2],2),',',
          round(solc[0][3],2),',',
          round(solc[0][4],2),'and',
          round(solc[0][5],2))
    print('')
    print('The static gain of the closed loop system (from the reference to the output) is: %.5f' %dcgain)
    print('The static gain of the closed loop ideal system (from the reference to the output) is: %.5f' %dcgain_id)

    X0w1 = numpy.matrix([[0],[0],[0],[X0w[0,0]],[X0w[1,0]],[X0w[2,0]]])
    T = numpy.linspace(0, 12, 1000)

    if selu == 'impulse': #selu
        U = [0 for t in range(0,len(T))]
        U[0] = u
        U1 = [0 for t in range(0,len(T))]
        U1[0] = u1
        U2 = [0 for t in range(0,len(T))]
        U2[0] = u2
        T, yout, xout = control.forced_response(sys,T,U1,X0w1)
        T, yout_id, xout_id = control.forced_response(sys_id,T,U2,[0, 0, 0])
    if selu == 'step':
        U = [u for t in range(0,len(T))]
        U1 = [u1 for t in range(0,len(T))]
        U2 = [u2 for t in range(0,len(T))]
        T, yout, xout = control.forced_response(sys,T,U1,X0w1)
        T, yout_id, xout_id = control.forced_response(sys_id,T,U2,[0, 0, 0])
    if selu == 'sinusoid':
        U = u*numpy.sin(2*numpy.pi/period*T)
        U1 = u1*numpy.sin(2*numpy.pi/period*T)
        U2 = u2*numpy.sin(2*numpy.pi/period*T)
        T, yout, xout = control.forced_response(sys,T,U1,X0w1)
        T, yout_id, xout_id = control.forced_response(sys_id,T,U2,[0, 0, 0])
    if selu == 'square wave':
        U = u*numpy.sign(numpy.sin(2*numpy.pi/period*T))
        U1 = u1*numpy.sign(numpy.sin(2*numpy.pi/period*T))
        U2 = u2*numpy.sign(numpy.sin(2*numpy.pi/period*T))
        T, yout, xout = control.forced_response(sys,T,U1,X0w1)
        T, yout_id, xout_id = control.forced_response(sys_id,T,U2,[0, 0, 0])
    # N.B. i primi 3 stati di xout sono quelli del sistema, mentre gli ultimi 3 sono quelli dell'osservatore

    fig = plt.figure(num='Simulation1', figsize=(16,17))
    mag, phase, omega = control.bode_plot(sys[0,0],Plot = False)
    mag = control.mag2db(mag)
    phase = phase*180/numpy.pi
    fig.add_subplot(321)
    plt.title('Bode plot: magnitude')
    plt.semilogx(omega,mag)
    plt.xlabel('$\omega$ [rad/s]')
    plt.ylabel('Mag. [dB]')
    plt.grid(True,which="both")

    fig.add_subplot(323)
    plt.semilogx(omega,phase)
    plt.title('Bode plot: phase')
    plt.xlabel('$\omega$ [rad/s]')
    plt.ylabel('Phase [deg]')
    plt.grid(True,which="both")

    fig.add_subplot(325)
    plt.title('Output response')
    plt.ylabel('Output')
    plt.plot(T,yout[0],T,yout_id,'g',T,U,'r--')
    plt.xlabel('$t$ [s]')
    plt.legend(['$y$','$y_{ideal}$','Reference'])
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

    fig.add_subplot(322)
    plt.title('First state response')
    plt.ylabel('$x_1$')
    plt.plot(T,yout[3],T,yout[0],T,xout_id[0],'g')
    plt.xlabel('$t$ [s]')
    plt.legend(['$x_{1est}$','$x_{1real}$','$x_{1ideal}$'])
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

    fig.add_subplot(324)
    plt.title('Second state response')
    plt.ylabel('$x_2$')
    plt.plot(T,yout[4],T,yout[1],T,xout_id[1],'g')
    plt.xlabel('$t$ [s]')
    plt.legend(['$x_{2est}$','$x_{2real}$','$x_{2ideal}$'])
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

    fig.add_subplot(326)
    plt.title('Third state response')
    plt.ylabel('$x_3$')
    plt.plot(T,yout[5],T,yout[2],T,xout_id[2],'g')
    plt.xlabel('$t$ [s]')
    plt.legend(['$x_{3est}$','$x_{3real}$','$x_{3ideal}$'])
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

alltogether2 = widgets.VBox([widgets.HBox([selm,
                                          sele,
                                          selu]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([widgets.Label('K:',border=3), Kw,
                                          widgets.Label(' ',border=3),
                                          widgets.Label(' ',border=3),
                                          widgets.Label('Eigenvalues:',border=3),
                                          eig1c,
                                          eig2c,
                                          eig3c,
                                          widgets.Label(' ',border=3),
                                          widgets.Label(' ',border=3),
                                          widgets.Label('X0 est.:',border=3), X0w]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([widgets.Label('L:',border=3), Lw,
                                          widgets.Label(' ',border=3),
                                          widgets.Label(' ',border=3),
                                          widgets.Label('Eigenvalues:',border=3),
                                          eig1o,
                                          eig2o,
                                          eig3o,
                                          widgets.Label(' ',border=3),
                                          widgets.VBox([widgets.Label('Inverse reference gain:',border=3),
                                                        widgets.Label('Inverse ideal reference gain:',border=3)]),
                                          widgets.VBox([gain_w2,gain_id_w2])]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([u,
                                          period,
                                          START])])
out2 = widgets.interactive_output(main_callback2, {'Aw':Aw, 'Bw':Bw, 'X0w':X0w, 'K':Kw, 'L':Lw,
                                                 'eig1c':eig1c, 'eig2c':eig2c, 'eig3c':eig3c, 'eig1o':eig1o, 'eig2o':eig2o, 'eig3o':eig3o,
                                                 'u':u, 'period':period, 'selm':selm, 'sele':sele, 'selu':selu, 'DW':DW})
out2.layout.height = '1120px'
display(out2, alltogether2)
```

    Output(layout=Layout(height='1120px'))

    VBox(children=(HBox(children=(Dropdown(index=1, options=('Set K and L', 'Set the eigenvalues'), value='Set the…

```python

```


```python
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>

<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Philosopher', sans-serif;
    font-weight: 400;
    font-size: 2.2em;
    line-height: 100%;
    color: rgb(0, 80, 120);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Philosopher', serif;
    font-weight: 400;
    font-size: 1.9em;
    line-height: 100%;
    color: rgb(245,179,64);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Philosopher', serif;
    margin-top:12px;
    margin-bottom: 3px;
    font-style: italic;
    color: rgb(94,127,192);
}

.text_cell_render h4 {
    font-family: 'Philosopher', serif;
}

.text_cell_render h5 {
    font-family: 'Alegreya Sans', sans-serif;
    font-weight: 300;
    font-size: 16pt;
    color: grey;
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h6 {
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 10pt;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 100%;
}

</style>

### Library import

```python
from sympy import init_printing, Matrix, symbols, eye, Rational
init_printing()
```

# Elimination

## A system of linear equations

In the previous lesson, we had a brief glimpse at linear systems.  The _linear_ in _linear systems_ refers to the fact that each variable appears on its own (i.e. to the power $1$ and not in the form $x \times y$ or the like) and it is not transcendental.  If a solution exists, it then satisfies all of the equations at once.  We will consider the linear system in (1).

$$ \begin{align} 1x+2y+1z &= 2 \\ 3x + 8y + 1z &= 12 \\ 0x + 4y + 1z &= 2 \end{align} \tag{1} $$

A possible solution for $x,y$, and $z$ is given in (2), where $x=2$, $y=1$, and $z = -2$.

$$ \begin{align} 1\left(2\right)+2\left(1\right)+1\left(-2\right) &= 2 \\ 3\left(2\right)+8\left(1\right)+1\left(-2\right) &= 12 \\ 0\left(2\right)+4\left(1\right)+1\left(-2\right) &= 2 \end{align} \tag{2} $$

Since (1) is a set (three) equations that have a solution ( or possibly solutions) for their variables in common, all left- and all right hand sides can be manipulated in certain ways.

We could simply exchange the order of the equations.  In (3) the second and third equations have been exchanged, called _row exchange_.

$$ \begin{align}1x+2y+1z &= 2 \\ 0x + 4y + 1z &= 2 \\ 3x + 8y + 1z &= 12 \end{align} \tag{3} $$

We could multiply both the left- and right-hand side of one of the equations with a scalar.  In (4) we multiply the first equation by $2$.

$$ \begin{align} 2x+4y+2z &= 4 \\ 3x + 8y + 1z &= 12 \\ 0x + 4y + 1z &= 2 \end{align} \tag{4}$$

Lastly, we can subtract a constant multiple of one equation from another.

These three _manipulations_ serve an excellent purpose, as it allows us to eliminate of one (or more) of the variables (that is, give it a coefficient of $0$).  Remember that we are trying to solve for all three equations and have three unknowns.  We can most definitely struggle by doing this problem algebraically by substitution, but linear algebra makes it much easier.

In (5) we have multiplied the first equation by $3$ (both sides, so that we maintain integrity of the equation) and subtracted the left hand side of this new equation from the left-hand side of the second equation and the new right-hand side of the first equation from the right-hand side of the second equation.  This is quite legitimate, as the left- and right-hand sides are equal (it is an equation after all) and so, when subtracting from the second equation, we are still doing the same thing to the left-hand side as the right-hand side.

$$ \begin{align} 1x+2y+1z &= 2 \\ 0x + 2y - 2z &= 6 \\ 0x + 4y + 1z &= 2 \end{align} \tag{5} $$

This has introduced a nice $0$ in the second equation.  Let's go further and multiply the second equation by $2$ and subtract that from the third equation as seen in (6) below.

$$ \begin{align} 1x+2y+1z &= 2 \\ 0x + 2y - 2z &= 6 \\ 0x + 0y + 5z &= -10 \end{align} \tag{6} $$

Now let last equation is easy to solve for $z$.

$$ z=-2 \tag{7}$$

Knowing this, we can go back up to the second equation and solve for $y$.

$$ \begin{align} 2y+2(-2) &= 6 \\ y &= 1 \end{align} \tag{8} $$

Finally, up to the first equation.

$$ \begin{align} x+2(1)+1(-2) &= 2 \\ x &= 2 \end{align} \tag{9} $$

W have solve the linear system by substitution.  We need to have gone straight for substitution, though.  Indeed, we could have tried to get zeros above all our leading (non-zero) coefficients.  Let's just clean up the third equation by multiplying throughout by $\frac{1}{5}$ as in (10) below.

$$ \begin{align} 1x+2y+1z &= 2 \\ 0x + 2y - 2z &= 6 \\ 0x + 0y + 1z &= -2 \end{align} \tag{10} $$

Now we have to get rid of the $-2z$ in the second equation, which we can do by multiplying the third equation by $-2$ and subtracting from the second equation.

$$ \begin{align}  1x+2y+1z &= 2 \\ 0x + 2y - 0z &= 2 \\ 0x + 0y + 1z &= -2 \end{align} \tag{11}$$

Multiplying the second equation by $\frac{1}{2}$ yields (12).

$$ \begin{align}1x+2y+1z &= 2 \\ 0x + 1y + 0z &= 1 \\ 0x + 0y + 1z &= -2 \end{align} \tag{12} $$

Now we can do the same to get rid of the $1z$ in the first equation (multiply the third equation by $1$ and subtract it from the first equation.

$$ \begin{align} 1x+2y+0z &= 4 \\ 0x + 1y + 0z &= 1 \\ 0x + 0y + 1z &= -2 \end{align} \tag{12}$$

Now tow get rid of the $2y$ in the first equation, which is above our leading $1y$ in the second equation.  Simple enough, we multiply he second equation by $2$ and subtract that from the first equation.

$$ \begin{align} 1x+0y+0z &= 2 \\ 0x + 1y + 0z &= 1 \\ 0x + 0y + 1z &= -2 \end{align} \tag{13} $$

The solution is now clear for $x,y$, and $z$.

We need not rewrite all of the variables all the time.  We can simply write the coefficients.  The augmented matrix of coefficients is in (14).

$$ \begin{bmatrix} 1&2&1&2\\3&8&1&12\\0&4&1&2 \end{bmatrix} \tag{14} $$

A matrix has rows and columns (attached, in position, to our algebraic equation above).  We simply omit the variables.  The left-upper entry is called the pivot.  Our aim is to get everything below it to be a zero (as we did with the algebra).  We do exactly the same as we did above, which is multiply row 1 by 3 and subtract these new values from row 2.

$$ \begin{bmatrix} 1&2&1&2\\0&2&-2&6\\0&4&1&2 \end{bmatrix} \tag{15} $$

Now $2$ times row 2 subtracted from row 3.

$$ \begin{bmatrix} 1&2&1&2\\0&2&-2&6\\0&0&5&-10 \end{bmatrix} \tag{16} $$

Multiply the last row with $\frac{1}{5}$.

$$ \begin{bmatrix} 1&2&1&2\\0&2&-2&6\\0&0&1&-2 \end{bmatrix} \tag{17} $$

This shows $z = -2$ in the last row of (17).

With this small matrix, it's easy to do back substitution as we did algebraically.  The first non-zero number in each row is the pivot (just like the upper-left entry).  The steps we have taken up to this point is called _Gauss elimination_ and the form we end up with is _row-echelon form_.  We could carry on and do the same sort of thing to get rid of all the non-zero entries above each pivot.  This is called _Gauss-Jordan elimination_ and the result is _reduced row-echelon form_ (see the computer code below).

All of these steps are called _elementary row operations_.  The only one we didn't do is _row exchange_.  We reserve this action so as not to have leading (in the pivot position) zeros.

Let's create some code to show-case elementary row operations.

```python
A_augmented = Matrix([[1, 2, 1, 2], [3, 8, 1, 12], [0, 4, 1, 2]])
A_augmented
```

We can ask `sympy` to simply get the augmented matrix in reduced row-echelon form and read off the solutions.  This is done with the `.rref()` method.

```python
A_augmented.rref() # The rref() method returns the reduced row-echelon form
```

## Elimination matrices

Matrices can only be multiplied by each other if in order we have the first column size equal the second row size.  Rows are usually called $m$ and columns $n$ when considering their dimensions.  So, our augmented matrix above will be $m \times n = 3 \times 4$.

Let's look at how matrices are multiplied by looking at two small matrices in (18).

$$ \begin{bmatrix} {a}_{11}&{a}_{12} \\ {a}_{21}&{a}_{22} \end{bmatrix} \\ \\ \begin{bmatrix} {b}_{11}&{b}_{12}\\{b}_{21}&{b}_{22} \end{bmatrix} \tag{18} $$

The subscripts refer to row and column position, i.e. $21$ means row $2$ column $1$>

We see that we have a two $ 2 \times 2 $ matrices.  The *inner* two values are the same ($2$ and $2$), so this multiplication is allowed.  The resultant matrix will have the size equal to the *outer* two values (first row and last columns); here also a $2 \times 2$ matrix.

So let's look at position $11$ (row $1$ and column $1$).  To get this we take the entries in row $1$ of the first matrix and multiply them by the entries in the first column of the second matrix.  We do this element by element and add the multiplication of each set of separate elements to each other.  The python code below shows you exactly how this is done.

```python
a11, a12, a21, a22, b11, b12, b21, b22 = symbols('a11 a12 a21 a22 b11 b12 b21 b22')
```

```python
A = Matrix([[a11, a12], [a21, a22]])
B = Matrix([[b11, b12], [b21, b22]])
A, B
```

```python
A * B
```

Let's constrain ourselves to the matrix of coefficients (this discards the right-hand side from the augmented matrix above).

```python
A = Matrix([[1, 2, 1], [3, 8, 1], [0, 4, 1]]) # I use the same computer variable above, which
# will change its value in the computer memory
A # A 3 by 3 matrix, which we call square
```

The _identity matrix_ is akin to the number $1$, i.e. multiplying by it leaves everything unchanged.  It has ones along what is called the main diagonal and zeros everywhere else.

```python
I = eye(3) # Identity matrices are always square and the argument
# here is 3, so it is a 3 by 3 matrix
I # Note what the main diagonal is
```

Let's multiply $I$ by $A$.

```python
I * A # Nothing will change
```

To get rid of the leading $3$ in the second row(because we want a $0$ under the first pivot in the first row), we multiply the first row by $3$ and subtracted that from the second row.  Interestingly enough, we can do the same to the identity matrix.

```python
E21 = Matrix([[1, 0, 0], [-3, 1, 0], [0, 0, 1]])
E21 # 21 because we are working on row 2, column 1
```

That gives us the required 3 times the first row and the negative shows that we subtract (add the negative).  It's a thing of beauty!

```python
E21 * A
```

Just what we wanted.  $E1$ is called the first elimination matrix.

Let's do something to the identity matrix to get rid of the $4$ in the third row (the second column).  It would require $2$ times the second row subtracted from the third row.  Look carefully at the positions.

```python
E32 = Matrix([[1, 0, 0], [0, 1, 0], [0, -2, 1]])
E32
```

```python
E32 * (E21 * A)
```

Spot on!  We now have nice pivots (leading non-zeros), with nothing under them (along the columns).  As a tip, try not to get fractions involved.  As far as the other two row operations are concerned, we can either exchange rows in the identity matrix or multiply the required row by a scalar constant.

Look at what happens when we multiply $E2$ and $E1$.

```python
L_inv = E32 * E21
L_inv
```

Later we'll call this matrix the inverse of $L$.  It is in triangular form, in this case lower triangular (note all the zeros above the main diagonal).

```python
L_inv * A # Later we'll call this result the matrix U
```

We now have the following, shown in (19).

$$ {L}^{-1}{A}={U} \tag{19} $$

Left-multiplying by $L$ leaves (20).

$$ {L}{L}^{-1}{A}={L}{U} \tag{20} $$

The inverse of a square matrix multiplied by itself gives the identity matrix.

$$ {I}{A}={L}{U} \\ {A}={L}{U} \tag{20} $$

We can construct $L$ from $E32$ and $E21$ above.

$$ {E}_{21}^{-1}{E}_{32}^{-1}{E}_{32}{E}_{21}={E}_{21}^{-1}{E}_{32}^{-1}{U} \\ \therefore {E}_{21}^{-1}{E}_{32}^{-1}={L} \tag{21} $$

```python
E21.inv() # The inverse is easy to understand in words
# We just want to add 3 instead of subtracting 3
```

```python
E32.inv()
```

```python
E21.inv() * E32.inv()
```

This is exactly the inverse of our inverse of $L$ above.

```python
L_inv.inv()
```

This is called _LU-decomposition_ of $A$.  More about this in two chapters from now (I_05_LU_decomposition).

As an aside we can also do elementary column operation, but then we have to multiply on the right of $A$ and not on the left as above.

## Example problems

### Example problem 1

Solve the linear system in (22).

$$ \begin{align} x-y-z+u &= 0 \\ 2x+2z &= 8 \\ -y-2z &= -8 \\ 3x-3y-2z+4u &= 7 \end{align} \tag{22} $$

#### Solution

```python
A_augm = Matrix([[1, -1, -1, 1, 0], [2, 0, 2, 0, 8], [0, -1, -2, 0, -8], [3, -3, -2, 4, 7]])
A_augm
```

```python
A_augm.rref()
```

Whoa! That was easy!  Let's take it a notch down and create some elementary matrices.  First off, we want the matrix of coefficients.

```python
A = Matrix([[1, -1, -1, 1], [2, 0, 2, 0], [0, -1, -2, 0], [3, -3, -2, 4]])
A
```

Now we need to get rid of the $2$ in position row `2`, column `1`.  We start by numbering the elementary matrix by this position and modifying the identity matrix.

```python
E21 = Matrix([[1, 0, 0, 0], [-2, 1, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]])
E21 * A
```

Now for position row `3`, column `2`.  We have to use row `2` to do this.  If we used row `1`, we would introduce a non-zero into position row `3`, column `1`.

```python
E32 = Matrix([[1, 0, 0, 0], [0, 1, 0, 0], [0, Rational(1, 2), 1, 0], [0, 0, 0, 1]])
E32 * (E21 * A)
```

Now for the $3$ in position row `4`, column `1`.

```python
E41 = Matrix([[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [-3, 0, 0, 1]])
E41 * (E32 * E21 * A)
```

Let's exchange rows `3` and `4`.

```python
Ee34 = Matrix([[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0]])
Ee34 * E41 * E32 * E21 * A
```

Let's see where that leaves $\underline{b}$.  After all, what we do to the left, we must do to the right.

$$ {Ee}_{34}\times{E}_{41}\times{E}_{32}\times{E}_{21}{A}{x}={Ee}_{34}\times{E}_{41}\times{E}_{32}\times{E}_{21}{b} \tag{23}$$

```python
b_vect = Matrix([[0], [8], [-8], [7]])
b_vect
```

```python
Ee34 * E41 * E32 * E21 * b_vect
```

Let's print them next to each other on the screen.

```python
Ee34 * E41 * E32 * E21 * A, Ee34 * E41 * E32 * E21 * b_vect
```

So we can simply do back substitution.  We note that $-1u = -4$ and thus $u = 4$.  From here, we work our way back up.

$$ \begin{align} -1(u) = -4 \quad &\therefore \quad u=4 \\ 1(z)+1(4) = 7 \quad &\therefore \quad z=3 \\  2(y) + 4(3) - 2(4) = 8 \quad &\therefore \quad y=2 \\ 1(x)-1(2)-1(3)+1(4)=0 \quad &\therefore \quad x=1 \end{align} \tag{24}$$

```python

```

Mathematics/Linear Algebra/0.0 MIT-18.06 - Jupyter/Lecture_2_Elimination.ipynb


```python
import sys
sys.path.append('..')
```

# Milestone 2

## Introduction

Automatic differentiation is a tool for calculating derivatives using machine accuracy. It has several advantages over traditional methods of derivative calculations such as symbolic and finite differentiation. Automatic differentiation is useful for calculating complex derivatives where errors are more likely with classical methods. For instance , with finite differentiation, h values that are too small will lead to accuracy errors though floating point roundoff error, while h values that are too large will start making vastly inaccurate approximations.

Automatic differentiation is useful due to its practicality in real world applications that involve thousands of parameters in a complicated function, which would take a long runtime as well as strong possibility for error in calculating the derivatives individually.

Our package allows users to calculate derivatives of complex functions, some with many parameters, allowing machine precision.

## Background

Essentially automatic differentiation works by  breaking down a complicated function and performing a sequence of elementary arithmetic such as addition, subtraction, multiplication, and division as well as elementary functions like exp, log, sin, etc. These operations are then repeated by the chain rule and the derivatives of these sequences are calculated. There are two ways that automatic differentiation can be implemented - forward mode and reverse mode.

### 2.1 The Chain Rule

The chain rule makes up a fundamental component of auto differentiation. The basic idea is:
For univariate function, $$ F(x) = f(g(x))$$

 $$F^{\prime} = (f(g))^{\prime} = f^{\prime}(g(x))g^{\prime}(x)$$

For multivariate function, $$F(x) = f(g(x),h(x))$$

$$ \frac{\partial F}{\partial x}=\frac{\partial f}{\partial g} \frac{\partial g}{\partial x}+\frac{\partial f}{\partial h} \frac{\partial h}{\partial x}$$

For generalized cases, if F is a combination of more sub-functions,  $$F(x) = f(g_{1}(x), g_{2}(x), …, g_{m}(x))$$

$$\frac{\partial F}{\partial x}=\sum_{i=1}^{m}\frac{\partial F}{\partial g_{i}} \frac{\partial g_{i}}{\partial x}$$

For F is a function f(g): f: $R^n$ -> $R^m$ and g: $R^m$ -> $R^k$,

$$\mathbf{J}_{\mathrm{gof}}(\mathbf{x})=\mathbf{J}_{\mathrm{g}}(\mathbf{f}(\mathbf{x})) \mathbf{J}_{\mathrm{f}}(\mathbf{x})$$

where $J(f) =\left[\begin{array}{ccc}{\frac{\partial \mathbf{f}}{\partial x_{1}}} & {\cdots} & {\frac{\partial \mathbf{f}}{\partial x_{n}}}\end{array}\right]=\left[\begin{array}{ccc}{\frac{\partial f_{1}}{\partial x_{1}}} & {\cdots} & {\frac{\partial f_{1}}{\partial x_{n}}} \\ {\vdots} & {\ddots} & {\vdots} \\ {\frac{\partial f_{m}}{\partial x_{1}}} & {\cdots} & {\frac{\partial f_{m}}{\partial x_{n}}}\end{array}\right]$ is the Jacobian Matrix.

### 2.2 Auto Differentiation: Forward Mode

The forward mode automatic differentiation is accomplished by firstly splitting the function process into one-by-one steps, each including only one basic operation. It focuses on calculating two things in each step, the value of scalar or vector x in $R^n$, and the 'seed' vector for the derivatives or Jacobian Matrix. From the first node, the value and derivatives will be calculated based on the values and derivatives of forward nodes. AD exploits the fact that every computer program, no matter how complicated, executes a sequence of elementary arithmetic operations (addition, subtraction, multiplication, division, etc.) and elementary functions (exp, log, sin, cos, etc.). By applying the chain rule repeatedly to these operations, derivatives of arbitrary order can be computed automatically, accurately to working precision, and using at most a small constant factor more arithmetic operations than the original program.

The automatic differentiation is superior to analytic or symbolic differentiation because it could be computed on modern machines. It is also superior to numerical differentiation because numeric method is not precise but AD deals with machine precision problems properly.

An example of computational graph and table for forward mode AD is shown as follows:

\begin{align}
  f\left(x,y\right) =\sin\left(xy\right)
\end{align}
We will be evaluating the function at $f(1, 0)$

Evaluation trace:

| Trace   | Elementary Function      | Current Value           | Elementary Function Derivative       | $\nabla_{x}$ Value  | $\nabla_{y}$ Value  |
| :---: | :-----------------: | :-----------: | :----------------------------: | :-----------------:  | :-----------------: |
| $x_{1}$ | $x_{1}$                  | $1$        | $\dot{x}_{1}$                        | $1$ | $0$ |
| $x_{2}$ | $x_{2}$                  | $0$        | $\dot{x}_{2}$                        | $0$ | $1$ |
| $x_{3}$ | $x_{1}x_{2}$                  | $0$        | $\dot{x}_{2}$                        | $0$ | $1$ |

### 2.3 Reverse Mode

The reverse mode automatic differentiation has a process similar to the forward mode auto differentiation, but has another reverse process. It does not apply the chain rule and only partial derivatives to a node are stored. First, for the forward process, the partial derivatives are stored for each node. For the reverse process, it starts with the differentiation to the last node, and then activations in the forward process are deployed in the differentiation step by step.

### 2.4 Forward Mode v.s. Reverse Mode

Two main aspects can be considered when choosing between Forward and Reverse mode auto differentiation.
* Memory Storage & Time of Computation

The forward mode needs memory storage for values and derivatives for each node, while the reverse mode only needs to store the activations of partial differentiation to each node. The forward mode do the computation at the same time as the variable evaluation, while the reverse mode do the calculation in the backward process.
* Input & Output Dimensionality

If the input dimension is much larger than output dimension, then reverse mode is more attractive. If the output dimension is much larger than the input dimension, the forward mode is much computational cheaper.

## Installation

Our package can be installed from our GitHub Repository at: https://github.com/VoraciousFour/cs207-FinalProject.

After the package is installed, it needs to be imported into their workspace. Doing so, will automatically download any dependencies that are required by our package such as math or numpy. Then, the user can create and activate a virtual envitronment to use the package in.

The user can set up and use our package using their terminal as follows.

1. Clone the VorDiff package from our Github Repository into your directory
        git clone https://github.com/VoraciousFour/cs207-FinalProject.git
2. Create and activate a virtual environment
        '''Installing virtualenv'''
        sudo easy_install virtualenv
        '''Creating the Virtual Environment'''
        virtualenv env
        '''Activating the Virtual Environment'''
        source env/bin/activate
3. Install required dependencies
        pip install -r requirements.txt
4. Importing VorDiff package for use
        import VorDiff

## How to use VorDiff

Our Automatic Differentiation package is called VorDiff. The two main objects you will interact with are `AutoDiff` and `Operator`. In short, the user will first instantiate a scalar variable as an `AutoDiff` object, and then feed those variables to operators specified in the `Operator` object. The `Operator` object allows users to build their own functions for auto-differentiation. Simple operations (e.g. addition, multiplication, power) may be used normally. More complex functions (e.g. log, sin, cos) must use the operations defined in the `Operator` class. Lastly, the user may retrieve the values and first derivatives from the objects defined above by using the `get()` method.

A short example is provided below:

```python
from VorDiff.autodiff import AutoDiff as ad
from VorDiff.operator import Operator as op

# Define variables
x = ad.scalar(3.14159)
y = ad.scalar(0)

# Build functions
fx = op.sin(x) + 3
fy = op.exp(y) + op.log(y+1)

# Get values and derivates
print(fx.get())
print(fy.get())
```

    (3.0000026535897932, -0.9999999999964793)
    (1.0, 2.0)

## Software Organization

### Directory Structure
The package's directory will be structured as follows:
```
VorDiff/
	__init__.py
    nodes/
        __init__.py
	    scalar.py
        reverse_scalar.py
        reverse_vector.py
	    vector.py
	tests/
        __init__.py
        test_autodiff.py
        test_node.py
        test_operator.py
        test_reverse_autodiff.py
        test_reverse_operator.py
        test_reverse_scalar.py
        test_reverse_vector.py
        test_scaler.py
        test_vector.py
    autodiff.py
    operator.py
    reverse_autodiff.py
    reverse_operator.py
    README.md
    ...
demo/
    demo_reverse.py
    demo_scalar.py
    demo_vector.py
docs/
    ...
```
### Modules
-   VorDiff: The VorDiff module contains the operator class to be directly used by users to evaluate functions and calculate their derivatives, and an autodiff class that acts as the central interface for automatic differentiation.

-   Nodes: The Nodes module contains the the scalar and vector classes, which define the basic operations that can be performed on scalar and vector variables for the autodiff class.

-   Test_Vordiff: The Test_Vordiff module contains the test suite for this project. TravisCI and CodeCov are used to test our operator classes, node classes, and auto-differentiator.

-   Demo: The Demo module contains python files demonstrating how to perform automatic differentiation with the implemented functions.

### Testing
In this project we use TravisCI to perform continuous integration testing and CodeCov to check the code coverage of our test suite. The status us TravisCI and CodeCov can be found in README.md, in the top level of our package. Since the test suite is included in the project distribution, users can also install the project package and use pytest and pytest-cov to check the test results locally.

### Distribution:
Our open-source VorDiff package will be uploaded to PyPI by using twine because it uses a verified connection for secure authentication to PyPI over HTTPS. Users will be able to install our project package by using the convential `pip install VorDiff`.

## Implementation

### Scalar
The `Scalar` class represents a single scalar node in the computational graph of a function. It implements the interface for user defined scalar variables. The object contains two hidden attributes, `._val` and `._der`, which can be retrieved with the `get()` method.

### Vector
The `Vector` class represents a single vector variable. Vectors are comprised of `Element` objects, which implement much of the computation necessary for vector automatic differentiation.

```python
import numpy as np

# Documentation Hidden
class Scalar():

    def __init__(self, value, *kwargs):
        self._val = value
        if len(kwargs) == 0:
            self._der = 1
        else:
            self._der = kwargs[0]

    def get(self):
        return self._val, self._der

    def __add__(self, other):
        try:
            return Scalar(self._val+other._val, self._der+other._der)
        except AttributeError:
            return self.__radd__(other)

    def __radd__(self, other):
        return Scalar(self._val+other, self._der)

    def __mul__(self, other):
        try:
            return Scalar(self._val*other._val, self._der*other._val+self._val*other._der)
        except AttributeError:
            return self.__rmul__(other)

    def __rmul__(self, other):
        return Scalar(self._val*other, self._der*other)

    def __sub__(self, other):
        return self + (-other)

    def __rsub__(self, other):
        return -self + other

    def __truediv__(self, other):
        try:
            return Scalar(self._val/other._val, (self._der*other._val-self._val*other._der)/(other._val**2))
        except AttributeError:
            return Scalar(self._val/other, self._der/other)

    def __rtruediv__(self, other):
        return Scalar(other/self._val, other*(-self._der)/(self._val)**2)

    def __pow__(self, other):
        try:
            return Scalar(self._val**other._val, (other._val*self._der/self._val+np.log(self._val)*other._der)*(self._val**other._val))
        except AttributeError:
            return Scalar(self._val**other, other*(self._val**(other-1))*self._der)

    def __rpow__(self, other):
        return Scalar(other**self._val, (other**self._val)*np.log(other)*self._der)

    def __neg__(self):
        return Scalar((-1)*self._val, (-1)*self._der)
```

### Operator
The operator class contains all mathematical operations that users can call to build their functions. Each function returns a `Scalar` object or a numeric constant, depending on the input type. Each function raises an erro if its input falls outside its domain. All functions in the class are static.

In this implementation, we include the following elementary functions. Derivatives are calculated with the chain rule.

```python
import numpy as np
from VorDiff.nodes.scalar import Scalar

# Documentation Hidden
class Operator():

    @staticmethod
    def sin(x):
        try: # If scalar variable
            return Scalar(np.sin(x._val), x._der*np.cos(x._val))

        except AttributeError: # If contant
            return np.sin(x)

    @staticmethod
    def cos(x):
        try: # If scalar variable
            return Scalar(np.cos(x._val), -np.sin(x._val)*x._der)

        except AttributeError: # If contant
            return np.cos(x)

    @staticmethod
    def tan(x):
        try: # If scalar variable
            return Scalar(np.tan(x._val), x._der/np.cos(x._val)**2)

        except AttributeError: # If contant
            return np.tan(x)

    @staticmethod
    def arcsin(x):
        try: # If scalar variable
            if x._val<-1 or x._val>1:
                raise ValueError('out of domain')
            else:
                return Scalar(np.arcsin(x._val), 1/(x._der*(1-x._val**2)**.5))

        except AttributeError: # If contant
            if x<-1 or x>1:
                raise ValueError('out of domain')
            else:
                return np.arcsin(x)

    @staticmethod
    def arccos(x):
        try: # If scalar variable
            if x._val<-1 or x._val>1:
                raise ValueError('out of domain')
            else:
                return Scalar(np.arccos(x._val), -x._der/(1-x._val**2)**.5)

        except AttributeError: # If contant
            if x<-1 or x>1:
                raise ValueError('out of domain')
            else:
                return np.arccos(x)

    @staticmethod
    def arctan(x):
        try: # If scalar variable
            return Scalar(np.arctan(x._val), x._der/(1+x._val**2))

        except: # If contant
            return np.arctan(x)

    @staticmethod
    def log(x):
        try: # If scalar variable
            return Scalar(np.log(x._val), x._der/x._val)

        except AttributeError: # If contant
            return np.log(x)

    @staticmethod
    def exp(x):
        try: # If scalar variable
            return Scalar(np.exp(x._val), x._der*np.exp(x._val))

        except AttributeError: # If contant
            return np.exp(x)

    @staticmethod
    def sinh(x):
        try: # if scalar variable
            return Scalar(np.sinh(x._val), x._der*(np.cosh(x._val)))

        except AttributeError: #if constant
            return np.sinh(x)

    @staticmethod
    def cosh(x):
        try: # if scalar variable
            return Scalar(np.cosh(x._val), x._der*(np.sinh(x._val)))

        except AttributeError: #if constant
            return np.cosh(x)

    @staticmethod
    def tanh(x):
        try: # if scalar variable
            return Scalar(np.tanh(x._val), x._der*(1-np.tanh(x._val)**2))

        except AttributeError: #if constant
            return np.tanh(x)

    @staticmethod
    def arcsinh(x):
        try: # if scalar variable
            return Scalar(np.arcsinh(x._val), x._der*(-np.arcsinh(x._val)*np.arctanh(x._val)))

        except AttributeError: #if constant
            return np.arcsinh(x)

    @staticmethod
    def arccosh(x):
        try: # if scalar variable
            return Scalar(np.arccosh(x._val), x._der*(-np.arccosh(x._val)*np.tanh(x._val)))

        except AttributeError: #if constant
            return np.arccosh(x)

    @staticmethod
    def arctanh(x):
        try: # if scalar variable
            return Scalar(np.arctanh(x._val), x._der*(1-np.arctanh(x._val)**2))

        except AttributeError: #if constant
            return np.arctanh(x)
```

### AutoDiff
The `AutoDiff` class will allow the user to easily create variables and build auto-differentiable functions, without having to interface with the `Node` class. It will make use of the auto-differentiator much more intuitive for the user.

```python
from VorDiff.nodes.scalar import Scalar

# Documentation Hidden
class AutoDiff():

    @staticmethod
    def scalar(val):

        return Scalar(val, 1)
```

## Future Features

### 1. Option for higher-order derivatives

There are plenty of ways we could improve our package. The first is to grant users the option to compute higher-order derivatives like Hessians. We can recursively apply AD first to the target function, i.e., producing the first-order derivative, then moving the operations of the first-order derivatives into a new computational graph then applying AD again. In short, higher order derivatives would be accomplished by repeatedly applying automatic differentiation to function and its derivatives.

### 2. Application using AD library to find the roots of functions

A second way we could extend our work is by writing a separate library to find the roots of given functions. For example, this could include an implementation of Newton’s Method that calculates the exact Hessian matrix of a function using AD to get second-order partial derivatives. We would use Newton's Method to search for the approximations by calculating the exact Hessian matrix of the function using AD to get the second-order partial derivatives.

### 3. Backpropagation in neural networks

We can also extend our implementation of automatic differentiation to the neural networks. Neural networks are able to gradually increase accuracy with every training session through the process of gradient descent. In gradient descent, we aim to minimize the loss (i.e. how inaccurate the model is) through tweaking the weights and biases.

By finding the partial derivative of the loss function, we know how much (and in what direction) we must adjust our weights and biases to decrease loss. In that series, we calculate the derivative mean squared error loss function of a single-neuron neural network.

For computers to calculate the partial derivatives of an expression in neural networks, we can implement the automatic differentiation for both forward pass and back propagation. Then we can calculate the partial derivatives in both scalar and vector modes.

```python

```


##### Copyright 2020 The TensorFlow Authors.

```
#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

# Calculate gradients

    View on TensorFlow.org

    Run in Google Colab

    View source on GitHub

    Download notebook

This tutorial explores gradient calculation algorithms for the expectation values of quantum circuits.

Calculating the gradient of the expectation value of a certain observable in a quantum circuit is an involved process. Expectation values of observables do not have the luxury of having analytic gradient formulas that are always easy to write down—unlike traditional machine learning transformations such as matrix multiplication or vector addition that have analytic gradient formulas which are easy to write down. As a result, there are different quantum gradient calculation methods that come in handy for different scenarios. This tutorial compares and contrasts two different differentiation schemes.

## Setup

```
!pip install tensorflow==2.3.1
```

Install TensorFlow Quantum:

```
!pip install tensorflow-quantum
```

Now import TensorFlow and the module dependencies:

```
import tensorflow as tf
import tensorflow_quantum as tfq

import cirq
import sympy
import numpy as np

# visualization tools
%matplotlib inline
import matplotlib.pyplot as plt
from cirq.contrib.svg import SVGCircuit
```

## 1. Preliminary

Let's make the notion of gradient calculation for quantum circuits a little more concrete. Suppose you have a parameterized circuit like this one:

```
qubit = cirq.GridQubit(0, 0)
my_circuit = cirq.Circuit(cirq.Y(qubit)**sympy.Symbol('alpha'))
SVGCircuit(my_circuit)
```

Along with an observable:

```
pauli_x = cirq.X(qubit)
pauli_x
```

Looking at this operator you know that $⟨Y(\alpha)| X | Y(\alpha)⟩ = \sin(\pi \alpha)$

```
def my_expectation(op, alpha):
    """Compute ⟨Y(alpha)| `op` | Y(alpha)⟩"""
    params = {'alpha': alpha}
    sim = cirq.Simulator()
    final_state_vector = sim.simulate(my_circuit, params).final_state_vector
    return op.expectation_from_state_vector(final_state_vector, {qubit: 0}).real

my_alpha = 0.3
print("Expectation=", my_expectation(pauli_x, my_alpha))
print("Sin Formula=", np.sin(np.pi * my_alpha))
```

 and if you define $f_{1}(\alpha) = ⟨Y(\alpha)| X | Y(\alpha)⟩$ then $f_{1}^{'}(\alpha) = \pi \cos(\pi \alpha)$. Let's check this:

```
def my_grad(obs, alpha, eps=0.01):
    grad = 0
    f_x = my_expectation(obs, alpha)
    f_x_prime = my_expectation(obs, alpha + eps)
    return ((f_x_prime - f_x) / eps).real

print('Finite difference:', my_grad(pauli_x, my_alpha))
print('Cosine formula:   ', np.pi * np.cos(np.pi * my_alpha))
```

## 2. The need for a differentiator

With larger circuits, you won't always be so lucky to have a formula that precisely calculates the gradients of a given quantum circuit. In the event that a simple formula isn't enough to calculate the gradient, the `tfq.differentiators.Differentiator` class allows you to define algorithms for computing the gradients of your circuits. For instance you can recreate the above example in TensorFlow Quantum (TFQ) with:

```
expectation_calculation = tfq.layers.Expectation(
    differentiator=tfq.differentiators.ForwardDifference(grid_spacing=0.01))

expectation_calculation(my_circuit,
                        operators=pauli_x,
                        symbol_names=['alpha'],
                        symbol_values=[[my_alpha]])
```

However, if you switch to estimating expectation based on sampling (what would happen on a true device) the values can change a little bit. This means you now have an imperfect estimate:

```
sampled_expectation_calculation = tfq.layers.SampledExpectation(
    differentiator=tfq.differentiators.ForwardDifference(grid_spacing=0.01))

sampled_expectation_calculation(my_circuit,
                                operators=pauli_x,
                                repetitions=500,
                                symbol_names=['alpha'],
                                symbol_values=[[my_alpha]])
```

This can quickly compound into a serious accuracy problem when it comes to gradients:

```
# Make input_points = [batch_size, 1] array.
input_points = np.linspace(0, 5, 200)[:, np.newaxis].astype(np.float32)
exact_outputs = expectation_calculation(my_circuit,
                                        operators=pauli_x,
                                        symbol_names=['alpha'],
                                        symbol_values=input_points)
imperfect_outputs = sampled_expectation_calculation(my_circuit,
                                                    operators=pauli_x,
                                                    repetitions=500,
                                                    symbol_names=['alpha'],
                                                    symbol_values=input_points)
plt.title('Forward Pass Values')
plt.xlabel('$x$')
plt.ylabel('$f(x)$')
plt.plot(input_points, exact_outputs, label='Analytic')
plt.plot(input_points, imperfect_outputs, label='Sampled')
plt.legend()
```

```
# Gradients are a much different story.
values_tensor = tf.convert_to_tensor(input_points)

with tf.GradientTape() as g:
    g.watch(values_tensor)
    exact_outputs = expectation_calculation(my_circuit,
                                            operators=pauli_x,
                                            symbol_names=['alpha'],
                                            symbol_values=values_tensor)
analytic_finite_diff_gradients = g.gradient(exact_outputs, values_tensor)

with tf.GradientTape() as g:
    g.watch(values_tensor)
    imperfect_outputs = sampled_expectation_calculation(
        my_circuit,
        operators=pauli_x,
        repetitions=500,
        symbol_names=['alpha'],
        symbol_values=values_tensor)
sampled_finite_diff_gradients = g.gradient(imperfect_outputs, values_tensor)

plt.title('Gradient Values')
plt.xlabel('$x$')
plt.ylabel('$f^{\'}(x)$')
plt.plot(input_points, analytic_finite_diff_gradients, label='Analytic')
plt.plot(input_points, sampled_finite_diff_gradients, label='Sampled')
plt.legend()
```

Here you can see that although the finite difference formula is fast to compute the gradients themselves in the analytical case, when it came to the sampling based methods it was far too noisy. More careful techniques must be used to ensure a good gradient can be calculated. Next you will look at a much slower technique that wouldn't be as well suited for analytical expectation gradient calculations, but does perform much better in the real-world sample based case:

```
# A smarter differentiation scheme.
gradient_safe_sampled_expectation = tfq.layers.SampledExpectation(
    differentiator=tfq.differentiators.ParameterShift())

with tf.GradientTape() as g:
    g.watch(values_tensor)
    imperfect_outputs = gradient_safe_sampled_expectation(
        my_circuit,
        operators=pauli_x,
        repetitions=500,
        symbol_names=['alpha'],
        symbol_values=values_tensor)

sampled_param_shift_gradients = g.gradient(imperfect_outputs, values_tensor)

plt.title('Gradient Values')
plt.xlabel('$x$')
plt.ylabel('$f^{\'}(x)$')
plt.plot(input_points, analytic_finite_diff_gradients, label='Analytic')
plt.plot(input_points, sampled_param_shift_gradients, label='Sampled')
plt.legend()
```

From the above you can see that certain differentiators are best used for particular research scenarios. In general, the slower sample-based methods that are robust to device noise, etc., are great differentiators when testing or implementing algorithms in a more "real world" setting. Faster methods like finite difference are great for analytical calculations and you want higher throughput, but aren't yet concerned with the device viability of your algorithm.

## 3. Multiple observables

Let's introduce a second observable and see how TensorFlow Quantum supports multiple observables for a single circuit.

```
pauli_z = cirq.Z(qubit)
pauli_z
```

If this observable is used with the same circuit as before, then you have $f_{2}(\alpha) = ⟨Y(\alpha)| Z | Y(\alpha)⟩ = \cos(\pi \alpha)$ and $f_{2}^{'}(\alpha) = -\pi \sin(\pi \alpha)$. Perform a quick check:

```
test_value = 0.

print('Finite difference:', my_grad(pauli_z, test_value))
print('Sin formula:      ', -np.pi * np.sin(np.pi * test_value))
```

It's a match (close enough).

Now if you define $g(\alpha) = f_{1}(\alpha) + f_{2}(\alpha)$ then $g'(\alpha) = f_{1}^{'}(\alpha) + f^{'}_{2}(\alpha)$. Defining more than one observable in TensorFlow Quantum to use along with a circuit is equivalent to adding on more terms to $g$.

This means that the gradient of a particular symbol in a circuit is equal to the sum of the gradients with regards to each observable for that symbol applied to that circuit. This is compatible with TensorFlow gradient taking and backpropagation (where you give the sum of the gradients over all observables as the gradient for a particular symbol).

```
sum_of_outputs = tfq.layers.Expectation(
    differentiator=tfq.differentiators.ForwardDifference(grid_spacing=0.01))

sum_of_outputs(my_circuit,
               operators=[pauli_x, pauli_z],
               symbol_names=['alpha'],
               symbol_values=[[test_value]])
```

Here you see the first entry is the expectation w.r.t Pauli X, and the second is the expectation w.r.t Pauli Z. Now when you take the gradient:

```
test_value_tensor = tf.convert_to_tensor([[test_value]])

with tf.GradientTape() as g:
    g.watch(test_value_tensor)
    outputs = sum_of_outputs(my_circuit,
                             operators=[pauli_x, pauli_z],
                             symbol_names=['alpha'],
                             symbol_values=test_value_tensor)

sum_of_gradients = g.gradient(outputs, test_value_tensor)

print(my_grad(pauli_x, test_value) + my_grad(pauli_z, test_value))
print(sum_of_gradients.numpy())
```

Here you have verified that the sum of the gradients for each observable is indeed the gradient of $\alpha$. This behavior is supported by all TensorFlow Quantum differentiators and plays a crucial role in the compatibility with the rest of TensorFlow.

## 4. Advanced usage
All differentiators that exist inside of TensorFlow Quantum subclass `tfq.differentiators.Differentiator`. To implement a differentiator, a user must implement one of two interfaces.  The standard is to implement `get_gradient_circuits`, which tells the base class which circuits to measure to obtain an estimate of the gradient.  Alternatively, you can overload `differentiate_analytic` and `differentiate_sampled`; the class `tfq.differentiators.Adjoint` takes this route.

The following uses TensorFlow Quantum to implement the gradient of a circuit.  You will use a small example of parameter shifting.

Recall the circuit you defined above, $|\alpha⟩ = Y^{\alpha}|0⟩$.  As before, you can define a function as the expectation value of this circuit against the $X$ observable, $f(\alpha) = ⟨\alpha|X|\alpha⟩$.  Using [parameter shift rules](https://pennylane.ai/qml/glossary/parameter_shift.html), for this circuit, you can find that the derivative is
$$\frac{\partial}{\partial \alpha} f(\alpha) = \frac{\pi}{2} f\left(\alpha + \frac{1}{2}\right) -  \frac{ \pi}{2} f\left(\alpha - \frac{1}{2}\right)$$
The `get_gradient_circuits` function returns the components of this derivative.

```
class MyDifferentiator(tfq.differentiators.Differentiator):
    """A Toy differentiator for <Y^alpha | X |Y^alpha>."""

    def __init__(self):
        pass

    def get_gradient_circuits(self, programs, symbol_names, symbol_values):
        """Return circuits to compute gradients for given forward pass circuits.

        Every gradient on a quantum computer can be computed via measurements
        of transformed quantum circuits.  Here, you implement a custom gradient
        for a specific circuit.  For a real differentiator, you will need to
        implement this function in a more general way.  See the differentiator
        implementations in the TFQ library for examples.
        """

        # The two terms in the derivative are the same circuit...
        batch_programs = tf.stack([programs, programs], axis=1)

        # ... with shifted parameter values.
        shift = tf.constant(1/2)
        forward = symbol_values + shift
        backward = symbol_values - shift
        batch_symbol_values = tf.stack([forward, backward], axis=1)

        # Weights are the coefficients of the terms in the derivative.
        num_program_copies = tf.shape(batch_programs)[0]
        batch_weights = tf.tile(tf.constant([[[np.pi/2, -np.pi/2]]]),
                                [num_program_copies, 1, 1])

        # The index map simply says which weights go with which circuits.
        batch_mapper = tf.tile(
            tf.constant([[[0, 1]]]), [num_program_copies, 1, 1])

        return (batch_programs, symbol_names, batch_symbol_values,
                batch_weights, batch_mapper)
```

The `Differentiator` base class uses the components returned from `get_gradient_circuits` to calculate the derivative, as in the parameter shift formula you saw above.  This new differentiator can now be used with existing `tfq.layer` objects:

```
custom_dif = MyDifferentiator()
custom_grad_expectation = tfq.layers.Expectation(differentiator=custom_dif)

# Now let's get the gradients with finite diff.
with tf.GradientTape() as g:
    g.watch(values_tensor)
    exact_outputs = expectation_calculation(my_circuit,
                                            operators=[pauli_x],
                                            symbol_names=['alpha'],
                                            symbol_values=values_tensor)

analytic_finite_diff_gradients = g.gradient(exact_outputs, values_tensor)

# Now let's get the gradients with custom diff.
with tf.GradientTape() as g:
    g.watch(values_tensor)
    my_outputs = custom_grad_expectation(my_circuit,
                                         operators=[pauli_x],
                                         symbol_names=['alpha'],
                                         symbol_values=values_tensor)

my_gradients = g.gradient(my_outputs, values_tensor)

plt.subplot(1, 2, 1)
plt.title('Exact Gradient')
plt.plot(input_points, analytic_finite_diff_gradients.numpy())
plt.xlabel('x')
plt.ylabel('f(x)')
plt.subplot(1, 2, 2)
plt.title('My Gradient')
plt.plot(input_points, my_gradients.numpy())
plt.xlabel('x')
```

This new differentiator can now be used to generate differentiable ops.

Key Point: A differentiator that has been previously attached to an op must be refreshed before attaching to a new op, because a differentiator may only be attached to one op at a time.

```
# Create a noisy sample based expectation op.
expectation_sampled = tfq.get_sampled_expectation_op(
    cirq.DensityMatrixSimulator(noise=cirq.depolarize(0.01)))

# Make it differentiable with your differentiator:
# Remember to refresh the differentiator before attaching the new op
custom_dif.refresh()
differentiable_op = custom_dif.generate_differentiable_op(
    sampled_op=expectation_sampled)

# Prep op inputs.
circuit_tensor = tfq.convert_to_tensor([my_circuit])
op_tensor = tfq.convert_to_tensor([[pauli_x]])
single_value = tf.convert_to_tensor([[my_alpha]])
num_samples_tensor = tf.convert_to_tensor([[5000]])

with tf.GradientTape() as g:
    g.watch(single_value)
    forward_output = differentiable_op(circuit_tensor, ['alpha'], single_value,
                                       op_tensor, num_samples_tensor)

my_gradients = g.gradient(forward_output, single_value)

print('---TFQ---')
print('Foward:  ', forward_output.numpy())
print('Gradient:', my_gradients.numpy())
print('---Original---')
print('Forward: ', my_expectation(pauli_x, my_alpha))
print('Gradient:', my_grad(pauli_x, my_alpha))
```

Success: Now you can use all the differentiators that TensorFlow Quantum has to offer—and define your own.


```python
# Erasmus+ ICCT project (2018-1-SI01-KA203-047081)

# Toggle cell visibility

from IPython.display import HTML
display(HTML(''))
```

Toggle cell visibility here.

```python
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
import sympy as sym
import scipy.signal as signal
from ipywidgets import widgets, interact
import control as cn
```

## Root locus

Root locus is a plot of the location of closed-loop system poles in relation with a certain parameter (typically amplification). It can be shown that the curves start in the open-loop poles and end up in the open-loop zeros (or infinity). The location of closed-loop system poles not only gives an indication of system stability, but other closed-loop system response properties such as overshoot, rise time and settling time can also be inferred from pole location.

---

### How to use this notebook?
1. Click on *P0*, *P1*, *I0* or *I1* to toggle between the following objects: proportional of the zeroth, first or second order, or an integral one of zeroth or first order. The transfer function of P0 object is $k_p$ (in this example $k_p=2$), of PI object $\frac{k_p}{\tau s+1}$ (in this example $k_p=1$ and $\tau=2$), of IO object $\frac{k_i}{s}$ (in this example $k_i=\frac{1}{10}$) and of I1 object $\frac{k_i}{s(\tau s +1)}$ (in this example $k_i=1$ and $\tau=10$).
2. Click on the *P*, *PI*, *PD* or *PID* button to toogle between proportional, proportional-integral, proportional-derivative or proportional–integral–derivative control algorithm types.
3. Move the sliders to change the values of proportional ($K_p$), integral ($T_i$) and derivative ($T_d$) PID tunning coefficients.
4. Move the slider $t_{max}$ to change the maximum value of the time on x axis.

```python
A = 10
a=0.1
s, P, I, D = sym.symbols('s, P, I, D')

obj = 1/(A*s)
PID = P + P/(I*s) + P*D*s#/(a*D*s+1)
system = obj*PID/(1+obj*PID)
num = [sym.fraction(system.factor())[0].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system.factor())[0], gen=s)))]
den = [sym.fraction(system.factor())[1].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system.factor())[1], gen=s)))]
system_func_open = obj*PID
num_open = [sym.fraction(system_func_open.factor())[0].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func_open.factor())[0], gen=s)))]
den_open = [sym.fraction(system_func_open.factor())[1].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func_open.factor())[1], gen=s)))]

# make figure
fig = plt.figure(figsize=(9.8, 4),num='Root locus')
plt.subplots_adjust(wspace=0.3)

# add axes
ax = fig.add_subplot(121)
ax.grid(which='both', axis='both', color='lightgray')
ax.set_title('Time response')
ax.set_xlabel('t [s]')
ax.set_ylabel('input, output')

rlocus = fig.add_subplot(122)

# plot step function and responses (initalisation)
input_plot, = ax.plot([],[],'C0', lw=1, label='input')
response_plot, = ax.plot([],[], 'C1', lw=2, label='output')
ax.legend()

rlocus_plot, = rlocus.plot([], [], 'r')

plt.show()

system_open = None
system_close = None
def update_plot(KP, TI, TD, Time_span):
    global num, den, num_open, den_open
    global system_open, system_close
    num_temp = [float(i.subs(P,KP).subs(I,TI).subs(D,TD)) for i in num]
    den_temp = [float(i.subs(P,KP).subs(I,TI).subs(D,TD)) for i in den]
    system = signal.TransferFunction(num_temp, den_temp)
    system_close = system
    num_temp_open = [float(i.subs(P,KP).subs(I,TI).subs(D,TD)) for i in num_open]
    den_temp_open = [float(i.subs(P,KP).subs(I,TI).subs(D,TD)) for i in den_open]
    system_open = signal.TransferFunction(num_temp_open, den_temp_open)

    rlocus.clear()
    r, k, xlim, ylim = cn.root_locus_modified(system_open, Plot=False)
#     r, k = cn.root_locus(system_open, Plot=False)
    #rlocus.scatter(r)
    #plot closed loop poles and zeros
    poles = np.roots(system.den)
    rlocus.plot(np.real(poles), np.imag(poles), 'kx')
    zeros = np.roots(system.num)
    if zeros.size > 0:
        rlocus.plot(np.real(zeros), np.imag(zeros), 'ko', alpha=0.5)
    # plot open loop poles and zeros
    poles = np.roots(system_open.den)
    rlocus.plot(np.real(poles), np.imag(poles), 'x', alpha=0.5)
    zeros = np.roots(system_open.num)
    if zeros.size > 0:
        rlocus.plot(np.real(zeros), np.imag(zeros), 'o')
    #plot root locus
    for index, col in enumerate(r.T):
        rlocus.plot(np.real(col), np.imag(col), 'b', alpha=0.5)

    rlocus.set_title('Root locus')
    rlocus.set_xlabel('Re')
    rlocus.set_ylabel('Im')
    rlocus.grid(which='both', axis='both', color='lightgray')

    rlocus.axhline(linewidth=.3, color='g')
    rlocus.axvline(linewidth=.3, color='g')
    rlocus.set_ylim(ylim)
    rlocus.set_xlim(xlim)

    time = np.linspace(0, Time_span, 300)
    u = np.ones_like(time)
    u[0] = 0
    time, response = signal.step(system, T=time)

    response_plot.set_data(time, response)
    input_plot.set_data(time, u)

    ax.set_ylim([min([np.min(u), min(response),-.1]),min(100,max([max(response)*1.05, 1, 1.05*np.max(u)]))])
    ax.set_xlim([-0.1,max(time)])

    plt.show()

controller_ = PID
object_ = obj

def calc_tf():
    global num, den, controller_, object_, num_open, den_open
    system_func = object_*controller_/(1+object_*controller_)

    num = [sym.fraction(system_func.factor())[0].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func.factor())[0], gen=s)))]
    den = [sym.fraction(system_func.factor())[1].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func.factor())[1], gen=s)))]

    system_func_open = object_*controller_
    num_open = [sym.fraction(system_func_open.factor())[0].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func_open.factor())[0], gen=s)))]
    den_open = [sym.fraction(system_func_open.factor())[1].expand().coeff(s, i) for i in reversed(range(1+sym.degree(sym.fraction(system_func_open.factor())[1], gen=s)))]

    update_plot(Kp_widget.value, Ti_widget.value, Td_widget.value, time_span_widget.value)

def transfer_func(controller_type):
    global controller_
    proportional = P
    integral = P/(I*s)
    differential = P*D*s/(a*D*s+1)
    if controller_type =='P':
        controller_func = proportional
        Kp_widget.disabled=False
        Ti_widget.disabled=True
        Td_widget.disabled=True
    elif controller_type =='PI':
        controller_func = proportional+integral
        Kp_widget.disabled=False
        Ti_widget.disabled=False
        Td_widget.disabled=True
    elif controller_type == 'PD':
        controller_func = proportional+differential
        Kp_widget.disabled=False
        Ti_widget.disabled=True
        Td_widget.disabled=False
    else:
        controller_func = proportional+integral+differential
        Kp_widget.disabled=False
        Ti_widget.disabled=False
        Td_widget.disabled=False

    controller_ = controller_func
    calc_tf()

def transfer_func_obj(object_type):
    global object_
    if object_type == 'P0':
        object_ = 2
    elif object_type == 'P1':
        object_ = 1/(2*s+1)
    elif object_type == 'I0':
        object_ = 1/(10*s)
    elif object_type == 'I1':
        object_ = 1/(s*(10*s+1))
    calc_tf()

style = {'description_width': 'initial'}

def buttons_controller_clicked(event):
    controller = buttons_controller.options[buttons_controller.index]
    transfer_func(controller)
buttons_controller = widgets.ToggleButtons(
    options=['P', 'PI', 'PD', 'PID'],
    description='Select control algorithm type:',
    disabled=False,
    style=style)
buttons_controller.observe(buttons_controller_clicked)

def buttons_object_clicked(event):
    object_ = buttons_object.options[buttons_object.index]
    transfer_func_obj(object_)
buttons_object = widgets.ToggleButtons(
    options=['P0', 'P1', 'I0', 'I1'],
    description='Select object:',
    disabled=False,
    style=style)
buttons_object.observe(buttons_object_clicked)

Kp_widget = widgets.FloatLogSlider(value=.5,min=-3,max=2.1,step=.001,description=r'\(K_p\)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.3f')
Ti_widget = widgets.FloatLogSlider(value=1.,min=-3,max=1.8,step=.001,description=r'\(T_{i} \)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.3f')
Td_widget = widgets.FloatLogSlider(value=1.,min=-3,max=1.8,step=.001,description=r'\(T_{d} \)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.3f')

time_span_widget = widgets.FloatSlider(value=10.,min=.5,max=50.,step=0.1,description=r'\(t_{max} \)',
    disabled=False,continuous_update=True,orientation='horizontal',readout=True,readout_format='.1f')

transfer_func(buttons_controller.options[buttons_controller.index])
transfer_func_obj(buttons_object.options[buttons_object.index])

display(buttons_object)
display(buttons_controller)

interact(update_plot, KP=Kp_widget, TI=Ti_widget, TD=Td_widget, Time_span=time_span_widget);
```

    <IPython.core.display.Javascript object>

    ToggleButtons(description='Select object:', options=('P0', 'P1', 'I0', 'I1'), style=ToggleButtonsStyle(descrip…

    ToggleButtons(description='Select control algorithm type:', options=('P', 'PI', 'PD', 'PID'), style=ToggleButt…

    interactive(children=(FloatLogSlider(value=0.5, description='\\(K_p\\)', max=2.1, min=-3.0, readout_format='.3…