Development of New Methods and Applications of Analog Computation: I - An Experimental Electronic Generalized Integrator; II - Nonstationary Noise for Monte Carlo Studies
I
~
-
-
~
~
DEVELOPMENT OF
NEW METHODS AND APPLICATIONS OF ANALOG CUMPU!TATION
I
I1
- An Ekperimental Electronic Generalized I n t e g r a t o r
- Nonstatiowry Noise f o r Monte Carlo Stud2es
---_
--- --
R. S. Johnson
F. R. Williamson
R.tD.
-
Loftin
-.
~.
-
~
- ~ -
-.
OTS PRICE(S) $
I
GlbRGE C. W W SPACE FLIG€l$' CBNTEX
Hunt mille, Alabama.
"
Engineering Experiment Station
Georgia Institute ol Techndogy
GBORGIA INSTITUTE OF TECHNOLOGY
Engineering Experliment S t a t i o n
Atlanta 13, Georgia
1 2 January 1962
DEVELOPMENT OF
NEM METHODS AND APPLICATIONS OF ANALOG COMPUTATION
I - An Experimental Electronic Generalized I n t e g r a t o r
I1 - Nonstationary Noise f o r Monte Carlo Studies
by
R. S. Johnson
F. R. Williamson
and
R. D. L o f t i n
QUARTERLY PROGRAM REPORT
on
Contract No. NAS8-2473
1 2 September 1961 - 1 2 December 1961
For
GEORGE C. MARSHALL SPACE FLJGHT CENTER
Huntsville, Alabama
Submitted:
Approved :
Robert S. J o h s o n
P r o j e c t D i r e c t o r 8-588
F. Dixon, Head
Special Problems Group
Physical Sciences Division
CONTENTS
. ,. .
1-1. I n t r o d u c t i o n
.
.
..
1-2. P r o j e c t Objectives . . . .
. .....
I1 ELECTRONIC GENERALIZED INTEGRATOR . . .
.,
. .
. .. ......
2-1, I n t r o d u c t i o n . . . . .
..
... .
2-2. Generalized I n t e g r a t i o n
The E l e c t r o n i c Generalized I n t e g r a t o r
. ..
2-3
. ,.
.
.
2-4* The Incremental Detector
I
BACKGROUND INFORMATION
.
..
Page
,
,
.
e
e
e
,
e
e
e
e
e
e
e
,.
e
,.
e
e
e
a
,.
2-5.
e
The E l e c t r o n i c Sampling Gate
e
a
e
..
e
. . . . . . . . . . . ..
. .
..
2-7. Work Completed
...
. . .... .
2-8 Future Work
2-6. The Analog Program
,.
,
e
e
e
e
e
a
,.
e
e
e
a
e
e
I11 GENERATION OF NONSTJLTIONARY NOISE
FOR ANALOG-COMPUTER MONTE CARLO STUDIES
1
1
2
2
2
4
7
9
9
13
13
. . 14
. . , . . 14
e
a
3-1.
Analysis of Certain S t o c h a s t i c Processes
e
3-2.
3-3.
3-4.
Functionals Useful i n Specifying Nonstationary Noise
e
Linear Networks f o r Nonstationary Shaping
e
. .
Planned Work f o r t h e Coming Q u a r t e r
1
,.
e
e
.
20
. 23
e
27
L i s t of Figures
....
3
. .
3
3 Basic Block Diagram o f the E l e c t r o n i c Generalized I n t e g r a t o r
6
1 Geometrical I n t e r p r e t a t i o n of a D e f i n i t e I n t e g r a l
e
2 Block Diagram of Analog Program Used f o r Generalized
Integration
,.
.
a
e
e
... . .. .,
e
+
,
,
4 Typical Waveforms Found i n t h e EleT'Jronic Generalized
Integrator
e
,.
..
e
e
.
e
, ,
. .
,
a
, ,
,.
.
~
~
5 Block Diagram of t h e Threshold D e t e c t o r s and Logic Section .
6 C i r c u i t Diagram of t h e Threshold D e t e c t o r s
7
e
. .
C i r c u i t Diagram of t h e Electronic Sampling Gates
..
e
,
8 Analog Program o f t h e Electronic Generalized I n t e g r a t o r
..
8
e
10
11
11
e
12
.
Chapter I
BACKGROUND INFORMATION
1-1.
Introduction
Georgia Tech eesearch P r o j e c t NQ. A-588 was e s t a b l i s h e d on 1 2 September
1961 t o assist t h e F l i g h t Simulation Branch of t h e Computation Division a t
t h e George C, Marshall Space F l i g h t Center (MSFC) in t h e i n v e s t i g a t i o n and
development of new methods and a p p l i c a t i o n s of analog computation w i t h i n t h e
following a r e a s of i n t e r e s t :
A - Developmen% 01 a Generalized I n t e g r a t o r
B
C
- Analog-Computes S t a t i s t i c a l Analysis
-I
Analog--Computer P a r t i a l D i f f e r e n t i a l Equation Solution
This p r o j e c t has been assigned t o the Georgia Tech Analog Computer Laboratory
(ACL), which operates under the Special Problems Group of t h e Engineering
Experiment S t a t i o n ' s Physical Sciences Division.
1-2.
P r o j e c t Objectives
I n accordance with d e c i s i o n s reached during t h e p r o j e c t organizational
meeting with t h e Contracking O f f i c e r ' s Representative ( D r . W.P.
Krause of
It
MSFC), primary emphasis h a s been placed on a r e a s A and B mentioned above.
was agreed t h a t t h e following s p e c i f i c assignments would be undertaken i n i t i a l ally:
Task I.
I n v e s t i g a t i o n of e l e c t r o n i c techniques f o r i n t e g r a t i n g analog
voltage s i g n a l s with r e s p e c t t o a r b i t r a r y v a r i a b l e s .
The t a s k includes
construc%ion of a working model of a generalized i n t e g r a t o r s u i t a b l e
f o r use with e x i s t i n g analog equipment,
Task 11.
-o___
Lnvestigation o f analog techniques f o r the generation of non-
s t a t i o n a r y noise voi-bages t o be used i n analog Monte Carlo studies.
Of
p a r t i c u l a r inbeyest i s t h e problem of nonstationary shaping of Gaussian,
band-limited,
white noise.
-1-
Chapter I1
ELECTRONIC GEN?3RALIZED INTEGRATOR
2-1
Introduction
The a b i l i t y t o perform i n t e g r a t i o n with r e s p e c t t o an a r b i t r a r y v a r i a b l e
can i n c r e a s e t h e f l e x i b i l i t y of t h e general-purpose analog computer.
We may
d i s t i n g u i s h t h e process of i n t e g r a t i o n with r e s p e c t t o v a r i a b l e s o t h e r than
An a l l - e l e c t r o n i c device i s being developed
time as "generalized i n t e g r a t i o n . ?
under t h i s p r o j e c t t o perform such i n t e g r a t i o n .
a s %he E l e c t r o n i c Generalized I n t e g r a t o r ,
It w i l l be r e f e r r e d t o h e r e i n
Electromechanical devices designed
f o r t h e same purpose, but with more l i m i t e d bandwidth, have been previously
described i n t h e l i t e r a t u r e .
2-2.
Generalized I n t e g r a t i o n
I n order t o understand b e t t e r t h e operation of t h e Electronic Generalized
I n t e g r a t o r , we should b r i e f l y examine t h e d e f i n i t i o n of a d e f i n i t e i n t e g r a l .
This i s expressed i n Equation ( 2 - l ) , where t h e f u n c t i o n f(X) i s defined t o be
continuous over t h e i n t e r v a l between a and b,
- Xi-l
where
AXi = Xi
and
xi-l 5 xi -< xi
:
The ' L i m i t i s evaluated as n i s increased and a s t h e maximum of t h e numbers
AXiJoo.Bx n i s made t o approach zero.
A geometrical i n t e r p r e t a t i o n of Equation
( 2 - 1 ) i s i l l u s t r a t e d i n Figure 1. If we now d e f i n e AX t o be a constant and
make .Lhis value very small, Equation (2-1) may be replaced by t h e following
approximation :
b
(2-2)
f(X) dX
AX
a
n
2 f(X;)
ill
This approximation d e s c r i b e s t h e i n t e g r a l a s t h e summation of a number of
-2-
xi
xi -1
Figure L.
Geometrical Interpretation of a Definite Integral
Figure 2: Slock Diagru of Analog Program Used for Generalized Integration
-3 -
4
samples of f ( X ) times t h e constant AX.
Since X and f(X) w i l l both be f u n c t i o n s
of time, we may make t h e s u b s t i t u t i o n of a sampling width of A t seconds f o r the
AX appearing i n Equation (2-2)-
This s u b s t i t u t i o n w i l l r e q u i r e t h a t t h e length
of t h e time A t f o r which t h e sample i s taken be less than t h e s h o r t e s t time r e quired f o r a change i n t h e v a r i a b l e X of amount AX.
The instrumentation of Equation (2-2) was described by Bekey i n 1958".
simplified block diagram of t h e analog program he
mentation i s i l l u s t r a t e d i n Figure 2.
A
used t o perform t h i s i n s t r u -
I n t h i s program, t h e v a r i a b l e X i s
examined f o r changes of a p r e s e t amount which i s defined t o be AXe
These
changes a r e detected by s t o r i n g t h e value of t h e X input a t some time t
0
i n the
Sample and Hold Circuit,arid comparing t h i s value with t h e value of X a t some
l a t e r t i m e tle The comparison i s made by t h e Difference Amplifier and t h e output of t h i s section may be described a s i n Equation (2-3) :
El
(2-3)
KIX(tl)
- X(to)]
e
When t h e magnitude of t h i s value reaches t h e predetermined level, an output s i g n a l i s generated i n t h e AX Detector.
This s i g n a l causes a sample of f(X) t h a t
i s A t seconds long t o appear a t t h e i n p u t of t h e Accumulator.
The time, to, a t
which t h e Sample and Hold C i r c u i t i s r e s e t t a t h e value of t h e v a r i a b l e X i s the
i n s t a n t j u s t a f t e r t h a t when f(X) i s sampled.
Since t h e change i n t h e X input
may be e i t h e r p o s i t i v e o r negative, t h e output, of t h e Difference Amplifier i s
simultaneously examined f o r i t s p o l a r i t y by t h e Sign Detector.
The Sign De-
t e c t o r c o n t r o l s t h e p o l a r i t y of the f(X) samples t h a t appear a t the input of
t h e Accumulator.
i s sampled.
If t h e incremental change i s negative, t h e
negative of f(X)
The output of t h e Accumulator then contains t h e summation of a l l
t h e i n p u t samples f o r X on t h e i n t e r v a l between a and b.
If t h e value of AX
t h z t i s p r e s e t i n t o t h e AX Detector is s u f f i c i e n t l y small when compared t o t h e
d i f f e r e n c e between t h e two l i m i t s of i n t e g r a t i o n , then t h e output of t h e Accumulator i s a good approximation t o t h e i n t e g r a l as expressed i n Equation ( 2 - 2 ) .
-2-3
The Electronic Generalized I n t e g r a t o r
The Electronic Generalized I n t e g r a t o r c o n s t i t u t e s an instrumentation of
Equation (2-2) which i s similar i n many r e s p e c t s t o t h a t used by Bekey.
__
It
"Bekey, George A. , '!Generalized I n t e g r a t i o n on t h e Analog Computertl, presented
a t t h e National Simulation Conference, Dallas, Texas, 23-25 October 1958.
-4-
d i f f e r s primarily i n t h e method whereby t h e incremental changes i n t h e X input
a r e d e t e c t e d and i n t h e use of e l e c t r o n i c switches f o r t h e sampling gates.
The
basic block diagram i s shown i n Figure 3 0
The i n t e g r a t o r m a y be divided i n t o two separate p a r t s , t h e f i r s t being t h e
s e c t i o n used f o r t h e d e t e c t i o n of incremental changes i n t h e X input,
This In-
cremental Detector s e c t i o n i s a simplified version of t h e A I D Converter".
The
information derived from t h i s section may be considered an incremental d i g i t a l
r e p r e s e n t a t i o n of t h e X input.
When t h e input t o t h i s s e c t i o n changes by a
prechosen amount, the output signal r e l a y s t h i s event and information on t h e
s i g n of t h i s change t o t h e sampling c i r c u i t s contained i n t h e o t h e r s e c t i o n
of t h e i n t e g r a t o r .
The Incremental Detector is a closed-loop c i r c u i t t h a t uses an o p e r a t i o n a l
a m p l i f i e r with c a p a c i t i v e feedback f o r t h e analog storage element.
The i n p u t s
t o t h i s a m p l i f i e r a r e t h e Reset Pulses and t h e analog v a r i a b l e X.
The analog
i n p u t t o t h e a m p l i f i e r i s coupled through a s e r i e s capacitor which r e s u l t s i n
t h e analog s i g n a l appearing a t t h e output of t h i s a m p l i f i e r with a sign inver-
sion and a s u i t a b l e s c a l e f a c t o r .
The Reset Pulses are introduced t o t h e g r i d
of t h i s amplifier through a s e r i e s r e s i s t o r which r e s u l t s i n t h e i n t e g r a t i o n of
t h e s e pulses.
The volt-second area of t h e Reset Pulses and t h e i n t e g r a t i n g g a i n
of t h e operational amplifier c i r c u i t are s o chosen t h a t t h e change i n a m p l i f i e r
output voltage r e s u l t i n g from one Reset Pulse i s equal t o t h a t produced by a
change i n t h e input analog s i g n a l of an amount equal t o AX.
The output voltage
of t h i s amplifier i s t h e e r r o r voltage of t h e closed l o o p c i r c u i t .
When it ex-
ceeds an amount equal t o AX9 t h i s event i s sensed by t h e Threshold Detector.
The output of the Threshold Detector i s used t o generate a Reset Pulse and t o
c l o s e t h e sampling g a t e when t h i s event occurs.,
The p o l a r i t y of t h e Reset
P u l s e i s selected t o reduce t h e magnitude of t h e voltage a t t h e output of t h e
o p e r a t i o n a l a m p l i f i e r providing negative feedback i n t h e closed-loop c i r c u i t of
t h e Incremental Detector section. An i l l u s t r a t i o n of t y p i c a l waveforms i s
given i n Figure
The second p o r t i o n of t h e Electronic Generalized I n t e g r a t o r h a s c i r c u i t r y
similar t o t h a t described i n Section 2-2.
E l e c t r o n i c switches are used f o r t h e
>L
"llNew Methods and Application of Analog Cornputationl1, F i n a l Summary Report on
15 may 1961.
Contract NO. DA-01-009 Om-853, GIT/mS Report &97/T1,
-5-
TIMING
I
I r-----------1
INCREMENTAL
DETECTOR
I
I
I
I
I
'
SECTION
I
TKRESHOLD
DETECTOR AND
LOGIC SECTION
I
I
I
STORAGE
I
INTEGRATOR
L--
I
I
I
RESET
PULSES
L
Figare 3:
Basic Block Diagram of" t h e Electronic Generalized I n t e g r a t o r
-6-
sampling g a t e s i n order t o allow t h e sampling time of t h e v a r i a b l e f(X) t o be
decreased.
This decrease i s necessary i n order t o increase t h e bandwidth of the
Generalized I n t e g r a t o r .
The accumulation of t h e samples of f ( X ) i s accomplished
with an operational a m p l i f i e r with capacitive feedback and r e s i s t i v e input.
The
output voltage of t h i s amplifier changes i n a s t a i r c a s e manner,with each voltage
s t e p being proportional t o t h e amplitude of t h e corresponding sample of f(X) e
This e f f e c t i s i l l u s t r a t e d i n Figure 4. For t h e output of t h e Electronic
Generalized I n t e g r a t o r t o be a useful representation of t h e i n t e g r a l of f(X),
t h e s i z e of AX must be made small so t h a t t h e curve pictured i n Figure 4
approaches a smooth curve.
2-4.
The Incremental Detector
It w a s pointed o u t i n t h e previous section t h a t t h e Incremental Detector
of t h e E l e c t r o n i c Generalized I n t e g r a t o r i s a simplified version of t h e Aid
Converter.
The d e t a i l e d block diagram of t h i s s e c t i o n i s shown i n Figure 5.
Since t h e need f o r d i g i t a l readout c i r c u i t s has been eliminated, a simplificat i o n in t h e l o g i c c i r c u i t may be made t o reduce t h e t o t a l number of s t a g e s required.
The o t h e r s i m p l i f i c a t i o n i s based on t h e elimination of t h e absolute-
value c i r c u i t from t h e analog program.
This w a s done i n an e f f o r t t o avoid some
of t h e d i f f i c u l t i e s experienced i n t h e frequency response i n t h i s section of the
A I D Converter.
The absolute value circuit has been replaced by separate p o s i t i v e
and negative threshold d e t e c t o r s .
The threshold d e t e c t o r c i r c u i t is i d e n t i c a l
t o t h a t used i n t h e A I D Converter,
It i s b a s i c a l l y a NOR c i r c u i t i n which t h e
condition a t t h e output i s determined by both of t h e input signals.
i s shown i n Figure 6.
The c i r c u i t
One input s i g n a l i s a square wave t h a t i s supplied a s an
i n t e r r o g a t i o n pulse t o both detectors.
If and only i f t h e e r r o r voltage i s
above t h e threshold l e v e l of a d e t e c t o r during t h e i n t e r v a l of t h e i n t e r r o g a t i o n
pulse, a p u l s e appears a t t h e output of t h i s d e t e c t o r .
The P o s i t i v e Threshold
Detector i s a complementary c i r c u i t of t h e Negative Threshold Detector.
That
i s , t h e NPN t r a m i s t o r i s replaced by a PNP t r a n s i s t o r and a l l voltages a r e reversed i n sign.
It then performs i d e n t i c a l l y t o t h e Negative Threshold Detector
f o r e r r o r v o l t a g e s of t h e opposite p o l a r i t y .
A pulse from the output of one de-
t e c t o r i n d i c a t e s t h a t t h e X variable h a s changed by an amount equal t o AX i n t h e
p o s i t i v e direction,while a pulse from t h e output of t h e o t h e r d e t e c t o r i n d i c a t e s
t h a t t h e X v a r i a b l e has changed by an amount equal t o AX i n t h e negative d i r e c -
-7-
/-
TO ax
AMOUNTEQUAL
X
INPUT
0
RESET
0
PULSES
/\
uu
TIME
nn n
uuuuu u u
U
TIME
AMOUNT EQUAL
TO AX
ERROR
VOLTAGE
0
TIME
f(X)
INWT
0
TIME
0
Figure 4:
\
f
TIME
'TypicalW a v e f o r m s Found i n t h e Electronic Generalized I n t e g r a t o r
-8-
tion.
The r e s e t pulse then reduces the magnitude of t h e e r r o r voltage below
t h e threshold value.
A pulse occuring a t t h e output of a given Threshold Detector i s amplified
and used t o s e t a f l i p - f l o p c i r c u i t t h a t i s associated with t h a t d e t e c t o r .
A
pulse from a timing source contained i n t h e Electronic Generalized I n t e g r a t o r
i s applied simultaneously t o t h e r e s e t i n p u t s of both f l i p - f l o p c i r c u i t s .
This
pulse follows t h e I n t e r r o g a t i o n Pulse a f t e r an a c c u r a t e l y controlled i n t e r v a l .
This arrangement allows t h e output pulses of t h e f l i p - f l o p s t o be used t o c o n t r o l
t h e length of t h e Reset Pulses i n the Incremental Detector and the length of t h e
f ( X ) sampling p u l s e s appearing a t the i n p u t o f t h e Accumulating I n t e g r a t o r ,
2-5’.
The Electronic Sampling Gate
The Sampling Gate i s a transformer-driven switching c i r c u i t using two t r a n s i s t o r s as shown i n the c i r c u i t diagram of Figure 7.
The transformer connections
t o t h e t r a n s i s t o r s a r e so phased t h a t an irnput s i g n a l i n t o t h e primary of t h e
transformer t u r n s on both t r a n s i s t o r s and causes them t o s a t u r a t e .
I n the
absence of t h i s input signal, no base curren3 i s supplied t o t h e t r a n s i s t o r s ahd
This condition i s equivalent t o having
t h e y may be considered i n a c u t o f f state.
a v e r y l a r g e r e s i s t a n c e between t h e i n p u t and t h e output terminals of the g a t e
circuit.
When t h e pulse i s applied t o t h e input of t h e transformer t h e two
t r a n s i s t o r s s a t u r a t e , reducing t h e equivalent r e s i s t a n c e of t h e gate c i r c u i t by
a f a c t o r of approximately a million.
Since t h e two t r a n s i s t o r s are connected
with t h e i r emitters together, they form a b i d i r e c t i o n switch and w i l l accept
signals of both p o l a r i t i e s . ,
The time required f o r t h e gate t o change states
may be made small compared t o t h e time t h e gate i s closed.
E r r o r s due t o t h e
f i n i t s r e s i s t a n c e i n t h e g a t e c i r c u i t i n t h e open condition are somewhat compens a t e d by t h e second gate c i r c u i t , which i s connected t o a voltage t h a t i e equal
i n magnitude but opposite i n sign t o t h a t connected t o t h e f i r s t gate.
If t h i s
compensation i s n o t complete, t h e remainder of the analog signal may be n u l l e d
out.
The r e s i s t a n c e i n t h e gate c i r c u i t remaining i n t h e on period i s i n s e r i e s
with t h e r e s i s t o r t o t h e g r i d of the Accumulating Amplifier and i s accounted f o r
i n a d j u s t i n g t h e gain of t h i s c i r c u i t .
2-6.
The Analog Program
The analog program included i n t h e Electronic Generalized I n t e g r a t o r i s
i l l u s t r a t e d i n Figure 8.
Two operational a m p l i f i e r s a r e being included i n t h e
-9-
x
.
1
p- - 8 VOLTS
NEGATIVE
b
2N711
THRESHOLD
PNP
DETECTOR
1
EmCR
VOLTAGZ
INPUT
I
I
r+
L
u
r
NPN
POSITIVE
“RESHOIJ>
DETECTOR
Figure 6: C i r c x i t Diagrm of t h e Threshold Eetectors
Figure 7: C i r c u i t Diagram of t h e Eleefronic Sampling Gates
-11-
111 6
RELAY "A"
Ill
TO "HOLD" CONTROL
c
INTEGRATOH
-Y
I3 PUT
Figure 8:
Analog Program of t h e E l e c t r o n i c Generalized I n t e g r a t o r
-12
-
i n i t i a l design.
If d r i f t problems i n t h e Storage I n t e g r a t o r contained i n t h e
Incremental Detector s e c t i o n become significant, it may be necessary t o share
t h e a m p l i f i c a t i o n i n t h i s stage over two amplifiers.
Two r e l a y s a r e included
i n t h e c i r c u i t s t o provide a means of slaving t h e Electronic Generalized I n t e g r a t o r t o t h e c o n t r o l s of an analog computer.
The relay i n t h e c i r c u i t of t h e
Accumulator I n t e g r a t o r i s used t o b r i n g t h e output of t h i s a m p l i f i e r t o zero,
This relay i s intended t o
which resets t h e Electronic Generalized I n t e g r a t o r .
be c o n t r o l l e d by t h e RESET c o n t r o l on t h e analog computer.
The r e l a y i n t h e
c i r c u i t of t h e Storage I n t e g r a t o r of t h e Incremental Detector i s used t o place
t h e Electronic Generalized I n t e g r a t o r i n a HOLD condition.
This r e l a y r e s e t s
t h e Storage Integrator,holding t h e output voltgge of t h i s a m p l i f i e r a t zero,
but n o t d i s t u r b i n g t h e voltage stored i n t h e Accumulator I n t e g r a t o r .
2-7.
Work Completed
The major portion of t h e design of t h e Electronic Generalized I n t e g r a t o r
has been completed and t h e p a r t s necessary f o r construction of a working model
have been placed on order,
It i s intended t h a t t h i s working model be a complete,
self-contained u n i t capable of being used with a standard general-purpose analog
computer ,
Construction has begun on a minor portion of t h e c i r c u i t , but t h i s work has
n o t progressed very f a r due t o t h e lack of p a r t s .
A breadboard of the c i r c u i t
used i n t h e Sampling Gate has been b u i l t and preliminary evaluation
of t h i s
c i r c u i t has shown t h a t it w i l l operate with a l i n e a r i t y of b e t t e r than 0.2 percent over t h e intended f u l l - s c a l e range.
B e t t e r performance i s expected by ad-
j u s t i n g t h e operating p o i n t of t h e g a t e t r a n s i s t o r s .
If s t i l l f u r t h e r improve-
ment i s needed, matched t r a n s i s t o r s w i l l be used,
2-8.
Future Work
The work during t h e next quarter should include t h e completion of a work-
i n g u n i t of t h e Electronic Generalized I n t e g r a t o r and a preliminary evaluation
of i t s performance.
-13-
Chapter I11
GENERATION OF NONSTATIONARY NOISE FOR ANALOG-COMPUTER MONTE CARLO STUDIES
Task I1 of GIT/EES P r o j e c t A-5'88 concerns t h e i n v e s t i g a t i o n of analog techniques f o r t h e generation of nonstationary noise v o l t a g e s t o be used i n analog
Monte Carlo s t u d i e s . O f p a r t i c u l a r i n t e r e s t i s t h e problem of nonstationary
shaping of Gaussian, band-limited, white noise. A s discussed i n t h e Sections
below, e f f o r t s during t h e p a s t quarter have centered about t h r e e p r i n c i p a l
topics--analysis of c e r t a i n s p e c i f i c s t o c h a s t i c processes, s e l e c t i o n of statist i c a l f u n c t i o n a l s u s e f u l i n specifying nonstationary noise, and i n v e s t i g a t i o n of
l i n e a r networks f o r nonstationary shaping,
Current plans a r e t o i s s u e separately a p r o j e c t t e c h n i c a l note covering the
fundamentals of s t o c h a s t i c process theory as it p e r t a i n s t o Task 11. This note
w i l l serve as a p r o j e c t reference manual and w i l l d e f i n e several f u n c t i o n a l s
which a r e i n c o n s i s t e n t l y defined i n the open l i t e r a t u r e .
3-1.
Analysis of Certain Stochastic Processes
It i s n o t possible t o e x h i b i t a convenient representation of t h e arbitrary
It i s i n s t r u c t i v e , however, t o examine t h e s t a t i s t i c s of
s t o c h a s t i c process.
a few "closed-form" processes which are expressible a s time f u n c t i o n s with random parameters *
a.
Sinewave of Random Phase and Amplitude
The s t o c h a s t i c process represented by
X(t) = x s i n ( t + y ) j
where x and y a r e independent random v a r i a t e s , mfght a r i s e through t h e use of a
sinewave generator whose output amplitude v a r i e s slowly i n a random way s o t h a t
f o r each sample function t h e amplitude may be considered a constant.
The random
v a r i a t e y t a k e s i n t o account t h e random phase of t h e s i g n a l with r e s p e c t t o t h e
beginning of t h e sample.
(To make t h i s process more n e a r l y an e x a c t representa-
t i o n of t h e experiment, t h e i n t e r v a l between sample f u n c t i o n s should be s u f f i c i e n t l y long t o provide independence. )
(3-1)
EX(t) = Ex Esin( t + y ) = ( s i n t ) Ex E(cos y) .t. ( c o s t ) Ek E( s i n y)
which,in general, i s a function of t.
The covariance function i s given by
(3-2)
Cov(X) = CX(T,t) = EX(t)X(t*T)
7
Ex
2
E[x
2
s i n ( t + y ) sin(t+T+y))
[ c o s T - cos(2t*T) Ecos2y
* sin(2t.t.T) Esin2yl ,
,
which, i n general, i s a l s o a function o f time, t.
The time average of X ( t ) i s given by
i n v a r i a n t with r e s p e c t t o t h e sample function chosen,jc
The t i m e - s t a t i s t i c equivalent of t h e covariance function, t h e c o r r e l a t i o n
function, i s
a
%(T,x,y)
= AX(t)X(t+T) = lim
a m
1
-a
x 2 s i n ( t + y ) sin(t+y+T) d t ,
I
which reduces t o a form i n v a r i a n t with respect t o y:
%(T,x)
(3-4)
= 1
2 x 2COS T.
If we assume t h a t t h e random phase, y, i s uniformly d i s t r i b u t e d over t h e
i n t e r v a l (-n,+n) with p r o b a b i l i t y d e n s i t y function
then t h e process mean, as given by Equation (3-1), reduces t o zero i d e n t i c a l l y
in t.
That is,
EX(t) = AX(t) = 0, a l l t.
Also, under u e assumption of (3-5'), t h e covariance becomes i n v a r i a n t i n
t and Equation (3-2) reduces t o
(3-6)
CX(T) = ;(cos
T) Ex2
which i s c h a r a c t e r i s t i c of t h e "wide-sense s t a t i o n a r y f 1 process.
__
"The symbol IlA11 i s used h e r e i n t o denote t h e time-average operator, as cont r a s t e d with t h e ensemble-average o p e r a t o r rlE1le
-15-
If t h e power s p e c t r a l d e n s i t y function, fX,, i s defined as t h e Fourier
>c
transform (with r e s p e c t t o T) of the covariance function,”
fX(w,t)
(3-7)
rC,(T,t)
-a
exp(-jwT) dT
,
then Equation (3-6) y i e l d s
Note t h a t t h e d e f i n i t i o n of (3-7) applied t o t h e nonstationary procegd of
Equation (3-2) y i e l d s a complex power s p e c t r a l d e n s i t y function which i s of
l i t t l e physical significance.
Other d e f i n i t i o n s i n common use, such as those
have o t h e r drawbacks which l i m i t t h e i r
suggested by Pagedk and Middletonrc’‘x,
usefulness.
Page’s tlinstantaneous power spectrumtt y i e l d s a random v a r i a t e and
Middleton’s “ i n t e n s i t y d e n s i t y ” i s i d e n t i c a l l y zero f o r a l l finite-energy processes.
Considerable a t t e n t i o n i s given t o t h i s problem in t h e forthcoming
t e c h n i c a l note.
b.
Sinewave of Random Phase and Frequency
L e t x and y be independent random v a r i a t e s with t h e p r o b a b i l i t y d e n s i t y
f u n c t i o n of y given by
Py(Y) = 0
J
lYb
and t h a t of x s a t i s f y i n g
”There does n o t appear t o be a c o n s i s t e n t d e f i n i t i o n of t h e power s p e c t r a l
d e n s i t y function f o r nonstationary processes. The d e f i n i t i o n used here i s
e s s e n t i a l l y t h a t of Kharkevich: A.A. Kharkevich, Spectra and Analysis, Cons u l t a n t s Bureau, New York, 1960 (Translated from R-I,P.
‘r
*Y
C .H. .Page, Vnstantaneous Power Spectrat1, Journal of Applied Physics, Vol.
23, No. 1, Jan. 1952.
-:FH
-
David Middleton, An Introduction t o S t a t i s t i c a l Communication Theory,
McGraw-Hill Book Comzny, New York, 1960.
-16-
Cons.truct t h e s t o c h a s t i c process
X ( t ) = 2 s i n ( x t 4- y) ,
where t h e f a c t o r It2'l i s chosen (without loss of g e n e r a l i t y ) t o make t h e average
power u n i t y .
This process would a r i s e , f o r example, through t h e use of a sine-
wave generator whose frequency v a r i e s slowly i n a random way so t h a t f o r each
sample f u n c t i o n t h e frequency may be considered a constant,
A s i n Section +la,
v a r i a t e y accounts f o r t h e random phase of t h e s i g n a l with r e s p e c t t o t h e
origin.
The expected value of X i s zero f o r a l l t and t h e covariance function i s
i n v a r i a n t i n t:
oc,
CX(T) = E(cos xT) =
1
p,(x)
c o s xT dx
-a,
The process i s t h e r e f o r e wide-sense s t a t i o n a r y and t h e d e f i n i t i o n of power
s p e c t r a l d e n s i t y given by Equation (3-7) i s applicable,
Applying t h i s d e f i n i -
t i o n , we o b t a i n
03
fx(w) =
CX(T) exp(-joT) dT =
-03
03
-03
Interchanging t h e order of integration,
f,(o)
= r P x ( x ) r e x p ( - j c o T ) cos XT dT dx
-OD
-00
r
= n]
03
px(x) [S(w-x) + S(w+x>] dx
-OD
fx(w) = 2npx(w)
a3
1 exp( -JOT) J px(x) cos xT dx dT
r
P
f
-03
Thus, t h i s process h a s the unusual property t h a t t h e power s p e c t r a l d e n s i t y
( i n t h e sense of Equation (3-7)) i s proportional t o t h e p r o b a b i l i t y d e n s i t y of
the p r i n c i p a l primitive v a r i a t e .
c.
Gaussian, Band-Limited, White Noise
Most commercially a v a i l a b l e noise generators d e l i v e r an output voltage
which approximates the Gaussian, band-limited,
white noise process.
f u l , therefore, t o express t h i s process i n closed form.
I t i s use-
Such a r e p r e s e n t a t i o n
can be developed a s shown below.
It i s convenient t o d i s c u s s f i r s t t h e f o l l o w i n g lemma:
03
Lemma I:
Let S =
s i n ( u-nn)
C
n= -00 u -nn
s i n ( u+v-m)
u*v-nn
.
sin v
then S = 7
Proof:
Write S i n t h e form
00
s i n u cos nn
u -nn
n- -00
s= z
e
sin(u*v) cos nn
u+v-pl
= s i n u sin(u*v) C
Expand t h e summand by p a r-t i a l f r a c t i o n s :
- sin u sin(u+v) & hPG - V
u+v-nrr
I.
Now
and
2
1
1 +:
ctn(u*v) - 2 nn
u+v-nn
7
n+0
-
x
"See, f o r example, ,DSH. Menzel, Fundamental Formulas of Physics, Section 14.8,
Dover Publications, New York, 1960,
-18-
.
Then S may be expressed as
- s i n u sin(u+v)
V
sin v
s i n u sin(u+v)
For t h e construction of t h e Gaussian, band-limited,
white n o i s e process,
l e t xn be t h e generic symbol for a s e t of independent, zero-mean, Gaussian
2
Form t h e s t o c h a s t i c process,
random v a r i a t e s with common variance, o
5,
- JJ
sin(at-nn)
- n=-N xn at-nn
'
'N
Then
N.
XN( tl),
. ,XN( t i )
e
,. ,XN( tk) i s a k-variahe Gaussian v a r i a b l e f o r every
Since
EX.X
I J
and
= 0, i + j J
2
2
hi = o , a l l i,
and
''
s i n ( at-nn)
n=-a at-nn
+a,
3
2
= 1
( a s p e c i a l case of Lemma I ) ,
then t h e r e e x i s t s a l i m i t i n g random variable, X(t), given by
x ( t ) = l i m xN(t) = O0
Z xn sin(at-nx)
at.nn
m
n=-00
2
Further, X ( t ) i s Gaussian with z e r o mean and variance o
X i s given by
CX(T) = EX(t)X(t+T)
-19-
The covapiance of
Because of t h e independence of the x Is, t h i s may be w r i t t e n
n
2
CX(T) = E n xn
s i n ( at-nn)
at-nn
s i n ( at+aT-nn)
at+aT-nn
~
o2 nz
s i n ( at-nn)
at-nn
s i n ( at+aT-nn)
at*aT-nn
By Lemma I, t h i s reduces t o
2 sin aT
aT
'
0 -
which i s i n v a r i a n t i n t and, therefore, r e p r e s e n t s a wide-sense s t a t i o n a r y
process.
But X ( t ) has been shown t o be a Gaussian procesk and i s then
"
s t r i c t l y stationary.
ic
The power s p e c t r a l d e n s i t y may now be computed as t h e Fourier transform
of CX'
Hence, t h e s t o c h a s t i c process represented by
00
(3-10)
X(t) =
2 xn
n= -a,
s i n ( a t -nn )
at-nn
Y
where t h e x 1 s a r e independent Gaussian v a r i a t e s with zero mean and common
n
variance, i s a s t a t i o n a r y Gaussian band-limited white n o i s e process with bandwidth a.
3-2.
Functionals Useful i n Specifying Nonstationary Noise
The s p e c i f i c a t i o n of s t a t i s t i c a l p r o p e r t i e s i n s t a t i o n a r y processes i s
n o t a p a r t i c u l a r l y d i f f i c u l t proposition.
I n a given Monte Carlo study, f o r
example, we might r e q u i r e a noise souce which d e l i v e r s a s t a t i o n a r y process
having a s p e c i f i e d power s p e c t r a l d e n s i t y ( o r , equivalently, a c e r t a i n covariance f u n c t i o n ) and a specified u n i v a r i a t e p r o b a b i l i t y d i s t r i b u t i o n .
d!ifficult,
It may be
i n cases, t o generate t h e d e s i r e d process, but i t i s g e n e r a l l y not
hard t o determine what i s required.
__
"J.H. Laning and R. H. Battin, Random Processes i n Automatic Control, McGrawHill Book Company, New York, 19~~~
-
-20-
The s e l e c t i o n of appropriate f u n c t i o n a l s t o d e s c r i b e a nonstationary
I n t h e f i r s t place,
process, on t h e o t h e r hand, may require much e f f o r t .
s e v e r a l of t h e f u n c t i o n a l s commonly used a r e not u n i v e r s a l l y defined i n t h e
same way.
Secondly, t h e n o n s t a t i o n a r i t y r u l e s out t h e p o s s i b i l i t y of ergod-
i c i t y and r e q u i r e s t h a t meaningful s p e c i f i c a t i o n s be made on t h e ensemble
basis.
Considerable e f f o r t has been devoted t o t h i s problem during t h e p a s t quar-
ter, with t h e following t e n t a t i v e conclusions:
( a ) The covariance function, a s defiRed by Equation (3-2), appears t o be
a more u s e f u l and meaningful function than t h e power s p e c t r a l d e n s i t y . A s
mentioned i n Section 3-1, t h i s l a t t e r q u a n t i t y i s n o t w e l l defined and may
lead t o a complex functional o r a random v a r i a t e ,
( b ) Generally speaking, t h e complete s p e c i f i c a t i o n of a nonstationary
process i s impractical. If X ( t ) r e p r e s e n t s t h e process, a complete descript i o n i s provided only by c i t i n g t h e k-variate d i s t r i b u t i o n of [ X ( t , ) , . . . , x ( t
f o r every i n t e g r a l k ( o r t h e equivalent information)
e
k )]
Thus it i s necessary t o
s e l e c t c a r e f u l l y t h e s t a t i s t i c a l parameters of p r i n c i p a l i n t e r e s t .
(This i s
t r u e of many s t a t i o n a r y processes, also, but appears t o be a more v i t a l quest i o n i n t h e nonstationary case.)
With regard t o t h e i n p u t random process, two d e c i s i o n s must be made when
s e t t i n g up a Monte Carlo study:
(1) Which f u n c t i o n a l s o r parameters w i l l be
used t o describe t h e random process, and ( 2 ) What i s t h e d e t a i l e d form of t h e
chosen functionals.
Decision (1) i s u s u a l l y made on t h e b a s i s of t h e response
v a r i a b l e s of i n t e r e s t i n t h e system under study.
Decision ( 2 ) i s d i c t a t e d by
t h e physical nature of the s t o c h a s t i c process being simulated.
To i l l u s t r a t e
t h e way i n which t h e s e l a t t e r s p e c i f i c a t i o n s a r e derived, we consider the f o l lowing very general example.
A rocket i s t o be f i r e d i n a v e r t i c a l a t t i t u d e and assumed t o r i s e a t a
constant r a t e .
A s it moves through t h e atmosphere, i t i s subject t o b u f f e t i n g
winds which a r e random i n nature.
It i s desired t o simulate t h e rocket system
and observe i t s response t o t h e simulated random wind.
A number of computation-
a l runs w i l l be performed t o gather s t a t i s t i c a l d a t a on t h e rocket performance.
Conceptually, a t l e a s t , t h e required d a t a on t h e s t a t i s t i c a l behavior of
t h e wind could be obtained by placing sensing elements a t various a l t i t u d e s
and recording sample f u n c t i o n s of wind velocity.
-21-
If we assume t h a t t h e speed
and d i r e c t i o n of t h e wind a r e independent s c a l a r q u a n t i t i e s , then t h e s e may
be simulated s e p a r a t e l y and combined i n t h e proper r e l a t i o n s h i p by a resolver.
F o r s i m p l i c i t y we w i l l consider only t h e s c a l a r wind speed.
Viewed as a whole, t h e wind speed may be thought of as a s t o c h a s t i c process of two parameters--the
a l t i t u d e , h, and time, t.
Let the process be
denoted by X(h,t) and symbolize t h e mean and covariance by b ( h , t ) and
Cx(h,H,t,T):
= EX( h, t ) X ( h+H, t+T)
Cx( h,H, t, T)
I f it i s understood t h a t t h e rocket w i l l be launched only under good
weather conditions and a t a p a r t i c u l a r time of day, then it may be assumed
t h a t t h e wmd a t a given a l t i t u d e may be represented by a s t a t i o n a r y stochas\c
t i c process."
Under these conditions, t h e mean and covariance a r e i n v a r i a n t
i n t and may be w r i t t e n as px( h) and CX(h,H,T)
Since t h e rocket i s t o r i s e a t a constant r a t e , v, we have f o r t h e a l t i tude
h = vt
and
H
vT.
The process m a y now be w r i t t e n i n terms of a s i n g l e parameter, t:
X(vt,t),
This i s expressed more concisely a s a new process Y(t) having mean and covariance given by
py(t)
EY(.t) = h ( v t ) = FX(vt,t)
Cy(Tst) := EY(t)Y(t+T) = C&vt,vT,T) = EX(vt,t)X[v(t+T),t+T]
.
If t h e s e q u a n t i t i e s seem appropriate t o t e s t t h a t p o r t i o n of t h e rocket i n which
w e have an i n t e r e s t , then e f f o r t s may be made t o generate a nonstationary proc e s s having t h e s e p r o p e r t i e s ,
Othemise, more appropriate f u n c t i o n a l s may be
derived i n t h e same way and attempts made t o generate a s u i t a b l e process.
--
"However, t h e rocket w i l l be changing a l t i t u d e c o n s t a n t l y and t h i s considerat i o n w i l l lead t o a nonstationary process,
-22-
3-3.
Linear Networks f o r Nonstationary Shaping
There appear t o be a t least t h r e e techniques which may be employed(sepa-
c a t e l y o r i n combination) t o d e r i v e nonstationary s t o c h a s t i c processes having
prescribed p r o p e r t i e s : nonlinear networks ( p o s s i b l y time-varying), time varyi n g l i n e a r networks, and u t i l i z a t i o n of t r a n s i e n t e f f e c t s i n time-invariant
networks,
During t h e p a s t q u a r t e r a t t e n t i o n h a s been given p r i m a r i l y t o t h e
-u-
use of l i n e a r networks."
a.
The r e s u l t s are summarized below.
Time-Varying Linear Networks
The a n a l y s i s of time-varying networks has been extensively studied by
several a u t h o r i t i e s , notably Zadeh and Darlington.
Xadeh i n p a r t i c u l a r h a s
developed techniques f o r obtaining the system f u n c t i o n and impulsive response
of l i n e a r v a r i a b l e networks from t h e so-called fundamental equation of t h e
network--i.e.,
t h e d i f f e r e n t i a l equation which r e l a t e s t h e input and output.
If t h e fundamental equation of t h e network i s of t h e form
where x ( t ) and y ( t ) are t h e i n p u t and output of t h e network and L and K are
l i n e a r d i f f e r e n t i a l operators, then t h e system function, H( Joyt ) , f o r t h e network must satisfy t h e following d i f f e r e n t i a l equation:
The boundary conditions for t h e above equation must e i t h e r be given o r be derivable from t h e network.
I n practice, even f o r r e l a t i v e l y simple networks,
t h e equation f o r H ( j 9 ) i s s o complex as t o preclude t h e p o s s i b i l i t y of obtain-
fng a closed-form s o l u t i o n and r