Hi there!
I am trying to explore tensorflow and apply it for signal detection - I need to design a neural network that uses a measured signal as an input and finds resonances in that signal. Actually finds the coordinates of resonances - peaks or specific regions where the resonance can be…
Dataset I have
Signals are stored in separate csv files, let’s say: signal1.csv, signal2.csv… signal1000000.csv.
Each signal is represented using the columns T, E, sigmaT…
T - is the value of a signal, E - x coordinate
each row stands for point of E_i, where i = 1,2…N (N - total number of signal points)
It’s pretty easy to detect each large resonance visually, but when signals are noised and amplitudes are low - it becomes much harder to detect and evaluate. And evaluators use a large number of rules and parameters to detect the signal and obtain the coordinates of resonance in the signal.
These rules use information about resonance amplitude, width, relatives (distances to the closest neighbors) and some other information.
The link for the visualization of a signal is:
(I don’t know why I can’t add an image to the post, sorry).
Green lines show the position of real resonance in data - points are additionally labeled using classes but it’s not important on the first stage.
An example of data in one signal:
one_example_signal = pd.read_csv(f, index_col=0)
Here is the data example:
E exp_trans exp_trans_unc marked_points
index
4883 20.005036 0.710289 0.057905 0
4882 20.012073 0.780100 0.060669 0
4881 20.019113 0.808801 0.063798 0
4880 20.026157 0.676023 0.053775 0
4879 20.033205 1.006920 0.072483 0
The example signal can be found here: example.transmission - Google Drive
Each signal in this dataset is evaluated by a specialist and I know the solution for each signal - e.g. signal1.solution.csv - each row of that file has a field - E_l - the coordinate of evaluated resonance.
How to construct a neural network to detect that resonances?
What I did and tried
I have labeled the points in each signal of the dataset I have. I used marked_points field: 0 class stands for point - if it doesn’t belong to the resonance, and 1 - if it’s very close or belongs to the body of the resonance (really there are a lot of classes - but let’s try with the binary classification of points in the signal).
And I want to make such a NN or maybe one can propose something better?
input() - two numpy arrays - exp_trans, E - representation of T(E) - see the pic.
Several intermediate layers - ( e.g. 2-3 dense layers )
output layer - has the same size as input - it will classify each point of the input signal - if it’s resonance or not…
The problem is that I have signals with varying size: e.g. one signal can have 1000 points, and one - 10000 points… Another issue is that signals have nonuniform grid in E-axis - so the distance between each point varies.
I can theoretically standardize the size of all signals - so they will have max. 5000 points each, so I can give the network signals with the same size.
-
But maybe there is a way to use networks with various input size?
-
How to define an output layer of a net if it must give the class of each input point of a signal?
Now I have only 2 classes of points: ‘resonance’ - 1, and ‘not a resonance’ - 0. (in real life there are a lot of classes that are used in application).
A simplified case and representation are that the output neuron will give the possibility that point with coordinates T_i , E_i can be classified as ‘resonance point’. So after processing of the signal we will have an output array P (the same size as the input, e.g. in the input we had 1000 points, so we will have 1000 outputs). Each output value - P_i is the possibility that point number i with coordinate E_i belongs to the resonance.
And we can construct and visualize a curve P_i (E_i) - curve as output - the possibility that point number i = 0, 1…N-1, belongs to the resonance.
Thanks for any comments and suggestions.